Serverless Inference
An OpenAI-compatible API for chat, image, and video models. Pay per token, scale to zero, and ship in one line of code — no instances to launch or shut down.
Because the API is OpenAI-compatible, the official SDKs, curl, and our CLI all work against it out of the box.
from openai import OpenAI
client = OpenAI(
base_url="https://api.gpu.ai/v1",
api_key="gpuai_live_...",
)
stream = client.chat.completions.create(
model="gpuai/qwen2.5-7b-instruct",
messages=[{"role": "user", "content": "Explain WireGuard in one sentence"}],
stream=True,
)
for chunk in stream:
print(chunk.choices[0].delta.content or "", end="")Get a key with gpu login or mint one in the dashboard. See the full serverless inference guide.
All the capability of dedicated GPUs, none of the fleet management.
Point any OpenAI SDK, curl, or the gpu CLI at our endpoint. Swap the base URL and key — your existing code works unchanged.
No instances to launch, patch, or shut down. You're billed per token, per image, or per second of video — nothing when you're idle.
The default serverless tier keeps models warm, so requests return in milliseconds. Opt into the economy tier when price matters more than latency.
The catalog is probe-verified and refreshed hourly from what's actually serving warm right now — GET /v1/models always reflects what you can call this minute.
Capacity flexes with your traffic automatically. Send one request or ten thousand per second — there's no fleet to provision or tune.
Chat completions, image generation, and video generation share the same base URL, the same auth, and the same billing.
Same base URL, same key, same billing — pick the modality your product needs.
Streaming chat completions across open-weight LLMs. Drop-in replacement for the chat completions endpoint you already call.
Text-to-image generation from fast, high-quality diffusion models. Submit a prompt, get pixels back.
Text-to-video generation. Submit a job, poll for progress, and download the finished clip when it's ready.
Representative starting prices — the live catalog surfaces the cheapest partner per model.
| Model | Modality | From |
|---|---|---|
| Qwen2.5 7B Instruct | Chat | $0.20 / 1M tokens |
| Llama 3.3 70B Instruct | Chat | $0.60 / 1M tokens |
| FLUX.1 [schnell] | Image | $0.003 / Mpxl |
| Wan 2.2 T2V | Video | $0.05 / output sec |
Pricing updates live across partners. Run gpu models list for current rates, or opt into the :economy tier for cheaper, compute-priced inference. Need a dedicated instance instead? On-Demand gives you the whole GPU.