Inference API.

§ 05.1Base URL & authentication¶

Everything lives under https://api.gpu.ai/v1 and authenticates with Authorization: Bearer gpuai_live_.... Get a key with gpu login or mint one in the dashboard. Scopes: serverless:read lists models, serverless:write runs chat, image, and video generation, and billing:read reads usage; a full_access key covers all three.

Because the API is OpenAI-compatible, the official OpenAI SDKs work unchanged for chat; just override the base URL and API key. The video endpoints follow the same submit → poll → download layout as OpenAI's video API.

§ 05.2Models¶

The catalog is live: built from the models actually being served warm right now, refreshed roughly hourly and probe-verified. Each entry carries a modality (chat, image, or video), pricing, and the request parameters the model accepts. Filter with ?modality=video.

Most chat models offer two tiers: serverless (default: low latency, no cold start; use the bare model id) and economy (cheaper, compute-priced, may cold-start ~30–60s; opt in with the :economy suffix, e.g. gpuai/qwen2.5-7b-instruct:economy).

§ 05.3Chat completions¶

Standard OpenAI chat completions, including stream: true for server-sent events.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gpu.ai/v1",
    api_key="gpuai_live_...",
)

stream = client.chat.completions.create(
    model="gpuai/qwen2.5-7b-instruct",
    messages=[{"role": "user", "content": "Explain FRP in one sentence"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

§ 05.4Image generation¶

Image generation is synchronous and OpenAI-compatible. v1 returns images as base64 only: data[].b64_json carries the bytes, and response_format: "url" is rejected. The response adds usage.image_count, the metered unit.

curl https://api.gpu.ai/v1/images/generations \
  -H "Authorization: Bearer gpuai_live_..." \
  -H "Content-Type: application/json" \
  -d '{"model":"gpuai/flux.1-schnell","prompt":"a watercolor fox","n":1,"size":"1024x1024"}'

{ "created": 1733800000, "data": [{ "b64_json": "iVBORw0KGgo..." }], "usage": { "image_count": 1 } }

§ 05.5Video generation¶

Video generation is asynchronous: POST /v1/videos returns a job in the queued state immediately. Poll GET /v1/videos/{id} until the status reaches a terminal value, then download the mp4 from GET /v1/videos/{id}/content.

# 1. Submit: returns a queued job immediately
curl https://api.gpu.ai/v1/videos \
  -H "Authorization: Bearer gpuai_live_..." \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "model": "gpuai/wan-2.2-t2v",
    "prompt": "ocean waves at sunset",
    "seconds": 5
  }'

# 2. Poll until status is terminal
curl https://api.gpu.ai/v1/videos/vid_... \
  -H "Authorization: Bearer gpuai_live_..."

# 3. Download the mp4
curl https://api.gpu.ai/v1/videos/vid_.../content \
  -H "Authorization: Bearer gpuai_live_..." -o out.mp4

{
  "id":           "vid_...",
  "object":       "video",
  "model":        "gpuai/wan-2.2-t2v",
  "status":       "completed",
  "progress":     100,
  "seconds":      5,
  "cost_cents":   18,
  "created_at":   1733800000,
  "completed_at": 1733800240,
  "expires_at":   1733886640
}

§05.5.1Job lifecycle¶

§05.5.2Parameters¶

Each model declares the request parameters it accepts in its catalog entry (parameters on GET /v1/models), typically seconds, the clip length, whose bounds and default come from the model. Parameters a model does not declare are rejected with an invalid_request_error rather than silently ignored: live video models generate at their default output resolution, so most do not accept size. Omitting seconds uses the model's default clip length.

§05.5.3Billing, idempotency & retention¶

You're charged by seconds of output video, measured when the job completes. Jobs that never complete (failed, cancelled, or expired before finishing) are never billed. The authoritative charge is the cost_cents field on the job object; it stays on the job even after the artifact expires.

Completed videos are retained for 24 hours; the job carries an expires_at (unix seconds). After that, the artifact is gone and GET /v1/videos/{id}/content returns 410 Gone with an expired error code. Download promptly and store the file on your side.

Stop an in-flight job with POST /v1/videos/{id}/cancel (a cancel verb, not DELETE). Cancellation is idempotent: a terminal job is returned unchanged, and a job whose output is already committed can no longer be cancelled.

§ 05.6Embeddings¶

Embeddings are synchronous and OpenAI-compatible. Pass input as a single string or an array of up to 2048 strings, bounded so the response stays under 64 MiB (about 680 inputs at 4096 dimensions; over-limit requests are rejected with a 400 before any processing); you get one vector per input, in request order, with data[].index matching the input's position. List the served models with GET /v1/models?modality=embedding.

curl https://api.gpu.ai/v1/embeddings \
  -H "Authorization: Bearer gpuai_live_..." \
  -H "Content-Type: application/json" \
  -d '{"model":"gpuai/qwen3-embedding-8b","input":["hello","world"]}'

{ "object": "list",
  "data": [{ "object": "embedding", "index": 0, "embedding": [-0.019, 0.0071, ...] }],
  "model": "gpuai/qwen3-embedding-8b",
  "usage": { "prompt_tokens": 4, "total_tokens": 4 } }

Billed on input tokens only, at the model's listed input rate, rounded up to the next whole cent per request — every call is billed at least 1¢, so batch inputs into one call to pay the listed per-token rate. An embeddings call emits no completion tokens, so usage carries only prompt_tokens and total_tokens. encoding_format supports float only; base64 is rejected with unsupported_parameter. A chat model id here returns 404 model_not_found.

§ 05.7Audio generation¶

§ 05.8Tool calling, JSON mode & structured outputs¶

{ "error": { "message": "Parameter \"tools\" is not supported in v1.1", "type": "invalid_request_error", "param": "tools", "code": "unsupported_parameter" } }

They are rejected rather than silently ignored, so a request that asked for a constrained answer never returns an unconstrained one. The best-effort workaround is to ask for JSON in the system prompt at temperature: 0, then parse and validate the reply against your own schema client-side and retry when it does not match — the serverless inference guide carries a full curl + Python example. Native pass-through (including streaming delta.tool_calls) is planned for the serverless-parity work; if it blocks you, tell us at support@gpu.ai — demand ordering is how this gets scheduled.

§ 05.9Errors¶

Errors use the OpenAI envelope, so SDK error handling works as-is:

{ "error": { "message": "...", "type": "invalid_request_error", "code": "model_not_found" } }

• model_not_found (404): the live catalog no longer serves that model id. Re-fetch GET /v1/models and pick another. On streaming this arrives as the final SSE error event before [DONE].
• stream_limit_exceeded (429): too many inference requests open at once on this API key. Close streams you're no longer reading or retry shortly; this is separate from the per-second rate limit (rate_limit_exceeded).
• upstream_unavailable (503): a model's serving capacity is temporarily degraded. Returned promptly; retry after a short interval, since the platform routes around degraded capacity automatically.