GPU Instances

Renting a GPU on GPU.ai is nine HTTP calls: find a GPU, check its price, upload an SSH key, create the instance, poll the operation until it settles, read the connection details, SSH in, terminate, confirm it's gone. This page walks the whole flow as raw curl — no SDK, no CLI required — and shows the SDK equivalent only at the two steps where a typed client genuinely saves you something.

Everything here is the same surface the gpu CLI and the official SDKs are built on; they are conveniences over these endpoints, not a different API.

This flow spends money. A created instance bills by the second from the moment it boots until you terminate it. Every example caps the price with max_price_per_hour, and the flow ends with terminate and a verify-gone check — a walkthrough that leaves an instance running is a walkthrough with a bug in it.

Base URL & authentication

Base URL	https://api.gpu.ai/v1
Auth	Authorization: Bearer gpuai_live_…

Get a key with gpu login (it stores a gpuai_live_… key in ~/.config/gpu/credentials.json) or mint one in the dashboard.

The catalog endpoints (/v1/gpu-types, /v1/pricing) need no key at all — steps 1 and 2 below run with no Authorization header. Everything from step 3 on is account-scoped and needs the bearer token. Reading spend from /v1/billing/spending-limit additionally needs the billing:read scope.

Keep the key in an environment variable rather than pasting it into scripts:

export GPUAI_API_KEY=gpuai_live_...

Endpoints

Method	Path	Auth	Purpose
GET	/v1/gpu-types	none	What GPU models exist, with VRAM / CPU / RAM
GET	/v1/pricing	none	Live offers: price, region, tier, availability
POST	/v1/ssh-keys	bearer	Register a public key to inject at launch
DELETE	/v1/ssh-keys/{id}	bearer	Remove a registered key
POST	/v1/instances	bearer	Create an instance — 202 + Operation-Id
GET	/v1/operations/{id}	bearer	Poll the async operation to a terminal state
GET	/v1/instances/{id}	bearer	Status + connection details (host, port, ssh command)
DELETE	/v1/instances/{id}	bearer	Terminate — 202 + Operation-Id, idempotent
GET	/v1/usage	bearer	Time-bucketed gpu_seconds / cost_cents
GET	/v1/billing/spending-limit	billing:read	Monthly limit and spend to date

Every list endpoint is cursor-paginated: the response carries data plus a next_cursor that is null on the last page.

1. List GPU types

No key needed — this one runs in ten seconds from a cold start.

# What GPU models does GPU.ai carry?
curl "https://api.gpu.ai/v1/gpu-types?limit=100"

{ "data": [ { "gpu_type": "h100_sxm", "vram_gb": 80, "cpu_cores": 26, "ram_gb": 200 } ],
  "next_cursor": null }

gpu_type is the identifier you pass everywhere else in this flow.

2. Check pricing

Also keyless. Offers with zero availability are omitted unless you ask for them with include_unavailable=true; results are ordered cheapest-first.

# What can I actually launch right now, and what does it cost?
curl "https://api.gpu.ai/v1/pricing?include_unavailable=false&limit=200"

{ "data": [ { "gpu_type": "rtx_4090", "gpu_count": 1, "region": "us-east",
              "tier": "on_demand", "price_per_hour": 0.44, "available": 6,
              "instant_boot": true, "disk_configurable": true,
              "disk_price_per_gb_hour": 0.00011 } ],
  "next_cursor": null }

Field notes that matter for the next step:

• available is a count of units, not a boolean — 0 means nothing to launch.
• price_per_hour covers the whole listed configuration, all gpu_count

GPUs of it. A gpu_count: 2 row is not twice a gpu_count: 1 row's price.

• tier is on_demand or spot. Spot is cheaper and can be reclaimed.
• instant_boot: false capacity is real, just slower to come up (minutes,

occasionally ~15 for baremetal) — size your poll deadline accordingly.

3. Register an SSH key

Generate a keypair locally and register the public half. The private half never leaves your machine.

ssh-keygen -t ed25519 -N '' -f ~/.ssh/gpuai_demo

# Register the public key — the response id is what you pass at create time
curl https://api.gpu.ai/v1/ssh-keys \
  -H "Authorization: Bearer gpuai_live_…" \
  -H "Content-Type: application/json" \
  -d "{\"name\":\"laptop\",\"public_key\":\"$(cat ~/.ssh/gpuai_demo.pub)\"}"

{ "id": "key-abc123", "name": "laptop", "fingerprint": "SHA256:…",
  "created_at": "2026-08-06T12:00:00Z" }

Keys can be removed later with DELETE /v1/ssh-keys/{id}.

4. Create the instance

Creation is asynchronous. POST /v1/instances returns 202 Accepted with an Operation-Id response header (and the same operation_id in the body) — not an instance. The instance id arrives later, from the operation.

gpu_type, gpu_count, and tier are required; ssh_key_ids and max_price_per_hour are optional but you want both. Always send max_price_per_hour — placement re-checks it on every attempt, so a launch is never billed above your cap even if the cheapest offer disappears mid-placement and the engine falls back to the next one.

# Create — 202 Accepted; -i so you can see the Operation-Id header
curl -i https://api.gpu.ai/v1/instances \
  -H "Authorization: Bearer gpuai_live_…" \
  -H "Content-Type: application/json" \
  -d '{"gpu_type":"rtx_4090","gpu_count":1,"tier":"on_demand","ssh_key_ids":["key-abc123"],"max_price_per_hour":1.00}'

HTTP/2 202
operation-id: 7a1e9d84-2c50-4f6b-9b31-0c5f2a6d8e19

{ "operation_id": "7a1e9d84-2c50-4f6b-9b31-0c5f2a6d8e19", "kind": "instance.create",
  "state": "pending", "created_at": "2026-08-06T12:00:01Z" }

In a script, capture the header:

# Capture the operation id for the poll loop in step 5
OP_ID=$(curl -sS -D - -o /dev/null https://api.gpu.ai/v1/instances \
  -H "Authorization: Bearer $GPUAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"gpu_type":"rtx_4090","gpu_count":1,"tier":"on_demand","ssh_key_ids":["key-abc123"],"max_price_per_hour":1.00}' \
  | grep -i '^operation-id:' | tr -d '\r' | awk '{print $2}')

With the SDK. Create is one of the two steps where the typed client earns its keep: the request body is a validated model, so a misspelled field or a bad tier fails at the call site instead of coming back as a 422 you have to read, and the returned Operation is a typed object rather than a JSON blob you have to remember the shape of.

import os

import gpuai_sdk

cfg = gpuai_sdk.Configuration(
    host="https://api.gpu.ai/v1",
    access_token=os.environ["GPUAI_API_KEY"],
)

with gpuai_sdk.ApiClient(cfg) as client:
    op = gpuai_sdk.InstancesApi(client).create_instance(
        gpuai_sdk.CreateInstanceRequest(
            gpu_type="rtx_4090",
            gpu_count=1,
            tier="on_demand",
            ssh_key_ids=["key-abc123"],
            max_price_per_hour=1.00,
        )
    )
    print(op.operation_id, op.state)

const { Configuration, InstancesApi } = require('@gpuai/sdk');

const cfg = new Configuration({
  basePath: 'https://api.gpu.ai/v1',
  accessToken: process.env.GPUAI_API_KEY,
});

async function main() {
  const op = await new InstancesApi(cfg).createInstance({
    createInstanceRequest: {
      gpuType: 'rtx_4090',
      gpuCount: 1,
      tier: 'on_demand',
      sshKeyIds: ['key-abc123'],
      maxPricePerHour: 1.0,
    },
  });
  console.log(op.operationId, op.state);
}

main();

5. Poll the operation

One poll is a plain GET:

# Where is the create up to?
curl https://api.gpu.ai/v1/operations/7a1e9d84-2c50-4f6b-9b31-0c5f2a6d8e19 \
  -H "Authorization: Bearer gpuai_live_…"

{ "operation_id": "7a1e9d84-…", "kind": "instance.create", "state": "in_progress",
  "created_at": "2026-08-06T12:00:01Z" }

Poll it until state reaches a terminal value. Use this cadence — it is exactly what the official CLI and SDKs do (PollOperation in internal/apiclient/instances.go): start at 2 seconds, add 1 second per tick, cap at 10 seconds, with a client-side deadline. It is a *linear* ramp, not an exponential backoff; matching it keeps your script's load and latency identical to the first-party tools.

# Poll to terminal — the CLI's exact cadence: 2s, +1s per tick, capped at 10s
API_BASE=https://api.gpu.ai/v1
interval=2
deadline=$(( $(date +%s) + 600 ))
while [ "$(date +%s)" -lt "$deadline" ]; do
  state=$(curl -s "$API_BASE/operations/$OP_ID" \
    -H "Authorization: Bearer $GPUAI_API_KEY" | jq -r .state)
  case "$state" in
    succeeded|failed|cancelled) break ;;
  esac
  sleep "$interval"
  [ "$interval" -lt 10 ] && interval=$((interval + 1))
done

# Only a succeeded operation carries an instance id. Without this guard a
# failed/cancelled/timed-out create leaves INSTANCE_ID as the string "null"
# and every following call goes to /v1/instances/null.
[ "$state" = "succeeded" ] || { echo "create ended in ${state:-timeout}" >&2; exit 1; }

INSTANCE_ID=$(curl -s "$API_BASE/operations/$OP_ID" \
  -H "Authorization: Bearer $GPUAI_API_KEY" | jq -r '.resource_id // empty')

The 600 second deadline suits instant_boot capacity; give slow-boot baremetal more. There is no server-side recovery worker chasing your operation, so the deadline is yours to choose and yours to enforce.

With the SDK. This is the other step worth a typed client. Be clear about what it does and doesn't give you: the generated SDKs are spec-generated and ship no poll-to-terminal helper — you still write the loop. What you get is get_operation returning a typed Operation whose state and resource_id are real fields, so the loop is a handful of lines with no jq, no string scraping, and no chance of misreading a field name. (If you want the loop written for you, that's the CLI: gpu operations get --wait-timeout 10m.)

import time

import gpuai_sdk

DEADLINE_SECS = 600
TERMINAL = {"succeeded", "failed", "cancelled"}

with gpuai_sdk.ApiClient(cfg) as client:
    ops = gpuai_sdk.OperationsApi(client)
    interval, end = 2, time.time() + DEADLINE_SECS
    while time.time() < end:
        op = ops.get_operation(op_id)
        if op.state in TERMINAL:
            break
        time.sleep(interval)
        interval = min(interval + 1, 10)

    if op.state != "succeeded":
        raise RuntimeError(f"{op.state}: {op.error.detail if op.error else 'no detail'}")
    instance_id = op.resource_id

const { OperationsApi } = require('@gpuai/sdk');

const TERMINAL = new Set(['succeeded', 'failed', 'cancelled']);
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function pollToTerminal(cfg, opId, deadlineSecs = 600) {
  const ops = new OperationsApi(cfg);
  const end = Date.now() + deadlineSecs * 1000;
  let interval = 2;
  let op;
  while (Date.now() < end) {
    op = await ops.getOperation({ id: opId });
    if (TERMINAL.has(op.state)) break;
    await sleep(interval * 1000);
    interval = Math.min(interval + 1, 10);
  }
  if (!op || op.state !== 'succeeded') {
    throw new Error(`${op ? op.state : 'timed out'}: ${op?.error?.detail ?? 'no detail'}`);
  }
  return op.resourceId; // the instance id
}

6. Read the connection details

# Status, price, and how to reach it
curl https://api.gpu.ai/v1/instances/gpu-abc123 \
  -H "Authorization: Bearer gpuai_live_…"

{ "id": "gpu-abc123", "status": "running", "gpu_type": "rtx_4090", "gpu_count": 1,
  "region": "us-east", "tier": "on_demand", "price_per_hour": 0.44,
  "connection": { "hostname": "frp.gpu.ai", "port": 41022,
                  "ssh_command": "ssh root@frp.gpu.ai -p 41022" },
  "created_at": "2026-08-06T12:00:01Z", "ready_at": "2026-08-06T12:01:30Z" }

status moves allocating → starting → running; connection is absent until there is something to connect to — the field is omitted from the body entirely, not sent as null.

A succeeded create operation does not mean a booted machine. The operation completes when the platform has *placed* your instance — the provider accepted the launch and the instance record exists. Booting happens after that, and the status tells you where you are: allocating means the node is still being acquired from the provider, starting means the node exists and is coming up. So step 5 and this step are two separate waits, and this one is the longer of the two — minutes, not seconds, and up to ~13 for a baremetal physical allocation.

Give it its own deadline rather than a fixed number of retries, and stop early if the instance reaches a terminal state:

# Wait for reachable: status=running AND connection details present
API_BASE=https://api.gpu.ai/v1
ready_deadline=$(( $(date +%s) + 900 ))
while [ "$(date +%s)" -lt "$ready_deadline" ]; do
  inst=$(curl -s "$API_BASE/instances/$INSTANCE_ID" \
    -H "Authorization: Bearer $GPUAI_API_KEY")
  status=$(printf '%s' "$inst" | jq -r .status)
  host=$(printf '%s' "$inst" | jq -r '.connection.hostname // empty')
  [ "$status" = "running" ] && [ -n "$host" ] && break
  case "$status" in
    error|terminated) echo "instance went to $status" >&2; break ;;
  esac
  sleep 10
done

Once status is running and connection is present, give sshd a few more seconds to start listening — step 7 retries the connection for that reason.

7. SSH in

connection.ssh_command is a ready-to-run command; add -i for the private key you generated in step 3.

ssh -i ~/.ssh/gpuai_demo -o StrictHostKeyChecking=accept-new \
  -p 41022 root@frp.gpu.ai nvidia-smi

The connection goes through an encrypted tunnel — hostname and port are the tunnel endpoint, not a raw machine address. accept-new is the right host-key policy for a freshly created, short-lived instance you just provisioned yourself: it pins the key on first connect and fails loudly if it ever changes.

If nvidia-smi prints your GPU, the instance is genuinely yours and working. That is the only proof that matters.

8. Terminate

# Terminate — 202 Accepted, and idempotent (404 once it's already gone)
curl -X DELETE https://api.gpu.ai/v1/instances/gpu-abc123 \
  -H "Authorization: Bearer gpuai_live_…"

Terminate is asynchronous too: 202 plus an Operation-Id you can poll exactly like step 5 (kind: "instance.delete"). It is idempotent — terminating an already-terminated instance is not an error, and once the record is gone the call returns 404.

9. Verify it's gone

Do not treat the 202 as the end. Read the instance back until it reports terminated (or 404s):

# Confirm — don't assume
curl https://api.gpu.ai/v1/instances/gpu-abc123 \
  -H "Authorization: Bearer gpuai_live_…" | jq -r .status

Billing stops when the instance actually stops, so this check is the difference between "I asked it to stop" and "it stopped". In a script, terminate from a cleanup handler that runs on failure paths too — a crash between create and terminate is exactly how a GPU gets left running for a weekend.

Operation lifecycle

Both create and terminate go through the same async operation model.

States. pending → in_progress → succeeded (or failed / cancelled). pending and in_progress are non-terminal; the three terminal states are the only ones that end a poll loop.

resource_id is absent until the platform knows the instance id, then carries it — like connection, the key is omitted from the body rather than sent as null, so test for presence, not for null. On a succeeded instance.create, that is your instance.

error is populated on failed with { "code": …, "detail": … } — read detail for the human-readable reason (out of capacity, price moved, a requested environment the placed machine cannot serve).

Scoping. Operations are readable by the key's own organization only; an operation id from someone else's account reads as not-found.

Billing basics

Instances are metered per second of runtime, at the whole-instance price_per_hour you saw at create time (disk beyond the included allowance is folded into that same hourly rate). There is no minimum billing period and no charge after termination completes — which is why step 9 exists.

Read your own usage with GET /v1/usage. bucket is hour, day, week, or month; group_by accepts instance_id or gpu_type; start / end take RFC 3339 timestamps.

# Per-day GPU-seconds and cost
curl "https://api.gpu.ai/v1/usage?bucket=day" \
  -H "Authorization: Bearer gpuai_live_…"

{ "data": [ { "bucket_start": "2026-08-06T00:00:00Z", "gpu_seconds": 372,
              "cost_cents": 5 } ],
  "next_cursor": null }

gpu_seconds is metered runtime and cost_cents is the charge for that bucket, both integers. Metering can lag a just-terminated instance by a short interval, so a run that ends at zero is normal — read the bucket again a few minutes later.

For guardrails rather than history, GET /v1/billing/spending-limit (requires the billing:read scope) returns monthly_limit_dollars, current_month_spend_dollars, and current_day_spend_cents.

Run this whole page as a script

The flow above is committed as a runnable harness: samples/gpu-instances/provision_flow.sh. It makes the same calls in the same order — keyless catalog reads, cheapest in-cap offer, ephemeral key, create, the exact poll loop, the deadline-bounded readiness wait, SSH + nvidia-smi, terminate, verify-gone, and the /v1/usage read — with an EXIT trap that terminates the instance on failure. The trap is best-effort and bounded, not absolute: if the run dies before the create operation has revealed the instance id, cleanup waits up to ~2 minutes for the operation to reveal it, and tells you to check for a stray instance by hand if it never can. Both waits are tunable: POLL_DEADLINE_SECS (operation, default 600) and READY_DEADLINE_SECS (boot to reachable, default 900).

GPUAI_API_KEY=gpuai_live_... MAX_PRICE_PER_HOUR=1.00 \
  ./docs/samples/gpu-instances/provision_flow.sh

It provisions a real GPU, so it costs real money (a few cents at the cheapest tier). It is also how this page gets re-verified: one command, and the output is the evidence.

Doing this from the CLI

gpu instances create -t rtx_4090 --tier on_demand --ssh-key-id key-abc123 — see gpu instances create.

gpu instances ssh gpu-abc123 — see gpu instances ssh.

gpu operations get 7a1e9d84-… --wait-timeout 10m — see gpu operations get.

Related

• Getting started — key, first call, first SDK call
• Python SDK quick-start — the typed client used in the asides
• TypeScript SDK quick-start — the same surface for Node
• gpu CLI reference — the command-line client
• Serverless Inference guide — pay-per-token

inference with no instance to manage

---

Samples on this page run-verified against demo on 2026-08-06, against https://api.demo.gpu.ai/v1 (the only substitution: the base URL). The whole flow ran for real — cheapest single-GPU offer created, operation polled to succeeded, SSH in, nvidia-smi, terminate, verify-gone — and left nothing running. Evidence — exact commands and captured output, including the nvidia-smi transcript — is committed at .planning/phases/79-missing-api-guides-sow-m4/evidence/gpu-instances/. The SDK install lines referenced above are the one thing not verified — see the package note in step 4.