CH·02CLI reference
gpu endpoints
Serve an open model as an OpenAI-compatible endpoint
Serve an open model as an OpenAI-compatible endpoint
Synopsis
An endpoint serves a Hugging Face model with vLLM on a GPU type you pick, behind a stable OpenAI-compatible URL. Point any OpenAI client's base_url at it and use a GPU.ai API key.
Machines start on the first request, scale out under load up to --max-replicas, and are released after --idle-timeout seconds without requests (down to --min-replicas, zero by default). You pay for machine time while a machine is up, at the instance price.
Endpoints are off by default. On a deployment that does not offer them, these commands fail with this server has no such endpoint. See the endpoints guide.
Examples
# Create, then call it
gpu endpoints create qwen-small --model Qwen/Qwen2.5-0.5B-Instruct --gpu rtx_4090
gpu endpoints get qwen-small
# Delete it (its machines are released)
gpu endpoints delete qwen-small --yes
Options
-h, --help help for endpoints
Options inherited from parent commands
--api-base string API base URL (override with GPUAI_API_BASE env) (default "https://api.gpu.ai/v1")
--debug Enable debug logging to stderr
-o, --output string Output format: table|json (default table on TTY, json otherwise)
SEE ALSO
- gpu - GPU.ai command-line interface
- gpu endpoints create - Create an endpoint
- gpu endpoints delete - Delete an endpoint and release its machines
- gpu endpoints get - Show an endpoint: status, URL, example, and recent machines
- gpu endpoints list - List endpoints