CH·02CLI reference
gpu endpoints create
Create an endpoint
Create an endpoint
Synopsis
Creates an endpoint named NAME (lowercase letters, digits, and hyphens). Nothing starts until the first request, unless --min-replicas keeps machines warm. A cold start takes a few minutes while the image and the model download.
For a gated model, store a Hugging Face token first (gpu secrets set HF_TOKEN) and pass --hf-token-secret HF_TOKEN.
The output shows the endpoint's URL and an example curl.
gpu endpoints create NAME --model HF_ID --gpu GPU_TYPE [flags]
Examples
gpu endpoints create qwen-small --model Qwen/Qwen2.5-0.5B-Instruct --gpu rtx_4090
gpu endpoints create llama --model google/gemma-2-9b-it --gpu l40s \
--hf-token-secret HF_TOKEN --max-replicas 2 --idle-timeout 600
Options
--gpu string GPU type for each machine (24 GB or more), such as rtx_4090; see gpu gpu-types
-h, --help help for create
--hf-token-secret string Name of a secret holding a Hugging Face token, delivered as HF_TOKEN
--idle-timeout int Seconds without requests before a machine is released (at least 60) (default 300)
--max-price float Highest hourly price per machine, in USD
--max-replicas int Most machines under load (at most 4) (default 1)
--min-replicas int Machines kept warm even with no requests
--model string Hugging Face model id, such as Qwen/Qwen2.5-0.5B-Instruct
--target-concurrency int Requests in flight per machine before another starts (default 16)
Options inherited from parent commands
--api-base string API base URL (override with GPUAI_API_BASE env) (default "https://api.gpu.ai/v1")
--debug Enable debug logging to stderr
-o, --output string Output format: table|json (default table on TTY, json otherwise)
SEE ALSO
- gpu endpoints - Serve an open model as an OpenAI-compatible endpoint