CH·GGuides

Deployable Templates

One-click templates: launch Jupyter, vLLM, and other prebuilt stacks on any GPU.

Deployable templates launch a curated open-source app (ComfyUI, vLLM, …) onto a GPU instance and give you a clean HTTPS URL (https://gpu-xxxx.apps.gpu.ai) behind basic-auth — one command, no Docker, no reverse-proxy setup. A template deploy *is* an instance: it provisions a GPU, runs the app, and tears down with the same lifecycle as any other instance.

You can also deploy from the dashboard at /cloud/deploy — browse the Hub, filter by category or search, pick an eligible GPU, and the running app appears under Instances with an Open app link.

Base URL & authentication


Base URL	`https://api.gpu.ai/v1`
Auth	`Authorization: Bearer gpuai_live_…`

Get a key with gpu login (it stores a gpuai_live_… key in ~/.config/gpu/credentials.json) or mint one in the dashboard. Scopes: instances:read lists the template catalog, instances:write deploys a template; a full_access key covers both.

Browse the catalog

# List every deployable template
gpu templates list

# Inspect one — shows the published ports, min VRAM, and overridable env keys
gpu templates get comfyui

Every template publishes a one-line description and an includes list — what's inside the image (the app, its serving stack, SSH access) — on the catalog API, the deploy hub cards, and gpu templates get. The catalog deliberately omits the container image path and start command — those are internal orchestration details.

Each template carries a kind. Every *deployable* template today is a long-running service — a web app you open and that runs until you terminate it. Current services:

ComfyUI — node-based Stable Diffusion / Flux pipelines.
vLLM — high-throughput, OpenAI-compatible LLM inference.
Axolotl Fine-Tuning — a GPU box running Jupyter Lab with

Axolotl preinstalled. Rent the GPU, open the notebook, and train your own model (LoRA / QLoRA / full fine-tunes) on any Hugging Face model and dataset. It ships with a getting-started notebook and an example QLoRA config for Qwen2.5-7B.

Faster Whisper — speech-to-text powered by

speaches on faster-whisper. One port serves both a web UI (drag in an audio file, get a transcript) and an OpenAI-compatible API (/v1/audio/transcriptions, /v1/audio/translations, /v1/audio/speech). Whisper models download on first use and unload after idle — no model configuration required:

  curl -u gpuai:<pass> https://gpu-abcd1234.apps.gpu.ai/v1/audio/transcriptions \
    -F file=@meeting.wav -F model=Systran/faster-whisper-large-v3

Stable Diffusion WebUI (A1111) — the classic

AUTOMATIC1111 interface: txt2img, img2img, inpainting, upscaling, LoRAs, and its extension ecosystem, with the REST API enabled (/docs). On first boot with no checkpoint present it downloads Stable Diffusion 1.5 (~4 GB) before the UI comes up, so the first deploy takes a few extra minutes.

Ollama — run open LLMs with one command. Pull

Llama, Gemma, Qwen and more over the simple Ollama API (/api/generate, /api/chat) — point any Ollama-compatible client at your app URL. OLLAMA_KEEP_ALIVE tunes how long a model stays in VRAM after the last request. (source, MIT)

Open WebUI + Ollama — a polished, ChatGPT-style chat interface with a

bundled Ollama server in the same instance: pull models from the UI, chat, and RAG over your own documents, fully self-hosted. The first visit creates the admin account. (source, BSD-3 with a branding clause)

SGLang — ultra-fast, OpenAI-compatible LLM serving with RadixAttention

prefix caching. MODEL picks the Hugging Face model (defaults to Qwen/Qwen2.5-7B-Instruct); HF_TOKEN unlocks gated models. (source, Apache-2.0)

Text Generation WebUI — the Swiss-army knife for local LLMs: chat,

notebook, and instruct modes across multiple backends, LoRA loading, and a deep extension ecosystem. TGW_ARGS appends extra launch flags (e.g. --api). (source, AGPL-3.0)

InvokeAI — a studio-grade Stable Diffusion & FLUX workspace: unified

canvas, board-based gallery, and professional inpainting/outpainting tools. Install models from the in-app model manager; HF_TOKEN unlocks gated weights. (source, Apache-2.0)

SwarmUI — a modern, modular web UI for image & video generation (FLUX,

Stable Diffusion, Wan) with powerful grid tooling and a node engine underneath. The first-boot wizard installs its backend and models in-app. (source, MIT)

Kohya_ss — the community-standard GUI for training SD & SDXL LoRAs:

DreamBooth, fine-tuning, and textual inversion with deep parameter control. HF_TOKEN unlocks gated base models. (source, Apache-2.0)

AI Toolkit — train FLUX and video-model LoRAs from a sleek web UI, by

ostris. HF_TOKEN is needed for gated weights (e.g. FLUX.1-dev); AI_TOOLKIT_AUTH (optional) adds the UI's own auth token on top of the basic-auth gate. (source, MIT)

LLaMA-Factory — fine-tune 100+ LLMs from the LLaMA Board web GUI: LoRA,

QLoRA, DPO, and full training with zero code. HF_TOKEN unlocks gated base models. (source, Apache-2.0)

The catalog may also list coming_soon placeholders for capabilities still in the works — those appear in the /cloud/deploy Hub but are not yet deployable.

Deploy

# Deploy ComfyUI (auto-selects the cheapest eligible GPU)
gpu deploy comfyui --tier on_demand --ssh-key-id <key-id>

gpu deploy submits the deploy, polls the operation until the app is ready, then prints the instance plus a one-time credential block:

App URL:   https://gpu-abcd1234.apps.gpu.ai
Username:  gpuai
Password:  <generated-password>
⚠ Save this password now — it is shown once and cannot be retrieved later.

The app URL and basic-auth user/pass are shown once on deploy. They are returned only on the deploy operation result — they never appear on a later gpu instances get, and they are never logged. Save them when you see them.

Reaching your app

Your app is served over HTTPS at https://.apps.gpu.ai, gated by HTTP basic-auth with the credential printed on deploy:

curl -u gpuai:<pass> https://gpu-abcd1234.apps.gpu.ai

Open the same URL in a browser and the browser prompts for the username and password.

Env overrides

Some templates expose env vars you can override at deploy time (e.g. vLLM's MODEL). Only keys the template marks user_overridable are accepted — pass anything else and the API returns a 422. Check gpu templates get for the overridable keys.

gpu deploy vllm --env MODEL=meta-llama/Llama-3.1-8B-Instruct --env HF_TOKEN=hf_xxx

# Axolotl: HF_TOKEN unlocks gated models/datasets; JUPYTER_TOKEN (optional) adds
# a Jupyter token on top of the basic-auth gate.
gpu deploy axolotl-finetuning --env HF_TOKEN=hf_xxx

# A1111: A1111_ARGS appends extra webui flags — e.g. enable extension installs
# (the UI disables them under --listen) or low-VRAM mode.
gpu deploy automatic1111 --env A1111_ARGS="--enable-insecure-extension-access --medvram"

# SGLang: pick the served model (defaults to Qwen/Qwen2.5-7B-Instruct).
gpu deploy sglang --env MODEL=meta-llama/Llama-3.1-8B-Instruct --env HF_TOKEN=hf_xxx

# Ollama: keep a loaded model in VRAM for an hour after the last request.
gpu deploy ollama --env OLLAMA_KEEP_ALIVE=1h

# Text Generation WebUI: TGW_ARGS appends extra launch flags.
gpu deploy text-generation-webui --env TGW_ARGS="--api"

Env values the template marks secret (such as a Hugging Face token) are redacted from the operation metadata snapshot before it is persisted or logged — the raw value reaches only the provisioning request that launches your app.

Model size vs GPU VRAM

When a deploy names a Hugging Face model via MODEL (the vLLM and SGLang templates), the API estimates the model's serving footprint from its published weights size before launching anything:

Explicit GPU (--type): a model that cannot fit the machine's total

VRAM is rejected immediately with a 422 model_too_large explaining the numbers (e.g. *"Model … needs ~160 GB VRAM to serve; a40 has 48 GB"*) — instead of a deploy that crash-loops until it times out ~30 minutes later. Fix it by picking a larger GPU, raising --count, or serving a smaller model.

Auto-select (no --type): the GPU auto-selection floor is raised to the

model's estimated per-GPU requirement, so placement only considers machines the model actually fits. If no available GPU is large enough at your --count, the deploy is rejected with the same actionable message.

Gated, private, or unknown repos: the deploy always proceeds. The 202

response carries a warnings entry (printed by gpu deploy at submit time) noting the size could not be verified. Local paths and non-Hub model ids are never checked.

The estimate is deliberately generous (weights + ~20% runtime overhead for KV cache and CUDA context); borderline fits are allowed through and left to the serving engine.