CH·GGuides

Deployable Templates

One-click templates: launch Jupyter, vLLM, and other prebuilt stacks on any GPU.

Deployable templates launch a curated open-source app (ComfyUI, vLLM, …) onto a GPU instance and give you a clean HTTPS URL (https://gpu-xxxx.apps.gpu.ai) behind basic-auth — one command, no Docker, no reverse-proxy setup. A template deploy *is* an instance: it provisions a GPU, runs the app, and tears down with the same lifecycle as any other instance.

You can also deploy from the dashboard at /cloud/deploy — browse the Hub, filter by category or search, pick an eligible GPU, and the running app appears under Instances with an Open app link.

Base URL & authentication


Base URL	`https://api.gpu.ai/v1`
Auth	`Authorization: Bearer gpuai_live_…`

Get a key with gpu login (it stores a gpuai_live_… key in ~/.config/gpu/credentials.json) or mint one in the dashboard. Scopes: instances:read lists the template catalog, instances:write deploys a template; a full_access key covers both.

Browse the catalog

# List every deployable template
gpu templates list

# Inspect one — shows the published ports, min VRAM, and overridable env keys
gpu templates get comfyui

Every template publishes a one-line description and an includes list — what's inside the image (the app, its serving stack, SSH access) — on the catalog API, the deploy hub cards, and gpu templates get. The catalog deliberately omits the container image path and start command — those are internal orchestration details.

Each template carries a kind. Every *deployable* template today is a long-running service — a web app you open and that runs until you terminate it. Current services:

JupyterLab — a GPU notebook in your browser. CUDA-matched PyTorch plus

the everyday data stack (pandas, matplotlib, scikit-learn, transformers, datasets, accelerate) behind JupyterLab on the app URL — deploy, click Open in browser, and you're in a live notebook with a GPU attached. Ships with a getting-started notebook (GPU sanity check + quick benchmark). Set HF_TOKEN for gated Hugging Face pulls; set JUPYTER_TOKEN to require a Jupyter token on top of the basic-auth gate.

ComfyUI — node-based Stable Diffusion / Flux pipelines.
vLLM — high-throughput, OpenAI-compatible LLM inference.
Axolotl Fine-Tuning — a GPU box running Jupyter Lab with

Axolotl preinstalled. Rent the GPU, open the notebook, and train your own model (LoRA / QLoRA / full fine-tunes) on any Hugging Face model and dataset. It ships with a getting-started notebook and an example QLoRA config for Qwen2.5-7B.

Faster Whisper — speech-to-text powered by

speaches on faster-whisper. One port serves both a web UI (drag in an audio file, get a transcript) and an OpenAI-compatible API (/v1/audio/transcriptions, /v1/audio/translations, /v1/audio/speech). Whisper models download on first use and unload after idle — no model configuration required:

  curl -u gpuai:<pass> https://gpu-abcd1234.apps.gpu.ai/v1/audio/transcriptions \
    -F file=@meeting.wav -F model=Systran/faster-whisper-large-v3

Stable Diffusion WebUI (A1111) — the classic

AUTOMATIC1111 interface: txt2img, img2img, inpainting, upscaling, LoRAs, and its extension ecosystem, with the REST API enabled (/docs). On first boot with no checkpoint present it downloads Stable Diffusion 1.5 (~4 GB) before the UI comes up, so the first deploy takes a few extra minutes.

Ollama — run open LLMs with one command. Pull

Llama, Gemma, Qwen and more over the simple Ollama API (/api/generate, /api/chat) — point any Ollama-compatible client at your app URL. OLLAMA_KEEP_ALIVE tunes how long a model stays in VRAM after the last request. (source, MIT)

Open WebUI + Ollama — a polished, ChatGPT-style chat interface with a

bundled Ollama server in the same instance: pull models from the UI, chat, and RAG over your own documents, fully self-hosted. The first visit creates the admin account. (source, BSD-3 with a branding clause)

SGLang — ultra-fast, OpenAI-compatible LLM serving with RadixAttention

prefix caching. MODEL picks the Hugging Face model (defaults to Qwen/Qwen2.5-7B-Instruct); HF_TOKEN unlocks gated models. (source, Apache-2.0)

Text Generation WebUI — the Swiss-army knife for local LLMs: chat,

notebook, and instruct modes across multiple backends, LoRA loading, and a deep extension ecosystem. TGW_ARGS appends extra launch flags (e.g. --api). (source, AGPL-3.0)

InvokeAI — a studio-grade Stable Diffusion & FLUX workspace: unified

canvas, board-based gallery, and professional inpainting/outpainting tools. Install models from the in-app model manager; HF_TOKEN unlocks gated weights. (source, Apache-2.0)

SwarmUI — a modern, modular web UI for image & video generation (FLUX,

Stable Diffusion, Wan) with powerful grid tooling and a node engine underneath. The first-boot wizard installs its backend and models in-app. (source, MIT)

Kohya_ss — the community-standard GUI for training SD & SDXL LoRAs:

DreamBooth, fine-tuning, and textual inversion with deep parameter control. HF_TOKEN unlocks gated base models. (source, Apache-2.0)

AI Toolkit — train FLUX and video-model LoRAs from a sleek web UI, by

ostris. HF_TOKEN is needed for gated weights (e.g. FLUX.1-dev); AI_TOOLKIT_AUTH (optional) adds the UI's own auth token on top of the basic-auth gate. (source, MIT)

LLaMA-Factory — fine-tune 100+ LLMs from the LLaMA Board web GUI: LoRA,

QLoRA, DPO, and full training with zero code. HF_TOKEN unlocks gated base models. (source, Apache-2.0)

The catalog may also list coming_soon placeholders for capabilities still in the works — those appear in the /cloud/deploy Hub but are not yet deployable.

Deploy

# Deploy ComfyUI (auto-selects the cheapest eligible GPU)
gpu deploy comfyui --tier on_demand --ssh-key-id <key-id>

gpu deploy submits the deploy, polls the operation until the app is ready, then prints the instance plus its credential block:

App URL:   https://gpu-abcd1234.apps.gpu.ai
Username:  gpuai
Password:  <generated-password>

The app URL and basic-auth user/pass also stay available for the instance's lifetime: they appear on gpu instances get, on the dashboard instance page (the Web Access section of the Connection card), and gpu instances open opens the URL in your browser directly. They are never logged. (Instances deployed before credential persistence only ever showed their credential once, at deploy time — for those, the deploy output remains the only copy.)

API access to the credential: the password is redacted from bare reads — GET /v1/instances/{id} returns connection.app_user and the URLs but omits app_password unless you pass ?include=credentials (the CLI and dashboard do this for you). The instance list and operation results (GET /v1/operations/{id}) never include the password; the operation result carries only app_url, terminal_url, and app_basic_auth_user.

Web console (browser terminal)

Every newly provisioned instance — template deploy or raw launch — also serves a web console: a shell on the instance in your browser, with no SSH key setup, at https://-term.apps.gpu.ai. It is gated by the same basic-auth login as the app URL and appears alongside it on the dashboard's Web Access panel and in gpu instances get (the Console: line).

# Open a shell on the instance in your browser
gpu instances open gpu-abcd1234 --terminal

Instances provisioned before the console shipped don't have it (there is no terminal on board); for those, SSH remains the only shell.

Reaching your app

Your app is served over HTTPS at https://.apps.gpu.ai, gated by HTTP basic-auth with the credential printed on deploy:

curl -u gpuai:<pass> https://gpu-abcd1234.apps.gpu.ai

Open the same URL in a browser and the browser prompts for the username and password.

Env overrides

Some templates expose env vars you can override at deploy time (e.g. vLLM's MODEL). Only keys the template marks user_overridable are accepted — pass anything else and the API returns a 422. Check gpu templates get for the overridable keys.

gpu deploy vllm --env MODEL=meta-llama/Llama-3.1-8B-Instruct --env HF_TOKEN=hf_xxx

# Axolotl: HF_TOKEN unlocks gated models/datasets; JUPYTER_TOKEN (optional) adds
# a Jupyter token on top of the basic-auth gate.
gpu deploy axolotl-finetuning --env HF_TOKEN=hf_xxx

# A1111: A1111_ARGS appends extra webui flags — e.g. enable extension installs
# (the UI disables them under --listen) or low-VRAM mode.
gpu deploy automatic1111 --env A1111_ARGS="--enable-insecure-extension-access --medvram"

# SGLang: pick the served model (defaults to Qwen/Qwen2.5-7B-Instruct).
gpu deploy sglang --env MODEL=meta-llama/Llama-3.1-8B-Instruct --env HF_TOKEN=hf_xxx

# Ollama: keep a loaded model in VRAM for an hour after the last request.
gpu deploy ollama --env OLLAMA_KEEP_ALIVE=1h

# Text Generation WebUI: TGW_ARGS appends extra launch flags.
gpu deploy text-generation-webui --env TGW_ARGS="--api"

Env values the template marks secret (such as a Hugging Face token) are redacted from the operation metadata snapshot before it is persisted or logged — the raw value reaches only the provisioning request that launches your app.

Template VRAM floor vs your GPU

Every template declares a minimum per-GPU VRAM (min_vram_gb — shown by gpu templates show ). It is enforced on both deploy paths:

Auto-select (no --type): placement only considers GPUs at or above the

floor, and picks the cheapest of them.

Explicit GPU (--type): naming a GPU below the floor is rejected

immediately with a 422 gpu_vram_too_small giving both numbers and the GPU types that do fit — e.g. *"template sglang requires at least 24 GB of VRAM per GPU; rtx_a4000 has 16 GB. Compatible GPU types: a30, l4, rtx_3090, …"*.

The comparison uses each type's published nameplate VRAM — the same figure GET /v1/gpu-types shows — so a machine whose upstream reports 22 GB for a 24 GB L4 still clears a 24 GB floor.

Behavior change (2026-08-31). Explicit-GPU deploys below the floor used

to be accepted with a 202 and then died on boot a minute or two later.

They are now refused up front, before any instance is created or billed.

Model size vs GPU VRAM

When a deploy names a Hugging Face model via MODEL (the vLLM and SGLang templates), the API estimates the model's serving footprint from its published weights size before launching anything:

Explicit GPU (--type): a model that cannot fit the machine's total

VRAM is rejected immediately with a 422 model_too_large explaining the numbers (e.g. *"Model … needs ~160 GB VRAM to serve; a40 has 48 GB"*) — instead of a deploy that crash-loops until it times out ~30 minutes later. Fix it by picking a larger GPU, raising --count, or serving a smaller model.

Auto-select (no --type): the GPU auto-selection floor is raised to the

model's estimated per-GPU requirement, so placement only considers machines the model actually fits. If no available GPU is large enough at your --count, the deploy is rejected with the same actionable message.

Gated, private, or unknown repos: the deploy always proceeds. The 202

response carries a warnings entry (printed by gpu deploy at submit time) noting the size could not be verified. Local paths and non-Hub model ids are never checked.

The estimate is deliberately generous (weights + ~20% runtime overhead for KV cache and CUDA context); borderline fits are allowed through and left to the serving engine.