how it will work
How it fits into GPU.ai
KServe is an open source project for serving models on Kubernetes. GPU.ai plans to run it on the Kubernetes Orchestration layer, so a model deployed with KServe can add replicas as requests grow and remove them as traffic falls, on GPUs rented through GPU.ai.
- Requeststo one endpoint
- KServeadds or removes replicas
- Run:ai on GPU.aiplaces them on GPUs
Deploy a model as a service
Describe the model and its runtime once, and KServe will run it behind an endpoint.
Scales with requests
Replicas are added as traffic grows and removed as it falls.
Placed by Run:ai
Run:ai decides where each replica runs on the cluster's GPUs.
KServe is an open source project. GPU.ai is not affiliated with or endorsed by its maintainers.