GPU Memory Scheduler

how it will work

How it fits into GPU.ai

The GPU Memory Scheduler will run batch jobs, each a container and a command, and pack the ones that opt in onto a GPU you already pay for when their memory fits. It decides by how much GPU memory each job needs. A job with no memory hint or history gets a whole GPU.

• Packed by memory

  Jobs that opt in to sharing are placed side by side when their memory fits on one GPU.

• Shared only with your own jobs

  Sharing is opt in, per job. A job never shares a GPU with another customer.

• Optional hard isolation

  For work that needs it, a job's memory reservation can be enforced as a hard limit.

• Retried when memory runs out

  A job that exits out of memory runs again with a larger reservation, and a checkpoint directory lets it resume.