Reserved or on-demand GPU compute: how the contracts differ

A reservation commits you to a set number of GPUs for a fixed term, usually months or years, in return for guaranteed availability and a lower unit price. On-demand capacity is paid by the hour or day with no commitment, at a higher price and with no promise that GPUs are free when you want them. Training runs and steady inference tend to suit reservations; experiments and bursts suit on-demand.

A reserved capacity agreement typically sets out:

• the GPU model, the number of GPUs and how they are grouped into nodes;
• the network between nodes, such as InfiniBand or RoCE, and the storage;
• the term, the start date and what happens if delivery is late;
• payment, often with a deposit or prepayment for large clusters;
• service levels for availability and for replacing failed hardware.

Our reserved GPU capacity template shows how these terms are commonly written.

On-demand is flexible: start and stop when you like, pay only for use. In return, capacity is first come, first served, large contiguous clusters are rarely available on demand, and the hourly price is higher. Check how billing is measured, what notice the provider gives before reclaiming capacity, and what happens to your data when you stop. The on-demand template covers these points.

Either contract can come as bare-metal servers you control fully or as virtual machines on the provider's platform. Bare metal gives full performance and control of the software stack; virtualised capacity is quicker to start and easier to resize. Our guide to bare metal or virtualised GPU compute compares them in detail.

State the GPU model and count, bare metal or virtualised, the interconnect, the storage, the software stack, the use case, the start date and the contract term. A GPU request on the marketplace asks for exactly these, so providers can answer with a comparable offer instead of a sales call.