Bare metal or virtualised GPU compute: how to choose

With bare metal you get whole physical servers to yourself, with no virtualisation layer between your software and the GPUs. Virtualised GPU compute gives you virtual machines on shared hardware, managed by the provider.

Large training runs that spread across many servers usually want bare metal: full use of each GPU, the fastest path between them, and control over drivers and the software stack. It suits teams that run their own scheduling and operations.

Inference, experiments and smaller jobs often do well on virtual machines: quicker to set up, easier to size up or down, and with more of the operations handled for you. If you aren't sure, say "no preference" and let providers propose.

For training across servers, the interconnect between GPUs decides how well the job scales. InfiniBand is the common choice for large clusters; RoCE carries the same kind of low-latency traffic over Ethernet. For inference on single servers, the interconnect matters much less.

• The GPU model and how many, or the workload if you are open on the model.
• Bare metal, virtualised or no preference.
• The interconnect, if your jobs span servers.
• When you need it and for how long.
• Where it may run, and whether the power must be renewable.

Providers whose clusters could fit see your request, scored against their capacity, without your company name. You see the ones that say they are interested, and names open only when both sides say yes. If you could use either GPU compute or space for your own hardware, choose "capacity or GPU compute" and both kinds of operator can answer.