GPU instances: how they are priced and how to choose

Par Rohit Athaide · Publié le 3 nov. 2026

A GPU is a processor built for large amounts of parallel arithmetic. Clouds rent them by the hour for training and running machine learning models, rendering and some scientific work. They are the most expensive common instance type, so choosing and using them carefully pays quickly.

How they are billed

A GPU instance is a whole virtual machine with one or more GPUs attached, billed per hour like any other instance, and the price covers the GPU, the CPU, the memory and the local storage together. Most are sold whole: one, two, four or eight GPUs per instance. A few sizes offer a fraction of a GPU for smaller jobs, and our catalog lists those with fractional counts.

The price per GPU does not scale perfectly with count. Larger instances may be priced differently per GPU, and the high-end ones include fast links between GPUs.

What to compare

• GPU memory. For machine learning, memory often decides what fits at all. A model that does not fit in GPU memory cannot run, whatever the speed. In the price list this is usually the total across the GPUs on the instance, not per GPU.
• GPU count. Training can spread across several GPUs. Inference often needs fewer.
• GPU type and generation. Newer generations do more per hour and cost more per hour. The vendor’s price list gives the instance family and not always the chip model, so read the vendor’s instance page for the hardware.
• Interconnect. Fast links between GPUs matter for multi-GPU training, and not for single-GPU inference.
• CPU, memory and storage. Data loading can bottleneck a fast GPU.
• Other accelerators. Some clouds sell their own chips for machine learning, such as AWS Trainium and Inferentia. They are priced like GPU instances and need software support for the chip. Our catalog marks them as “ML ASIC”, to separate them from GPUs.

Availability matters as much as price

• Quotas. New accounts often have a zero quota for GPU instances. You must request an increase, and approval takes time.
• Capacity. The most sought-after GPUs can be hard to get on demand in a given region. Check more than one region, or reserve capacity.
• Commitments. Many vendors offer reserved or committed GPU capacity at a lower rate. That is the usual route for steady training or serving. See reserved instances and savings plans.
• Spot. GPUs on spot can be much cheaper and can disappear. Training that checkpoints regularly suits it. See spot and preemptible instances.

Keeping the bill under control

1. Turn them off. An idle GPU instance costs the full hourly price. Stop or delete it when a job ends.
2. Right-size. Use the smallest GPU memory that fits your model.
3. Batch inference requests so the GPU stays busy.
4. Checkpoint long jobs so interruptions cost little.
5. Set budgets with forecast alerts. GPU overruns are fast. See budgets and cost alerts.

Using the catalog

The GPU accelerators catalog lists the instances we have prices for on AWS, with GPU count, GPU memory where the vendor gives it, vCPU, memory and a source link on each row. Use the minimum GPU filter, and sort by monthly price. The catalog does not cover the other clouds yet.

Rent a GPU or pay per token

If you want to run a language model, you can rent GPUs and serve it yourself, or pay a vendor per token. The trade is covered in self-hosting an LLM versus using an API, and the token side in AI model token pricing.