How to choose a GPU cloud for LLM inference
Start with model memory, then compare the billable allocation, region and price type. A low GPU-hour rate alone does not establish whether a model fits or what the full instance costs.
Current means eligible observations in this build’s dated database snapshot, not a new availability check or a market-wide minimum. Whole-GPU USD quotes only; standard and unspecified regions, tax excluded or unknown. Spot stays separate. Stale, fractional, advertised starting prices and GPU-only components do not enter the calculations. Source, tax, marketplace, FX and billing qualifiers remain on the offer cards.
Price observations dated: 2026-11-18
The table computes ideal weight storage from nominal model-card parameter counts stored as database references. FP16 uses two bytes per parameter; packed four-bit weights use half a byte. Results are decimal GB, not GiB. These are nominal-size arithmetic estimates, not measured VRAM requirements: quantization metadata, KV cache, activations and runtime memory add overhead. Context length and concurrency change the requirement. Extra GPU memory does not prove throughput, and memory across multiple GPUs requires a supported sharding setup. Check the linked estimator and cache documentation before choosing capacity.
| GPU model | Nominal parameters (billions) | FP16 weights only (GB) | Packed four-bit weights only (GB) |
|---|---|---|---|
| Llama 2 7B | 7 | 14 | 3.5 |
| Llama 2 13B | 13 | 26 | 6.5 |
| Llama 2 70B | 70 | 140 | 35 |
Keep headroom above weight storage and test the intended context, batch and precision. The classes below are non-overlapping memory bands per GPU. Their cheapest observed quotes are a shortlist, not a recommendation or proof that a named model fits. Compare the linked exact-model and provider pages for allocation, location and source conditions.
| VRAM band per GPU | Price type | Lowest observed USD/GPU-hour |
|---|---|---|
| 16 GB – < 24 GB | On-demand | RTX A4000 16GB · clore · $0.062 · On-demand |
| 16 GB – < 24 GB | Spot | RTX A4000 16GB · clore · $0.062 · Spot |
| 24 GB – < 48 GB | On-demand | V100 SXM2 32GB · clore · $0.074 · On-demand |
| 24 GB – < 48 GB | Spot | RTX 3090 24GB · clore · $0.067 · Spot |
| 48 GB – < 80 GB | On-demand | CMP 170HX 64GB · clore · $0.231 · On-demand |
| 48 GB – < 80 GB | Spot | CMP 170HX 64GB · clore · $0.231 · Spot |
| 80 GB+ | On-demand | A100 PCIe 80GB · clore · $0.354 · On-demand |
| 80 GB+ | Spot | A100 PCIe 80GB · clore · $0.250 · Spot |
Live-data examples and quote conditions
RTX A4000 16GB 1×
JP
USD (stablecoin) • platform fees may apply
$0.062 per GPU-hour
Minimum billable instance cost: USD $0.062/hr · Minimum billing duration unknown
Details & source
SKU: 113131 · Availability: Available · A · Official API
Reported GPU name: NVIDIA RTX A4000
Reported VRAM: 16 GB
Hourly amount in source scope: USD $0.062/hr · Instance
Source billing unit: Instance-hour
2/2
16 GB
16 GB
Listed whole-server daily price divided by 24 and GPU count. Platform service fees may apply; tax treatment unknown. Stablecoin-denominated USD price (USD-Blockchain).
API record pageRTX A4000 16GB 1×
JP
USD (stablecoin) • platform fees may apply
$0.062 per GPU-hour
Minimum billable instance cost: USD $0.062/hr · Minimum billing duration unknown
Details & source
SKU: 113131 · Availability: Available · A · Official API
Reported GPU name: NVIDIA RTX A4000
Reported VRAM: 16 GB
Hourly amount in source scope: USD $0.062/hr · Instance
Source billing unit: Instance-hour
2/2
16 GB
16 GB
Listed whole-server daily price divided by 24 and GPU count. Platform service fees may apply; tax treatment unknown. Stablecoin-denominated USD price (USD-Blockchain).
API record pageRTX 3090 24GB 1×
PL
USD (stablecoin) • platform fees may apply
$0.067 per GPU-hour
Minimum billable instance cost: USD $0.067/hr · Minimum billing duration unknown
Details & source
SKU: 112743 · Availability: Available · A · Official API
Reported GPU name: NVIDIA GeForce RTX 3090
Reported VRAM: 24 GB
Hourly amount in source scope: USD $0.067/hr · Instance
Source billing unit: Instance-hour
16/32
64 GB
24 GB
Listed whole-server daily price divided by 24 and GPU count. Platform service fees may apply; tax treatment unknown. Stablecoin-denominated USD price (USD-Blockchain).
API record pageV100 SXM2 32GB 4×
RU
USD (stablecoin) • platform fees may apply
$0.074 per GPU-hour
Minimum billable instance cost: USD $0.295/hr · Minimum billing duration unknown
Details & source
SKU: 122717 · Availability: Available · A · Official API
Reported GPU name: Tesla V100SXM232GB
Reported VRAM: 32 GB
Hourly amount in source scope: USD $0.295/hr · Instance
Source billing unit: Instance-hour
24/48
192 GB
32 GB
Listed whole-server daily price divided by 24 and GPU count. Platform service fees may apply; tax treatment unknown. Stablecoin-denominated USD price (USD-Blockchain).
API record pageCMP 170HX 64GB 9×
FI
USD (stablecoin) • platform fees may apply
$0.231 per GPU-hour
Minimum billable instance cost: USD $2.08/hr · Minimum billing duration unknown
Details & source
SKU: 115963 · Availability: Available · A · Official API
Reported GPU name: NVIDIA CMP 170HX
Reported VRAM: 64 GB
Hourly amount in source scope: USD $2.08/hr · Instance
Source billing unit: Instance-hour
4/8
32 GB
64 GB
Listed whole-server daily price divided by 24 and GPU count. Platform service fees may apply; tax treatment unknown. Stablecoin-denominated USD price (USD-Blockchain).
API record pageCMP 170HX 64GB 9×
FI
USD (stablecoin) • platform fees may apply
$0.231 per GPU-hour
Minimum billable instance cost: USD $2.08/hr · Minimum billing duration unknown
Details & source
SKU: 115963 · Availability: Available · A · Official API
Reported GPU name: NVIDIA CMP 170HX
Reported VRAM: 64 GB
Hourly amount in source scope: USD $2.08/hr · Instance
Source billing unit: Instance-hour
4/8
32 GB
64 GB
Listed whole-server daily price divided by 24 and GPU count. Platform service fees may apply; tax treatment unknown. Stablecoin-denominated USD price (USD-Blockchain).
API record pageA100 PCIe 80GB 1×
US
USD (stablecoin) • platform fees may apply
$0.250 per GPU-hour
Minimum billable instance cost: USD $0.250/hr · Minimum billing duration unknown
Details & source
SKU: 124543 · Availability: Available · A · Official API
Reported GPU name: NVIDIA A100 80GB PCIe
Reported VRAM: 80 GB
Hourly amount in source scope: USD $0.250/hr · Instance
Source billing unit: Instance-hour
24/24
216 GB
80 GB
Listed whole-server daily price divided by 24 and GPU count. Platform service fees may apply; tax treatment unknown. Stablecoin-denominated USD price (USD-Blockchain).
API record pageA100 PCIe 80GB 1×
US
USD (stablecoin) • platform fees may apply
$0.354 per GPU-hour
Minimum billable instance cost: USD $0.354/hr · Minimum billing duration unknown
Details & source
SKU: 124543 · Availability: Available · A · Official API
Reported GPU name: NVIDIA A100 80GB PCIe
Reported VRAM: 80 GB
Hourly amount in source scope: USD $0.354/hr · Instance
Source billing unit: Instance-hour
24/24
216 GB
80 GB
Listed whole-server daily price divided by 24 and GPU count. Platform service fees may apply; tax treatment unknown. Stablecoin-denominated USD price (USD-Blockchain).
API record pageOfficial sources
Know the full job cost before you rent: GPU Budget Kit ($12)
Related guides and comparisons
- Spot vs on-demand GPU prices
- Per-GPU-hour vs instance price
- GPU cloud price glossary
- Cheapest GPU per VRAM GB right now
- Llama models
- Best cloud GPU for Llama 3.1 8B LLM inference (FP16)
- Best cloud GPU for Llama 3.1 8B LLM inference (4-bit)
- Best cloud GPU for Llama 2 13B LLM inference (FP16)
- Best cloud GPU for Llama 2 13B LLM inference (4-bit)
- Best cloud GPU for Llama 3.1 70B LLM inference (FP16)
- Best cloud GPU for Llama 3.1 70B LLM inference (4-bit)
- Qwen models
- Best cloud GPU for Qwen2.5 7B Instruct LLM inference (FP16)
- Best cloud GPU for Qwen2.5 7B Instruct LLM inference (4-bit)
- Best cloud GPU for Qwen2.5 14B Instruct LLM inference (FP16)
- Best cloud GPU for Qwen2.5 14B Instruct LLM inference (4-bit)
- Best cloud GPU for Qwen2.5 32B Instruct LLM inference (FP16)
- Best cloud GPU for Qwen2.5 32B Instruct LLM inference (4-bit)
- Best cloud GPU for Qwen2.5 72B Instruct LLM inference (FP16)
- Best cloud GPU for Qwen2.5 72B Instruct LLM inference (4-bit)
- Mistral models
- Whisper speech recognition