OpenAI gpt-oss

gpt-oss-20b hardware requirements

Fits on one Instinct MI350X OAM at INT4 / FP4. For 1,000 tokens/s at 32k context you need 1 x H100 SXM5 80GB at $1.79/hr or 1 x RTX PRO 6000 Blackwell Max-Q Workstation Edition at $0.50/hr.

Parameters

20.9B

Active per token

3.6B

mixture of experts

KV cache per token

48 KB

16-bit cache

Native precision

MXFP4

Rent the cheapest fit: 1 x RTX A5000 24GB from $0.27/hr Model card

Bare minimum

Fewest GPUs

1 x Instinct MI350X OAM

INT4 / FP4, 4,096 tokens, one stream

Tokens/s: 2.8k
Per hour: $6.16
Per million tokens: $0.61
Memory used: 4%

Rent on DigitalOcean

Cheapest per hour

1 x RTX A5000 24GB

INT4 / FP4, 4,096 tokens, one stream, live price

Tokens/s: 271
Per hour: $0.27
Per million tokens: $0.28
Memory used: 54%

Rent on RunPod

Cheapest per million tokens

1 x RTX PRO 6000 Blackwell Max-Q Workstation Edition

INT4 / FP4, 32,768 tokens, 32 streams

Tokens/s: 1.5k
Per hour: $0.50
Per million tokens: $0.09
Memory used: 41%

Rent on RunPod

Configurations

Every datacenter GPU in the catalog, sized for these settings. Change precision, context and concurrency and the whole page follows. Each point is one replica: the smallest tensor-parallel group of that GPU that holds the model.

Precision Context Streams

Vendor

Priced only

Priced configurations only; 37 more fit without a live price and appear in the table. Marker size grows with the tensor-parallel width. Roofline estimates.

GPU	TP	Memory used	Tokens/s	Per stream	Per hour	Per M tokens	kW
Instinct MI455X OAM	1	9%	~20k	626				Specs
Rubin SXM	1	14%	19k	591				Specs
Instinct MI355X OAM	1	14%	6.9k	215			1.4	Specs
GB300 NVL72 GPU 288GB	1	14%	6.9k	215	$8.62	$0.35		Rent
GB200 NVL72 GPU 186GB	1	21%	6.9k	215	$10.50	$0.42		Rent
Instinct MI350X OAM	1	14%	6.9k	215	$6.16	$0.25	1.0	Rent
B300 SXM 262GB	1	15%	6.9k	215	$7.40	$0.30	1.4	Rent
B200 SXM 180GB	1	22%	6.6k	207	$4.09	$0.17	1.0	Rent
B100 SXM 192GB	1	20%	6.9k	215			0.7	Specs
Instinct MI325X OAM	1	15%	~5.2k	161	$3.80	$0.20	1.0	Rent
Instinct MI300X 192GB	1	20%	~4.6k	142	$2.39	$0.15	0.8	Rent
BR100	1	62%	~1.4k	43.0			0.6	Specs
H200 SXM 141GB	1	28%	~4.1k	129	$2.99	$0.20	0.7	Rent
H100 SXM5 80GB	1	49%	~2.9k	90.1	$1.79	$0.17	0.7	Rent
Data Center GPU Max 1550 128GB	1	31%	~2.8k	88.1			0.6	Specs
H200 NVL 141GB	1	28%	~4.1k	129	$3.79	$0.25	0.6	Rent
H100 NVL 94GB	1	42%	~3.4k	105	$3.19	$0.26	0.4	Rent
H100 PCIe 80GB	1	49%	~1.7k	53.8	$2.50	$0.40	0.3	Rent
H800 PCIe 80GB	1	49%	~1.7k	53.8			0.3	Specs
RTX PRO 6000 Blackwell Workstation Edition	1	41%	1.5k	48.2	$1.80	$0.32	0.6	Rent
RTX PRO 6000 Blackwell Server Edition	1	41%	~1.4k	42.9	$0.59	$0.12	0.6	Rent
RTX PRO 6000 Blackwell Max-Q Workstation Edition	1	41%	1.5k	48.2	$0.50	$0.09	0.3	Rent
Data Center GPU Max 1100 48GB	1	83%	~1.1k	33.0			0.3	Specs
GeForce RTX 5090 32GB	2	63%	2.9k	91.5	$1.98	$0.19	1.1	Rent
MI250X 128GB	1	31%	~2.8k	88.1			0.6	Specs
RTX 6000 Ada 48GB	1	83%	~826	25.8	$0.84	$0.28	0.3	Rent
Instinct MI250 128GB	1	31%	~2.8k	88.1			0.5	Specs
L40S 48GB	1	83%	~743	23.2	$0.87	$0.33	0.3	Rent
A800 PCIe 80GB	1	49%	~1.7k	52.0			0.3	Specs
A100 PCIe 80GB	1	49%	~1.7k	52.0	$1.35	$0.23	0.3	Rent
A100 SXM4 80GB	1	49%	~1.8k	54.8	$1.40	$0.22	0.4	Rent
A100 PCIe 40GB	2	50%	~2.5k	79.4			0.5	Specs
A100 SXM4 40GB	2	50%	~2.5k	79.4	$2.58	$0.28	0.8	Rent
GeForce RTX 5090 D V2 24GB	2	85%	~2.2k	68.7			1.1	Specs
RTX 5880 Ada 48GB	1	83%	~826	25.8			0.3	Specs
RTX 5000 Ada 32GB	2	63%	~942	29.4	$1.66	$0.49	0.5	Rent
RTX PRO 5000 Blackwell 48GB	1	83%	~1.2k	36.1	$0.96	$0.23	0.3	Rent
RTX PRO 5000 Blackwell 72GB	1	55%	~1.2k	36.1			0.3	Specs
RTX PRO 4500 Blackwell 32GB	2	63%	~1.5k	45.8	$1.44	$0.27	0.4	Rent
Radeon AI PRO R9700 32GB	2	63%	~1.0k	32.7			0.6	Specs
Instinct MI100 32GB	2	63%	~2.0k	62.8			0.6	Specs
Arc Pro B70 32GB	2	63%	~994	31.1			0.5	Specs
L40 48GB	1	83%	~743	23.2	$0.82	$0.31	0.3	Rent
Instinct MI210 PCIe	1	62%	~1.4k	44.1			0.3	Specs
GeForce RTX 4090 24GB	2	85%	~1.6k	51.5	$1.20	$0.20	0.9	Rent
A30 24GB	2	85%	~1.5k	47.7	$1.47	$0.27	0.3	Rent
RTX 4500 Ada 24GB	2	85%	~706	22.1			0.4	Specs
RTX A6000 48GB	1	83%	~661	20.6	$0.50	$0.21	0.3	Rent
A40 48GB	1	83%	~599	18.7	$0.49	$0.23	0.3	Rent
H20 141GB HBM3e	1	28%	~3.4k	108			0.4	Specs
H20 96GB	1	41%	~3.4k	108			0.4	Specs
RTX PRO 4000 Blackwell 24GB	2	85%	~1.1k	34.3	$1.14	$0.29	0.3	Rent
BI-V100 Tiangai 100	2	63%	~2.0k	61.3			0.5	Specs
RTX A5500 24GB	2	85%	~1.3k	39.2			0.5	Specs
V100S PCIe 32GB	2	63%	~1.9k	57.9	$1.76	$0.26	0.5	Rent
A10 24GB	2	85%	~981	30.7	$2.58	$0.73	0.3	Rent
Radeon PRO W7900 Dual Slot 48GB	1	83%	~743	23.2			0.3	Specs
Radeon PRO W7900 48GB	1	83%	~743	23.2			0.3	Specs
Radeon RX 7900 XTX 24GB	2	85%	~1.6k	49.0			0.7	Specs
L4 24GB	2	85%	~490	15.3	$0.98	$0.56	0.1	Rent
L20 48GB	1	83%	~743	23.2			0.3	Specs
Arc Pro B65 32GB	2	63%	~994	31.1			0.4	Specs
Arc Pro B60 24GB	2	85%	~745	23.3			0.4	Specs
L2 24GB	2	85%	~490	15.3				Specs
RTX PRO 4000 Blackwell SFF 24GB	2	85%	~706	22.1			0.1	Specs
Radeon PRO W7800 48GB	1	83%	~743	23.2			0.3	Specs
Radeon PRO W7800 32GB	2	63%	~942	29.4			0.5	Specs
GeForce RTX 3090 24GB	2	85%	~1.5k	47.8	$1.00	$0.18	0.7	Rent
A10G 24GB	2	85%	~981	30.7			0.6	Specs
GeForce RTX 3090 Ti 24GB	2	85%	~1.6k	51.5	$0.92	$0.16	0.9	Rent
Radeon PRO W6800 32GB	2	63%	~837	26.2			0.5	Specs
RTX A5000 24GB	2	85%	~1.3k	39.2	$0.54	$0.12	0.5	Rent
Radeon PRO V710 28GB	2	72%	~732	22.9			0.3	Specs

AMD vs NVIDIA

At INT4 / FP4 the NVIDIA pick is 1 x RTX PRO 6000 Blackwell Max-Q Workstation Edition at about 1.5k tokens/s for $0.50/hr; the AMD pick is 1 x Instinct MI300X 192GB at about 4.6k tokens/s for $2.39/hr. Per rental dollar NVIDIA delivers 1.62x the tokens of AMD here.

NVIDIA 49 parts fit

Best value: 1 x RTX PRO 6000 Blackwell Max-Q Workstation Edition, 1.5k tok/s for $0.50/hr

Fewest GPUs: 1 x Rubin SXM, 19k tok/s

AMD 17 parts fit

Best value: 1 x Instinct MI300X 192GB, 4.6k tok/s for $2.39/hr

Fewest GPUs: 1 x Instinct MI455X OAM, 20k tok/s

Intel 5 parts fit

Fewest GPUs: 1 x Data Center GPU Max 1550 128GB, 2.8k tok/s

Other 2 parts fit

Fewest GPUs: 1 x BR100, 1.4k tok/s

Fleet what-if

Compare whole fleets at these settings: a thousand of one part against a hundred of another. Each fleet splits into replicas of its tensor-parallel width; GPUs left over sit idle.

A GPUs Part

B GPUs Part

Fleet	Replicas	Idle	Tokens/s	Per hour	Power	Per M tokens
A 1,000 x H200 SXM 141GB	1,000 x TP1	0	4.13M	$2,990	700 kW	$0.20
B 100 x GB200 NVL72 GPU 186GB	100 x TP1	0	688k	$1,050	kW	$0.42

Size gpt-oss-20b for a tokens per second target

The calculator starts from your traffic instead of a fleet: set a target rate and read the replica count for every GPU.

Open in the LLM GPU Calculator

Frequently asked questions

How much GPU memory does gpt-oss-20b need?

The weights take 11.8 GB at the native INT4 / FP4 precision (41.8 GB at BF16, 20.9 GB at FP8, 11.8 GB at INT4). Each concurrent stream adds 48 KB of KV cache per token: 0.1 GB at 4,096 tokens and 0.8 GB at 32,768 tokens.

What is the bare minimum to run gpt-oss-20b?

1 x Instinct MI350X OAM at INT4 / FP4 with 4,096 tokens of context and one stream. The cheapest live rental for that footprint is 1 x RTX A5000 24GB at $0.27 per hour.

How many H100s do you need to run gpt-oss-20b?

one H100 at INT4 / FP4 for 4,096 tokens of context and one stream. For 32,768 tokens and 32 concurrent streams the tensor-parallel group is one H100, and 1,000 tokens per second takes 1 H100s across 1 replicas.

How much does gpt-oss-20b cost per million tokens on an H100?

About $0.17 per million output tokens at INT4 / FP4, 32,768 tokens of context and 32 streams, using the cheapest live on-demand price of $1.79 per GPU-hour. The cheapest part per token is 1 x RTX PRO 6000 Blackwell Max-Q Workstation Edition at $0.09.

What precision does gpt-oss-20b ship in?

The published checkpoint is MXFP4. That 4-bit format is what the lab validated, so the INT4 column is the native one; BF16 figures describe a dequantised copy nobody would deploy.

All figures are roofline estimates from the model's config.json and each GPU's published memory, bandwidth and tensor-core peak, with fixed efficiency factors. Prices are the cheapest live on-demand listing per GPU when the site has one. They are a planning floor; a tuned serving stack can do better. Active parameters per the OpenAI model card. Half the layers attend over a 128-token sliding window.