OpenAI gpt-oss

gpt-oss-120b hardware requirements

Fits on one Instinct MI350X OAM at INT4 / FP4. For 1,000 tokens/s at 32k context you need 2 x H100 SXM5 80GB at $3.58/hr or 2 x RTX PRO 6000 Blackwell Max-Q Workstation Edition at $1.00/hr.

Parameters

117B

Active per token

5.1B

mixture of experts

KV cache per token

72 KB

16-bit cache

Native precision

MXFP4

Rent the cheapest fit: 1 x RTX PRO 6000 Blackwell Max-Q Workstation Edition from $0.50/hr Model card

Bare minimum

Fewest GPUs

1 x Instinct MI350X OAM

INT4 / FP4, 4,096 tokens, one stream

Tokens/s: 2.0k
Per hour: $6.16
Per million tokens: $0.86
Memory used: 24%

Rent on DigitalOcean

Cheapest per hour

1 x RTX PRO 6000 Blackwell Max-Q Workstation Edition

INT4 / FP4, 4,096 tokens, one stream, live price

Tokens/s: 444
Per hour: $0.50
Per million tokens: $0.31
Memory used: 72%

Rent on RunPod

Cheapest per million tokens

2 x RTX PRO 6000 Blackwell Max-Q Workstation Edition

INT4 / FP4, 32,768 tokens, 32 streams

Tokens/s: 2.0k
Per hour: $1.00
Per million tokens: $0.14
Memory used: 57%

Rent on RunPod

Configurations

Every datacenter GPU in the catalog, sized for these settings. Change precision, context and concurrency and the whole page follows. Each point is one replica: the smallest tensor-parallel group of that GPU that holds the model.

Precision Context Streams

Vendor

Priced only

Priced configurations only; 37 more fit without a live price and appear in the table. Marker size grows with the tensor-parallel width. Roofline estimates.

GPU	TP	Memory used	Tokens/s	Per stream	Per hour	Per M tokens	kW
Instinct MI455X OAM	1	25%	~13k	419				Specs
Rubin SXM	1	38%	13k	396				Specs
Instinct MI355X OAM	1	38%	4.6k	144			1.4	Specs
GB300 NVL72 GPU 288GB	1	38%	4.6k	144	$8.62	$0.52		Rent
GB200 NVL72 GPU 186GB	1	58%	4.6k	144	$10.50	$0.63		Rent
Instinct MI350X OAM	1	38%	4.6k	144	$6.16	$0.37	1.0	Rent
B300 SXM 262GB	1	41%	4.6k	144	$7.40	$0.45	1.4	Rent
B200 SXM 180GB	1	60%	4.4k	139	$4.09	$0.26	1.0	Rent
B100 SXM 192GB	1	57%	4.6k	144			0.7	Specs
Instinct MI325X OAM	1	42%	~3.5k	108	$3.80	$0.31	1.0	Rent
Instinct MI300X 192GB	1	57%	~3.1k	95.4	$2.39	$0.22	0.8	Rent
BR100	2	86%	~1.8k	54.7			1.1	Specs
H200 SXM 141GB	1	77%	~2.8k	86.4	$2.99	$0.30	0.7	Rent
H100 SXM5 80GB	2	69%	~3.7k	115	$3.58	$0.27	1.4	Rent
Data Center GPU Max 1550 128GB	1	85%	~1.9k	59.0			0.6	Specs
H200 NVL 141GB	1	77%	~2.8k	86.4	$3.79	$0.38	0.6	Rent
H100 NVL 94GB	2	58%	~4.3k	133	$6.38	$0.42	0.8	Rent
H100 PCIe 80GB	2	69%	~2.2k	68.4	$5.00	$0.63	0.7	Rent
H800 PCIe 80GB	2	69%	~2.2k	68.4			0.7	Specs
RTX PRO 6000 Blackwell Workstation Edition	2	57%	2.0k	61.3	$3.60	$0.51	1.2	Rent
RTX PRO 6000 Blackwell Server Edition	2	57%	~1.7k	54.6	$1.18	$0.19	1.2	Rent
RTX PRO 6000 Blackwell Max-Q Workstation Edition	2	57%	2.0k	61.3	$1.00	$0.14	0.6	Rent
Data Center GPU Max 1100 48GB	4	58%	~2.5k	79.6			1.2	Specs
GeForce RTX 5090 32GB	4	88%	3.7k	116	$3.96	$0.30	2.3	Rent
MI250X 128GB	1	85%	~1.9k	59.0			0.6	Specs
RTX 6000 Ada 48GB	4	58%	~2.0k	62.2	$3.36	$0.47	1.2	Rent
Instinct MI250 128GB	1	85%	~1.9k	59.0			0.5	Specs
L40S 48GB	4	58%	~1.8k	56.0	$3.48	$0.54	1.4	Rent
A800 PCIe 80GB	2	69%	~2.1k	66.2			0.6	Specs
A100 PCIe 80GB	2	69%	~2.1k	66.2	$2.70	$0.35	0.6	Rent
A100 SXM4 80GB	2	69%	~2.2k	69.7	$2.80	$0.35	0.8	Rent
A100 PCIe 40GB	4	70%	~3.2k	101			1.0	Specs
A100 SXM4 40GB	4	70%	~3.2k	101	$5.16	$0.44	1.6	Rent
GeForce RTX 5090 D V2 24GB	8	59%	~5.3k	164			4.6	Specs
RTX 5880 Ada 48GB	4	58%	~2.0k	62.2			1.1	Specs
RTX 5000 Ada 32GB	4	88%	~1.2k	37.3	$3.32	$0.77	1.0	Rent
RTX PRO 5000 Blackwell 48GB	4	58%	~2.8k	87.1	$3.84	$0.38	1.2	Rent
RTX PRO 5000 Blackwell 72GB	2	76%	~1.5k	46.0			0.6	Specs
RTX PRO 4500 Blackwell 32GB	4	88%	~1.9k	58.0	$2.88	$0.43	0.8	Rent
Radeon AI PRO R9700 32GB	4	88%	~1.3k	41.5			1.2	Specs
Instinct MI100 32GB	4	88%	~2.5k	79.6			1.2	Specs
Arc Pro B70 32GB	4	88%	~1.3k	39.4			0.9	Specs
L40 48GB	4	58%	~1.8k	56.0	$3.28	$0.51	1.2	Rent
Instinct MI210 PCIe	2	86%	~1.8k	56.0			0.6	Specs
GeForce RTX 4090 24GB	8	59%	~3.9k	123	$4.80	$0.34	3.6	Rent
A30 24GB	8	59%	~3.7k	114	$5.86	$0.45	1.3	Rent
RTX 4500 Ada 24GB	8	59%	~1.7k	52.9			1.7	Specs
RTX A6000 48GB	4	58%	~1.6k	49.8	$2.00	$0.35	1.2	Rent
A40 48GB	4	58%	~1.4k	45.1	$1.96	$0.38	1.2	Rent
H20 141GB HBM3e	1	77%	~2.3k	72.0			0.4	Specs
H20 96GB	2	57%	~4.4k	137			0.8	Specs
RTX PRO 4000 Blackwell 24GB	8	59%	~2.6k	82.2	$4.56	$0.48	1.2	Rent
BI-V100 Tiangai 100	4	88%	~2.5k	77.7			1.0	Specs
RTX A5500 24GB	8	59%	~3.0k	94.0			1.8	Specs
V100S PCIe 32GB	4	88%	~2.4k	73.5	$3.52	$0.42	1.0	Rent
A10 24GB	8	59%	~2.3k	73.4	$10.32	$1.22	1.2	Rent
Radeon PRO W7900 Dual Slot 48GB	4	58%	~1.8k	56.0			1.2	Specs
Radeon PRO W7900 48GB	4	58%	~1.8k	56.0			1.2	Specs
Radeon RX 7900 XTX 24GB	8	59%	~3.8k	117			2.8	Specs
L4 24GB	8	59%	~1.2k	36.7	$3.92	$0.93	0.6	Rent
L20 48GB	4	58%	~1.8k	56.0			1.1	Specs
Arc Pro B65 32GB	4	88%	~1.3k	39.4			0.8	Specs
Arc Pro B60 24GB	8	59%	~1.8k	55.8			1.6	Specs
L2 24GB	8	59%	~1.2k	36.7				Specs
RTX PRO 4000 Blackwell SFF 24GB	8	59%	~1.7k	52.9			0.6	Specs
Radeon PRO W7800 48GB	4	58%	~1.8k	56.0			1.0	Specs
Radeon PRO W7800 32GB	4	88%	~1.2k	37.3			1.0	Specs
GeForce RTX 3090 24GB	8	59%	~3.7k	115	$4.00	$0.30	2.8	Rent
A10G 24GB	8	59%	~2.3k	73.4			2.4	Specs
GeForce RTX 3090 Ti 24GB	8	59%	~3.9k	123	$3.68	$0.26	3.6	Rent
Radeon PRO W6800 32GB	4	88%	~1.1k	33.2			1.0	Specs
RTX A5000 24GB	8	59%	~3.0k	94.0	$2.16	$0.20	1.8	Rent
Radeon PRO V710 28GB	8	50%	~1.8k	54.8			1.3	Specs

AMD vs NVIDIA

At INT4 / FP4 the NVIDIA pick is 2 x RTX PRO 6000 Blackwell Max-Q Workstation Edition at about 2.0k tokens/s for $1.00/hr; the AMD pick is 1 x Instinct MI300X 192GB at about 3.1k tokens/s for $2.39/hr. Per rental dollar NVIDIA delivers 1.54x the tokens of AMD here.

NVIDIA 49 parts fit

Best value: 2 x RTX PRO 6000 Blackwell Max-Q Workstation Edition, 2.0k tok/s for $1.00/hr

Fewest GPUs: 1 x Rubin SXM, 13k tok/s

AMD 17 parts fit

Best value: 1 x Instinct MI300X 192GB, 3.1k tok/s for $2.39/hr

Fewest GPUs: 1 x Instinct MI455X OAM, 13k tok/s

Intel 5 parts fit

Fewest GPUs: 1 x Data Center GPU Max 1550 128GB, 1.9k tok/s

Other 2 parts fit

Fewest GPUs: 2 x BR100, 1.8k tok/s

Fleet what-if

Compare whole fleets at these settings: a thousand of one part against a hundred of another. Each fleet splits into replicas of its tensor-parallel width; GPUs left over sit idle.

A GPUs Part

B GPUs Part

Fleet	Replicas	Idle	Tokens/s	Per hour	Power	Per M tokens
A 1,000 x H200 SXM 141GB	1,000 x TP1	0	2.76M	$2,990	700 kW	$0.30
B 100 x GB200 NVL72 GPU 186GB	100 x TP1	0	461k	$1,050	kW	$0.63

Size gpt-oss-120b for a tokens per second target

The calculator starts from your traffic instead of a fleet: set a target rate and read the replica count for every GPU.

Open in the LLM GPU Calculator

Frequently asked questions

How much GPU memory does gpt-oss-120b need?

The weights take 65.7 GB at the native INT4 / FP4 precision (234 GB at BF16, 117 GB at FP8, 65.7 GB at INT4). Each concurrent stream adds 72 KB of KV cache per token: 0.2 GB at 4,096 tokens and 1.2 GB at 32,768 tokens.

What is the bare minimum to run gpt-oss-120b?

1 x Instinct MI350X OAM at INT4 / FP4 with 4,096 tokens of context and one stream. The cheapest live rental for that footprint is 1 x RTX PRO 6000 Blackwell Max-Q Workstation Edition at $0.50 per hour.

How many H100s do you need to run gpt-oss-120b?

one H100 at INT4 / FP4 for 4,096 tokens of context and one stream. For 32,768 tokens and 32 concurrent streams the tensor-parallel group is 2 H100s, and 1,000 tokens per second takes 2 H100s across 1 replicas.

How much does gpt-oss-120b cost per million tokens on an H100?

About $0.27 per million output tokens at INT4 / FP4, 32,768 tokens of context and 32 streams, using the cheapest live on-demand price of $1.79 per GPU-hour. The cheapest part per token is 2 x RTX PRO 6000 Blackwell Max-Q Workstation Edition at $0.14.

What precision does gpt-oss-120b ship in?

The published checkpoint is MXFP4. That 4-bit format is what the lab validated, so the INT4 column is the native one; BF16 figures describe a dequantised copy nobody would deploy.

All figures are roofline estimates from the model's config.json and each GPU's published memory, bandwidth and tensor-core peak, with fixed efficiency factors. Prices are the cheapest live on-demand listing per GPU when the site has one. They are a planning floor; a tuned serving stack can do better. Active parameters per the OpenAI model card. Half the layers attend over a 128-token sliding window, so the cache grows at half the rate the head count suggests.