Qwen3.8-2.4T-A95B hardware requirements
Needs at least 16 GPUs at BF16 (Instinct MI455X).
Top options for Qwen3.8-2.4T-A95B
The strongest part each vendor offers for Qwen3.8-2.4T-A95B at BF16 with 32,768 tokens per user, rentable parts first. Each card is one replica, the fewest accelerators that hold the model, and the line it draws on the graph below. Select a card to plot it.
Tokens per second as users grow
Each line is one replica, the fewest accelerators that hold the model, serving more users at once. It climbs while memory bandwidth is the limit, flattens once compute is, and stops where memory runs out at 32,768 tokens per user. Hover a line to bring it forward.
16 x Instinct MI455X: peaks at 4.3k tokens per second when memory fills at 482 users.
Memory fills at: 16 x Instinct MI455X 482 users. Roofline estimates at BF16.
Bare minimum
- Tokens/s
- 1.1k
- Per hour
- no live price
- Per million tokens
- n/a
- Memory used
- 73%
No configuration qualifies.
No configuration qualifies.
Configurations
Every datacenter GPU in the catalog, sized for these settings. Change precision, context and concurrency and the whole page follows. Each point is one replica: the smallest tensor-parallel group of that GPU that holds the model.
No priced configuration fits at these settings.
| Rent | |||
|---|---|---|---|
Instinct MI455X 2 nodes | 16 | 1.9k tok/s | Specs → |
| 16 | ~1.4k tok/s | Specs → |
By vendor
At BF16 only AMD parts fit: 16 x Instinct MI455X at about 1.9k tokens/s. No NVIDIA part holds the model at these settings.
AMD: no live price, no power figure (16 x Instinct MI455X).
Fleet what-if
Compare whole fleets at these settings: a thousand of one part against a hundred of another. Each fleet splits into replicas of its tensor-parallel width; GPUs left over sit idle.
| Fleet | Tokens/s |
|---|
Size Qwen3.8-2.4T-A95B for a tokens per second target
The calculator starts from your traffic instead of a fleet: set a target rate and read the replica count for every GPU.
Frequently asked questions
How much GPU memory does Qwen3.8-2.4T-A95B need?
The weights take 4,892 GB at the native BF16 precision (4,892 GB at BF16, 2,446 GB at FP8, 1,376 GB at INT4). Each concurrent stream adds 92 KB of KV cache per token: 0.4 GB at 4,096 tokens and 3.1 GB at 32,768 tokens.
What is the bare minimum to run Qwen3.8-2.4T-A95B?
16 x Instinct MI455X at BF16 with 4,096 tokens of context and one stream.
How many H100s do you need to run Qwen3.8-2.4T-A95B?
more than sixteen H100s at BF16 for 4,096 tokens of context and one stream. For 32,768 tokens and 32 concurrent streams the tensor-parallel group is more than sixteen H100s.
How much does Qwen3.8-2.4T-A95B cost per million tokens on an H100?
No live H100 rental price is on file right now, so the cost per token cannot be quoted. The configuration table lists every part that has one.
What precision does Qwen3.8-2.4T-A95B ship in?
The published checkpoint is BF16. FP8 and INT4 figures describe post-training quantisations that halve and quarter the footprint at a small accuracy cost.
All figures are roofline estimates from the model's config.json and each GPU's published memory, bandwidth and tensor-core peak, with fixed efficiency factors. Prices are the cheapest live on-demand listing per GPU when the site has one. They are a planning floor; a tuned serving stack can do better. Only 23 of 92 layers keep a per-token cache; the 69 linear-attention layers hold a fixed 579 MB of FP32 state per sequence. Active parameters per the Qwen model card.