GPU & Architecture Glossary

Also known as: BFloat16, Brain Float

Brain Float 16 - A 16-bit floating-point format developed by Google with the same exponent range as FP32 but reduced man...

General-purpose parallel processing units in NVIDIA GPUs that handle a wide range of computational tasks. Unlike Tensor ...

Also known as: Half Precision, Float16

16-bit floating-point precision format (half precision). Commonly used in AI training and inference to reduce memory usa...

Also known as: single precision

Single-precision 32-bit floating point, the long-standing default for general-purpose computing and the baseline most GP...

Also known as: 4-bit floating point

A 4-bit floating-point format introduced with NVIDIA Blackwell, halving the memory and bandwidth footprint of FP8 again....

Also known as: double precision

Double-precision 64-bit floating point, required for scientific and engineering workloads where rounding error compounds...

Also known as: Float8, 8-bit Float

8-bit floating-point precision format introduced in Hopper architecture. Enables up to 2x throughput improvement over FP...

Also known as: High Bandwidth Memory, HBM2, HBM2e, HBM3, HBM3e

High Bandwidth Memory - A type of 3D-stacked memory technology that provides much higher bandwidth than traditional GDDR...

Also known as: 8-bit integer,quantization

An 8-bit integer format for quantized inference, mapping weights and activations onto 256 discrete levels to cut memory ...

Also known as: Chiplet, Multi-chip

A GPU design that uses multiple silicon dies connected via ultra-fast interconnects to function as a single GPU. Blackwe...

Also known as: NVIDIA FP4

NVIDIA's own 4-bit floating-point format for Blackwell, using finer-grained scaling blocks than standard FP4 to hold acc...

Also known as: NVLink-C2C

NVIDIA's proprietary high-bandwidth, low-latency interconnect technology for connecting multiple GPUs. Enables direct GP...

A high-speed switch fabric that enables full NVLink connectivity between multiple GPUs in a system. Allows any GPU to co...

Also known as: PCI Express, PCI-E

Peripheral Component Interconnect Express - Standard interface for connecting GPUs to motherboards. PCIe Gen 4 provides ...

Also known as: Streaming Multiprocessor, SMX

Streaming Multiprocessor - The fundamental processing block in NVIDIA GPU architecture. Each SM contains multiple CUDA c...

Also known as: structured sparsity,dense vs sparse,2:4 sparsity

Structured sparsity lets a GPU skip half the weights in a 2:4 pattern, doubling theoretical throughput. Vendors usually ...

A system-on-module that combines different processor types (typically CPU + GPU) into a unified package with high-bandwi...

Also known as: TensorFloat32, Tensor Float 32

TensorFloat-32 - A 19-bit precision format introduced with NVIDIA Ampere architecture. Combines the range of FP32 with r...

Also known as: TeraFLOPS, Tera FLOPS

Trillion Floating-Point Operations Per Second - A measure of computational performance. Used to quantify GPU processing ...

Specialized processing units in NVIDIA GPUs designed to accelerate matrix multiplication operations used in deep learnin...

Also known as: TE

Hardware and software technology in Hopper architecture designed to accelerate Transformer models (GPT, BERT, etc.). Aut...