SambaNova SN40L

The SambaNova SN40L is the fourth-generation Reconfigurable Dataflow Unit and the chip behind the SambaRack system. Rather than a fixed matrix pipeline it is a coarse-grained reconfigurable array of 1,040 Pattern Compute Units and 1,040 Pattern Memory Units on a two-dimensional mesh, delivering 638 TFLOPS of BF16. Its defining feature is a three-tier memory system: 520 MB of on-chip distributed SRAM, 64 GB of on-package HBM at 1.8 TB/s, and 1.5 TB of off-package DDR at 200 GB/s. That third tier is the point of the design, letting a single socket hold many models in DDR and switch between them by moving weights into HBM in microseconds, which is how SambaNova runs a 150-expert, trillion-parameter Composition of Experts on eight sockets. The chip is built on TSMC 5nm as a dual-die socket using CoWoS packaging. Note that SambaNova publishes no specification table on its website and no datasheet; these figures come from Table II of its own architecture paper, and the memory tiers are independently confirmed by the SambaRack totals in SambaNova's product brief. SambaNova states the die size only as under 650 mm2 and the clock only as under 2 GHz, so neither is recorded here as a precise value, and no power figure is published anywhere. Neither the paper nor the brief labels the compute figure dense or sparse, and no sparse figure is published.