Azure OpenAI GPT-3 Cluster

Specifications

Additional Information

"The supercomputer developed for OpenAI is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server. Compared with other machines listed on the TOP500 supercomputers in the world, it ranks in the top five, Microsoft says." "We understand the supercomputer uses AMD second-generation Rome Epyc server processors, and Nvidia V100 GPUs, from sources in the industry."

It has "since been upgraded to include tens of thousands of A100 chips". Rumors suggest that it is in Washington State