Start-Up Cerebras Builds Massive AI Supercomputer

Matthew Broersma,
Linkedin
Cerebras founder and chief executive Andrew Feldman. Image credit: Cerebras

California start-up Cerebras builds exaflop supercomputer for AI workloads by linking together 16 of its CS-2 systems

Artificial intelligence chip start-up Cerebras Systems has formally announced a supercomputer system called Andromeda constructed from 16 of its CS-2 computing systems.

The CS-2 is a self-contained supercomputing cluster based on Cerebras’ AI chip, the Wafer-Scale Engine 2, the world’s largest semiconductor with 850,000 compute cores operating in parallel.

The Wafer-Scale Engine 2 is about the size of a semiconductor manufacturing wafer, roughly comparable to a dinner plate.

The Wafer-Scale Engine chip. Image credit: Cerebras
The Wafer-Scale Engine 2 chip. Image credit: Cerebras

Building blocks

Cerebras said its approach of linking such building blocks together to create the supercomputer means reduced cost and complexity when compared with conventional supercomputers such as the world’s most powerful, the Frontier system that came into operation earlier this year at the Oak Ridge Leadership Computing Facility (OLCF) in Tennessee.

Frontier took years to build and cost hundreds of millions, while Andromeda cost $30 million (£25m) and was assembled in three days, said Cerebras co-founder and chief executive Andrew Feldman.

Both are exascale systems, meaning they can perform more than one quintillion operations per second, although Frontier uses 64 bit double precision and Andromeda operates at 16-bit half-precision.

Cerebras is one of a handful of companies challenging the dominance of Nvidia in AI chips, which are often used in supercomputers, such as Meta’s RSC system, announced in January, which is powered by more than 6,000 Nvidia A100 Tensor Core GPUs.

Image credit: Cerebras
Image credit: Cerebras

Linear scaling

The company said its system provides near-perfect linear scaling for large language model (LLM) workloads, meaning AI training time is reduced in a near-perfect proportion as more CS-2 units are added to the system.

The systems are linked together using a data switch Cerebras introduced last year, Swarm-X, which connects the CS-2 units to a central repository called Memory-X

Andromeda is hosted at California data centre Colovore and is available for remote access by organisations and researchers, including those from US national labs.

The University of Edinburgh’s Edinburgh Parallel Computing Centre (EPCC) in January of last year became the first European customer to use the CS-2’s predecessor, the CS-1, at the Edinburgh International Data Facility.