Cerebras deploys trillion-parameter AI models
Archived — this story has rotated out of today’s deck. It is kept here in full.
The gist
Cerebras published an engineering post splitting a ~3-trillion-parameter model across 59 wafers at 1,000-2,000+ tokens per second.
The pitch targets the memory bottleneck and Nvidia GPU scarcity driving AI serving costs.
Background
Cerebras Systems makes wafer-scale AI chips; its current chip, the WSE-3, packs 4 trillion transistors and 900,000 cores on a single wafer built on a 5-nanometer TSMC process. The post, written by Sarah Chieng, answers the practical objection that limited on-chip memory rules out wafer-scale chips for trillion-parameter models, which need terabytes just for their weights. It explains an architecture rather than introducing a newly trained model, using Moonshot AI's open-weight Kimi K3 (~3 trillion parameters) as an illustrative example. It lands as Nvidia H100 rental prices climbed roughly 40% since last October, from about $1.70 to $2.35 per GPU-hour per SemiAnalysis data, with on-demand H100 capacity effectively sold out.
How it unfolded
- May (year not stated)Cerebras went public, pricing its IPO at $185 a share and closing its first day up nearly 70%, pushing market capitalization to roughly $95 billion.
- January (year not stated)OpenAI signed a multiyear agreement worth more than $20 billion to deploy 750 megawatts of Cerebras wafer-scale systems from 2026 through 2028.
- Sep 29, 2026General Compute signed a multiyear Cerebras inference deal for agentic coding workloads.
- Oct 13, 2026CEO Andrew Feldman argued at TechCrunch Disrupt 2026 in San Francisco that the industry's dependence on Nvidia GPUs is cracking under its own weight.
- Oct 22, 2026Cerebras published an engineering post explaining how it splits a large frontier model across multiple WSE-3T wafers, using Kimi K3 as an example.
- Oct 22, 2026The illustrative deployment spans 59 wafers — 35 for 69 linear-attention layers and 24 for 24 dense layers — with 44 GB on-chip memory per wafer and ~16 GB per layer at 4-bit weights.
- Oct 22, 2026Cerebras cited generation rates of 1,000 to more than 2,000 tokens per second for the configuration; shares had dipped into the $140s by mid-October.
Who’s saying what
- Official
- Cerebras says each layer fits in a wafer's 44 GB of on-chip SRAM so weights never move, only activations cross between wafers via RDMA over Ethernet at roughly 2 microseconds per hop.
- Official
- Cerebras claims the approach needs 97% less code than GPU clusters to train the same models and that LLMs on its hardware respond up to 15 times faster than on GPU-based systems.
- Analysts
- The post addresses the objection that limited on-chip memory rules out wafer-scale chips for trillion-parameter models, which need terabytes for weights alone.
Still unverified
Cerebras cites generation rates of 1,000 to more than 2,000 tokens per second without establishing a universal production guarantee; the deployment was a technical architecture example, not a public demonstration serving real users.