Definition
An AI accelerator is not a chip. It is a dozen pieces of silicon bonded to a shared carrier and sold as one component. Open an NVIDIA Blackwell Ultra package and you find two processor dies — each one as large as a lithography scanner can physically print — welded together by a 10 TB/s die-to-die link, surrounded by eight towers of stacked memory, all mounted on a common slab of silicon (NVIDIA).
Advanced packaging is the manufacturing step that assembles those pieces into one part. It is not finishing work, and it is not the box the chip ships in. It is a distinct industrial capability with its own factories, its own queue and its own yield — and through most of the AI build-out it, rather than the wafer fab, has been the thing you could not get. When an accelerator is described as "sold out", the shortage is frequently downstream of the transistors.
How It Works
The reticle limit: you are not allowed to make the chip bigger
A lithography scanner prints one rectangle of pattern at a time, and that rectangle has a maximum size: 26 mm × 33 mm, or 858 mm² (Vik's Newsletter). This is the reticle limit, and it is a property of the machine's optics, not of anyone's ambition — the scanners themselves are the subject of EUV Lithography. No single die can exceed it in one exposure.
Accelerators are already pressed flat against it. NVIDIA's H100 die measures 26 × 31 mm — 814 mm² (Silicon Analysts) — which is 94.9% of the largest area a scanner can expose. There is no headroom left. Anyone who wants more compute in a single accelerator has no lithographic option at all; they must place more than one die in the package. This is why Blackwell exists in the shape it does: two reticle-sized dies bonded edge to edge, communicating fast enough that software addresses them as one GPU. Going bigger stopped being a design preference and became a packaging requirement.
Yield falls exponentially with area, which makes small dies cheaper
The second force is economic, and it is the one that turns "we are allowed to use several dies" into
"we would use several dies even if we were allowed one". Defects land on a wafer at random, and
Semiconductor Manufacturing explains where they come from;
what matters here is only the shape of the resulting curve. Under the standard Poisson model, a die's
yield is Y = e^(−D₀ × A), where D₀ is defects per cm² and A is die area
(Vik's Newsletter). The
exponent is the whole story: yield does not decline gently with size, it collapses.
Take a defect density of 0.1 defects/cm² — a normal figure for a mature leading-edge node, where
the cited range is 0.05–0.15 (Silicon Analysts) —
and a 300 mm wafer with 3 mm of edge exclusion, so the usable radius is 147 mm. Gross dies per wafer
follow the standard approximation GDPW = πR²/S − πR/√(2S) (Silicon Analysts, above).
One monolithic 800 mm² accelerator die:
- Gross dies:
π(147)²/800 − π(147)/√1600 = 84.9 − 11.5 ≈ 73 - Yield:
e^(−0.1 × 8.0 cm²) = e^(−0.8) = 44.9% - Good dies:
73 × 0.449 ≈ 33
The same accelerator as four chiplets. Real chiplet designs pay a silicon tax for the die-to-die interfaces and redundancy — AMD measured it at about 10% (The Next Platform) — so give each chiplet 220 mm², for 880 mm² of silicon per accelerator instead of 800:
- Gross dies:
π(147)²/220 − π(147)/√440 = 308.7 − 22.0 ≈ 286 - Yield:
e^(−0.1 × 2.2 cm²) = e^(−0.22) = 80.3% - Good chiplets:
286 × 0.803 ≈ 230, which is230 ÷ 4 ≈ 57accelerators' worth
Fifty-seven versus thirty-three, from the same wafer, while spending 10% more silicon. Per accelerator, the chiplet approach costs `33 ÷ 57 ≈ 0.57× the silicon of the monolithic one — it nearly halves the bill of the most expensive material in the industry.
That is not a toy result. When AMD published the real numbers for EPYC, its quad-chiplet design (213 mm² chiplets in place of a hypothetical 777 mm² monolith, using ~10% more silicon in total) came out at approximately 0.59× the cost of the monolithic approach (The Next Platform, above). The estimate above lands within two points of a figure measured in a real factory — and the gap is roughly what the packaging itself costs.
The mechanism is worth stating plainly, because it is the entire case for chiplets: a defect that lands on an 800 mm² die destroys 800 mm² of silicon; the same defect on a 220 mm² chiplet destroys 220 mm². Splitting the area replaces one brutal exponent with four mild ones. And because chiplets are tested before they are bonded — the industry calls a chiplet that passes a known good die — the survivors are not dragged down by their neighbours.
Putting it back together: the interposer, the bumps, and the bond
Having deliberately broken the chip apart, someone now has to reassemble it well enough that the pieces behave as one. TSMC's CoWoS — Chip-on-Wafer-on-Substrate — names the two moves. First the compute dies and the memory stacks are bonded onto a silicon interposer, a slab of silicon that acts as a miniature circuit board inside the package (chip-on-wafer). Then the interposer is mounted on an ordinary organic package substrate (on-substrate). The memory side of this is owned by High Bandwidth Memory; the point here is the other side of the bond, and one consequence that follows from it.
Interposer area is a budget, and it is the real limit on how much memory you can attach. Today's CoWoS runs to about 3.3× the reticle size, which is what allows eight HBM stacks to sit around the compute dies; TSMC's roadmap goes to 5.5× reticle with twelve HBM4 stacks in 2025–2026, and a 9× reticle interposer of up to 7,722 mm² by 2027 (3D InCites). Notice what those numbers are measuring. Not transistors, not nanometres — floor space inside a package. That is now a first-class design constraint.
The connections themselves are where the newest gains are. Dies have traditionally been joined by
microbumps, tiny solder balls, and it is extremely hard to push their pitch below 10 µm
(TSPA Semiconductor).
Hybrid bonding removes the solder entirely and fuses copper directly to copper: Intel's Foveros
Direct went from a 9 µm pitch in its first generation to 3 µm in its second (TSPA, above).
Connection density scales as the inverse square of pitch — (1000/p)² connections per mm² — so a
10 µm microbump pitch gives about 10,000 connections per mm² while a 3 µm hybrid bond gives about
111,000, more than eleven times as many wires through the same area. No transistor got
smaller to earn that.
The reframing
Moore's law slowing did not stop performance from scaling; it moved where the scaling comes from. Density used to be bought by shrinking features. It is now also bought by stacking silicon and bonding it more finely — and both of those are packaging. The most consequential number on a modern accelerator's spec sheet may be how much silicon fits in the package, not how small the transistors on it are.
Real-World Applications
Forecasting AI chip supply. This is the concept's cash value, and it is why the term escaped the trade press. An accelerator's build rate is the minimum of three independent supply lines: wafer starts, HBM stacks, and packaging slots. Packaging has repeatedly been the minimum. In January 2025, Nvidia's CEO said the industry's advanced packaging capacity was "probably four times" what had been available less than two years earlier (Taipei Times) — and it was still the constraint. As of mid-2026, TSMC's CoWoS supply–demand gap is reported at around 20%, projected to narrow to about 10% by year end, with monthly CoWoS capacity heading for 120,000–140,000 wafers plus 50,000–60,000 more from outsourced assembly partners (TrendForce). Treat those capacity figures as perishable — they are revised every quarter and they are the first thing on this page that will go stale. The structural claim is the durable one: packaging is a separate factory with a separate queue, and it does not scale just because the fab did.
Designing the accelerator. Chiplets change what a design team can do. A compute chiplet can be built on the newest, most expensive node while I/O and memory controllers — which barely benefit from shrinking — stay on an older, cheaper one, so the costly silicon is spent only where it pays. The same chiplet can be reused across a whole product line, with the number bonded into the package setting the SKU. And a design can now exceed the reticle limit at all, which is the only reason a 208-billion-transistor part exists (NVIDIA, above).
Nations and export controls. Because packaging is a distinct chokepoint, it has become a distinct policy target: a country can hold wafer capacity and still be unable to build a competitive accelerator without advanced packaging lines. Any analysis of who can build AI chips that stops at the fab is reading half the supply chain.
Key Concepts
- Known good die: a chiplet tested before bonding. This is the hinge the whole chiplet economy turns on — without it, you would assemble packages out of untested parts and the yield advantage of small dies would be thrown away at the bonding step.
- The interposer is the new motherboard: a wide, short bus is only possible over millimetres of silicon. Anything that wants HBM-class bandwidth has to move onto the interposer, and thereby inherits its area budget and its capacity queue.
- Packaging yield multiplies, it does not add: a package with two compute dies and eight memory stacks has ten chances to fail assembly, and a failure at the last step scraps every component already bonded — the most valuable silicon in the industry, in one piece.
Challenges
What breaks: planning capacity from wafer supply alone. This is the mistake the concept exists to prevent, and it is made constantly — by buyers estimating when their GPUs will arrive, by analysts modelling a foundry's output, by governments counting fab capacity. A fab can have wafers free while the packaging line is booked for a year, and the accelerator still does not ship. Wafer starts are a necessary condition for accelerator supply and have not, for most of the AI build-out, been the binding one. If your model has one number in it, it is modelling the wrong step.
Packaging capacity is not fungible. It is tempting to treat "CoWoS wafers per month" as a single pool, and it is not. Hopper is packaged with CoWoS-S and Blackwell largely with CoWoS-L, and Nvidia's CEO has described converting CoWoS-S capacity into CoWoS-L (Taipei Times, above). A line qualified for one variant cannot absorb demand for another without being retooled and requalified, so headline capacity can rise while the capacity that matters for a specific product does not.
The whole package scraps as one. There is no field repair and no upgrade path. A single bad memory stack can write off a package containing two reticle-sized compute dies, so packaging turns independent component yields into a compounding product yield — and it means the compute and the memory in a fleet are bought, deployed and retired as one unit.
Heat and warpage do not shrink. Bonding hundreds of watts of logic millimetres from thinned DRAM, across a silicon interposer several times the reticle size, on a substrate over 100 mm on a side (3D InCites, above), is a mechanical problem as much as an electrical one: the materials expand at different rates, the assembly bends, and the bonds are microns wide. Package size and package reliability pull against each other, and every generation has to renegotiate the trade.
Future Trends
The roadmap is unusually legible, because it is mostly about area: 5.5× reticle interposers with twelve HBM4 stacks, then 9× reticle and 7,722 mm² by 2027, on substrates beyond 120 × 120 mm, with different process nodes stacked vertically inside the same package — 1.6 nm logic on top of 2 nm (3D InCites, above). The unit of design is drifting from the die to the package, and the interesting engineering is following it there.
The sharpest reason to expect packaging to matter more, not less, comes from an unexpected direction: the next generation of lithography makes the problem worse. High-NA EUV uses anamorphic optics, and the price is a field half the size — 26 × 16.5 mm, a maximum die of 429 mm² (IOP / Jpn. J. Appl. Phys.). The most advanced scanner ever built will print dies smaller than today's accelerators. Whatever is printed on it will have to be stitched or bonded back together.
That is the whole trajectory in one fact. The lithography gets finer, the printable chip gets smaller, and the accelerator gets bigger anyway — in the package.