Definition
An Application-Specific Integrated Circuit (ASIC) is a chip whose circuitry is permanently wired for a single task, which lets it beat a general-purpose processor badly at that task and makes it useless for everything else. Anyone chooses one over a GPU for an economic reason rather than a technical one: the design is paid for in full before a single working chip exists, so an ASIC only makes sense above a break-even volume — which is why Google builds TPUs for an operation it runs billions of times a day, and why bitcoin mining, whose algorithm can never change, abandoned general-purpose hardware within a few years and never looked back.
The word "specific" is doing the work. A CPU can run any program. A GPU can run any program that is embarrassingly parallel. An ASIC runs the thing it was built for. That is a strictly worse deal on flexibility and a strictly better one on efficiency, and the whole subject is about when the trade is worth making.
What breaks if you get it wrong is expensive and slow to discover. Spend two years and a few hundred million dollars specialising silicon for a workload, and if the workload moves in the meantime you own a warehouse of excellent chips for a problem nobody has any more. That failure has happened repeatedly and to well-funded teams, which is the part of this page worth remembering.
How It Works
Where the efficiency actually comes from
The gain is not magic and it is not better transistors — a custom chip and a GPU are often built on the same process node at the same foundry. The gain comes from deleting the machinery that makes a processor programmable.
Mark Horowitz's ISSCC 2014 keynote put numbers on this that are still the clearest illustration available. At a 45nm node, an 8-bit integer addition costs about 0.03 picojoules. The instruction fetch, decode and register-file traffic needed to tell a programmable core to perform that addition costs about 70 picojoules — roughly 2,300 times the energy of the arithmetic itself. Programmability is not free; it is almost the entire energy budget.
Every architecture is a strategy for amortising that 70 pJ across more useful work. Take the figure and divide it, as order-of-magnitude arithmetic rather than a measurement of any real chip:
- A scalar CPU instruction spreads it over one operation: ~70 pJ of overhead per op.
- A GPU issues one instruction across a warp of 32 lanes: ~2.2 pJ per op, a 32x improvement, and this is most of why GPUs beat CPUs on neural networks at all.
- A systolic matrix multiply unit spreads it over its entire array. Google's first TPU had a 65,536-element 8-bit multiply-accumulate array, so a single control decision drives 65,536 multiply-accumulates: ~0.001 pJ per op.
That is the mechanism behind the headline result in Google's ISCA 2017 TPU paper — the chip ran about 15-30x faster than its contemporary CPU and GPU with performance per watt 30-80x higher. A ratio like that looks implausible until you notice it is mostly the overhead ratio above, not a claim about better multipliers.
An ASIC also gets to hard-wire choices a general chip must leave open: the exact numeric format (int8, quantised formats, bespoke floating-point widths), the exact dataflow between compute and memory, the exact on-chip buffer sizes. TPU v1 carried 28 MiB of software-managed on-chip memory because the designers knew what would be sitting in it. A GPU cannot know, so it spends area on caches and coherence logic instead.
Why it takes years
The design flow is long because almost all of it must be right the first time. Architecture and RTL design, then verification — which typically consumes more engineering effort than the design itself — then physical design and timing closure, then tapeout, then the foundry builds a photomask set, then wafers, then bring-up and characterisation, then production ramp. A bug found after tapeout means a respin: new masks, another pass through the fab, months lost.
The consequence that matters is the freeze point. The workload has to be pinned down at the start of that pipeline, not the end. You are not designing for the model architectures you can see; you are designing for the ones that will be running when the chip lands. Google's first TPU was unusually fast at 15 months from design to deployed silicon, and 15 months is a long time in machine learning.
The break-even calculation
Here is the whole economic argument in one worked example. The cost structure of a chip has two parts that behave completely differently: non-recurring engineering (NRE), paid once and up front, and the marginal cost per die, paid per chip and comparatively small.
Industry estimates put total design NRE at roughly $100-200 million for a 7nm-class design and $400-600 million or more at 3nm; the photomask set alone is $10-20 million at 3nm. Take $150 million as a working figure. Assume the finished accelerator costs about $3,000 to actually build — large die, HBM stacks, advanced packaging, test — and that it replaces a data-centre GPU you would otherwise buy for around $25,000. These last two are illustrative; plug in your own and the shape does not change.
Each deployed chip saves $22,000 of capital expenditure, so break-even arrives at $150,000,000 ÷ $22,000 per chip ≈ 6,800 chips. Below that, you should have bought GPUs. The amortisation curve either side of it is brutal in both directions:
- 1,000 chips deployed — NRE is $150,000 per chip. Your custom accelerator effectively costs six GPUs each.
- 10,000 chips — $15,000 per chip. Roughly break-even; you did enormous work to draw level.
- 1,000,000 chips — $150 per chip. The design cost is 5% of the hardware and no longer part of the conversation.
This is why the list of organisations building AI silicon is short and consists almost entirely of companies that operate their own data centres at six-figure accelerator counts. It is not that everyone else lacks the expertise. It is that for a company running a 500-GPU cluster, the arithmetic is not close.
Energy savings ride on top but are smaller than people assume at these volumes. A chip drawing 400 W that saves 40% of that runs 160 W cooler; over three years at $0.08/kWh that is about 4,200 kWh, or roughly $340 per chip — real, but only 1.5% of the $22,000 capital delta. The reason power efficiency dominates hyperscaler thinking anyway is that a data centre's constraint is usually megawatts of grid connection, not dollars. Efficient chips do not mainly lower the electricity bill; they let you install more chips inside a power budget you cannot expand.
Types
Two taxonomies are in common use, and they answer different questions.
The first is the generality ladder, which places ASICs relative to everything else. A CPU executes arbitrary sequential code. A GPU executes arbitrary data-parallel code, trading branch flexibility for thousands of lanes. An FPGA has no fixed function at all — it is a fabric of configurable logic blocks wired at load time, so it can be re-specialised in hours and needs no foundry, at the price of running perhaps an order of magnitude slower and hungrier than the same design hardened into silicon. An ASIC is the end of the ladder: maximum efficiency, zero re-specialisation. Efficiency rises left to right; the cost of being wrong rises with it.
The second is the design-style spectrum, which describes how much of the chip is actually custom:
- Full-custom — transistors placed and sized individually. Reserved for a handful of blocks where the last 20% of performance justifies enormous labour, such as high-speed SerDes or memory cells.
- Standard-cell — the mainstream. The design is synthesised from RTL onto a library of pre-characterised logic cells the foundry supplies, then placed and routed automatically. Nearly every AI accelerator you have heard of is a standard-cell ASIC.
- Gate-array and structured ASIC — most mask layers are fixed in advance and only the upper metal layers are customised. Far cheaper NRE and faster turnaround, at a real cost in density and speed. It is the compromise for volumes too high for FPGAs and too low for a full mask set.
Worth naming honestly: "AI ASIC" is used loosely, and most AI accelerators are not fixed-function the way a bitcoin miner is. A TPU has an instruction set, a compiler and enough programmability to run architectures nobody had invented when the silicon was designed. It is specialised to a domain — dense linear algebra on tensors — rather than to an algorithm. That middle ground is not a fudge; it is the compromise the industry deliberately settled on, precisely because of the freeze-point problem above.
Real-World Applications
Bitcoin mining is the cleanest case that exists, because it satisfies all three conditions at once: the SHA-256 hash is fixed forever, demand for chips is effectively unbounded, and every joule saved converts directly into profit. The industry walked the entire generality ladder in about four years — CPUs in 2009, GPUs from 2010, FPGAs in 2011, ASICs from 2013 — and then stopped, because there was nowhere further to go. Mining on a CPU consumed on the order of 5,000,000 joules per terahash; current mining ASICs operate below 15 J/TH, an efficiency gain of roughly 300,000x. General-purpose hardware did not become less competitive at bitcoin. It became worthless for it.
Google's TPUs are the case that makes AI different, and the same 2017 paper that reported the 30-80x efficiency win also contains the warning. Google measured what its own data centres were actually running: multi-layer perceptrons were 61% of the inference workload and LSTMs 29%, while convolutional networks — the architecture the research community was overwhelmingly focused on at the time — were 5%. The chip earned its keep because it was designed around the workload rather than around the literature. TPUs now serve Google Search, Ads, Translate and Gemini, and are rented through Google Cloud.
Phones and edge devices are where ASICs are most numerous and least visible. An NPU in a phone exists because a general-purpose core cannot run face unlock, computational photography or on-device speech within a battery's power envelope at all — this is not a cost optimisation but the difference between a feature existing and not existing. That constraint is the whole reason edge AI is an ASIC story rather than a GPU story.
Fixed-function blocks inside chips you already own are the largest category by unit volume and rarely called ASICs at all: video encode/decode engines, image signal processors, cellular modem basebands, cryptographic accelerators. Each is a hard-wired circuit for one standardised algorithm, sitting on the same die as a general-purpose CPU. They are the proof that the ASIC bet pays off beautifully when the algorithm is genuinely frozen by a published standard.
Hyperscaler inference silicon — AWS Inferentia and Trainium, Microsoft Maia, Meta MTIA — exists for a reason that is only partly performance. A company spending billions on accelerators from a single supplier has a negotiating position problem as much as an efficiency problem, and a credible in-house alternative changes both. That motive is rarely stated in the press release and frequently the dominant one.
Key Concepts
A few pieces of vocabulary make the rest of the literature readable. NRE is the up-front, non-recurring engineering cost of the design; tapeout is the moment the design is sent to the foundry and stops being changeable; a mask set is the collection of photomasks used to pattern each layer, priced at $10-20 million at 3nm; a respin is what a post-tapeout bug forces — new masks and another fab cycle. PPA (power, performance, area) is how any design trade-off is described.
The one concept most often missed is that an ASIC's compute advantage does not automatically become a system advantage. Large-model inference is usually limited by memory bandwidth rather than arithmetic, so a chip with ten times the multiply throughput and the same memory system delivers nowhere near ten times the tokens. This is the memory wall, and it is why serious AI accelerators spend most of their engineering budget on the memory hierarchy and interconnect rather than the multipliers — and why inference optimisation at the software level often beats new silicon on cost per token.
Challenges
The freeze problem is the central one, and it is unforgiving. Specifying silicon for a workload two or three years out is a forecast, and the field has repeatedly proved the forecast hard. Transformers were published in mid-2017; chips specified in 2016 around convolutional workloads reached volume production into a world that had moved on, and were excellent at the wrong thing. Intel halted development of the Nervana NNP-T in February 2020, two months after paying $2 billion for Habana Labs. Tesla abruptly shut down Dojo in August 2025, with Musk describing the follow-on D2 as "an evolutionary dead end", weeks after signing a $16.5 billion deal with Samsung for a different chip. Neither failure was a manufacturing failure. Both were bets on a workload that resolved badly.
The software ecosystem is a larger barrier than the silicon. A chip is only as useful as the compiler, kernel library and framework integration around it, and CUDA has had nearly two decades of accumulated investment. New silicon must be fast enough to justify porting work that is annoying, ongoing and invisible to end users. Several technically strong accelerators have failed here rather than in the fab.
Verification cost scales worse than transistor count. Because a post-tapeout bug means a respin, verification typically consumes more effort than design, and it is the part of the schedule that resists being compressed by spending money.
Capacity is contested and geography is a risk. Leading-edge capacity, EUV lithography and advanced packaging capacity are concentrated in very few places, so allocation is negotiated years ahead and a design can be complete while its manufacturing slot is not. Export controls and trade policy now sit directly on top of that dependency.
Yield punishes exactly the chips AI wants. Defect density is roughly constant per unit area, so the probability a die is defect-free falls as its area grows — and AI accelerators want large dies. This is a first-order reason the industry moved to chiplets rather than a stylistic preference.
Future Trends
Chiplets decouple the NRE bet from the whole chip. Splitting a design into smaller dies connected through advanced packaging means a compute die can be revised without redoing the I/O and memory dies, each die is small enough to yield well, and different parts can sit on different process nodes. Economically this is the most important shift on this page: it lowers the break-even volume that the entire argument above turns on, and pulls custom silicon within reach of organisations for whom a monolithic design never penciled out.
Programmable-within-a-domain keeps winning over fixed-function. Every generation of AI accelerator has kept more flexibility than pure efficiency would suggest, because the freeze-point risk is priced in. Expect that to persist rather than reverse: the efficiency left on the table is cheaper than the risk of being wrong about the architecture.
Inference and training silicon continue to diverge. They have genuinely different bottlenecks — training is throughput-bound and communication-heavy across distributed clusters, while serving is latency-sensitive and memory-bandwidth-bound. Inference is also the far larger and more stable market by volume, which makes it much the better ASIC bet.
Numeric formats keep narrowing. Each step down in precision cuts both the arithmetic energy and, more importantly, the memory traffic. Hard-wiring a low-precision format is one of the safest specialisations available, because it is a decision about how numbers are represented rather than about which model architecture will win.
Watch whether the transformer stays still. The entire current wave of AI silicon is a bet that dense attention and feed-forward layers remain the dominant shape of the workload. If that holds, specialisation deepens and the economics improve. If a genuinely different architecture displaces it, this decade's accelerators will be studied the way the convolution-era chips are now — as excellent engineering aimed at the wrong target.