Definition
A neuron in an artificial neural network is a single arithmetic recipe: multiply each incoming number by a weight, add the results together, add one extra number called the bias, and push the total through one function. That is the entire object. Give a neuron the three inputs 2.0, -1.0 and 0.5, the three weights 0.4, 0.3 and -0.8, and a bias of 0.1, and it computes 0.8 - 0.3 - 0.4 + 0.1 = 0.2, then applies its activation function — ReLU leaves 0.2 alone — and emits 0.2. It holds no state, remembers nothing between calls, and has no sense of time.
The word sits at a particular scale, and confusing it with the two scales either side of it is the most common mistake a first tutorial produces. A dense layer with 512 inputs and 2,048 outputs has 2,048 neurons — one per output — and each of those neurons owns 512 weights plus its own single bias, which is 513 parameters. Multiply out: 2,048 × 513 = 1,050,624 parameters in the layer. A neuron is a slice of the layer's arithmetic; a parameter is a single cell inside it. When a model is advertised at 175 billion, nobody is counting neurons.
The name is a historical accident that has done real damage. It comes from a 1943 attempt to show that nerve cells could be treated as logic gates, and it stuck — so a reader arriving at the term expects a brain cell and gets a dot product. The resemblance is structural and shallow, and the rest of this page is mostly about how far it goes before it starts to mislead.
How It Works
Every neuron has a fan-in: the number of values it reads. In a dense layer that is the width of the previous layer, so a neuron in a 512-wide layer's successor reads 512 numbers and owns 512 weights, one per input. The weights are what the neuron has learned; the inputs are what it is being shown right now. Nothing distinguishes one neuron from its neighbour in the same layer except its own weights and bias — they all see the identical input vector.
The multiply-and-add step is a dot product, and it is worth seeing that it is a single number coming out of many numbers going in. The neuron has compressed everything it can see into one scalar, usually written z and called the pre-activation. Then one function is applied to z to give the activation a, which is what the next layer receives. Everything the neuron will ever contribute to the network's output passes through that one number.
Trace the same neuron twice, changing nothing but the bias:
inputs x = [ 2.0, -1.0, 0.5 ]
weights w = [ 0.4, 0.3, -0.8 ]
bias +0.1 -> z = (2.0)(0.4) + (-1.0)(0.3) + (0.5)(-0.8) + 0.1 = +0.2 ReLU -> 0.2000
bias -0.5 -> z = (2.0)(0.4) + (-1.0)(0.3) + (0.5)(-0.8) - 0.5 = -0.4 ReLU -> 0.0000
A shift of 0.6 in one parameter takes the neuron from speaking to silent. Under ReLU it now emits exactly zero for this input and contributes nothing at all downstream; under a sigmoid the same change moves its output from 0.5498 to 0.4013, quieter but still audible. This is why the bias is not decorative bookkeeping. It sets how much evidence the neuron requires before it says anything, independently of what the evidence is.
One neuron is one straight cut
The set of inputs for which z is exactly zero is a flat boundary through the input space — a line in two dimensions, a plane in three, a hyperplane in 512. On one side the neuron's pre-activation is positive, on the other negative. The weights set the boundary's orientation and the bias slides it back and forth without rotating it, which is exactly what the trace above shows: the same plane, moved 0.6 further out, with our input point now on the far side of it.
That geometry is the whole limit of a single unit, and XOR is the standard demonstration. Suppose one threshold neuron could compute it. Feeding (0,0) must give a non-positive z, so b ≤ 0. Feeding (1,0) must give a positive one, so w₁ + b > 0; likewise w₂ + b > 0. Add those two inequalities: w₁ + w₂ + 2b > 0, and since b ≤ 0 this forces w₁ + w₂ + b > 0. But (1,1) must come out non-positive, meaning w₁ + w₂ + b ≤ 0. The two conclusions contradict each other, so no such neuron exists — for any weights, any bias, any threshold. The ## Code Example below confirms it the blunt way, by trying 68,921 settings and finding that the best of them scores 3 out of 4. Everything a network does beyond drawing one straight cut comes from stacking these units, which is the subject of the neural network page rather than this one.
Why it is called a neuron
Warren McCulloch and Walter Pitts published "A logical calculus of the ideas immanent in nervous activity" in the Bulletin of Mathematical Biophysics in December 1943. Their opening premise is the source of the whole analogy: "Because of the 'all-or-none' character of nervous activity, neural events and the relations among them can be treated by means of propositional logic." Their unit summed its inputs and fired if a fixed threshold was crossed. It had no weights to learn and no learning rule — it was an argument that nervous systems can be described logically, not a proposal for building anything.
Frank Rosenblatt's perceptron, published in Psychological Review in November 1958, added the missing half: weights that adjust in response to mistakes. From there the object stopped being a model of a nerve cell and became a component. Replacing the hard threshold with a smooth function, decades later, was the change that mattered most, and it was made for a reason that has nothing to do with biology — a step function's derivative is zero everywhere it is defined, so backpropagation cannot pass through it. The modern neuron is shaped the way it is because gradient descent has to be able to differentiate it.
Is it anything like a brain cell?
In outline, and no further. Both take many inputs and produce one output, and both have something like a threshold. Past that, the two objects disagree on almost every property that matters.
A biological neuron communicates in spikes — discrete, near-identical electrical events. What varies is not the size of a spike but when it happens and how often, so timing itself carries information. An artificial neuron emits a real number, once, and the concept of "when" does not exist inside it: run the same input through it a million times and you get the same value with no history and no refractory period. A biological neuron is also not differentiable — a spike either occurs or does not — which is precisely the property the artificial version was designed to abandon.
The population figures are worth stating because they are so often quoted badly. Azevedo et al. counted 86 billion neurons in the adult male human brain, of which about 16 billion are in the cerebral cortex and 69 billion in the cerebellum; the familiar "100 billion" lies outside the range they measured, and the equally familiar claim that glia outnumber neurons ten to one is wrong too — the two counts are roughly equal. Now count the other side. Take the feed-forward hidden units of GPT-3 175B, the closest thing in a transformer to the textbook picture of a neuron: 49,152 per layer across 96 layers is 4,718,592 units, about 18,000 times fewer than a brain. That ratio is a real calculation and it means nothing, which is the point. The two things being counted are not the same kind of thing, so no ratio between them predicts anything about capability.
There is no neuron object in the code
Nothing in PyTorch, JAX or TensorFlow has a Neuron class, and no line of a training loop iterates over neurons. A layer is a matrix; a neuron is one column of it, plus one entry of the bias vector. The word survives because it names a useful slice — "this layer has 2,048 neurons" is a clearer statement of shape than "this weight matrix has 2,048 columns" — but a beginner who pictures little cells passing messages will be looking for something the code does not contain. It contains a matrix multiply.
Real-World Applications
A single artificial neuron has almost no standalone deployments, and saying so is more useful than assembling a list of things neural networks do. Image recognition, translation and fraud detection are applications of networks; attributing them to the neuron is like crediting the transistor with your spreadsheet. There are, however, three places where one unit really is the object of interest.
A single sigmoid neuron is logistic regression. This is an identity, not an analogy: one unit with a linear pre-activation and a sigmoid, trained on cross-entropy loss, has exactly the model class and exactly the loss surface of logistic regression. In PyTorch it is nn.Linear(n, 1) followed by a sigmoid. That is the one genuinely widespread deployment of a lone artificial neuron, and it is enormously widespread — everywhere a regulated setting needs a model whose coefficients can be read off and defended one at a time.
Output heads are single neurons, and the choice is a real decision. A binary classifier ends in exactly one unit whose activation is the predicted probability; a regression model ends in one unit with no activation at all, so it can emit any real number. Getting this wrong is a shipped bug rather than a subtlety: put a ReLU on a regression head and the model becomes incapable of predicting a negative value, silently, with a loss curve that looks fine.
Individual neurons are the unit of analysis in interpretability research. This is where single units are studied in earnest, and the findings are specific. Olah et al.'s circuits work on InceptionV1 catalogues named units — curve detectors, a pose-invariant dog-head detector — and also unit 4e:55, which "responds to cat faces, fronts of cars, and cat legs". Whether one unit corresponds to one human-meaningful concept is an empirical question about real models, and the answer is often no.
Key Concepts
Pre-activation and activation are different numbers. z is the weighted sum plus bias; a is what comes out of the activation function. Papers and framework code distinguish them constantly — gradients are computed with respect to z, normalization layers operate on z, and "the activation of neuron 47" almost always means a. Conflating them makes half the literature unreadable.
The bias buys a threshold, and removing it costs more than it looks. Set every bias to zero and every neuron's boundary is forced through the origin, so a neuron can no longer say "fire only when this input is well above zero" — only "fire when this direction is positive at all". That is one parameter per neuron out of hundreds, and it constrains the function class much more than its share of the parameter count suggests.
Fan-in decides what a neuron costs. A neuron reading 512 inputs performs 512 multiplications and holds 513 parameters; the same neuron in a 12,288-wide model performs 12,288 and holds 12,289. Both the arithmetic and the memory scale with the width of the layer feeding it, which is why widening a network is quadratically expensive while deepening it is only linear.
Neurons are not the unit of model size. Two networks with identical neuron counts can differ by orders of magnitude in parameters, because a neuron's cost depends entirely on its fan-in. This is the same reason a convolutional network can have millions of units and few weights: the units share weights instead of owning them.
Monosemantic versus polysemantic. A monosemantic unit responds to one identifiable thing; a polysemantic unit responds to several unrelated things. Both occur in real trained models, and which one you have is not something you can tell by looking at the architecture.
Challenges
Reading a neuron's activation as a concept is usually wrong. The tempting move — find the inputs that make unit 137 fire hardest, name what they have in common, and conclude that unit 137 detects that thing — fails whenever the unit is polysemantic. Olah et al. are blunt about their cat/car unit: "this neuron isn't responding to some commonality of cars and cat faces... it's looking for the eyes and whiskers of a cat, for furry legs, and for shiny fronts of cars — not some subtle shared feature." They also give the reason it matters for anyone trying to trace a computation: "if one neuron with five different meanings connects to another neuron with five different meanings, that's effectively 25 connections that can't be considered individually." An explainability story built on single-unit labels will be confidently wrong about a model that works fine.
Dead ReLU units are permanent, not slow. If a unit's pre-activation is negative for every example in the training set, ReLU outputs zero for all of them — and ReLU's derivative on that side is exactly zero, so backpropagation sends the unit no gradient at all and no update can revive it. The capacity is gone for the remainder of training. It is a real failure with real causes (a learning rate large enough to drive the bias sharply negative in one step is the classic one) and real fixes: lower the learning rate, initialise more carefully, or use a leaky variant that keeps a small slope on the negative side. Diagnose it by logging the fraction of units in each layer that are zero across a whole validation batch; a layer sitting at 90% zeros for every input is not sparse, it is dead.
"More neurons" is a poor thing to tune. Because a neuron's parameter cost is set by its fan-in, adding units to a wide layer is far more expensive than adding the same number to a narrow one, and neither is comparable to adding a layer. Anyone reasoning about capacity in units of neurons is working in a currency with a floating exchange rate.
The analogy imports intuitions that are false. Because the word says "neuron", readers expect memory, timing, spontaneous activity and energy efficiency, and a standard artificial neuron has none of them. The most damaging import is the idea that a bigger neuron count means a more brain-like system — a claim that has no content, since the two counts measure incomparable objects.
Future Trends
The unit of analysis is shifting from neurons to features. If polysemanticity comes from a network packing more distinct features than it has units, then the neuron is the wrong basis to interpret in — an arbitrary axis rather than a meaningful one. Olah et al. framed the goal as resolving "polysemantic neurons, perhaps by 'unfolding' a network to turn polysemantic neurons into pure features". Sparse dictionary methods pursuing exactly that have become a mainstream interpretability tool, and their consequence for this term is direct: the neuron may keep its place as a unit of computation while losing it as a unit of meaning.
Spiking units are a genuine alternative, held back by differentiability. A spiking neuron emits discrete events in time, which restores the property that makes biological neurons efficient — a unit that is silent costs nothing. The obstacle is the one that shaped the standard neuron in the first place: a spike is a step, and a step has no usable derivative, so spiking networks are trained with surrogate gradients that substitute a smooth function during the backward pass only. Neuromorphic hardware exists to run these, and the open question is not whether spikes are more efficient but whether the training story ever becomes as reliable as backpropagation through smooth activations.
Neurons are the granularity of structured pruning. Compressing a model by deleting individual weights produces a sparse matrix that most hardware runs no faster than a dense one. Deleting an entire neuron — a whole column, its bias, and the corresponding row of the next layer — produces a smaller dense matrix that every accelerator runs faster immediately. That makes the neuron, not the parameter, the practical unit of compression, and it is a rare case where the word earns its place in an engineering decision rather than a diagram.
Code Example
The whole neuron is three lines. Running it makes the bias flip and the XOR limit concrete rather than asserted.
import math
x = [2.0, -1.0, 0.5] # three numbers arriving from the previous layer
w = [0.4, 0.3, -0.8] # this neuron's three weights
relu = lambda z: max(0.0, z)
sigmoid = lambda z: 1.0 / (1.0 + math.exp(-z))
def neuron(x, w, b):
"""The entire artificial neuron: a dot product, a bias, and one function."""
return sum(xi * wi for xi, wi in zip(x, w)) + b
for b in (0.1, -0.5):
z = neuron(x, w, b)
print(f"bias {b:+.1f} -> z = {z:+.4f} relu(z) = {relu(z):.4f} sigmoid(z) = {sigmoid(z):.4f}")
# Can ONE threshold neuron compute XOR? Try 68,921 settings of (w1, w2, b).
xor = [((0, 0), 0), ((0, 1), 1), ((1, 0), 1), ((1, 1), 0)]
grid = [i / 4 for i in range(-20, 21)] # -5.00 to +5.00 in steps of 0.25
best, solutions = 0, 0
for w1 in grid:
for w2 in grid:
for b in grid:
score = sum((w1 * a + w2 * c + b > 0) == bool(y) for (a, c), y in xor)
best = max(best, score)
solutions += score == 4
print(f"settings tried: {len(grid) ** 3} best score: {best}/4 settings solving XOR: {solutions}")
Output:
bias +0.1 -> z = +0.2000 relu(z) = 0.2000 sigmoid(z) = 0.5498
bias -0.5 -> z = -0.4000 relu(z) = 0.0000 sigmoid(z) = 0.4013
settings tried: 68921 best score: 3/4 settings solving XOR: 0
The first two lines are one neuron falling silent because a single parameter moved by 0.6. The third is the reason there is more than one neuron in anything: 68,921 candidate units, and not one of them can separate four points on the corners of a square.