Definition
Pooling is a downsampling operation used inside a convolutional neural network that shrinks a feature map by summarizing each small region into a single value. A 2x2 max-pooling layer, for example, slides a 2x2 window across the map and keeps only the largest number it sees in each window, throwing away the other three. The result has half the height and half the width — a quarter as many values — while the strongest activations survive.
The reason to do this is threefold, and none of it involves learning: pooling cuts the spatial resolution so the layers that follow have far less to compute, it keeps the most salient signal in each neighborhood rather than the exact pixel it came from, and that "roughly where, not exactly where" behavior gives the network a small amount of translation invariance. A pooling layer has zero learnable parameters — it applies a fixed rule, so unlike a convolution there is no weight to train.
How It Works
A pooling layer is defined by two numbers: the size of its window (usually 2x2) and its stride (usually 2, meaning the window jumps two pixels at a time so windows never overlap). It slides that window across the input feature map and replaces each window with one summary value. With a 2x2 window and stride 2, an input of width W produces an output of width floor((W - 2) / 2) + 1. Plug in the standard CNN input size and a 224x224 feature map becomes exactly 112x112 — each spatial dimension halves, so the area is quartered from 50,176 down to 12,544 values.
The arithmetic is worth doing by hand once, because it is the whole operation. Take this 4x4 feature map:
1 3 | 2 4 5 6 | 1 2 ---------+--------- 7 2 | 8 0 1 3 | 4 9
The dividers split it into four 2x2 windows. Max pooling reports the maximum of each window — max(1,3,5,6)=6, max(2,4,1,2)=4, max(7,2,1,3)=7, max(8,0,4,9)=9 — giving a 2x2 output:
6 4 7 9
Average pooling over the same four windows reports the mean of each instead — (1+3+5+6)/4 = 3.75, and so on — giving:
3.75 2.25 3.25 5.25
That is the entire mechanism. Sixteen numbers became four, no weights were consulted, and the largest response in each neighborhood (or its average) is what carried forward.
During training, gradients still have to flow back through this layer even though it learns nothing. For max pooling the answer is routing: the gradient for a window is passed only to the position that held the maximum on the forward pass, and the other three positions receive zero, because only the winning input actually influenced the output. For average pooling the gradient is split evenly across all cells in the window. This is why pooling is not "non-differentiable" in any way that stops training — it simply has a fixed, weightless backward rule.
Types
Three pooling operations account for almost everything you will meet in real architectures. They differ only in the summary they compute and the region they compute it over.
Max pooling keeps the single largest value in each window. Because an edge or texture detector fires with a high activation exactly where its pattern is present, keeping the maximum preserves whether a feature appeared in the neighborhood while discarding precisely where. It is the default inside classic CNNs — AlexNet and the VGG networks stack 2x2 max-pooling layers between their convolution blocks — and it tends to keep sharp features crisp.
Average pooling takes the mean of the window instead of the max. It produces a smoother output that reflects the whole region rather than its brightest point, which makes it a poorer edge detector but a gentler downsampler; it shows up where suppressing spiky, high-variance responses is the goal.
Global average pooling is the special case that collapses an entire feature map to one number by averaging all of its values — the 4x4 grid above becomes the single scalar 3.625. A convolutional stack that ends with, say, 512 feature maps of size 7x7 becomes a 512-element vector, one value per map, which then feeds the classifier directly. Introduced in the Network In Network paper (Lin, Chen, and Yan, 2013), global average pooling replaced the large fully connected layers that used to sit at the end of a CNN. Its two cited advantages are exactly the pooling story in miniature: "there is no parameter to optimize in the global average pooling thus overfitting is avoided at this layer," and it is "more native to the convolution structure by enforcing correspondences between feature maps and categories." ResNet, GoogLeNet and most modern classification backbones end this way.
Real-World Applications
Pooling is not an application in itself; it is a layer inside image models, and its concrete uses are the well-known architectures that rely on it. In AlexNet (2012) and the VGG family, a 2x2 max-pooling layer follows each convolution block, so a 224x224 input is halved to 112x112, then 56x56, 28x28, 14x14 and 7x7 as it descends the network — five halvings that shrink the spatial grid by 32x while the channel count grows. That progressive shrinking is what lets a deep CNN look at ever-larger parts of the image with a fixed-size filter.
Global average pooling is the piece that survives most strongly into current practice: ResNet and GoogLeNet end their convolutional stacks with it to produce the fixed-length vector a softmax classifier needs, regardless of the exact input resolution. Because that vector has one entry per feature map, it also underpins class activation mapping, a common technique for showing which region of an image drove a classification — a diagnostic that fully connected classifier heads cannot provide as directly. Outside classification, pooling appears in the encoder half of segmentation and detection networks wherever a feature map needs to be reduced before it is processed or upsampled again.
Challenges
The defining property of pooling is also its main cost: it deliberately throws information away and cannot get it back. Once a 2x2 max-pool has kept the value 6 and discarded the 1, 3 and 5 around it, the exact location of that response is gone. For classification, where the question is "is a cat present?", that loss is a feature. For tasks that need precise spatial detail — semantic segmentation, keypoint localization — it is a problem, which is why segmentation architectures pair each pooling step with a matching upsampling step and often store the pooling indices so the decoder can put activations back where they came from.
The translation invariance pooling provides is also smaller and more local than it is often described. A single 2x2 pool tolerates a one-pixel shift; genuine robustness to large translations comes from stacking many of them, not from any one layer, and pooling gives nothing toward invariance to rotation or scale. And because a pooling layer is a fixed rule, it cannot adapt: it applies the same maximum-or-mean everywhere, whether or not that is the right summary for a given region.
Future Trends
The clearest trend is that a dedicated pooling layer is no longer considered necessary for downsampling. In Striving for Simplicity: The All Convolutional Net (Springenberg, Dosovitskiy, Brox, and Riedmiller, 2014), the authors report that "max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks," and that "when pooling is replaced by an additional convolution layer with stride r=2 performance stabilizes and even improves on the base model." A strided convolution downsamples the same way pooling does — it steps its window two pixels at a time — but unlike pooling it learns how to summarize each region instead of applying a fixed max. Many modern CNNs, including much of the ResNet lineage, downsample this way and use no max-pooling layer at all.
Vision transformers push further in the same direction: they do not pool spatially the CNN way. A ViT splits the image into a grid of patches, treats each patch as a token, and reduces spatial resolution through patch merging or by attending over tokens rather than by taking a maximum over a 2x2 window. The one form of pooling that has proven durable across all of these designs is global average pooling — parameter-free, resolution-agnostic, and still the standard bridge from a stack of feature maps (or tokens) to a classifier.
Code Example
The forward pass is short enough to write from scratch. This slides a non-overlapping 2x2 window over the 4x4 map from the worked example above and applies both summaries. Note that there are no weights anywhere in the function — the layer has nothing to learn.
import numpy as np
fmap = np.array([
[1, 3, 2, 4],
[5, 6, 1, 2],
[7, 2, 8, 0],
[1, 3, 4, 9],
])
def pool_2x2(x, reduce):
H, W = x.shape
out = np.zeros((H // 2, W // 2))
for i in range(0, H, 2): # stride 2: windows never overlap
for j in range(0, W, 2):
window = x[i:i + 2, j:j + 2]
out[i // 2, j // 2] = reduce(window)
return out
print("max pooling:\n", pool_2x2(fmap, np.max))
print("average pooling:\n", pool_2x2(fmap, np.mean))
print("global average pooling:", fmap.mean())
Running this prints the values derived by hand earlier:
max pooling:
[[6. 4.]
[7. 9.]]
average pooling:
[[3.75 2.25]
[3.25 5.25]]
global average pooling: 3.625
In practice you would not write the loop — torch.nn.MaxPool2d(kernel_size=2, stride=2) and torch.nn.AdaptiveAvgPool2d((1, 1)) (global average pooling) do the same thing on batched, multi-channel tensors — but the framework layers hold exactly zero parameters, just like this one.