Convolutional Neural Network (CNN): Letting the Same Local Detector Scan Across Space

The core problem that a convolutional layer solves is enabling the same local pattern detector to work repeatedly at different spatial positions in the input. The input is a feature map X, and the output is a feature map Y. For an output position (i,j) and output channel o, the computation is:

Y[i,j,o] = Σᵤ,ᵥ,𝚌 K[u,v,c,o] × X[i+u,j+v,c] + b[o]

Here, i and j specify the spatial position in the output, and o specifies the output channel; u and v enumerate positions within the convolution kernel window, and c enumerates input channels. In the computation, first extract the local window from X based on the current position, then multiply each element in the window by the corresponding weight in kernel K element-wise, and finally sum across window positions and all input channels, adding the bias b[o] for that output channel. Therefore, each output value is a signed response indicating how well the current local window matches a particular kernel.

The key mechanism is weight sharing: the same set of weights K does not change with i and j but is reused at all spatial positions; it distinguishes only input channel c and output channel o. Local connectivity avoids connecting every output to the entire input, and weight sharing avoids learning a separate set of parameters for each position. Thus, when the same local pattern moves from one place in the image to another, the same kernel still detects it, and the response location moves accordingly.

Consider a 3×3 grayscale image X = [[1,2,0],[0,3,1],[2,1,0]] and a 2×2 kernel K = [[1,0],[0,−1]], using a stride of 1 and no padding. The kernel successively covers each 2×2 local window in the input; at each position, it performs the same element-wise multiplication and summation. The 1 in the kernel emphasizes the upper-left corner of the window, the −1 suppresses the lower-right corner, and the two zeros indicate that the other two positions do not directly contribute. Therefore, the sign of the output indicates the response direction of that local structure relative to the kernel, and the absolute value reflects the response strength; however, one cannot assert from the sign alone that it corresponds to any object with a fixed name.

Many deep learning libraries actually perform cross-correlation without flipping the kernel, but by convention it is still called convolution. When understanding convolutional layers, what matters is the three things: local windows, element-wise multiply-add, and spatial sharing, not whether the kernel is flipped before computation.

When using a 2×2 kernel with stride 1 and no padding on a 3×3 input, the kernel can stop at two positions horizontally and two positions vertically, so four local windows are formed in total and the output is a 2×2 feature map. The kernel K = [[1,0],[0,−1]] compares only the top-left and bottom-right corners of each window: the top-left corner is multiplied by 1, the bottom-right corner by −1, and the other two positions by 0, so every inner product can be written as "top-left value − bottom-right value".

The calculations for the four windows, in order, are:

Arranging these results by the spatial positions of the windows gives:

Y = [[1−3, 2−1],[0−1, 3−0]] = [[−2,1],[−1,3]]

This feature map preserves the distribution of the kernel's responses at each position. A positive value means the top-left corner of the window is brighter than the bottom-right corner, a negative value means the bottom-right corner is brighter than the top-left corner, and a larger absolute value indicates a more obvious difference between the two. For example, the bottom-right output 3 comes from 3 − 0, indicating a strong positive brightness difference along this diagonal in that window; the top-left output −2 comes from 1 − 3, indicating the opposite difference direction.

These values describe only the relationship between the local input structure and the current kernel; they cannot be directly interpreted as "having found some object." In actual training, many different kernels are learned according to the task loss, and their responses are then combined through nonlinear transformations and subsequent layers to gradually form more complex representations.

The output size of a convolutional layer is determined jointly by the input length, kernel width, padding, stride, and dilation. Along any spatial axis, the output length is:

out = ⌊(in + 2p − d(k−1) − 1) ÷ s⌋ + 1

Here, in is the input length, out is the output length, k is the kernel width, p is the amount of padding on each side, s is the stride, and d is the dilation rate. d(k−1)+1 is the effective coverage width of the dilated kernel; after padding both sides by p, the available length becomes in+2p; stride s determines the distance between starting positions of adjacent windows. The floor operation indicates that windows that do not fit at the end are not counted.

For example, applying a convolution with k=3, p=1, s=2, d=1 to a 32×32 input gives the following length along each spatial axis:

out = ⌊(32 + 2 − 2 − 1) ÷ 2⌋ + 1 = 16

Therefore the output spatial size is 16×16. This calculation must be matched exactly when designing network structures, because operations such as residual addition require the participating tensors to have the same shape; if any spatial axis differs by one element, direct element-wise addition is impossible.

Padding p adds extra positions at the input boundary; it can preserve more edge information and is often used to control the output size, but padding values introduce artificial boundaries that are not present in the original input. What is called same padding usually means keeping the output size unchanged when stride s=1; when the kernel width is even or s>1, the padding required on the left and right sides may be asymmetric, and the specific allocation rules may differ across frameworks, so you cannot infer the amount of padding on each side solely from "same".

Stride s controls the distance the window moves each time. Increasing the stride shrinks the output, implementing downsampling, but it may also cause aliasing or lose small objects. Dilation rate d widens the sampling interval between kernel elements, expanding the coverage range without directly increasing the number of kernel parameters; the cost is that sampling points may form a sparse grid, leading to a grid effect. The three respectively control the boundary, sampling interval, and intra-kernel spacing, but ultimately they all determine the output geometry by changing where effective windows land.

The number of parameters in a convolutional layer is determined by the kernel size and the number of input and output channels, not by the spatial area of the feature map. One output channel requires a set of kernels covering all input channels, so the number of kernel weights is kₕ×k𝓌×Cᵢₙ; if each output channel also has a bias, the total number of parameters is:

Parameter count = (kₕ×k𝓌×Cᵢₙ + 1) × Cₒᵤₜ

where kₕ and k𝓌 are the height and width of the kernel, Cᵢₙ is the number of input channels, and Cₒᵤₜ is the number of output channels. For example, a 3×3 convolution that transforms 64 input channels into 128 output channels:

(3×3×64 + 1) × 128 = 73,856

The 3×3×64 in parentheses is all of the kernel weights required for one output channel, the extra 1 is the bias for that output channel, and we multiply by 128 because we need to produce 128 different output channels.

These 73,856 parameters are shared across all spatial positions. Whether the input feature map is 16×16 or 256×256, the number of kernel parameters remains unchanged. However, a larger feature map contains more output positions, so the same set of parameters must perform multiply-add operations on more windows; therefore, the computational cost grows with the output spatial area. An unchanged parameter count does not mean the runtime cost is unchanged.

Different convolution forms change the organization of spatial mixing and channel mixing. A 1×1 convolution has a window with only one spatial position, so it does not mix adjacent positions; it only combines the input channels at each position. Depthwise convolution, by contrast, gives each input channel its own spatial kernel, and then uses pointwise convolution to mix channels. Such decomposition can often significantly reduce computational cost, but it reduces channel interaction during the spatial convolution stage; actual speed is also affected by hardware execution efficiency, so it cannot be judged solely by theoretical computational cost.

A shared convolution kernel performs the same kind of local detection at every spatial position, so after a pattern in the input shifts one cell to the right, the kernel's strong response to that pattern should also shift one cell to the right. When the input is translated, the position of the output feature translates in the same way; this correspondence is called translation equivariance.

Figure 1 illustrates this causal chain: the same convolution kernel first scans the original pattern and produces a response at the pattern's location; after the pattern moves, the convolution kernel itself is unchanged and still scans the entire input by the same rule, so it produces a corresponding response at the new pattern location. Weight sharing ensures that the detection rule does not change because of spatial position, while the local window lets the response location follow the detected local structure as it moves.

Translation equivariance describes that “wherever the input moves, the feature also moves there”; it is not equivalent to classification results being completely invariant to translation. Classification invariance requires that after the input moves, the final class output remains unchanged, which typically also requires global pooling, data augmentation, or other aggregation mechanisms to integrate position-shifted features into a position-insensitive result. Therefore, the convolution's shared kernel provides equivariant responses, but alone it cannot guarantee that the whole model obtains translation invariance.

The receptive field indicates how large an area of the original input a unit in a given layer can theoretically be influenced by. It is not simply equal to the current convolution kernel size, because after layers are stacked, a kernel position in an upper layer may already correspond to a region in the original input. When computing it, you need to simultaneously track the receptive field side length r and the jump distance j after adjacent units are mapped back to the original input:

rₗ = rₗ₋₁ + (kₗ−1) × jₗ₋₁ jₗ = jₗ₋₁ × sₗ

where l is the layer number, rₗ is the theoretical receptive field side length of a unit in layer l on the original input, jₗ is the distance between adjacent unit centers in layer l when mapped back to the original input, kₗ is the kernel width of that layer, and sₗ is the stride of that layer. The input layer starts with r₀=1, j₀=1, meaning that an input unit corresponds only to itself and the distance between adjacent input units is 1.

Three layers of 3×3 convolutions, all with stride 1, can be computed layer by layer as:

The first layer expands the receptive field from 1 to 3; the second layer adds two more positions at the original jump distance of 1, expanding it to 5; the third layer similarly expands it to 7. Therefore, the theoretical receptive field of a unit in the third layer is 7×7.

The stride changes how quickly subsequent layers expand the receptive field. If the stride of the second layer is 2, then the jump distance after the second layer becomes larger, and adjacent positions of the upper-layer convolution kernel also span a greater distance when mapped back to the original input; thereafter, each additional kernel position covers a more distant input region.

The theoretical receptive field only indicates the maximum range that may be covered, not that every position within the range contributes equally to the output; gradients tend to be more concentrated in the central region. For tasks such as detection and segmentation that require both deep semantics and precise spatial detail, multi-scale features are usually needed to connect high-resolution details with deep representations.

Strided convolution or pooling reduces the number of sampling points in a feature map, which is equivalent to lowering the spatial sampling rate. If the input contains details that change too rapidly relative to the new sampling rate, these high-frequency structures cannot be represented correctly and will fold into lower-frequency artifacts; this phenomenon is called aliasing. When a checkerboard texture is reduced, stripes or block-like patterns that do not exist in the original image appear, which is the result of high-frequency details being misinterpreted.

The new Nyquist frequency is the highest spatial frequency that can be represented unambiguously after downsampling. If texture above this limit is not attenuated before downsampling, the discretely sampled values may be identical to those of another, slower-varying pattern, so the model sees a fabricated low-frequency structure rather than the original detail. The causal chain is: lowering the sampling rate → the highest representable frequency decreases → excessively high frequencies fold → artifacts appear in the output.

Max pooling keeps the maximum response in each local region, which can make small positional changes less likely to change the result, thereby providing some local stability; but it may also retain accidental noise peaks. Averaging or low-pass processing first suppresses components that change too quickly and then performs downsampling, usually making it more resistant to aliasing. Filtering here is not about recovering information out of thin air; rather, before reducing the sampling points, it actively discards the high-frequency components that the new sampling rate cannot reliably represent.

Downsampling may also directly erase small objects. If an object is smaller than one downsampling unit, it may have no chance to form a stable response at the retained sampling positions and thus disappear completely. Therefore, a smaller feature map is not compression without cost: while it saves space and computation, it also changes the range of details that can be preserved.

When choosing the downsampling method and frequency, you need to conduct slice tests in combination with the real variations in the task, including object size, rotation, scaling, boundary position, and compression noise. Only by examining these conditions separately can you determine whether the observed performance comes from effective scale compression or is masked by aliasing and detail loss.

A CNN is not naturally insensitive to all geometric transformations. It is necessary to distinguish between "equivariance," in which features move with the input, and "invariance," in which the output remains unchanged after a transformation, and also to distinguish between properties provided by the network architecture and behaviors taught by training data.

Ordinary convolution shares the same kernel at different spatial positions, so when a local pattern moves, the feature response usually moves with it, approximating translation equivariance. However, this relationship does not hold unconditionally: when the pattern moves near the boundary it encounters different padding content, and stride sampling can break the one-to-one correspondence between window positions before and after the movement.

Translation invariance for classification is a different requirement. It expects the final class to remain unchanged after the input position changes, rather than the feature map moving along. To obtain a stable class from features that move with position, an aggregation mechanism and corresponding training are also needed. Rotation and scale changes are even less likely to automatically follow the sharing rules of ordinary convolution, because after rotation the local arrangement has changed or after scaling the spatial extent has changed, and the original kernel may not produce the same response.

The role of data augmentation is to explicitly tell the model, through training examples, which input transformations should not change the label. It can help the model learn desired invariances, but only if the transformation truly preserves the task semantics. If you rotate the digit 6 by 180° and still force the original label, you may treat a sample whose semantics have changed as an equivalent sample, injecting an incorrect invariance into the model. Therefore, the augmentation strategy must be designed according to whether the label still holds under the corresponding transformation.

After the emergence of attention mechanisms, convolution remains widely used because CNN and ViT process local patterns and global content relationships along different paths. When choosing a backbone network, one cannot compare only model categories; the final conclusion should come from empirical measurements consistent with the target data, resolution, and operating environment.

CNN uses local connections and weight sharing to let the same set of convolutional kernels repeatedly detect nearby patterns at different spatial positions. Stacking multiple layers gradually expands the receptive field, so global relationships require multi-layer local composition. ViT, by contrast, divides the image into patch tokens and uses self-attention to let tokens directly aggregate content from other positions, thereby establishing global content relationships more directly.

This comparison table describes mechanistic differences and does not declare in advance which architecture is necessarily better. Actual results also depend on specific training settings and the operating environment, so you should run empirical tests under the same target data, input resolution, and evaluation conditions before deciding whether to adopt CNN or ViT.

The complete mechanism from local input to task output in a CNN can be understood along a verifiable causal chain. The input is first divided into sliding local windows, and the same set of convolution kernels is reused across different spatial positions. Each window performs an inner product with the convolution kernel across channels, yielding responses at the corresponding position and output channel; these responses are arranged by spatial position into a feature map.

After the feature map undergoes nonlinear transformations and is stacked layer by layer, units in higher layers can combine more complex local responses, and their receptive fields gradually expand. The network thus transitions from input relationships over smaller ranges to representations covering larger regions. Downsampling further reduces spatial size and forms a multi-scale hierarchy, but it may also lose detail, so it is both a computational and hierarchical organization tool and an information bottleneck that needs to be verified.

After obtaining multi-layer features, the task head decides how to use them according to the objective. Tasks that require holistic judgment aggregate spatial features to form the final output; tasks that need to preserve location must restore or combine spatial information to produce predictions at corresponding positions. Thus the whole chain can be written as:

local windows → shared convolution kernels → channel inner product → feature map → nonlinear stacking and receptive field expansion → downsampling to form a multi-scale hierarchy → task head aggregation or restoration of spatial output

Each step introduces behaviors that can be checked individually. Whether shared kernels produce the expected shifted response must be checked by observing feature positions under input transformations; whether the receptive field and downsampling retain the information required by the task must be checked at the target scale; whether boundary padding changes the result must be checked by comparing the target at the center and at the edge. Final acceptance should be sliced by transformation, scale, and boundary position, rather than based on a single overall metric. Only then can structural design, internal responses, and actual task results be connected.