What does "shift invariant" mean in convolutional neural network?

by RockTheStar   Last Updated March 06, 2018 18:19 PM

I saw a term describing the feature detectors, i.e. shift invariant. What is that mean?

Paper: 1989 Generalization and Network Design Strategies

Answers 2

For CNNs, I think it means the invariance to small* displacements of the input image. For example in the character recognition task, if you train the system by shifting (i.e. sliding the images to left/right and up/down) a little bit, you learn a more generalizable detector, that works under difficult conditions, i.e. when the character is not perfectly aligned to the center of the image. Similar precautions are also taken for rotation, scale, etc.

$^*$ I Googled to be sure about "small" and saw a similar discussion here, it made me realize that CNNs can be resistant to big displacements too, since the pooling process summarizes the local features in a smaller vector (that is representing the whole), it doesn't matter where you see the objects in the image.

December 02, 2015 21:54 PM

Shift-invariance: this means that if we shift the input in time (or shift the entries in a vector) then the output is shifted by the same amount


March 06, 2018 18:06 PM

Related Questions

Neural Style Transfer

Updated April 11, 2018 16:19 PM