I saw a term describing the feature detectors, i.e. shift invariant. What is that mean?
Paper: 1989 Generalization and Network Design Strategies
For CNNs, I think it means the invariance to small* displacements of the input image. For example in the character recognition task, if you train the system by shifting (i.e. sliding the images to left/right and up/down) a little bit, you learn a more generalizable detector, that works under difficult conditions, i.e. when the character is not perfectly aligned to the center of the image. Similar precautions are also taken for rotation, scale, etc.
$^*$ I Googled to be sure about "small" and saw a similar discussion here, it made me realize that CNNs can be resistant to big displacements too, since the pooling process summarizes the local features in a smaller vector (that is representing the whole), it doesn't matter where you see the objects in the image.
Shift-invariance: this means that if we shift the input in time (or shift the entries in a vector) then the output is shifted by the same amount