Convolutional and Recurrent Networks for Structured Data

Every architecture in this topic so far has been a fully connected MLP: every neuron in a layer connects to every neuron in the previous layer, with no assumption about how the input's features relate to each other spatially or temporally. That assumption is wasteful — and sometimes actively harmful — for two extremely common data types: images, where nearby pixels are far more related than distant ones, and sequences, where an observation's meaning depends on what came before it in time. This note is an introductory survey of the two architecture families built around those priors: convolutional neural networks (CNNs) for grid data, and recurrent neural networks (RNNs) for sequences. Separate CNN and sequence topics in the Neural Networks and AI unit develop each family in operational depth.