Evenly Cascaded Convolutional Networks

We introduce Evenly Cascaded convolutional Network (ECN), a neural network taking in-
spiration from the cascade algorithm of wavelet analysis. ECN employs two feature streams – a
low-level and high-level steam. At each layer these streams interact, such that low-level features
are modulated using advanced perspectives from the high-level stream. ECN is evenly structured
through resizing feature map dimensions by a consistent ratio, which removes the burden of ad-hoc
specification of feature map dimensions. ECN produces easily interpretable features maps, a result
whose intuition can be understood in the context of scale-space theory. We demonstrate that ECN’s
design facilitates the training process through providing easily trainable shortcuts. We report
new state-of-the-art results for small networks, without the need for additional treatment such as
pruning or compression – a consequence of ECN’s simple structure and direct training. A 6-layered
ECN design with under 500k parameters achieves 95.24% and 78.99% accuracy on CIFAR-10 and
CIFAR-100 datasets, respectively, outperforming the current state-of-the-art on small parameter
networks, and a 3 million parameter ECN produces results competitive to the state-of-the-art.

Read more here: https://arxiv.org/pdf/1807.00456