Fórum Pirátské strany

Not very often a new ground breaking Neural Network (NN) architecture is published. A few days ago Geoffrey Hinton published the paper "Dynamic Routing Between Capsules" and the entire Deep Learning community is going crazy. The article describes new architectural models called "Capsules". A NN based on them shows a significant increase in performance in some cases, compared to the current state-of-the-art CNNs.

Dynamic Routing Between Capsules
Sara Sabour, Nicholas Frosst, Geoffrey E Hinton
(Submitted on 26 Oct 2017)

A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation paramters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discrimininatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher level capsules whose activity vectors have a big scalar product with the prediction coming from the lower-level capsule.