Enhancing deep extreme learning machines by error backpropagation

We present an extension to a previously proposed Deep ELM architecture, and make the network end-to-end trainable using backpropagation. This significantly increases the network's performance for lower numbers of hidden units, and hence is well suited for resource constrained applications. The new architecture offers classification results of over 98% on the MNIST handwritten digits dataset for hidden layer sizes of 200. This compares favourably with previous results. We achieve the same performance of a network with a total of 12,000 hidden units, but only using 5,000 hidden units. Therefore the execution times of the original Deep ELM have been significantly reduced, whilst maintaining performance.