Learning in Compressed Space

Alexander Fabisch; Yohannes Kassahun; Hendrik Wöhrle; Frank Kirchner

In: Neural Networks, Vol. 42, Pages 83-93, Elsevier, 2013.


We examine two methods which are used to deal with complex machine learning problems: compressed sensing and model compression. We discuss both methods in the context of feed-forward artificial neural networks andmdevelop the ackpropagation method in compressed parameter space. We further show that compressing the weights of a layer of a multilayer perceptron is equivalent to compressing the input of the layer. Based on this theoretical framework, we will use orthogonal functions and especially random projections for compression and perform experiments in supervised and reinforcement learning to demonstrate that the presented methods reduce training time significantly.


Weitere Links

140611_Learning_in_Compressed_Space_Journal_Fabisch.pdf (pdf, 485 KB )

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence