Evolving Neural Networks for Online Reinforcement Learning

Jan Hendrik Metzen, Mark Edgington, Yohannes Kassahun, Frank Kirchner

In: G. Rudolph , T. Jansen , S.M. Lucas , et al. (Hrsg.). Proceedings of the 10th International Conference on Parallel Problem Solving From Nature. International Conference on Parallel Problem Solving From Nature (PPSN-2008) September 13-17 Dortmund Germany Seiten 518-527 Lecture Notes in Computer Science (LNCS) 5199 Springer 2008.


For many complex Reinforcement Learning problems with large and continuous state spaces, neuroevolution (the evolution of artificial neural networks) has achieved promising results. This is especially true when there is noise in sensor and/or actuator signals. These results have mainly been obtained in offline learning settings, where the training and evaluation phase of the system are separated. In contrast, in online Reinforcement Learning tasks where the actual performance of the systems during its learning phase matters, the results of neuroevolution are significantly impaired by its purely exploratory nature, meaning that it does not use (i. e. exploit) its knowledge of the performance of single individuals in order to improve its performance during learning. In this paper we describe modifications which significantly improve the online performance of the neuroevolutionary method Evolutionary Acquisition of Neural Topologies (EANT) and discuss the results obtained on two benchmark problems.

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence