Advances in Artificial Neural Networks

Seminar an der Universität des Saarlandes, Fachrichtung Informatik, LVst 102658

Topics which have been CURRENTLY ASSIGNED to participants for presentation at the seminar are MARKED IN RED (current status: 10/13 slots are assigned).

On April 6, 2017 (registration deadline) all topics were assigned.

Please note that there are more topics (19) listed than seminar slots available (13).

Please also check the seminar schedule.

All papers and selected references on deep learning listed below are available online, and from the CS library of Saarland university.

Topic#

Topic

1

Multi-Layer Networks and Backpropagation

  • Introduction to ANN, multi-layer perceptrons, and backpropagation (mathematics and algorithms)
  • T. Mitchell (1997): Machine Learning. Chapter 4 (Artificial Neural Networks), McGrawHill
  • R.K. Srivastava, K. Greff, J. Schmidhuber (2015): Training Very Deep Networks. arXiv:1507.06228v2

2

Self-Organizing Maps /  Kohonen Networks

  • Introduction to SOMs; T. Kohonen, T. Honkela (2007): Kohonen network. Scholarpedia, 2(1):1568
  • O. Walter et al. (2015): Autonomous Learning of Representations. Künstliche Intelligenz, Springer
  • D. Nova and P.A. Estevez (2015): A Review of Learning Vector Quantization Classifiers. arXiv:1509.07093v1; also in: Neural Computation and Applications, 25, 2013

3

Deep Belief Networks

  • Introduction to DBNs (RBM, Auto-Encoder)
  • G.E. Hinton (2009): Deep Belief Networks. Scholarpedia, 4(5):5947; G.E. Hinton, S. Osindero, Y.W. Teh (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18; Y. Bengio (2009): Learning Deep Architectures for AI. Foundations and Trends in ML, 2
  • R.R. Salakhutdinov, G.E. Hinton (2007): Semantic Hashing. Proc. of SIGIR Workshop on Information Retrieval and Applications of Graphical Models

4

Recurrent Neural Networks (1)

  • Introduction to RNNs and LSTM
  • Stephen Grossberg (2013): Recurrent Neural Networks. Scholarpedia, 8(2):1888
  • S. Hochreiter, J. Schmidhuber (1997): Long Short-Term Memory. Neural computation, 9(8); K. Greff et al. (2015): LTSM: A Search Space Odyssey.  arXiv:1503.04069v1

5

Recurrent Neural Networks (2)

  • Introduction to plan and goal recognition
  • W. Min et al.  (2017): Deep LSTM-based Goal Recognition Models for Open-World Digital Games. Proc. of 10th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment

6

Recursive Neural Networks

  • Introduction to Recursive NNs
  • C. Goller, A. Küchler (1996): Learning Task-Dependent Distributed Representations by Backpropagation Through Structure. IEEE Trans. on Neural Networks; also TU Muenchen AR-95-02
  • F. Bisson, H. Larochelle, F. Kabanza (2015): Using a Recursive Neural Network to Learn an Agent's Decision Model for Plan Recognition. Proc. of International Joint Conference on Artificial Intelligence

7

Convolutional Neural Networks (1)

  • Introduction to ConvNets/CNNs; Y. LeCun et al. (1998): Gradient-based learning applied to document recognition. Proc. of the IEEE
  • S. Ren et al. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv:1506.01497v3; also in: Advances in Neural Information Processing Systems

8

Convolutional Neural Networks (2)

  • P. Tang, H. Wang, S. Kwong (2017): G-MS2F: GoogLeNet Based Multi-Stage Feature Fusion of Deep CNN for Scene Recognition. Neurocomputing, 225, Elsevier
  • V. Badrinarayanan, A. Kendall, R. Cipolla (2017): SegNet: A Deep Convolutional Encoder-Decoder Architecture for Scene Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence
  • J. Long, E. Shelhamer, T. Darrell (2015): Fully Convolutional Networks for Semantic Segmentation. Proc. of IEEE Conference on Computer Vision and Pattern Recognition

9

Convolutional Neural Networks (3)

  • Multi-Task Learning
  • Y. Kao, R. He, K. Huang (2017): Deep Aesthetic Quality Assessment with Semantic Information. IEEE Transactions on Image Processing. arXiv:1604.04970v3
  • X. Liu et al. (2015): Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval. HLT-NAACL

10

Deep Reinforcement Learning (1)

  • Introduction to Deep RL
  • V. Mnih et al. (2015): Human-Level Control Through Deep Reinforcement Learning. Nature 518(7540) [Deep Q-Network @Google DeepMind]
  • B. Bakker (2007): Reinforcement Learning by Backpropagation Through an LSTM model/critic. Proc. of IEEE International Symposium of Approximate Dynamic Programming and Reinforcement Learning

11

Deep Reinforcement Learning (2)

  • V. Mnih et al. (2016): Asynchronous Methods for Deep Reinforcement Learning. Proc. of 33rd International Conference on Machine Learning
  • M. Hausknecht, P. Stone (2015/2017): Deep Recurrent Q-learning for Partially Observable MDPs. arXiv:1507.06527

12

Deep Reinforcement Learning (3)

  • D. Silver et al. (2016): Mastering the Game of Go with Deep Neural Networks and Tree search. Nature, 529(7587)
  • T.P. Lillicrap et al. (2016): Continuous Control With Deep Reinforcement Learning. Proc. of International Conference on Learning Representations (ICLR)

13

Deep Reinforcement Learning (4)

  • H. He et al. (2016): Opponent Modeling in Deep Reinforcement Learning. Proc. of 33rd International Conference on Machine Learning,
  • J.N. Foerster et al. (2016): Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks. arXiv:1602.02672

14

Generative Adversarial Networks (1)

  • Introduction to GANs
  • I.J. Goodfellow et al. (2014): Generative Adversarial Nets. Advances in Neural Information Processing Systems; I.J. Goodfellow et al. (2015): Explaining and Harnessing Adversarial Examples. arXiv:1511.06434v2
  • I. Durugkar, I. Gemp, S. Mahadevan (2016): Generative Multi-Adversarial Networks. arXiv:1611.01673

15

Generative Adversarial Networks (2)

  • X. Huang et al. (2016): Stacked Generative Adversarial Networks. arXiv:1612.04357
  • M.Y. Liu, O. Tuzel (2016): Coupled Generative Adversarial Networks. Advances in Neural Information Processing Systems

16

Generative Adversarial Networks (3)

  • J. Ho and S. Ermon (2016): Generative Adversarial Imitation Learning. arXiv:1606.03476
  • A. Kuefler et al. (2017): Imitating Driver Behavior with Generative Adversarial Networks. arXiv:1701.06699

17

Domain Adaptation (1)

  • H. Ajakan et al. (2014): Domain-Adversarial Neural Networks. arxiv.org/abs/1412.4446
  • Y. Ganin and V. Lempitsky (2014): Unsupervised Domain Adaptation by Backpropagation. arXiv:1409.7495

18

Domain Adaptation (2)

  • K. Bousmalis et al. (2016): Domain Separation Networks. Proc. Neural Information Processing Systems (NIPS)
  • K. Bousmalis et al. (2016): Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks. arXiv:1612.05424

19

Semantic Deep Learning

  • N. Phan et al. (2015): Ontology-Based Deep Learning for Human Behavior Prediction in Health Social Networks. Information Sciences, 384.
  • L. Ferrone, F.M. Zanzotto (2017): Symbolic, Distributed and Distributional Representations for Natural Language Processing in the Era of Deep Learning: a Survey. arXiv:1702.00764.
  • S. Bowman (2014): Can Recursive Neural Tensor Networks Learn Logical Reasoning? arXiv:1312.6192v4

Selected references on deep learning: