Semantic Deep Learning

Seminar an der Universitšt des Saarlandes, Fachrichtung Informatik, LSF 116956

The topics for the seminar time slots are listed below. Currently assigned topics are marked in red; waiting list for topics is maintained.   

Selected background literature is indicated below; topic reference papers are available in the web or on request from seminar organizers. In the following, “DL” abbreviates “Deep Learning”.




DL-based link prediction in knowledge graphs for semantic scene descriptions:

S. Baier, Y. Ma, V. Tresp (2017): Improving Visual Relationship Detection using Semantic Modeling of Scene Descriptions. Proc. of International Semantic Web Conference (ISWC); LNCS, Springer.
https://arxiv.org/pdf/1809.00204.pdf  and https://arxiv.org/pdf/1808.08941.pdf  


DL-based translation from natural language to description logic (ALCQ) for ontology learning:

G. Petrucci, M. Rospocher, C. Ghidini (2018): Expressive Ontology Learning as Neural Machine Translation. Journal of Web Semantics; Elsevier.


DL-based natural language inference I

S. Bowman, C. Potts, CD. Manning (2015): Recursive Neural Networks Can Learn Logical Semantics. Proc. of 3rd Workshop on Continuous Vector Space Models and their Compositionality; ACL.
https://arxiv.org/pdf/1406.1827.pdf  and Tree-structured recursive neural networks for sentence meaning (recognition of textual entailment between sentences).


DL-based natural language inference II

(a) J. Devlin, M.-W. Chang, K. Lee,  K. Toutanova (2018):  Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805.
https://arxiv.org/pdf/1810.04805.pdf  [ref. in 10.]

(b) Q. Chen, X. Zhu, Z.-H. Ling, S. Wei, H. Jiang, D. Inkpen (2017): Enhanced LSTM for Natural Language Inference. Proc. of 55th Annual Meeting of the Association for Computational Linguistics; vol.1:1657–1668. https://arxiv.org/pdf/1609.06038.pdf  [ref. in 10.] 


Ontology-based deep learning of classification and prediction tasks:

(a) H. Wang, D. Dou, D. Lowd (2016): Ontology-Based Deep Restricted Boltzmann Machine.
Proc. International Conference on Database and Expert Systems Applications (DEXA); LNCS, Springer.

(b) N. Phan, D. Dou, H. Wang, D. Kil, B. Piniewski (2015): Ontology-Based Deep Learning for Human Behavior Prediction in Health Social Networks. Proc. of 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics; ACM.


Generation of image descriptions via DL-based alignment of visual and language data:

A. Karpathy, L. Fei-Fe (2015): Deep Visual-Semantic Alignments for Generating Image Descriptions.
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR); IEEE.


Generation of justification explanations of visual evidence and DL results:

LA. Hendricks, Z. Akata, M. Rohrbach, J. Donahue, B. Schiele, T. Darrell (2016): Generating Visual Explanations. Proc. of European Conference on Computer Vision; Springer.


Generation of introspective explanations via DL-based visual attention and annotation:

J. Kim, A. Rohrbach, T. Darrell, J. Canny, Z. Akata (2018): Textual Explanations for Self-Driving Vehicles. Proc. of European Conference on Computer Vision; Springer.
https://arxiv.org/pdf/1807.11546.pdf  and https://arxiv.org/abs/1612.04757 


Rationalization of DL result with backpropagation-based input feature relevance (attribution) methods:

M. Ancona, E. Ceolini, C. Oztireli, M. Gross (2018): Towards better understanding of gradient-based attribution methods for Deep Neural Networks. Proc. of 6th International Conference on Learning Representations (ICLR).


Visual commonsense reasoning:

R. Zellers, Y. Bisk, A. Farhadi, Y. Choi (2018): From Recognition to Cognition: Visual Commonsense Reasoning. arXiv:1811.10830. https://arxiv.org/pdf/1811.10830.pdf [ref. 4a+b]