Skip to main content Skip to main navigation


Fine-tuning BERT Models for Summarizing German Radiology Findings

Siting Liang; Klaus Kades; Matthias A. Fink; Peter M. Full; Tim F. Weber; Jens Kleesiek; Michael Strube; Klaus Maier-Hein
In: Tristan Naumann; Steven Bethard; Kirk Roberts; Anna Rumshisky (Hrsg.). Proceedings of the 4th Clinical Natural Language Processing Workshop. Clinical Natural Language Processing Workshop (ClinicalNLP-2022), located at NAACL 2022, July 14, Seattle, WA, USA, Association for Computational Linguistics, 7/2022.


Writing the conclusion section of radiology reports is essential for communicating the radiology findings and its assessment to physicians in a condensed form. In this work, we employ a transformer-based Seq2Seq model for generating the conclusion section of German radiology reports. The model is initialized with the pre-trained parameters of a German BERT model and fine-tuned in our downstream task on our domain data. We proposed two strategies to improve the factual correctness of the model. In the first method, next to the abstractive learning objective, we introduce an extraction learning objective to train the decoder in the model to both generate one summary sequence and extract the key findings from the source input. The second approach is to integrate the pointer mechanism into the transformer-based Seq2Seq model. The pointer network helps the Seq2Seq model to choose between generating tokens from the vocabulary or copying parts from the source input during generation. The results of the automatic and human evaluations show that the enhanced Seq2Seq model is capable of generating human-like radiology conclusions and that the improved models effectively reduce the factual errors in the generations despite the small amount of training data.


Weitere Links