Skip to main content Skip to main navigation



Consistent dynamic scene reconstruction and property transfer using priors and constraints

Consistent dynamic scene reconstruction and property transfer using priors and constraints

  • Duration:

The objective of DYNAMICS is to develop a new methodology for 4D reconstruction of real world scenes with a small number of cameras, as well as to learn statistical models from the captured data sets. A 4D reconstruction refers to a sequence of accurate 3D reconstructions (including geometry, topology and surface properties) of a dynamic (evolving in time) real-world scene. We aim to build a robust lightweight capture system that can be easily installed and used (e.g. in the living room of a house, in outdoor environments, and broadly under various spatial and temporal constraints).

We are developing a novel interactive software system for motion estimation capitalizing on our experience from the predecessor project DENSITY and exploring new directions (new hardware and machine learning methods).

Specifically, the project DYNAMICS can be subdivided into several work packages according to the target scenarios and concerned areas of computer vision:

1) Software for an interactive monocular 4D reconstruction of non-rigid scenes. The main components are modules dealing with non-rigid structure from motion (NRSfM) pipeline and non-rigid registration. Underlying technology will allow to reconstruct non-rigidly deforming scenes with a minimal number of assumptions from a single RGB camera. Target scenarios include endoscopy, capture of facial expressions, small motion and post-factum reconstructions.

2) Software for robust 4D reconstruction from multiple views incorporating optical flow and scene flow with additional assumptions. We plan to assemble a capture studio with five Emergent HT-4000C high-speed cameras (a multi-view setting). Here, we aim at the highest precision and richness of detail in the reconstructions.

3) 3D shape templates with attributes derived from real data using deep learning techniques. The main objective of this work package is to provide statistical models as a prior knowledge in order to increase the robustness and accuracy of reconstructions. Furthermore, the shape templates will allow for more accurate reconstructions of articulated motion (e.g. skeleton poses) from uncalibrated multi-view settings.

DYNAMICS is a BMBF project with an emphasis on development of core technologies applicable in other ongoing and forthcoming projects in the Augmented Vision Lab.


BMBF - Federal Ministry of Education and Research

BMBF - Federal Ministry of Education and Research

Publications about the project

Didier Stricker; Vladislav Golyanik; Christian Theobalt

In: 13. International Conference on Computer Vision. International Conference on Computer Vision (ICCV-2019), October 27 - November 2, Seoul, Korea, Republic of, 11/2019.

To the publication

Yongzhi Su; Vladislav Golyanik; Nareg Minaskan Karabid; Sk Aziz Ali; Didier Stricker

In: Proceedings of the 18th IEEE ISMAR. IEEE International Symposium on Mixed and Augmented Reality (ISMAR-2019), October 14-18, Beijing, China, IEEE, 2019.

To the publication

Jameel Malik; Ahmed Elhayek; Didier Stricker

In: Proceedings of EuroVR 2018 |. EuroVR (EuroVR-2018), October 22-23, London, United Kingdom, Springer, 11/2018.

To the publication