Project

SAGE

Percipient Storage for Exascale Data-Centric Computing

Percipient Storage for Exascale Data-Centric Computing

Das weltweite Datenvolumen wächst stark an, wodurch Lösungen, die Speicherung und Verarbeitung von Daten trennen, nicht skalierbar sind. Experimente und Simulationen in wissenschaftlichen Forschungsgebieten wie Physik, Weltraumforschung, Meteorologie, Genetik oder Biologie generieren immer größere Datensätze. Diese reichen bis in den Exabyte-Bereich (Milliarden Gigabyte), sodass für die Forschung auf diesen Gebieten neuartige Verfahren für die Speicherung, Verarbeitung und Analyse von Daten entwickelt werden müssen.

  • Duration:

Worldwide data volumes are exploding and islands of storage remote from compute will not scale. In scientific research domains such as physics, space sciences, meteorology, genetics and biology, experiments and simulations generate increasingly large data sets. As these scale into the range of exabytes (billions of gigabytes), novel storage, processing, and analytics solutions must be devised to continue deriving insights and innovation in research.

The SAGE project proposes an advanced object based storage solution, termed Percipient Storage, with a very flexible new API enabling applications to achieve Exascale I/O loads exploiting deep I/O hierarchies. The solution will have the capability to run computations on data from any tier – with a homogenous view of data throughout the stack. The SAGE architecture reflects the need for reducing data movement in order to improve energy efficiency, as well as the technology trend towards new non-volatile memory technologies. DFKI will work in cooperation with European partners at Allinea, Bull, CCFE, CEA, Diamond Light Source, FSZ Jülich, KTH, and STFC in this Horizon 2020-funded project lead by Seagate, to meet the requirements of exascale scientific computing.

In the course of the project, DFKI will integrate the advanced data analytics platform Apache Flink with the native object interface together with data-local computations. Apache Flink will benefit from the full performance and features offered by the storage platform, lifting its analytics processing capabilities to the exascale level.

The project has received funding from the European Union’s Horizon2020 Research & Innovation Programme under grant agreement 671500.

Partners

Seagate, Allinea, Bull, CCFE, CEA, Diamond Light Source, FSZ Jülich, KTH, STFC

Share project:

Contact Person
Dr.-Ing. Sebastian Breß

Images

[Translate to English:]

[Translate to English:]

Publications about the project

Tobias Behrens, Viktor Rosenfeld, Jonas Traub, Sebastian Breß, Volker Markl

In: Michael Böhlen, Reinhard Pichler, Norman May, Erhard Rahm, Shan-Hung Wu, Katja Hose (editor). Advances in Database Technology — EDBT 2018. International Conference on Extending Database Technology (EDBT-2018) 21th International Conference on Extending Database Technology March 26-29 Vienna Austria Pages 489-492 ISBN 978-3-89318-078-3 OpenProceedings Konstanz, Germany 2018.

To the publication
Clemens Lutz, Sebastian Breß, Tilmann Rabl, Steffen Zeuch, Volker Markl

In: Proceedings of the 14th International Workshop on Data Management on New Hardware. International Workshop on Data Management on New Hardware (DaMoN) (DaMoN-2018) 14th located at ACM SIGMOD International Conference on Management of Data June 10-15 Houston TX United States ISBN 978-1-4503-5853-8/18/06 ACM New York, NY, USA 2018.

To the publication
Clemens Lutz, Sebastian Breß, Tilmann Rabl, Steffen Zeuch, Volker Markl

In: Theo Härder, Ralf Schenkel (editor). Datenbank-Spektrum Schwerpunktbeitrag 13222 Pages 1-13 Springer 2018.

To the publication

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz