DB4ML - An In-Memory Database Kernel with Machine Learning SupportMatthias Jasny; Tobias Ziegler; Tim Kraska; Uwe Röhm; Carsten Binnig
In: David Maier; Rachel Pottinger; AnHai Doan; Wang-Chiew Tan; Abdussalam Alawini; Hung Q. Ngo (Hrsg.). Proceedings of the 2020 International Conference on Management of Data. ACM SIGMOD International Conference on Management of Data (SIGMOD-2020), June 14-19, Pages 159-173, ACM, 2020.
In this paper, we revisit the question of how ML algorithms can be best integrated into existing DBMSs to not only avoid expensive data copies to external ML tools but also to comply with regulatory reasons. The key observation is that database transactions already provide an execution model that allows DBMSs to efficiently mimic the execution model of modern parallel ML algorithms. As a main contribution, this paper presents DB4ML, an in-memory database kernel that allows applications to implement user-defined ML algorithms and efficiently run them inside a DBMS. Thereby, the ML algorithms are implemented using a programming model based on the idea of so called iterative transactions. Our experimental evaluation shows that DB4ML can support user-defined ML algorithms inside a DBMS with the efficiency of modern specialized ML engines. In contrast to DB4ML, these engines not only need to transfer data out of the DBMS but also hardcode the ML algorithms and thus are not extensible.