Optimizing cross-platform data movement

Sebastian Kruse, Zoi Kaoudi, Jorge Arnulfo Quiane-Ruiz, Sanjay Chawla, Felix Naumann, Bertty Contreras-Rojas

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations


Data analytics are moving beyond the limits of a single data processing platform. A cross-platform query optimizer is necessary to enable applications to run their tasks over multiple platforms efficiently and in a platform-agnostic manner. For the optimizer to be effective, it must consider data movement costs across different data processing platforms. In this paper, we present the graph-based data movement strategy used by Rheem, our open-source cross-platform system. In particular, we (i) model the data movement problem as a new graph problem, which we prove to be NP-hard, and (ii) propose a novel graph exploration algorithm, which allows Rheem to discover multiple hidden opportunities for cross-platform data processing.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE 35th International Conference on Data Engineering, ICDE 2019
PublisherIEEE Computer Society
Number of pages4
ISBN (Electronic)9781538674741
StatePublished - Apr 2019
Externally publishedYes
Event35th IEEE International Conference on Data Engineering, ICDE 2019 - Macau, China
Duration: Apr 8 2019Apr 11 2019

Publication series

NameProceedings - International Conference on Data Engineering
ISSN (Print)1084-4627


Conference35th IEEE International Conference on Data Engineering, ICDE 2019


  • Cross-platform
  • Data movement
  • Polystore
  • Query opimization


Dive into the research topics of 'Optimizing cross-platform data movement'. Together they form a unique fingerprint.

Cite this