Jannis Koch

SMiPE: Estimating the Progress of Recurring Iterative Distributed Dataflows

2017

Distributed dataflow systems such as Apache Spark allow the execution of iterative programs at large scale on clusters. In production use, programs are often recurring and have strict latency requirements. Yet, choosing appropriate resource allocations …