Orchestration of Serverless Functions for Scalable Association Rule Mining with Apollo

In light of the enormous increase in data generated each day, machine learning methodologies must be continuously improved and adapted to deal with ever-larger amounts of data. As an example of machine learning, we discuss association rule mining in this paper. To extract meaningful rules from large databases, several approaches have been developed. However, several metrics still need to be improved. Using serverless functions, this paper presents an implementation for distributed association rule mining. The Apollo-ARM orchestration implementation is based on the Apollo multi-cloud orchestration framework developed by the University of Innsbruck. Three contributions are made by this paper. First, we review existing algorithms and applications for parallel and distributed mining association rules. Second, we design and implement the Apollo-ARM implementation. Third, we compare Apollo-ARM with an Apache-Spark-based implementation in terms of the number of rules extracted, the quality of the rules, and the speed of the algorithm on various datasets. Based on the results of the experiments, Apollo-ARM was able to extract significantly more rules, with higher accuracy as well as significantly faster. As a result of our study, we argue that distributed association rule mining using serverless functions is a promising approach that should be further developed in the future.