Distributed task scheduling with security and outage constraints in MapReduce

The emergence of MapReduce, a simple software framework, is helping to deal with vast amount of data (multiterabyte data-sets) in-parallel on large clusters (thousands of nodes) of commodity hardware in a reliable, fault-tolerant manner. Extensive researches and popularity are gained by MapReduce recently. In this paper, we consider the MapReduce task scheduling problem with security and outage constraints, which are performance effected and not well resolved. The objective is to minimize the makespan while meet data locality and security requirement. A heuristic algorithm with three components is proposed for the problem under study. The simulated results verified the effectiveness of the proposed method, which is closely dependent on the outage probability and the number of worker nodes.