Delay Tolerant Bulk Transfers on Inter-Datacenter Networks

Large datacenter operators often run a large number of datacenters over the globe, and it is of their great interest to improve the bandwidth utilization of inter-datacenter networks. One general approach is to schedule delay tolerant background traffic over the residual bandwidth. However, the difficulty is that the residual bandwidth is time and spatial varying, and the traditional bandwidth forecast technique is not precise enough. We propose a back- pressure based joint flow scheduling and bandwidth allocation algorithm which does not require the prior- knowledge of residual bandwidth. The algorithm allows the data packets to be temporarily stored at intermediate datacenters and forwarded to the destination when there are available spare residual bandwidth. The main idea is to balance the buffers of two adjacent datacenter nodes as much as possible by pushing data across the link between the two nodes using the residual bandwidth, where the buffer size is regard as the pressure of the buffer. The novelty of our approach is that it allows random bandwidth, where in the original algorithm, the link bandwidth is fixed and static. In order to cope with the randomness of residual bandwidth, we redesign the buffer reallocation scheme and apply the Lyapunov drift technique with a completely new Lyapunov function to prove the stability of the algorithm. Our experiments show that the algorithm makes the buffers stable when the data arrival rates are inside the residual capacity region and achieves good throughput compared with the simple store-and-forward strategy and the BitTorrent technique.