A Stochastic Model for Analyzing Tail Latency of Multi-Tier Online Cloud Services

Tail latency is an important performance metric to characterize the quality of experience (QoE) of online cloud services. However, there still lacks an efficient and easy-to-use analytical model to analyze the tail latency of response time of online cloud services. A big challenge is that online cloud service typically consists of multiple tiers of components that interacts with each other, which is difficult to be accurately modeled. In this paper, we propose an efficient stochastic model for analyzing tail latency of online cloud services. First, a stochastic reward net (SRN) is used to model the online cloud services by taking the interaction between multi-tiers into account. A tagged customer model is then introduced into the SRN to compute the cumulative distribution of response time, based on which tail latency can be directly derived. We further implement an e-commerce site on both a private cloud and a public cloud to verify the accuracy and effectiveness of the proposed model without and with interferences from other applications, respectively. A series of experiments show that the differences between the tail latency obtained from the model and that from the actual experiments are generally less than 20% in both cases.