Shengnan Yue

Challenges and Opportunities of a Vastly Distributed Cloud Computing Infrastructure – In the Context of the Dong Shu Xi Suan (DSXS) Project of China

2023

It has been widely recognized that placing cloud data centers near clean energy sources or in cooler environments may reduce the operational cost and carbon footprint. But when the scale of the system, in terms …

RTT- or Bandwidth-Bound? Demystifying the KV Cache Transfer in Large Language Model Serving

2025

Modern large language model (LLM) serving systems increasingly adopt a prefill-decode disaggregation architecture to enhance inference efficiency. While this design improves resource utilization, it introduces latency due to the transfer of key-value (KV) cache. The …