Yu Yan

RTT- or Bandwidth-Bound? Demystifying the KV Cache Transfer in Large Language Model Serving

2025

Modern large language model (LLM) serving systems increasingly adopt a prefill-decode disaggregation architecture to enhance inference efficiency. While this design improves resource utilization, it introduces latency due to the transfer of key-value (KV) cache. The …

DistJoin: A Decoupled Join Cardinality Estimator based on Adaptive Neural Predicate Modulation

2025 · arXiv (Cornell University)

Research on learned cardinality estimation has made significant progress in recent years. However, existing methods still face distinct challenges that hinder their practical deployment in production environments. We define these challenges as the ``Trilemma of …