Yile Gu

Semantic Scheduling for LLM Inference

2025 · arXiv (Cornell University)

Conventional operating system scheduling algorithms are largely content-ignorant, making decisions based on factors such as latency or fairness without considering the actual intents or semantics of processes. Consequently, these algorithms often do not prioritize tasks …