Kunming Zhang

AGFT: An Adaptive GPU Frequency Tuner for Real-Time LLM Inference Optimization

2025 · arXiv (Cornell University)

The explosive growth of interactive Large Language Models (LLMs) has placed unprecedented demands for low latency on cloud GPUs, forcing them into high-power modes and causing escalating energy costs. Real-time inference workloads exhibit significant dynamic …