Guochao Jiang

RASD: Retrieval-Augmented Speculative Decoding

2025 · arXiv (Cornell University)

Speculative decoding accelerates inference in large language models (LLMs) by generating draft tokens for target model verification. Current approaches for obtaining draft tokens rely on lightweight draft models or additional model structures to generate draft …