Weida Wang

Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction

2024 · arXiv (Cornell University)

The attention operator is arguably the key distinguishing factor of transformer architectures, which have demonstrated state-of-the-art performance on a variety of tasks. However, transformer attention operators often impose a significant computational burden, with the computational …

Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoning

2025 · arXiv (Cornell University)

Large language models (LLMs) demonstrate strong reasoning abilities via Chain-of-Thought (CoT), but their token-level generation encourages local decisions and lacks global planning, often leading to redundant or inaccurate reasoning. Existing methods, such as tree-based search …