Feiyang Pan

RLKD: Distilling LLMs' Reasoning via Reinforcement Learning

2025 · arXiv (Cornell University)

Distilling reasoning paths from teacher to student models via supervised fine-tuning (SFT) provides a shortcut for improving the reasoning ability of smaller Large Language Models (LLMs). However, the reasoning paths generated by teacher models often …