Jerry Wei

Symbol tuning improves in-context learning in language models

2023 · arXiv (Cornell University)

We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar"). Symbol tuning leverages the intuition that when a model …