J. Lacey Smith

Progress measures for grokking via mechanistic interpretability

2023 · arXiv (Cornell University)

Neural networks often exhibit emergent behavior, where qualitatively new capabilities arise from scaling up the amount of parameters, training data, or training steps. One approach to understanding emergence is to find continuous \textit{progress measures} that …