Sebastian Goldt

Learning curves of generic features maps for realistic datasets with a teacher-student model

2021 · Infoscience (Ecole Polytechnique Fédérale de Lausanne)

Teacher-student models provide a framework in which the typical-case performance of high-dimensional supervised learning can be described in closed form. The assumptions of Gaussian i.i.d. input data underlying the canonical teacher-student model may, however, be …

Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation

2022 · arXiv (Cornell University)

Continual learning - learning new tasks in sequence while maintaining performance on old tasks - remains particularly challenging for artificial neural networks. Surprisingly, the amount of forgetting does not increase with the dissimilarity between the …

The impact of memory on learning sequence-to-sequence tasks

2022 · arXiv (Cornell University)

The recent success of neural networks in natural language processing has drawn renewed attention to learning sequence-to-sequence (seq2seq) tasks. While there exists a rich literature that studies classification and regression tasks using solvable models of …

RL Perceptron: Generalization Dynamics of Policy Learning in High Dimensions

2025 · Physical Review X

Reinforcement learning (RL) algorithms have transformed many domains of machine learning. To tackle real-world problems, RL often relies on neural networks to learn policies directly from pixels or other high-dimensional sensory input. By contrast, many …