Zhongwang Zhang

Implicit Regularization of Dropout

2024 · IEEE Transactions on Pattern Analysis and Machine Intelligence

It is important to understand how dropout, a popular regularization method, aids in achieving a good generalization solution during neural network training. In this work, we present a theoretical derivation of an implicit regularization of …

Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers

2025 · arXiv (Cornell University)

Transformers have demonstrated impressive capabilities across various tasks, yet their performance on compositional problems remains a subject of debate. In this study, we investigate the internal mechanisms underlying Transformers' behavior in compositional tasks. We find …