Zhi‐Qin John Xu

Limitation of Characterizing Implicit Regularization by Data-independent Functions

2022 · arXiv (Cornell University)

In recent years, understanding the implicit regularization of neural networks (NNs) has become a central task in deep learning theory. However, implicit regularization is itself not completely defined and well understood. In this work, we …

Implicit Regularization of Dropout

2024 · IEEE Transactions on Pattern Analysis and Machine Intelligence

It is important to understand how dropout, a popular regularization method, aids in achieving a good generalization solution during neural network training. In this work, we present a theoretical derivation of an implicit regularization of …

Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers

2025 · arXiv (Cornell University)

Transformers have demonstrated impressive capabilities across various tasks, yet their performance on compositional problems remains a subject of debate. In this study, we investigate the internal mechanisms underlying Transformers' behavior in compositional tasks. We find …