Mengdi Huai

Demystify Hyperparameters for Stochastic Optimization with Transferable Representations

2022 · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

This paper studies the convergence and generalization of a large class of Stochastic Gradient Descent (SGD) momentum schemes, in both learning from scratch and transferring representations with fine-tuning. Momentum-based acceleration of SGD is the default …

AdvST: Revisiting Data Augmentations for Single Domain Generalization

2023 · arXiv (Cornell University)

Single domain generalization (SDG) aims to train a robust model against unknown target domain shifts using data from a single source domain. Data augmentation has been proven an effective approach to SDG. However, the utility …

Towards Stable and Explainable Attention Mechanisms

2025 · IEEE Transactions on Knowledge and Data Engineering

Currently, attention mechanism has become a standard fixture in most state-of-the-art natural language processing (NLP) models, not only due to the outstanding performance it could gain but also due to plausible innate explanations for the …

Neuron Explanations for Conformal Prediction (Student Abstract)

2025 · Proceedings of the AAAI Conference on Artificial Intelligence

Conformal prediction (CP) has gained prominence as a popular technique for uncertainty quantification in deep neural networks (DNNs), providing statistically rigorous uncertainty sets. However, existing CP methods fail to clarify the origins of predictive uncertainties. …