Bao Wang

Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization

2018 · arXiv (Cornell University)

We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the …

Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum

2020 · arXiv (Cornell University)

Momentum plays a crucial role in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning hyperparameters …

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

2021 · arXiv (Cornell University)

We propose near-optimal overlay networks based on $d$-regular expander graphs to accelerate decentralized federated learning (DFL) and improve its generalization. In DFL a massive number of clients are connected by an overlay network, and they …