Zhangyang Wang

Self-Damaging Contrastive Learning

2021 · arXiv (Cornell University)

The recent breakthrough achieved by contrastive learning accelerates the pace for deploying unsupervised training on real-world data applications. However, unlabeled data in reality is commonly imbalanced and shows a long-tail distribution, and it is unclear …

Bringing Your Own View

2022 · Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Self-supervision is recently surging at its new frontier of graph learning. It facilitates graph representations beneficial to downstream tasks; but its success could hinge on domain knowledge for handcraft or the often expensive trials and …

Sparsity Winning Twice: Better Robust Generalization from More Efficient Training

2022 · arXiv (Cornell University)

Recent studies demonstrate that deep networks, even robustified by the state-of-the-art adversarial training (AT), still suffer from large robust generalization gaps, in addition to the much more expensive training costs than standard training. In this …

The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training

2022 · TU/e Research Portal

Random pruning is arguably the most naive way to attain sparsity in neural networks, but has been deemed uncompetitive by either post-training pruning or sparse training. In this paper, we focus on sparse training and …

Doubly Robust Instance-Reweighted Adversarial Training

2023 · arXiv (Cornell University)

Assigning importance weights to adversarial data has achieved great success in training adversarially robust networks under limited model capacity. However, existing instance-reweighted adversarial training (AT) methods heavily depend on heuristics and/or geometric interpretations to determine …

Graph Domain Adaptation via Theory-Grounded Spectral Regularization.

2023 · PubMed

, respectively. In a nut-shell, our study paves the way toward explicitly constructing and training GNNs that can capture more transferable representations across graph domains. Codes are released at https://github.com/Shen-Lab/GDA-SpecReg.

Graph Contrastive Learning with Augmentations

2020 · arXiv (Cornell University)

Generalizable, transferrable, and robust representation learning on graph-structured data remains a challenge for current graph neural networks (GNNs). Unlike what has been developed for convolutional neural networks (CNNs) for image data, self-supervised learning and pre-training …