Zhe Gan

Contextual Text Style Transfer

2020 · arXiv (Cornell University)

We introduce a new task, Contextual Text Style Transfer - translating a sentence into a desired style with its surrounding context taken into account. This brings two key challenges to existing style transfer approaches: ($i$) …

InfoBERT: Improving Robustness of Language Models from An Information\n Theoretic Perspective

2020 · arXiv (Cornell University)

Large-scale language models such as BERT have achieved state-of-the-art\nperformance across a wide range of NLP tasks. Recent studies, however, show\nthat such BERT-based models are vulnerable facing the threats of textual\nadversarial attacks. We aim to address …

Multimodal Autoregressive Pre-training of Large Vision Encoders

2024 · arXiv (Cornell University)

We introduce a novel method for pre-training of large-scale vision encoders. Building on recent advancements in autoregressive pre-training of vision models, we extend this framework to a multimodal setting, i.e., images and text. In this …

Patient Knowledge Distillation for BERT Model Compression

2019

Siqi Sun, Yu Cheng, Zhe Gan, Jingjing Liu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Domain Adaptive Text Style Transfer

2019

Dianqi Li, Yizhe Zhang, Zhe Gan, Yu Cheng, Chris Brockett, Bill Dolan, Ming-Ting Sun. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language …

FreeLB: Enhanced Adversarial Training for Natural Language Understanding

2019 · arXiv (Cornell University)

Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models. In this work, we propose a novel adversarial training algorithm, FreeLB, that …

Discourse-Aware Neural Extractive Text Summarization

2020

Recently BERT has been adopted for document encoding in state-of-the-art text summarization models. However, sentence-based extractive models often result in redundant or uninformative phrases in the extracted summaries. Also, long-range dependencies throughout a document are …