Olga Vechtomova

Stochastic Wasserstein Autoencoder for Probabilistic Sentence Generation

2018 · arXiv (Cornell University)

The variational autoencoder (VAE) imposes a probabilistic distribution (typically Gaussian) on the latent space and penalizes the Kullback--Leibler (KL) divergence between the posterior and prior. In NLP, VAEs are extremely difficult to train due to …

Disentangled Representation Learning for Non-Parallel Text Style Transfer

2019

This paper tackles the problem of disentangling the latent representations of style and content in language models. We propose a simple yet effective approach, which incorporates auxiliary multi-task and adversarial objectives, for style prediction and …

Distilling Task-Specific Knowledge from BERT into Simple Neural Networks

2019 · arXiv (Cornell University)

In the natural language processing literature, neural networks are becoming increasingly deeper and complex. The recent poster child of this trend is the deep language representation model, which includes BERT, ELMo, and GPT. These developments …

Generating Sentences from Disentangled Syntactic and Semantic Spaces

2019

Variational auto-encoders (VAEs) are widely used in natural language generation due to the regularization of the latent space. However, generating sentences from the continuous latent space does not explicitly model the syntactic information. In this …