Lala Li

Differentiable Product Quantization for End-to-End Embedding Compression

2020 · International Conference on Machine Learning

Embedding layers are commonly used to map discrete symbols into continuous embedding vectors that reflect their semantic meanings. Despite their effectiveness, the number of parameters in an embedding layer increases linearly with the number of …

Intriguing Properties of Contrastive Losses

2020 · arXiv (Cornell University)

We study three intriguing properties of contrastive learning. First, we generalize the standard contrastive loss to a broader family of losses, and we find that various instantiations of the generalized loss perform similarly under the …