Sanjiv Kumar

Pre-training Tasks for Embedding-based Large-scale Retrieval

2020 · arXiv (Cornell University)

We consider the large-scale query-document retrieval problem: given a query (e.g., a question), return the set of relevant documents (e.g., paragraphs containing the answer) from a large document corpus. This problem is often solved in …

On the Reproducibility of Neural Network Predictions

2021 · arXiv (Cornell University)

Standard training techniques for neural networks involve multiple sources of randomness, e.g., initialization, mini-batch ordering and in some cases data augmentation. Given that neural networks are heavily over-parameterized in practice, such randomness can cause {\em …

TPU-KNN: K Nearest Neighbor Search at Peak FLOP/s

2022 · arXiv (Cornell University)

This paper presents a novel nearest neighbor search algorithm achieving TPU (Google Tensor Processing Unit) peak performance, outperforming state-of-the-art GPU algorithms with similar level of recall. The design of the proposed algorithm is motivated by …

It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Models

2023 · arXiv (Cornell University)

Classical wisdom in machine learning holds that the generalization error can be decomposed into bias and variance, and these two terms exhibit a \emph{trade-off}. However, in this paper, we show that for an ensemble of …