Vikas Chandra

CPT: Efficient Deep Neural Network Training via Cyclic Precision

2021 · arXiv (Cornell University)

Low-precision deep neural network (DNN) training has gained tremendous attention as reducing precision is one of the most effective knobs for boosting DNNs' training time/energy efficiency. In this paper, we attempt to explore low-precision training …

Scaling Parameter-Constrained Language Models with Quality Data

2024 · arXiv (Cornell University)

Scaling laws in language modeling traditionally quantify training loss as a function of dataset size and model parameters, providing compute-optimal estimates but often neglecting the impact of data quality on model generalization. In this paper, …