Christopher De

A Two Pronged Progress in Structured Dense Matrix Multiplication

2016 · arXiv (Cornell University)

Matrix-vector multiplication is one of the most fundamental computing primitives. Given a matrix $A\in\mathbb{F}^{N\times N}$ and a vector $b$, it is known that in the worst case $Θ(N^2)$ operations over $\mathbb{F}$ are needed to compute …

MixML: A Unified Analysis of Weakly Consistent Parallel Learning

2020 · arXiv (Cornell University)

Parallelism is a ubiquitous method for accelerating machine learning algorithms. However, theoretical analysis of parallel learning is usually done in an algorithm- and protocol-specific setting, giving little insight about how changes in the structure of …

Low-Precision Reinforcement Learning: Running Soft Actor-Critic in Half Precision

2021 · International Conference on Machine Learning

Low-precision training has become a popular approach to reduce compute requirements, memory footprint, and energy consumption in supervised learning. In contrast, this promising approach has not yet enjoyed similarly widespread adoption within the reinforcement learning …

Understanding Accuracy-Efficiency Trade-Offs as a Means for Holding Distributed ML Systems Accountable

2020 · arXiv (Cornell University)

Trade-offs between accuracy and efficiency are found in multiple non-computing domains, such as law and public health, which have developed rules and heuristics to guide how to balance the two in conditions of uncertainty. While …

ModuLoRA: Finetuning 2-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers

2023 · arXiv (Cornell University)

We propose a memory-efficient finetuning algorithm for large language models (LLMs) that supports finetuning LLMs with 65B parameters in 2/3/4-bit precision on as little as one 24GB GPU. Our method, modular low-rank adaptation (ModuLoRA), integrates …