Manya Ghobadi

InfoShape: Task-Based Neural Data Shaping via Mutual Information

2022 · arXiv (Cornell University)

The use of mutual information as a tool in private data sharing has remained an open challenge due to the difficulty of its estimation in practice. In this paper, we propose InfoShape, a task-based encoder …

Rail-only: A Low-Cost High-Performance Network for Training LLMs with Trillion Parameters

2024

This paper presents a low-cost network architecture for training large language models (LLMs) at hyperscale. We study the optimal parallelization strategy of LLMs and propose a novel datacenter network design tailored to LLM's unique communication …