Ion Stoica

FastLane

2015

The drive towards richer and more interactive web content places increasingly stringent requirements on datacenter network performance. Applications running atop these networks typically partition an incoming query into multiple subqueries, and generate the final result …

numpywren: serverless linear algebra

2018 · arXiv (Cornell University)

Linear algebra operations are widely used in scientific computing and machine learning applications. However, it is challenging for scientists and data analysts to run linear algebra at scales beyond a single machine. Traditional approaches either …

Hoplite: Efficient Collective Communication for Task-Based Distributed Systems.

2020 · arXiv (Cornell University)

Collective communication systems such as MPI offer high performance group communication primitives at the cost of application flexibility. Today, an increasing number of distributed applications (e.g, reinforcement learning) require flexibility in expressing dynamic and asynchronous …

Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments

2021 · arXiv (Cornell University)

The capability of a reinforcement learning (RL) agent heavily depends on the diversity of the learning scenarios generated by the environment. Generation of diverse realistic scenarios is challenging for real-time strategy (RTS) environments. The RTS …

Grounded Graph Decoding Improves Compositional Generalization in Question Answering

2021 · arXiv (Cornell University)

Question answering models struggle to generalize to novel compositions of training patterns, such to longer sequences or more complex test structures. Current end-to-end models learn a flat input embedding which can lose input syntax context. …

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

2024 · arXiv (Cornell University)

Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry. However, as new and improved LLMs are developed, existing evaluation benchmarks (e.g., HumanEval, …

A Statistical Framework for Ranking LLM-Based Chatbots

2024 · arXiv (Cornell University)

Large language models (LLMs) have transformed natural language processing, with frameworks like Chatbot Arena providing pioneering platforms for evaluating these models. By facilitating millions of pairwise comparisons based on human judgments, Chatbot Arena has become …

Efficient Memory Management for Large Language Model Serving with PagedAttention

2023

High throughput serving of large language models (LLMs) requires batching sufficiently many requests at a time. However, existing systems struggle because the key-value cache (KV cache) memory for each request is huge and grows and …