Nikil Ravi

REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites

2025

We introduce REAL, a benchmark and framework for multi-turn agent evaluations on deterministic simulations of real-world websites. REAL comprises high-fidelity, deterministic replicas of 11 widely-used websites across domains such as e-commerce, travel, communication, and professional …