The LMSYS Chatbot Arena is a crowdsourced evaluation platform launched in 2023 by researchers at UC Berkeley. Users submit prompts that are answered by two randomly selected anonymous models, then vote for which response they prefer. The aggregated votes produce an Elo-style rating that ranks models by real-world user preference.
Arena ratings have become one of the most-watched model rankings because they capture something that closed benchmarks miss — how models perform on the diverse, messy prompts real users actually submit. The rankings are particularly influential for comparing closed-source commercial models (Claude, GPT, Gemini) where direct parameter-level benchmarking is impossible.
Arena has limitations. It measures preference, not accuracy. Users may prefer models that sound confident over models that are correct. Non-English prompt distributions are underrepresented. But as a directional signal for deployed-model quality, Arena ratings remain the most widely cited public ranking of frontier language models.