SWE-bench is a benchmark introduced in 2023 that evaluates AI coding agents on real issues from twelve popular Python open-source repositories. The agent is given a bug report or feature request and the full repository codebase, and must produce a patch that passes the maintainer's existing test suite.
The benchmark is considered the most credible software-engineering specific measure of AI agent capability. Unlike closed-form coding challenges, SWE-bench issues require reading and understanding an unfamiliar codebase, navigating project conventions, and producing changes that integrate cleanly with existing code. Human software engineers solve approximately 50-60 percent of SWE-bench issues under time pressure.
Frontier agent systems as of early 2026 score in the 60-75 percent range on SWE-bench Verified (a curated subset), meaning they now exceed the median human engineer on this specific task. The implications for software engineering workforce displacement — like those for financial analyst work — track the OSWorld-V threshold discussion.