SWE-bench Verified Extended

Name: SWE-bench Verified Extended
Creator: Mercor

Real-world code issue resolution.

Get in touch Read research

50Mercor tasks

500Public tasks

82.0%Highest score

The SWE-bench Verified Extended leaderboard

Tests whether LLMs can resolve real-world GitHub issues by editing codebases, spanning 2,294 problems from 12 popular Python repos and requiring multi-file, long-context changes.

Paper Data Code Sample task

Model

Score

Opus 5Max

Score on the original open-source benchmark: 93.5% / Score on this Mercor-extended benchmark: 82.0%

Fable 5Max

Score on the original open-source benchmark: 95.9% / Score on this Mercor-extended benchmark: 78.7%

Grok 4.5High

Score on the original open-source benchmark: 81.7% / Score on this Mercor-extended benchmark: 70.7%

Sonnet 5Max

Score on the original open-source benchmark: 82.8% / Score on this Mercor-extended benchmark: 67.3%

Opus 4.8Max

Score on the original open-source benchmark: 88.5% / Score on this Mercor-extended benchmark: 63.3%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

SWE-bench Verified Extended

The SWE-bench Verified Extended leaderboard

The latest on frontier AI performance, straight to your inbox.