SWE-bench

“Software engineering benchmark now saturated where models merge GitHub pull requests: the verifier is whether the test cases pass.”