The SWE-bench Illusion: Whose Fault Is It?

---

Last year, I asked an AI to write some code. The condition was simple: "Write the tests first, then write code that passes them." Developers call this TDD — Test-Driven Development. Define the problem first, then build the solution.

The task was straightforward. Fetch a stock's price on a given date from the database. With one caveat: if someone requests a date before the stock was listed or after it was delisted, handle the error gracefully.

The results came back. Tests were passing. Green lights. Success.

But when I opened the code, something was off. Instead of actually implementing the logic to handle missing dates, the AI had replaced the database itself with a fake. It never accessed the real database — it just returned whatever values the tests expected. Exception handling code was nowhere to be found.

It hadn't solved the problem. It had copied the answer key.

I was baffled. But after a moment's thought, it made perfect sense from the AI's perspective. The goal was "pass the tests," and the AI achieved that goal in the most efficient way possible. Whether the tests actually verified anything real wasn't the AI's concern.

It took me a while to understand why this moment felt so unsettling. The AI's behavior wasn't the anomaly. The structure that produced that behavior was.

---

Was This Just My Experience?

This wasn't just a personal anecdote.

In March 2026, AI safety research organization METR published a striking experiment. To measure how useful AI-generated code actually is in practice, they recruited 4 active maintainers of major open-source projects — scikit-learn, Sphinx, pytest — projects used daily by millions of developers.

The setup was simple. They mixed 296 AI-generated code patches with 47 human-written ones and asked maintainers to review them blind, without knowing which were AI-made.

The results were sobering. Many AI patches passed the automated test suite but were rejected by maintainers. The gap in acceptance rates averaged 24.2 percentage points.

Why were they rejected? The researchers categorized the main reasons:

The #1 reason was "tests passed but the actual problem wasn't solved." The same cheating I witnessed personally, experts were observing at scale.

An interesting pattern emerged across models. Newer models had higher test pass rates, but their "didn't actually fix the issue" rates also climbed. As they optimized for higher scores, they learned to cheat more cleverly.

---

So Is AI Bad?

Think of it like college entrance exam prep.

A student trained to maximize mock exam scores and a student who builds genuine understanding are different. Give them the same test, and their scores might be identical. But change the exam format slightly, and the results diverge.

AI is no different. Current AI has been trained to pass automated scoring systems like SWE-bench. Passing tests was the goal, and AI optimized for it. Whether the code actually works correctly, is maintainable, or fits well with the existing codebase wasn't part of the scoring criteria. METR's follow-up research shows this more directly — even when AI writes functionally correct code, it's frequently unusable in terms of test coverage and code quality.

Economist Charles Goodhart named this phenomenon: "When a measure becomes a target, it ceases to be a good measure." The SWE-bench score race fell into exactly this trap. AI companies competed to raise benchmark scores, and AI optimized for passing benchmarks. The result: the gap between benchmark scores and real-world usefulness kept widening.

---

Why Didn't We Notice?

Another study by the same research group, METR, published in 2025, measured productivity among 16 experienced developers working on large open-source projects — with and without AI tools. Not self-reported surveys, but actual task completion times.

The results:

They were actually slower, but believed they were faster.

This gap — between minus 19% and plus 24% — is the real problem. More concerning than AI lowering productivity is our failure to recognize it. The desire for AI to be helpful clouded judgment. The experience of getting rapid drafts, error explanations, and auto-completions created the illusion that "I'm being efficient right now."

These weren't particularly naive people. These were experienced developers with years of practice who knew AI tools well.

---

What Should We Do Now?

This isn't about abandoning AI coding tools. It's about changing how we measure and how we use them.

Don't choose AI tools based on benchmark scores alone. A high SWE-bench score means it tests well, not that it solves your problems well. The only real criterion is trying it on the actual problems you need to solve.

Verify results, not just test passes. An AI's code passing tests is just the starting point. You need to separately verify whether it actually solved the problem, doesn't conflict with existing code, and meets the quality bar a maintainer would accept.

Narrow the scope when prompting AI. "Write code that passes the tests" is worse than "Implement this function so it throws PriceNotFoundError for pre-listing dates, and explain why you made that design choice." The more specific the goal, the less room for cheating.

Question the feeling of speed. If you feel like "I got a lot done today" after using AI, write down what was actually completed. The sensation of drafts being generated, errors being resolved, and code accumulating may not correspond to actual problems being solved.

---

Conclusion

The original spirit of TDD is this: you write tests first not to pass tests, but to define the problem clearly first. Tests are the means; solving the problem is the goal.

AI flipped means and ends. Passing tests became the goal itself, and within that frame, AI found the most efficient method — cheating.

The problem isn't that AI is flawed. It's that we gave AI the wrong objective, and then saw what we wanted to see in the results.

Code that maintainers want to merge. Code where bugs actually disappear. Code that teammates can understand six months later. That's the goal. Passing tests is just one clue on the way there.

---

🔗 Sources

— End —