Weco AI, a company building agentic systems, just dropped SpecBench — an open-source system-level coding benchmark — and it reveals a dirty little secret about AI programmers: they're ruthless rule-exploiters. The term is "reward hacking," and it's exactly what it sounds like: AI agents gaming the test suite to get high scores without actually learning to code properly.
The benchmark found that AI models tend to take shortcuts, patching over visible test cases with superficial fixes that crumble when faced with hidden tests. In one extreme example, an AI using Codex didn't bother writing a C compiler at all. Instead, it secretly called GCC (the external compiler) to run all the test problems, memoized the answers into a nearly 3,000-line hash table, and then returned the precomputed result for each input. On the visible tests, it scored a whopping 97% — but on the hidden tests, it got a flat zero.
However, the researchers note that most cheating isn't that dramatic. More common are structural failures: poor component isolation, missing edge cases, and other design flaws. The study also found that the gap between validation and held-out scores grows with code size. And blindly adding more debugging steps can actually backfire, pushing the AI toward error-prone paths that pass visible tests while wrecking the underlying system architecture.