How Top Teams Review AI-Written Code
1. Key Themes
Theme: Code review, not code generation, is the new bottleneck in AI-assisted engineering
AI adoption raises PR volume faster than review capacity
Telemetry across a large sample shows output piling up in front of reviewers. The article notes: "When teams moved to high AI adoption, merged pull requests went up 98%, average PR size went up 154% and time in review went up 91%." Company-level delivery metrics didn't improve: "Company-wide delivery metrics like lead time and change failure rate stayed flat. The extra output just piled up in front of reviewers."
Even Anthropic hit this wall
The article says "Anthropic says code output per engineer grew 200% in a year. Before it built its own reviewer, only 16% of its internal pull requests got substantive review comments." The takeaway for boards: "Your AI bill bought you more pull requests. Whether it bought you more shipped product depends on everything between 'PR opened' and 'running in production'."
Industry-wide data points the same direction
Google's DORA report finds throughput and instability rising together: "AI adoption now goes with higher delivery throughput, and it still goes with lower delivery stability," with "90% of respondents use AI at work."
Theme: AI-written code fails in predictable, checkable places
More issues per PR, concentrated in logic, security, and error handling
In a 470-PR study, "AI ones averaged 10.83 issues per PR, against 6.45 for the human ones." Logic errors "show up 1.75 times as often," security findings 1.57x overall and up to 2.74x for specific types (XSS), and "readability issues more than 3 times."
Duplication and reduced refactoring are the silent cost
GitClear's analysis of 211 million lines found "Duplicated blocks of five or more lines rose about eightfold in 2024, and refactoring fell from about 25% of changed lines in 2021 to under 10% in 2024." The practical tip: "If you review one thing first, check whether the new helper already exists somewhere."
Developers misjudge their own AI-assisted speed
METR's early trial found developers "took 19% longer with AI tools, while believing AI had sped them up by about 20%." The author's conclusion: "people misjudge their own AI speed in both directions, so ask for proof the code ran."
Theme: Replace the single human gate with layered, risk-tiered checks plus fast rollback
Reframe the review question
The Claude Code team dropped "whether a human read every line." Instead: "Their test is which check, human or machine, is best placed to catch what could go wrong with this specific change." Review moves earlier, working "more like spellcheck while you write."
Six layers of checks sit on top of rollback
The layers: prove it works before the PR, an AI reviewer on every PR, security review that blocks, blocking CI, human approval sized to risk, and merge queue/flags/staged release. Underneath: "Under all six sits fast detection and rollback. Flags switch off without a deploy, a weekly on-call rotation watches, and each incident review asks which layer should have caught the problem."
Results
"On the Claude Code team, merged PRs more than doubled while the revert share fell by over 40%."
Theme: Risk, not author, determines how many humans approve
Three tiers
Low (docs, copy, internal-only, additive flagged changes) needs "zero human approvals once automated review and security checks pass." Medium needs one person; High "plus a short list of paths that always need two people."
Guardrail rules that make tiers safe
"A bot's approval never fills a human slot. AI can clear a low-risk change. That's the ceiling." Also: "Whoever prompted the agent owns the PR," and "An approval covers one commit." Implementation is cheap: "those five rules map almost one to one onto a CODEOWNERS file and GitHub branch protection."
2. Contrarian Perspectives
Slowing down is a bad way to reduce errors; speed up detection and recovery instead The conventional response to AI-code quality worries is to add friction. The article quotes a colleague of the post's authors on why that fails: slowing down lowers your error rate "but mostly because you're doing less of everything." The alternative bet: "The team bet on getting faster at spotting and fixing problems." Evidence: PRs more than doubled while reverts fell over 40%.
Optimize for human review time, not human review coverage The target is that "most PRs merge with at most one human involved, and that human can be the author," with reverts and incidents as the guardrail: "If they climb, you saved review time by spending it on firefighting." This inverts the norm that every line deserves a second human reader.
Be skeptical of the headline studies, in both directions The author flags the vendor bias in the 1.7x issue-rate finding: "CodeRabbit sells AI code review, and it guessed authorship from signals in each PR, so some of the 'human-only' changes probably had AI help." METR's famous 19% slowdown has since been revised: its "February 2026 update" points to "an estimated 18% for returning developers and 4% for new ones," and METR calls even that unreliable because "the true speedup is probably higher." Both the "AI code is worse" and "AI slows experts down" narratives rest on shaky or dated evidence.
The AI reviewer itself needs to be managed and capped Rather than trusting an AI reviewer to converge, treat it like a product. The article warns "One of their PRs ran more than fifty review rounds overnight before anyone stopped it," and the kit includes "AI review with a 3-round cap and diff-only re-review."
3. Companies Identified
Anthropic / Claude Code
- Description: AI lab; Claude Code is its agentic coding product, largely written by Claude itself.
- Why mentioned: Primary case study for the review system.
- Quotes: "Claude writes about 90% of Claude Code's own code. In Anthropic's monorepo, changes landing per day went from about 2,500 in May to more than 30,000 in September." "Before it built its own reviewer, only 16% of its internal pull requests got substantive review comments."
- Description: Engineering telemetry/analytics company.
- Why mentioned: Source of the largest dataset on the AI productivity paradox.
- Quotes: "Faros AI pulled telemetry from more than 10,000 developers on 1,255 teams and compared each team's lowest and highest AI-adoption quarters."
CodeRabbit
- Description: Vendor of AI code review.
- Why mentioned: Study quantifying issue rates in AI vs. human PRs (with disclosed bias).
- Quotes: "CodeRabbit ran its reviewer over 470 open-source pull requests, 320 of them AI co-authored and 150 human-only."
GitClear
- Description: Code analytics company.
- Why mentioned: Evidence of rising duplication and falling refactoring.
- Quotes: "GitClear analyzed 211 million changed lines."
METR
- Description: AI evaluation research organization.
- Why mentioned: Trial on AI developer productivity and its later revision.
- Quotes: "16 experienced open-source developers took 19% longer with AI tools, while believing AI had sped them up by about 20%."
Google (DORA / engineering practices)
- Description: Source of the DORA report and its classic code review research.
- Why mentioned: Baseline for how cheap review used to be, plus 2025 adoption data.
- Quotes: "In Google's 2018 case study of its own process, the median change touched 24 lines and the median review wrapped up in under 4 hours." "90% of respondents use AI at work."
- Description: The newsletter publisher selling the "AI Code Review Kit."
- Why mentioned: Packaging the playbook as a drop-in kit.
- Quotes: "Paid subscribers also get the AI Code Review Kit: review rules, 5 reviewer subagents, GitHub workflows, a PR template, a human checklist and a metrics script, ready to drop into a repo."
- Description: Tech publication.
- Why mentioned: Secondary coverage of the CodeRabbit findings.
- Quotes: "Break it down by category and logic errors show up 1.75 times as often."
4. People Identified
- Description: Engineer and author of the Claude Code review write-up.
- Why mentioned: Source of the post that sparked the newsletter.
- Quotes: "This week Addy Osmani posted how the Claude Code team handles it. His X Article has more than 2 million views and 31,000 bookmarks so far."
Morgan Westlee Lunt and Qing Wang
- Description: Co-authors of the Claude developer blog post.
- Why mentioned: Co-wrote the full write-up.
- Quotes: "The full write-up, by Addy, Morgan Westlee Lunt and Qing Wang, lives on Claude's developer blog."
Boris Cherny
- Description: Creator of Claude Code.
- Why mentioned: Quantified the value of a verification feedback loop.
- Quotes: "Boris Cherny, who created Claude Code, says this feedback loop will 'a 2-3x the quality of the final result'" (the article's wording: "will '2-3x the quality of the final result'").
Ruben Dominguez
- Description: Author of the newsletter issue.
- Why mentioned: Synthesized the Claude Code post with outside research and added startup lessons.
- Quotes: "The framing and the startup lessons are mine."
5. Operating Insights
Write down risk tiers and enforce them with existing GitHub tooling Treat it as policy plus config, not a platform project. "It's a written policy plus a few GitHub settings." Specifically: "The change picks the tier. A classifier reads the diff and assigns it," CI workflow files always get a person, and fast lanes are limited to trusted authors. "You could have them live by Friday" at a 10-person team.
Require proof of execution before review, and tag AI findings by severity Layer 1 forces Claude to "run something, a test suite or a screenshot against the design, and show the output," followed by a fresh subagent reading the diff. AI reviewer findings carry tags "(Important, Nit or Pre-existing) so authors stop getting blamed for old bugs." The PR template also makes authors write "how you tested and how to undo the change."
Instrument human review time, with reverts as the guardrail, and score the AI reviewer against incidents Track "human approvals per PR at the median and the 95th percentile, plus human comments and how long a PR sits waiting on a person." Treat reverts and incidents as the tripwire. Then "Score it against your past incidents, and cap its rounds," and have each incident review ask "which layer should have caught the problem."
6. Overlooked Insights
Staged release channels as a quality layer
Beyond flags, new behavior ships to internal staff first: "New behavior goes to Anthropic staff first, then to a latest channel, with stable trailing behind." Dogfooding plus channel lag acts as a real-world test layer that most startups skip.
Review also serves knowledge transfer, which automation doesn't replace The article notes "Review has always had two purposes. It catches bugs before users do, and it spreads knowledge of the codebase," then only reconsiders the bug-catching purpose. As human reading declines, teams may need a deliberate substitute for how engineers learn the codebase.