What is happening with code reviews?
👋 Hi, this is Gergely with a subscriber-only issue of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at Big Tech and startups through the lens of engineering managers and senior engineers. If you’ve been forwarded this email, you can subscribe here. What is happening with code reviews?AI generates more code than devs can track in 2026, so will the code review process have to adapt – or is it doomed? A look into this decades-old practice and the approaches that could replace it
One question haunting the minds of CTOs and heads of engineering whom I’ve been talking with, is how to deal with large quantities of code review which have only been growing now that AI agents generate most code at many tech companies. Since the end of 2025, it has seemed that the era of devs writing code by hand is over at startups and in Big Tech. AI agents work faster and generate more pull requests (PRs) than devs ever did, and the size of those pull requests is also increasing. Today’s article summarizes some approaches to code review at various workplaces in this new paradigm, covering:
Unsurprisingly, it’s clear there’s no one-size-fits-all solution to the question of how to handle a deluge of AI-generated code review. Please leave a comment below about how your team or company deals with this new, pressing issue! A snapshot of what’s going on in code review at this stage of AI development is provided by the graphic from GitHub, below. The background context it provides is pretty stark. It shows the stats for the number of PRs and commits over the course of three years on the popular platform:
Over that time, the number of PRs opened has increased fivefold, which is a lot! And growth sped up from the end of 2025, when PRs and commits nearly doubled just in that period alone! So, how are teams dealing with this avalanche of extra work? To find out more, I asked around. 1. Humans review the AI code reviewsThe most common approach is to add an AI code review step to every pull request in a variety of ways:
Typical processes:
In the above cases, engineers typically review the review itself, and not usually the code. Here’s Etienne Dilocker, cofounder and CTO at AI database software, Weaviate, explaining why he likes their approach:
Noise is a big problem with AI code reviews. WeTravel, a Series C travel tech company, decided to not use AI for code reviews because of the amount of noise it generated. In June, they did an updated evaluation which showed lots of improvement, but still not enough to justify adopting AI for the task. As things stand, custom tooling is probably needed to reduce code-review noise. Uber built a clever approach for this; an agentic pipeline called uReview:
What uReview does:
2. Triage by “blast radius” & choose an approachAnother common approach is to decide whether to review code by hand or with AI, based on how “risky” a change is:
This is the approach that Anthropic and OpenAI follow, which I confirmed by talking with both companies. At Anthropic, Jarred Sumner told me that a human merges even low-risk changes, but that their goal is eventually to get another Claude instance to merge low-risk changes. And it’s not just at leading AI labs: five-person startup, Duckbill Group, a cloud and AI cost management company, changed their process, as explained by cofounder and CEO Mike Julian:
Here’s how I’d visualize this approach:
Some companies have built additional tooling to make it easier for devs to know which reviews to focus on. For example, Uber’s custom-built Code Review Inbox highlights high-impact changes, so devs know to spend more time and effort on them:
3. Review the plan/tests/database schema, but not the implementationSome devs and teams have stopped reviewing the code (the implementation), and instead review the “before” and “after” states: Review the plan: spend a lot more time on the plan than before, to get a much more detailed spec. Using The /grill-me skill by Matt Pocock is a popular method, and I’m also a fan of it for thorough upfront planning, as is Andrea Francesco Speziale, Principal Engineer at Musixmatch:
Review the tests: via Test Driven Development (TDD) – which is much easier with agents when writing the tests upfront is a chore – or by focusing the review to ensure the software is tested. One argument for this approach is that customers and users of software usually don’t care about the code. There’s a caveat that automated tests can verify a lot of different software – and are great at verifying business logic – but they don’t do a good job at verifying whether a UI looks and feels good. Review the database schema. Jackie Luo, cofounder and CEO of AI startup Sigil, and formerly an engineer at Square, says:
Jackie’s point is that data (that is, the state) is the most “rigid” part of any system. Stateless business logic is now easy to change because it’s “just” code, and code is easy and fast to generate and regenerate. For startups, it’s worth getting the data schema – and thereby your state machine – right. Then, everything else will be easy and fast to iterate on. My sense is this approach makes perfect sense for a startup iterating to get product-market fit. However, once you have a business, you’ll want to “guard” the business logic with tests: else your product could break, and existing users will be unhappy when this happens! 4. Produce less code...Subscribe to The Pragmatic Engineer to unlock the rest.Become a paying subscriber of The Pragmatic Engineer to get access to this post and other subscriber-only content. A subscription gets you:
|
Comments
Post a Comment