What’s the ROI of software engineers’ spending on AI?
👋 Hi, this is Gergely with a subscriber-only issue of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at Big Tech and startups through the lens of engineering managers and senior engineers. If you’ve been forwarded this email, you can subscribe here. What’s the ROI of software engineers’ spending on AI?AI spending has greatly increased at most companies this year, but has it actually changed shipping velocity and PR output? We analyse data from 27 companies – with some interesting findings
A software engineer whose company is looking into its own spending on AI tools pinged me with a very timely question recently, asking what other companies spend on AI, per engineer. They were also keen to find out what the PR throughput was before and after the spending on AI at other places. Spending on AI agents is ballooning at all the tech companies I talk with this year. With 2027 coming around fast, many companies are seeking to put a realistic number on how much they really spend on AI. The change in PR throughput is a very simple metric, and one of those “readily-available” measurements at nearly all companies, It’s also one that probably has not been “gamed” by devs yet, and it’s possible to compare a “before AI” and “after AI” measurement with this metric. To find out more and to try and answer the reader’s question, I asked around and got to talk with engineering leaders and senior+ engineers at 27 workplaces. I also gathered aggregate data from CodeRabbit and Gitar. This article summarizes what I found, covering:
The bottom of this article could be cut off in some email clients. Read the full article uninterrupted, online. 1. Stats from CodeRabbit and GitarThe team at CodeRabbit polled data engineers using the platform, anonymized it, and has kindly shared it with us. They saw a 113% increase in weekly PRs:
There’s a 127% rise in monthly PRs:
When tracking the same group of 20,000 developers, the increase is nearly identical across the platform: a 128% increase:
Across customer groups, there seems to be “tiers” of PR frequency. Here’s data from Gitar (Sonar’s AI code verification agent) on PRs/week/dev across hundreds of customers:
I suspect that a lot has to do with company culture: whenever you join a new team, you pick up the pace pretty quickly. If the company is one where ~5 PRs per week is considered a high output, then new joiners get used to that pace. If it’s a place where most people ship 20-30 PRs to production per week, then new joiners pick up that pace! 2. AI Spending patterns of 27 companiesI got details on AI spending and changes in the rate of PRs from 27 different companies. I anonymized most names by request. At a glance, here’s how spend looks like:
A few common themes from this data set:
3. High spenders ($1,000+ per month)Here are details of companies spending considerable amounts each month on AI coding tools. Quant trading shop, ~5,000 staff (circa 2,000 devs)This company has seen its AI spend balloon overall: total spend was $8M/month in September, a 16x increase on the $500K/month spend in January! The average spend is $1,600/dev/month (up from around $80 in January). Most are going to Claude, and the biggest spenders are not devs, but traders. A current engineer there told me:
The company has a $25,000 limit per employee for AI, which is extremely high, more than at NVIDIA! The engineer explained this generous limit:
AI outlay is expected to grow: this firm is building its own internal GPU and CPU farms to run models, and are building models expected to beat state-of-the-art (SOTA) models like Fable, according to their own data. ROI is measured differently at trading firms. The ROI of any spend is the profit or loss made by a trading desk, and AI spend is now part of the costbase. So, provided the desk’s profit increases, no questions are expected. Small .NET startup, 2 devs, EUOutside of Big Tech where the monthly AI spend can be $10K+ for some people, the largest per-developer bill comes from a two-person .NET shop in Europe. This agency builds assessment games for a niche, global HR platform. Everything is built in C# and Blazor, running on Azure. A single tech stack was a great decision, especially now, they told me. $3,000/month/dev (€2,500/dev/month) is what they spend on AI tools. Most of the spend goes on Claude Opus, and GitHub Copilot. One of the devs told me:
PR frequency: up 5-10x. As this engineer told me:
Series D, hospitality sector, 1,000 employees, ~200 engineers, US$1,500/dev/month is the default spending cap, but devs can ask for an increase. A current software engineer told me they spent $2,200/month in September. with Opus 5. With the latest version, Opus 5.5, this dev expects their spending to be cut in half. PR frequency has not much changed. This engineer told me:
Publicly traded SaaS company, ~2,500 staff, ~600 engineers, US+EU$1,200/month/dev is the budget limit, but a Staff Engineer at the company reports that fewer than 5% of devs hit this mark. PR volume: it’s up (no data). However, the velocity of PRs merged has not seemed to change much! From the same engineer:
The engineer identified areas where AI has helped a lot:
When the focus is on quality, velocity probably does not increase all that much. The engineer had this observation:
Publicly traded company, 25,000+ employees, ~5,000+ devs, US+EU$1,000/dev/month spend is the average, according to an engineering lead. While not in place yet, the company is considering a $1,000/dev/month soft cap. PR frequency: 2x In 2025 there were around 4 PRs/dev/week; Today, this is closer to 8 PRs/dev/week. The lead engineer observed a 2-3x increase in PRs by experienced engineers compared to a year ago. Publicly traded company, 5,000 employees, US+EU$1,000/dev/month spending cap: this is the default, companywide, and can be spent on any provider, any way that devs want. It’s a “soft” spending cap and it’s trivial to request a larger budget: just fill out a short form to justify it, and it’s approved fast. A current team lead shared with me the numbers they’re seeing:
PR frequency: flat. The team lead observed no real increase compared to a year ago. PR size: 3x. Meanwhile, PRs have exploded in size and have 3x as many lines of code per change request as before. I wonder if this could be attributed to LLMs being pretty verbose, even when coding. AI SaaS, ~100 employees, US$2,000/month/dev allowance for devs. This is through a Claude or Codex Max subscription, plus credits used. This company has a public #ai-limits Slack channel, which publicly pings those nearing their limit, as well as top spenders and reminds them to rein in their AI usage, if possible; it sounds a bit like a “public embarrassment” strategy! Getting an increase is more complicated than at most companies. As a current engineer explained to me:
Velocity: unchanged. The engineer told me they don’t see anything getting done faster than before:
Multinational company, 10K+ employees, US / EU / Asia$4,000/dev/month maximum capped spend. This is a hard cap. Engineers at this company use GitHub Copilot and can choose between the models. Right now, Opus 5.5 is the top choice for devs across the company, a current team lead told me. $1,000/dev/month is what the average spend feels like, though from this team lead looking around the org. People are not getting close to the $4K monthly cap. PR frequency: 4x compared to the start of this year for a single team, led by a team lead, who told me:
Velocity: 3x more stories closed than last year. This team lead has been using Scrum and Kanban for years at the company, which helps compare “business value delivered”, not just PRs. They told me:
4. AI spend at Big TechSome of the most “AI-pilled” places are in Big Tech, which is also home to some of the least “AI-pilled” approaches of all. Here are details about Meta, NVIDIA, and Apple:... Subscribe to The Pragmatic Engineer to unlock the rest.Become a paying subscriber of The Pragmatic Engineer to get access to this post and other subscriber-only content. A subscription gets you:
|
Comments
Post a Comment