The Pulse: a trend of trying to cut back on AI spend within eng departments?
The Pulse: a trend of trying to cut back on AI spend within eng departments?I talked with engineering leaders at mid-sized and large companies, where spending on AI agents is being dampened via per-engineer monthly budgets, or smart model routing efforts to cut costs
Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from The Pulse issue from two weeks ago. Full subscribers received the article below fourteen days ago. If you’ve been forwarded this email, you can subscribe here. The Pulse below is interesting, as a week after the original was sent out, OpenAI CEO Sam Altman also said how AI budgeting is a huge issue for some companies – echoing findings from this analysis. In mid-May, Uber president, Andrew McDonald, was on the Rapid Response podcast for a conversation about the ridesharing giant with host Bob Safian, who raised the lack of hoped-for efficiency leverage from AI, citing the language learning app, Duolingo.
My hunch is that pretty much every company is starting to, or will do soon, ask questions about the massive growth in AI spend; starting with AI coding tools. I talked with a few folks at larger and smaller companies about it:
There’s a new bottom-up focus on AI efficiency. Most tech companies do a variety of internal knowledge-sharing things like regular team demos, lunch-and-learn sessions, and engineering all-hands. I’ve been noticing more AI efficiency-focused sessions in the past couple of months, coming from engineers: no top-down mandate! Engineering all-hands, CTOs, and even CEOs have started to raise concerns about increasing AI token costs, and now more engineers are experimenting with cheaper models for simpler tasks, model routing, more efficient token usage, etc. I’d expect that during the next performance review and promotion cycles, engineers who helped save on token costs might be rewarded, like two years ago, when engineering teams were rewarded for saving on third-party vendor bills. For an engineer, the best way to show impact in your work is to translate it to money: revenue generated, or costs saved. With AI spending as high as (or higher than) on observability, it should be straightforward to show massive savings with smart optimizations. There’s a touch of irony in how any savings – for which there might be promotions and pay rises – will come from the places that actually did the rocketing spending. Read the full issue in the previous The Pulse. Or check out this week’s The Pulse: Did Anthropic’s new model just boost rival Codex’s market share? You’re on the free list for The Pragmatic Engineer. For the full experience, become a paying subscriber. Many readers expense this newsletter within their company’s training/learning/development budget. If you have such a budget, here’s an email you could send to your manager. This post is public, so feel free to share and forward it. If you enjoyed this post, you might enjoy my book, The Software Engineer's Guidebook: navigating senior, tech lead, staff and principal positions at tech companies and startups.
|
Comments
Post a Comment