The Pulse: tech companies move to open AI models
The Pulse: tech companies move to open AI modelsUber, Pinterest, Stripe, Coinbase, Ramp, and AT&T are making large savings on their AI bills by dropping proprietary models and using smart model routing.
Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of five topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here. Before we start, as a reminder, I’ll be in New York, next week. Should you be in the city, you can meet me at:
In May, I covered an emerging trend of companies wanting to cut back their AI spending, starting with engineering departments. Different approaches were being tried:
A few months later, it seems that several companies have managed to achieve this, according to sources I’ve spoken with. Uber: AI costs down 50%Uber managed to blow through its annual AI budget in the first three months of this year, and it wasn’t a surprise to hear, in May, Uber’s COO say that it was getting harder to justify spending on tools like Claude Code without seeing benefits from the leading models. It wasn’t long until engineering teams at the ridesharing giant set to work on how to optimize AI spend, and their efforts weren’t in vain. Uber cut the cost per AI request by 34%, and the cost per AI session by 52%:
Of course, Uber keeps using more AI tokens and starting more AI sessions, but thanks to optimizations the cost has been flat since March, despite significantly more usage:
How did they pull it off at Uber? It was via a combination of different approaches:
From the outside, the single biggest win seems to be Uber’s transition to using open models for certain tasks. Open models cost 2-20x less, compared to frontier ones:
Pinterest makes 90%+ cost savings by dropping frontier modelsInteresting details from Pinterest’s earnings call last month reveal how much the social media platform saves by running open models. Here’s what CEO William Ready said (emphasis mine):
Basically, what used to cost Pinterest $100 to run on a closed, frontier model, they now spend $8 on by using open models on owned or rented inference! AT&T: 56% savings by swapping Claude for open modelsWith 100,000 employees, AT&T is a big spender on AI. The telco giant cut its AI bill by 56% while measuring a 2% decrease in the quality of AI’s output, after they moved workloads over to open models. From The Information:
Anthropic overpriced compared to the rest of the market?Only a few months ago, Anthropic was the preferred model (Claude) and harness (Claude Code) among engineers. But Anthropic’s models are becoming steeply more expensive at a time when open weight models – and also OpenAI – are getting much cheaper. Meanwhile, Opus 5 is 100x more expensive (!!) than models like GPT-5.6 Luna xhigh and DeepSeek. That may be simply too much to ignore for some tech companies:
Seeing this data, I’m not surprised that more tech companies are looking to run open weight providers on inference providers, due to the significant savings available from a model that’s similarly capable as one from Anthropic. What worked for Stripe, Coinbase, Uber & RampThe engineering team at Databricks interviewed engineers at Stripe, Coinbase, Uber, and Ramp, and collected how different approaches helped save costs for them. The summary:
To answer the question posed in the header of this report, it’s apparent that using open models is indeed the approach offering the biggest savings, followed by smart model routing. Spending controls and context optimization also bear down on costs, but they don’t come close to the first two techniques in results. A week after publishing this article, Ara Krahzian at Ramp has confirmed that AI spend in August, has, indeed, declined at the top 1% of businesses by 10%, based on Ramp data:
Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:
You’re on the free list for The Pragmatic Engineer. For the full experience, become a paying subscriber. Many readers expense this newsletter within their company’s training/learning/development budget. If you have such a budget, here’s an email you could send to your manager. This post is public, so feel free to share and forward it. If you enjoyed this post, you might enjoy my book, The Software Engineer's Guidebook: navigating senior, tech lead, staff and principal positions at tech companies and startups.
|
Comments
Post a Comment