Context engineering with Dex Horthy
Context engineering with Dex HorthyContext engineering is increasingly table stakes when building with AI. Dex Horthy coined the term "context engineering," and walks us through things worth knowing, as a dev
Stream the latest episodeListen and watch now on YouTube, Spotify, and Apple. See the episode transcript at the top of this page, and timestamps for the episode at the bottom. Brought to You by
• Antithesis — with Antithesis, you can use AI agents to work on critical systems without worrying about correctness. Going far beyond code review, you can run your complete system in a hostile environment, analyze its behavior, and reproduce every issue perfectly. Teams like Jane Street, Fly.io, and the etcd community use Antithesis to ship better code, faster. Learn more. • Buildkite — the CI platform trusted by OpenAI, Anthropic, Cursor, Meta, Uber, Ramp, Nvidia, Airbnb and many more. Buildkite was stress-tested at the largest scale inside companies solving some of the hardest engineering problems. It’s built to reliably manage whatever your coding agents throw at the build queue, today, next year and beyond. Learn more. • Sentry – application monitoring software built by developers, for developers. Sentry’s Seer AI agent is one of their new, neat tools, as a way to debug errors faster. Same with Sentry’s MCP server. Check out Sentry. In this episodeKnowing how LLM contexts work and how to work around context limitations – aka “context engineering” – is becoming more important for software engineers working with LLMs. Let’s look into what works and what doesn’t, today. In this episode of The Pragmatic Engineer podcast, I sit down with the CEO and cofounder of HumanLayer, Dex Horthy, who coined the term “context engineering”. We discuss the ideas behind this context engineering, harness engineering, loop engineering, software factories, why his approach to AI-assisted software development has evolved, and how HumanLayer is helping engineering teams automate more of the software development lifecycle without sacrificing code quality. Key observations from DexHere are 12 useful points Dex made in our conversation: 
1. Dex talked with ~100 “real” AI Engineers and wrote the popular ‘12-Factor Agents - Principles for building reliable LLM applications’ based on what he learned. Around August 2024, he started to build AI agents when the common approach was to use frameworks like LangChain and CrewAI, and also talked with around 100 AI engineers doing tangible work such as taking on $100K+ contracts to ship AI solutions within enterprises. They tried those frameworks and discarded them in favor of building their own pipelines. Dex shared the learnings from the conversations in his hit book, 12-Factor Agents. It’s a great read! 2. Lesson learned: Shipping unread code spells disaster within months. Dex experimented with having the model write the code and humans not reviewing anything in July 2025. Four months later, they shut things down and threw the whole system out. Production broke, and no matter how much the team prompted Opus 4.1, the model could not find the root cause. It took days of wading through spaghetti code to discover the primary key wrongly routed through the complete codebase. Once fixed, it took three weeks to re-onboard to a codebase no human had ever read. Today, Dex thinks this problem wouldn’t even take four months to develop, now that newer models produce code a lot faster than a year ago. 3. Today’s coding models are most likely trained in a way that makes codebases worse over time. Dex believes that the reason we see LLMs “degrade” existing codebases is because they are optimized to do well on SWE-bench-style benchmarks. These benchmarks reward reproducing a known fix in codebases like Django, but cannot measure poor architecture decisions. This is because the cost function of bad architecture and bad program design cannot be evaluated by running a unit test. It’s a tricky problem to solve: Dex’s best guess is an eval of a model building 20 features in a row in a codebase without knowing what’s coming next. 4. Context engineering 101: figure out where the “dumb zone” begins. As a rule of thumb, the less of the context window that is used, the better the outcomes are. This is because the attention mechanism is quadratic: the more that goes into the context window, the more compute is required to process it all. We cover more about self-attention scalability challenges in our ChatGPT deepdive. For a model with a 1M context window, Dex pushes it to around 300-400K when it feels right. For smaller models, he stops at around 100K. You hit the “dumb zone” when its performance starts to degrade because the context window fills up beyond this heuristic limit, and the model begins doing increasingly stupid things like deleting your .env file, for example. 5. A larger context window does not mean a smarter model. Models’ intelligence is behind the ability to use the tokens in the context window, by deciding which parts of the context are relevant for the next decision. You have to get a feel for how much context usage makes sense, and experiment when you hit the “dumb zone.” 6. Frequent, intentional compaction is a technique Dex uses for more complex projects. He will take a long and noisy context, compress it into a Markdown document, then start a new session fresh, pointing the model to this “compressed context” that is in the Markdown. A workflow he uses:
7. Don’t bother optimizing LLM usage until business is booming, there’s massive scale, or high costs. Dex suggests to always start building software with the smartest available model to solve the problem, since engineering time is almost always the bottleneck. Begin to optimize LLM usage and context usage only when at real scale and costs are high enough. That’s when it can be worth using GPT-OSS-120B (1/1,000th the cost of Opus) for the simpler steps in your process. 8. “You’re completely right!” or “you’re right to push back on that” are phrases that mean it’s time to start a new session. These responses mean the LLM session is trajectory-poisoned, and you’re wasting time and tokens to continue. Models are autoregressive, so if you get into this loop of:
9. Only four things matter in the context window:
10. Slow loops are Dex’s favorite way to do ‘loop engineering.’ The HumanLayer team started with a nightly automation setup that kicks off an agent to fix one thing in the codebase, and open a pull request. In the mornings, they woke up to a PR waiting to be merged. They tweaked it over time and now have four agents open a total of four PRs by the morning, with the focus on code quality improvements. A person still reads all of them before merging. 11. “Token harder” vs. “token smarter”: Dex is in a group chat named ‘Hyper Engineering’, where members share advice on how to max out their Claude subscriptions. This approach, he calls “token harder”. On the other side is “token smarter”: aiming to get maximum value from AI while keeping control. Smarter is harder to pull off. 12. Three ways to run a “software factory.” Here’s options Dex sees as viable:
The Pragmatic Engineer deepdives relevant for this episode• How Uber uses AI for development: inside look • Are AI agents actually slowing us down? • AI Tooling for Software Engineers in 2026 • Vibe Coding as a software engineer • AI Engineering in the real world • How AI-assisted coding will change software engineering: hard truths • The creator of OpenClaw: “I ship code I don’t read” Timestamps00:00 Intro 01:33 Dex’s path into tech 03:34 Early work in platform engineering 05:28 Replicated 11:24 Metalytics 12:36 12-factor agents 18:27 Context engineering 23:38 Harness engineering 26:11 Context overload 30:45 Loop engineering 44:34 Software factories before and after AI 50:33 Automation limits 55:18 Three options for automating 59:00 RPI framework 1:04:16 Intentional compaction 1:11:48 Token harder vs. token smarter 1:16:44 AI slop 1:19:15 HumanLayer 1:29:09 Book recommendation ReferencesWhere to find Dex Horthy: • LinkedIn: linkedin.com/in/dexterihorthy • Website: https://www.humanlayer.dev Mentions during the episode: • HumanLayer: https://www.humanlayer.dev • Jet Propulsion Laboratory (JPL): https://www.jpl.nasa.gov • Dykstra’s projection algorithm: https://en.wikipedia.org/wiki/Dykstra’s_projection_algorithm • Replicated: https://www.replicated.com • Docker: https://www.docker.com • HashiCorp: https://www.hashicorp.com • DataStax: https://www.ibm.com/products/datastax • Puppet: https://www.puppet.com • Travis CI: https://www.travis-ci.com • Circle CI: https://circleci.com • Randy Newman’s website: https://www.randynewman.com • 12-Factor Agents - Principles for building reliable LLM applications: https://github.com/humanlayer/12-factor-agents • The creator of Clawd: “I ship code I don’t read”: https://newsletter.pragmaticengineer.com/p/the-creator-of-clawd-i-ship-code • Vaibhav Gupta on LinkedIn: https://www.linkedin.com/in/vaigup • Tobi Lutke’s post on X about context engineering: • Andrej Karpathy’s post on X about context engineering:
• Improving Deep Agents with harness engineering: https://www.langchain.com/blog/improving-deep-agents-with-harness-engineering • # Skill Issue: Harness Engineering for Coding Agents: https://www.humanlayer.dev/blog/skill-issue-harness-engineering-for-coding-agents • Harness engineering for coding agent users: https://martinfowler.com/articles/harness-engineering.html • How AI will change software engineering – with Martin Fowler: https://newsletter.pragmaticengineer.com/p/martin-fowler • Dex’s post on X about context reality check:
• Laurie Voss on LinkedIn: https://www.linkedin.com/in/seldo • Everything is a ralph loop: https://ghuntley.com/loop • Dex’s post on X about feedback loops:
• Dex’s post on X about token harder vs. token smarter:
• Dex’s post on X about AI slop:
• GitHub: https://github.com • Paul Graham, Live from Stockholm: https://www.ycombinator.com/library/Q7-paul-graham-live-from-stockholm • Refactoring: Improving the Design of Existing Code: https://www.amazon.com/dp/0134757599 • Clean Code: A Handbook of Agile Software Craftsmanship: https://www.amazon.com/dp/0132350882 • The Pragmatic Programmer: From Journeyman to Master: https://www.amazon.com/dp/020161622X — Production and marketing by Pen Name. You’re on the free list for The Pragmatic Engineer. For the full experience, become a paying subscriber. Many readers expense this newsletter within their company’s training/learning/development budget. If you have such a budget, here’s an email you could send to your manager. This post is public, so feel free to share and forward it. If you enjoyed this post, you might enjoy my book, The Software Engineer's Guidebook: navigating senior, tech lead, staff and principal positions at tech companies and startups.
|
%26stroke%3Dnone%26strokeWidth%3D3.6){kind=small}
Comments
Post a Comment