What is Growth Engineering?
👋 Hi, this is Gergely with a subscriber-only issue of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at Big Tech and startups through the lens of engineering managers and senior engineers. If you’ve been forwarded this email, you can subscribe here. What is Growth Engineering?A deep dive into the field of growth engineering, which is often positioned between product engineering and marketing. With former head of growth engineering at MasterClass, Alexey Komissarouk
Before we start: if you’ve already filled out the What is your tech stack? survey: thank you! If you’ve not done so, your help will be greatly appreciated. It takes 5-15 minutes to complete. Those filling out will receive results before anyone else, and additional analysis from myself and Elin. Fill out this survey here. Growth engineering was barely known a decade ago, but today, most scaleups and many publicly traded tech companies have dedicated growth teams staffed by growth engineers. However, some software engineers are still suspicious of this new area because of its reputation for hacky code with little to no code coverage. For this reason and others, I thought it would be interesting to learn more from an expert who can tell us all about the practicalities of this controversial domain. So I turned to Alexey Komissarouk, who’s been in growth engineering since 2016, and was in charge of it at online education platform, MasterClass. These days, Alexey lives in Tokyo, Japan, where he advises on growth engineering and teaches the Growth Engineering course at Reforge. In today’s deep dive, Alexey covers:
The bottom of this article could be cut off in some email clients. Read the full article uninterrupted, online. With that, it’s over to Alexey: I’ll never forget the first time I made my employer a million dollars. I was running a push notification A/B test for meal delivery startup Sprig, trying to boost repeat orders.
Initial results were unpromising; the push notification was not receiving many opens. Still, I wanted to be thorough: before concluding the idea was a failure, I wrote a SQL query to compare order volume for subsequent weeks between customers in test vs control.
As it turned out, our test group “beat” the control group by around 10%:
I plugged the numbers into a significance calculator, which showed it was statistically significant – or “stat-sig” – and therefore highly unlikely to be a coincidence. This meant we had a winner on our hands! But how meaningful was it, really, and what would adding the push notification mean for revenue, if rolled out to 100% of users? It turned out this experiment created an additional $1.5 million dollars, annually, with just one push notification. Wow! I was hooked. Since that day, I've shipped hundreds of experimental “winners” which generated hundreds of millions of incremental revenue for my employers. But you never forget the first one. Moments like this is what growth engineering is all about. 1. What is Growth Engineering?Essentially, growth engineering is the writing of code to make a company money. Of course, all code produced by a business on some level serves this purpose, but while Product Engineers focus on creating a Product worth paying for, Growth Engineers instead focus on making that good product have a good business. To this end, they focus on optimizing and refining key parts of the customer journey, such as:
What kinds of companies employ Growth Engineers? Places you’ve heard of, like Meta, LinkedIn, DoorDash, Coinbase, and Dropbox, are some of the ones I’ve had students from. There’s also OpenAI, Uber, Tiktok, Tinder, Airbnb, Pinterest… the list of high-profile companies goes on. Most newer public consumer companies you’ve heard have a growth engineering org, too. Typically, growth engineering orgs are started by companies at Series B stage and beyond, so long as they are selling to either consumers or businesses via SaaS. These are often places trying to grow extremely fast, and have enough software engineers that some can focus purely on growth. Before the Series B stage, a team is unlikely to be ready for growth for various reasons; likely that it hasn’t found product-market fit, or has no available headcount, or lacks the visitor traffic required to run A/B tests. Cost is a consideration. A fully-loaded growth team consisting of a handful of engineers, a PM, and a designer costs approximately 1 million dollars annually. To justify this, a rule of thumb is to have at least $5 million dollars in recurring revenue – a milestone often achieved at around the Series B stage. Despite the presence of growth engineering at many public consumer tech companies, the field itself is still quite new, as a discipline and as a proper title. Brief history of growth engineeringWhen I joined Opendoor in 2016, there was a head of growth but no dedicated growth engineers, but there were by the time I left in 2020. At MasterClass soon after, there was a growth org and a dozen dedicated growth engineers. So when did growth engineering originate? The story is that its origins lie at Facebook in 2007. The team was created by then-VP of platform and monetization Chamath Palihapitiya. Reforce founder and CEO Brian Balfour shares:
Rather than focus on a particular product or feature, the growth team at Facebook focused on moving the needle, and figuring out which features to work on. These days, Meta employs hundreds if not thousands of growth engineers. 2. What do Growth Engineers work on?Before we jump into concrete examples, let’s identify three primary focus areas that a growth engineer’s work usually involves.
Let’s go through all three: Business-facing workThis is the bread and butter of growth engineering, and follows a common pattern:
Experiments can lead to sweeping or barely noticeable changes. A famous “I can’t believe they needed to test this” was when Google figured out which shade of blue generates the most clicks. At MasterClass, we tested things across the spectrum:
EmpowermentOne of the best ways an engineer can move a target metric is by removing themselves as a bottleneck, so colleagues from marketing can iterate and optimize freely. To this end, growth engineers can either build internal tools or integrate self-serve MarTech (Marketing Technology) vendors. With the right tool, there’s a lot that marketers can do without engineering’s involvement:
We go more into detail about benefits and applications in the MarTech section of Tech Stack, below. Platform workAs a business scales, dedicated platform teams help improve stability and velocity for the teams they support. Within growth, this often includes initiatives like:
When I worked at MasterClass, having monitoring at the ad layer prevented at least one six-figure incident. One Friday, a marketer accidentally broadened the audience for a particular ad from US-only, to worldwide. In response, the Facebook Ad algorithm went on a spending spree, bringing in plenty of visitors from places like Brazil and India, whom we knew from past experience were unlikely to purchase the product. Fortunately, our monitoring noticed the low-performing campaign within minutes, and an alert was sent to the growth engineer on-call, who immediately reached out to the marketer and confirmed the change was unintentional, and then shut down the campaign. Without this monitoring, a subtle targeting error like this could have gone unnoticed all weekend and would have eaten up $100,000+ of marketing budget. This episode shows that platform investment can benefit everyone; and since growth needs them most, it’s often the growth platform engineering team which implements them. As the day-to-day work of a Growth Engineer shows, A/B tests are a critical tool to both measure success and learn. It’s a numbers game: the more A/B tests a team can run in a given quarter, the more of them will end up winners, making the team successful. It’s no wonder, then, that Growth Engineering will pull out all the stops to improve velocity. 3. Why Growth Engineers move faster than Product EngineersOn the surface, growth engineering teams look like product engineering ones; writing code, shipping pull requests, monitoring on-call, etc. So how do they move so much faster? The big reason lies in philosophy and focus, not technology. To quote Elena Verna, head of growth at Dropbox:
Real-world case: price changes at MasterclassA few years ago at MasterClass, the growth team wanted to see if changing our pricing model to multiple tiers would improve revenue.
From a software engineering perspective, this was a highly complex project because:
The software engineering team estimated that becoming a “multi-pricing-tier” company would take months across numerous engineering teams, and engineering leadership was unwilling to greenlight that significant investment. We in growth engineering took this as a challenge. As usual, our goal was not just to add the new pricing model, but to learn how much money it might bring in. The approach we ended up proposing was a Fake Door test, which involves offering a not-yet-available option to customers to gauge interest level. This was risky, as taking a customer who’s ready to pay and telling them to join some kind of waiting list is a colossal waste, and risks making them feel like the target of a “bait and switch” trick. We found a way. The key insight was that people are only offended about a “bait and switch”, if the “switch” is worse than the “bait.” Telling customers they would pay $100 and then switching to $150 would cause a riot, but starting at $150 and then saying “just kidding, it’s only $100” is a pleasant surprise.
So long as every test “pricing tier” is less appealing – higher prices, fewer features – than the current offering, we could “upgrade” customers after their initial selection. A customer choosing the cheapest tier gets extra features at no extra cost, while a customer choosing a more expensive tier is offered a discount.
The best thing about this was that no backend changes were required. There were no real, new, back-end pricing plans; everybody ended up purchasing the same version of MasterClass for the same price, with the same features. The entirety of the engineering work was on building a new pricing page, and the “congratulations, you’ve been upgraded” popup. This took just a few days. Within a couple of weeks, we had enough data to be confident the financial upside of moving to a multi-pricing-tier model would be significant. With this, we’re able to convince the rest of engineering’s leadership to invest in building the feature properly. In the end, launching multiple pricing tiers turned out to be one of the biggest revenue wins of the year. Building a skyscraper vs building a tentThe MasterClass example demonstrates the spirit of growth engineering; focusing on building to learn, instead of building to last. Consider building skyscrapers versus tents. Building a tent optimizes for speed of set-up and tear-down over longevity. You don’t think of a tent as one that is shoddy or low-quality compared to skyscrapers: it’s not even the same category of buildings! Growth engineers maximize use of lightweight materials. To stick with the tents vs skyscraper metaphor: we prioritize lightweight fabric materials over steel and concrete whenever possible. We only resort to traditional building materials when there’s no other choice, or when a direction is confirmed as correct. Quality is important – after all, a tent must keep out rain and mosquitoes. However, the speed-vs-durability tradeoff decision results in very different approaches and outcomes. 4. Tech stackAt first glance, growth and product engineers use the same tooling, and contribute to the same codebases. But growth engineering tends to be high-velocity, experiment-heavy, and with limited test coverage. This means that certain “nice to have” tools for product engineering are mission-critical for growth engineers... Subscribe to The Pragmatic Engineer to unlock the rest.Become a paying subscriber of The Pragmatic Engineer to get access to this post and other subscriber-only content. A subscription gets you:
|
Comments
Post a Comment