Growth Guide: How to Set Up, Staff and Scale a Growth Program
Metrics and Data You Need to See if You Have Good Retention To Do Examples □ Pick the right set of metrics Pick a leading indicator of revenue and repeat behavior. Don’t pick a vanity metric (like app downloads). If it’s a marketplace with two sides you need to have metrics for both the supply and the demand side.
Demand side metrics: “Rebook rate” – % of customers that rebook after the first booking “Nights booked per user” – # of nights booked per user over time Supply side metrics: “Active hosts” – % of hosts that are active (i.e. have a booking) “Bookings per active host” – # of bookings per unique active host over time across cohorts 
Demand side metrics: Rider retention – % of riders that ride after the first transaction Trips per active rider – # of rides/active rider over time Supply side metrics: Driver retention – % of drivers that drive after the first transaction Trips per active driver – # of rides/active driver over time □ Pick the right period for your cohort This will be typically be a day, a week, or a month depending on the business (shorter time periods typically make sense for younger businesses, and longer ones for more mature businesses). In Airbnb’s case given the velocity of use is low, and people don’t travel often, the focus is on measuring retention on an annual basis In Uber’s case, given the velocity of use is high and people use it often, the focus is on measuring retention on a monthly and weekly basis □ Identify an intial user action within Period 1 100% of the install base takes some action that is a leading indicator for revenue. For Airbnb, this is booking a room for at least one night (only a portion of the install base “rebook” each year) For Uber, this is equivalent to riding with Uber for the first time or driving with Uber for the first time □ Identify a follow-on user action in Period 2 Calculate the % of install base that is still engaging in that action at period 2 (the following day, week, month, or year) In the case of Airbnb, % of the install base each year that rebooked since initial action In the case of Uber, % of riders that ride with Uber every month since initial action Good Retention vs. Bad RetentionThe biggest question to ask at this point is: Is your retention good? To determine if your retention is good, run through these 3 steps: 1. Stable long-term retention: Long-term retention should be stable and parallel to the x-axis (the y-axis represents the retention metric). It is common to see a dip after the first period (e.g., month 2 for high-velocity1 products or year 2 for low-velocity2 products), but the most important thing is to make sure that the long-term retention is stable and parallel to the x-axis (see this in the Cohort Analysis graph below). 2. Long-term retention in line with “average or median” benchmarks in your specific vertical: It is important to benchmark your retention against companies in your specific vertical. For example, stable long-term retention of 10% is poor if you are a social network. 3. Newer cohorts should perform better: “Cohort” refers to the group of new customers that started using your service that particular month. Determine whether newer cohorts are performing progressively better than older cohorts. If the retention of newer cohorts are better than older cohorts, it implies that you are improving your product and value proposition. Below is an example of how Airbnb has performed–which would qualify as great retention. The graph below demonstrates stable long-term retention. Each new cohort did better than the previous one. For example, year 2 and year 3 retention rates are better than the year 1 retention rate. Airbnb’s long-term retention rate is better than the median retention from competitors in the same vertical.
It is important to benchmark your retention against companies in your specific vertical. Below, we have included average long-term retention targets for 5 business verticals. Vertical3 Period Long Term Period Long Term Target Median Social Network4 Monthly Month 12 45% – 65% 55% On-Demand Monthly Month 12 20% – 30% 22% Travel Annual Year 2 20% – 35% 29% E-commerce Monthly Month 12 10% – 25% 16% Subscription Monthly Month 12 25% – 35% 33% Once you’ve passed these checks and know that you have good retention, you can take the first steps to build your formal growth team, which we cover in the next section. Building Your Growth TeamIn the early days at a company, pretty much everyone is responsible for growth as they are solidifying product market fit, and some companies treat this as a shared responsibility even past product market fit. The reason a company forms a dedicated growth team is to pour gasoline into product market fit by launching structured experiments to drive a desired behavior/action.If you have proven sustainable retention, you can focus on building a dedicated team to improve retention even further while acquiring and activating and retaining incremental new users. Here’s the most common makeup of an initial, Year 1 Growth Team: Year 1 Growth Team = 1 Growth-focused PM + 2-3 Growth Engineers + 1-2 Growth Data Scientists When to hire your initial growth team:
Most companies made their first hire when they had about 15 engineers on the team working on product. If you have strong retention, then the Growth PM (your first growth hire) is likely to be the 3rd or 4th PM on the team. The most common mistake CEOs make is waiting too long before they hire a growth-focused PM. The trend is moving toward investing in building a growth team earlier on, with many starting to invest as soon as they have strong product market fit and retention. Additionally, there is considerable evidence supporting the argument that a formal growth team created at the right time accelerates the growth trajectory of a product. A good growth team can also play the role of “defense” really well. Launch of new features and enhancements can often go sideways and impact usage. The growth team has the ability to understand the root cause within minutes (not days) and course correct the problem and thereby limit the negative impact. Facebook’s growth team is considered one of the best at defense and this has consistently helped them differentiate from competition since the early days. Your first hires are critical as the initial team members will establish your company’s experiment framework and growth culture. 100% of growth experts refer to the first few hires as “magnets” for hiring and scaling the team. It’s no accident that many accomplished data scientists work at Stitch Fix as they are motivated to work and learn from the leadership of Eric Colson (former VP of data science and engineering from Netflix and one of the early hires Stitch Fix made). While success cannot be attributed to the growth team alone, having a growth team in place early on helps accelerate the overall growth trajectory of the company. Typically, the first growth team hire is a Product Manager (PM). We found some strong trends in PMs, Engineers and Data Scientist traits highlighted by growth experts who built successful growth teams: The Ideal Growth PM Candidate
Data-oriented: The ideal candidate is intensely data-driven and inquisitive. All of the experts we spoke with said this is a must-have. You want someone in this role who will constantly ask “Why?” – even when growth numbers are up. One of the experts we spoke to said, “The scariest day is when numbers are down, the second scariest day is when numbers are up and you don’t know why.” Prior growth experience: It’s important that the PM has experience at a company focused on driving growth in a competitive space (e.g., e-commerce, dating apps, gaming apps, social networks). This means that you won’t be recruiting out of a company like Google or Apple, as those teams didn’t scale based on competitive growth strategies. More than 90% of the experts mentioned that prior growth experience is an important characteristic for the team lead to have. Former startup founder (bonus) : Interestingly, 60% of growth experts in our interviews were former founders. Why are they great PMs? Because people who’ve started companies tend to be able to think independently, are comfortable with taking risks, and have high levels of perseverance. This is important as many experiments will fail. Existing PM (bonus): If an existing PM has the above characteristics, then you might have the opportunity to appoint them as the Growth PM (as Facebook and Slack did). The growth team has to work with all stakeholders within the company and having someone who has already built up social capital within the company can accelerate the team’s progress. This is great, but not a must have. 40% of the growth leads we spoke with were already PMs at the company prior to leading the growth team. Others, like Airbnb and Uber, hired a Growth PM especially for this function. The Ideal Growth Engineer Candidate 
Self-starter: Since a big chunk of the work involves running experiments to determine what really works, the engineer should be proactive about coming up with their own hypotheses and experiments and iterating. Similar to the growth PM, they should have infinite curiosity and constantly ask “Why?” to uncover hidden insights. Doesn’t cry over lost code: This is someone who should be very comfortable with experimentation, knowing that a large amount of work won’t make it into the final product. 3. OK doing things that don’t scale: Many of the tests will be small and without much impact– so an engineer who is fairly new – just 2-4 years of experience – might fit better with this mentality vs. someone one with many years of experience that may train toward rigid requirements and roadmaps. 4. Great communicator: Growth engineers should be particularly comfortable working with teams with several functions – design, copywriting, data, etc. The Ideal Data Scientist Candidate 
Lastly, a data scientist is a vital hire for a well-rounded growth team. Data scientists are in such demand that Airbnb announced recently that they have an internal university dedicated to training up data scientists. Fluency with experimental design and interpretation: Since growth is about running a lot of experiments — more so than other data science roles — it is important to test for this during the interview process. You can pose questions like, “Under a particular scenario, roughly how large of a sample size would you need?” and “How would you correct for multiple comparisons in this case?” You can also pull together a sample data set and run through the analysis live in a pair coding interview. Coding Ability: More so than other data science roles, growth requires more work to get & prepare data. This is simply because growth is often dealing with new data sets, and new data logging. Some suggested testing for this in an interview by doing live coding on cleaning up a data set together in Python or R. Great Communication Skills: The two most important elements of communication are (a) Communicating the results of experiments — especially what can and what cannot be validly deduced from an experiment and (b) articulating the persuasive case for investing in certain growth initiatives. Someone with a strong business background and a strong familiarity with causal inference (econometric and experimental backgrounds are ideal). To Do: Your Growth Team’s First YearOnce you have a team, there are five key initiatives you (and the team) will need to tackle in the first year. Here they are with additional detail on each below. Set absolute goal (with CEO) & Define key metrics Identify growth channels Establish systems & tools Establish user research Continue to iterate 1. Set an absolute goal and define key metrics
The most important thing is to identify your absolute goal and drive every aspect of the funnel toward improving your goal. Casey Winters, former Growth Product Lead at Pinterest, wrote an excellent post about this. What we mean by absolute is that goals cannot be percentage changes or rate changes (for example, you should not have a goal like “improve conversion rates by 10%”). The goal needs to be an absolute number. (For example, “achieve 5M first-time room nights this year”). Note that this is an absolute milestone the entire team needs to hit. An important next step is to break down an absolute goal into subgoals – for example, if Airbnb’s goal is 15M incremental room nights per year, it would need to achieve sub-goals with an absolute number of bookings from both new users and existing users. Jonathan Hsu, Partner at Social Capital (also part of Facebook’s early growth team), has shared his growth accounting equation — here’s how Airbnb’s equation would break down: [x] Room Nights = [A] Room nights from new users + [B] Room nights from existing users Similarly, Facebook’s absolute goal of monthly active users (MAU) incorporates both new and existing users. Here’s Facebook’s growth accounting equation: [x] Monthly Active Users = [A] New monthly active users + [B] Retained monthly active users + [C] Resurrected monthly active users For marketplaces, the companies would have absolute goals (and sub-goals) for both the supply and demand sides, and sometimes companies will have separate teams working on each side. For example, in the case of Airbnb the supply side metrics would include Host Activation, Quality and retention. At times, teams make the goal too unrealistic or set it in such a way that it is too easy to achieve. The most common advice from growth experts is to set a goal that is halfway between “Sandbagging” and “Too hard to achieve”. You want to set something that is a stretch, but at the same time motivate the team such that it is realistic to achieve. 100% of the growth experts said that the CEO must be aligned when setting and defining the absolute goal. The goal also needs to be communicated with the entire company so all teams are aware what the company plans to accomplish that year. Often CEOs wait too long or don’t fully endorse the goal and as a result, aligning teams within the company takes too long. This could severely hinder the growth team’s progress in the first year. 2. Identify growth channels
Once an absolute goal and subgoals have been defined, the next step is for the team to identify channels for their first few experiments. The most common framework growth experts use to identify channels is based on existing user behavior. The two key questions to ask are the following: How do customers find solutions / solve this issue today? How do your best users use your product today? Can you do something to get more such users to discover the product quickly? The below behaviors were highlighted by Linkedin’s Aatif Awan, and we share some examples of companies that used those channels. User Behavior Channels to Explore Example Companies To use the product you need to connect with another user Product itself Facebook, PayPal, Slack Existing users talk about the product Referrals, Community Uber, Airbnb, Dropbox Use search to find a solution to their pain point SEO, SEM Airbnb Look for inspiration from experts Affiliate bloggers, Pinterest, Partnerships, Content Stitch Fix, Glossier, Intercom High LTV users Paid acquisition (social, search, native, offline) Airbnb, Expedia, Uber Not every channel is relevant for all companies. Most products find 1-2 relevant channels early on that really work for them. ~70% of experts mentioned that referrals were the top channel within the first year. Over time (as brand awareness increased) other online advertising channels were more fruitful. There are some exceptions to this rule were referrals do not work as well. For example – you can’t offer a $20 discount and expect team members to persuade other team members to join Slack. 3. Establish Systems & Tools
The 4 most important elements you need to kick off a growth team are the following: Clean data set to track key metrics and goals Segmentation tools to be able to understand and segment the customer and activity at a granular level Rigorous experiment dashboard to analyze the experiment results and the statistical significance behind them Peer review process to discuss and analyze findings It is critical for teams to have the right systems and tools in place to run experiments at scale. Especially key in the first year is the experiment dashboard. Experiment dashboards are essentially a single destination to track experiments/results, and allow for easy analysis by lots of people at the company. Dashboards contain: Experiment group metrics Control group metrics A set of metrics defined to track and measure statistical significance The dashboard helps the team to run various experiments and test the results before proposing every single idea to be added to the product. As the growth team scales, the number of engineers increase and it becomes unwieldy without an experiment dashboard. A company at scale typically runs 1 experiment per growth engineer per week. With that future state in mind, it’s vital to start early with a solid growth experiment dashboard. The dashboard also becomes an invaluable archive of past experiments that is also immensely helpful when adding new team members or iterating on past experiments. 100% of the experts we spoke with emphasized their decision to build their own internal tools at scale. Initially, you can use tools like Mixpanel, Optimizely, Superset and Chartio to track your experiments. Here’s a screenshot of Airbnb’s internal experiment dashboard:
It can take several iterations to formalize the experiment dashboard. For example – one of the experts cited that the experiment dashboard was formalized after several iterations only after they had ~25 to 30 growth engineers on the team. Peer review & Individual Experimentation Teams often set up an internal experiment review process on a biweekly basis. Team members present their hypothesis and share the results of the experiment they ran to test the hypothesis. Peers ask a lot of questions to decide whether they agree or disagree with the findings. Growth teams that run 100+ experiments per year cite that only a third of their experiments turn out to be positive. Though the success rate is only 20% to 30%, the point of this exercise is to encourage engineers to take more risks. A common contention is whether engineers are allowed to run experiments independently. Companies in their early stages often encourage engineers to run growth experiments on their own. However some of them require PM oversight as they scale, especially as they get more rigid with quality standards. Another important element is to make sure you set heuristics for the growth team. Growth teams are constantly testing hypotheses and running experiments. One of the most common heuristic experts use is: “Don’t test things you wouldn’t ship to everybody” 4. Establish User ResearchData alone cannot answer all the questions. It is equally important to have user researchers in place to really understand what is happening behind the numbers. Your first 100M users will look a lot different from the second 100M users. Therefore it is important to do the following: Solicit real time feedback from users Use tools like Inspectlet to track UX Meet users outside of San Francisco, especially if it was your first core market. Other markets will look a lot different from SF Pay attention to how users use the product internationally. There may be cultural nuances in addition to language gaps (for example, people in Japan do not like to post photos of people without their permission and products may need to adapt to local taste). Document every single use case. What is perfectly normal for one group can be very different for another group of users. As you scale it is important to add dedicated user researchers to the growth team 5. Continue to Iterate
While the above roadmap items will help set the foundation for a strong growth program, a lot of the tools, processes and systems will evolve at scale. Where Should the Growth Team Sit?
This has been the biggest source of debate among companies. Facebook pioneered the concept of a separate growth team (meaning: Growth is essentially a department within the company). The rationale behind it was if they didn’t assign sole responsibility for growth of MAUs, then no one would own it. This has worked very well for Facebook, which recently hit 2 billion MAUs (the only social network in the world to have achieved this). Facebook was also really good at clarifying responsibilities across various teams. Advocates of separate growth teams cite that it is important for the Head of Growth to report directly to the CEO. However other companies like Uber, Airbnb and Slack, started with separate growth teams but later merged them with product team. Growth is not about just looking at data to drive insight. The growth team also experiments and makes subtle changes to the product to fuel growth, and this becomes increasingly important at scale. Therefore, advocates of this approach cite that it is crucial that the product and growth teams are within the same org. In these cases the Head of Growth reports to Head of Product. Traditionally, a company’s marketing team has been responsible for driving user acquisition (and the associated budget), so this is sometimes a default department in which to house a growth team. Often this evolves from prior functions that have lived in the marketing department (like performance marketing and user acquisition). In these cases, the Head of Growth would report to the Head of Marketing. The general sentiment about this approach is that the line of reporting is a bit rooted in the past, and most growth experts cited this as the least-favorable option. The commonality, regardless of their department, is that the growth team can be more than 100 cross-functional people. It is roughly composed of the following: 10% Product Managers 50% Engineers, 10%-15% Data Scientists, 10% Product Marketing, 10%-15% Designers ~5% Researchers The End Goal: Growth is in the Company’s DNAHopefully, when you’re ready to create a scalable growth program, this will be helpful. This is the newest frontier in the cross-section of marketing and product, so it’s still evolving. When done right, an amazing growth program will permeate the entire organization, making an evidence-based mindset part of the company’s DNA. If you have any other growth program advice to share, please reach out @YCombinator or @AnuHariharan on Twitter or on Hacker News. I want to thank the growth experts who have pioneered this practice and whose combined insights allowed us to draw these trends: Gustaf Alstromer, Former Product lead, growth, at Airbnb (who recently joined us as a Partner at YC) Merci-Victoria Grace, Growth Lead at Slack Ed Baker, Former Head of Growth at Uber Mike Duboe, Head of Growth at Stitch Fix Casey Winters, Former Growth Product Lead at Pinterest George Lee, Former Head of Growth at Instagram Julie Zhou, Former Head of growth at YikYak Ray Ko, Partner at Social Capital and Former Director of Growth @ Facebook Jonathan Hsu, Partner at Social Capital, Former Data Scientist @ Facebook Beau Hartshorne, Former Growth Engineer at Facebook John McDonnell, Data Science Manager at Stitch Fix Slater Stich, Former Data Scientist at Square Max Mullen, Cofounder at Instacart Fareed Mosavat, Senior PM, Growth at Slack Othman Laraki, Cofounder Color Genomics, Former VP of Product @ Twitter Dennis Goedegebuure, VP of Growth & SEO @ Fanatics And several others who opted to stay anonymous Thank you also to Sharon Pope, Nic Dardenne, Craig Cannon, Ali Rowghani, Daniel Gackle and Scott Bell for contributing to this guide. Notes 1. High velocity implies frequent usage – for example daily, weekly or even monthly.
2.Low velocity implies occasional usage -for example, annually or once in 6 months. 3.Second Measure (anonymized credit card transaction data) unless otherwise stated. 4.Business insider; http://www.businessinsider.com/whatsapp-engagement-chart-2014-2. Sign up for weekly updates from Y Combinator.
Source: blog.ycombinator.com
