# data[].attributes.attachment_link.readable_text
For a product to achieve its full potential and create the most value for its users, it must grow sustainably and reach a high level of penetration with respect to the total addressable market.

Ideally, the users engage deeply with the product forever and keep coming back to the product. That's retention  - the measure of people who tried the product and liked it enough to return.

The product is also sticky when people keep coming back to the product. While stickiness generally drives retention, a product that is retaining and growing well may not necessarily be sticky. Retention is simply about users returning to your product; stickiness is about them returning of their own volition. Stickiness helps reduce your dependency on tactics such as push notifications. For example, Facebook is sticky because of users' urge to share, and because of their curiosity about other people's lives.

Engagement is key to the success of any product, engaged users return for repeat experiences - driving stronger retention and ultimately helps a product grow in a sustainable way. Ultimately, engagement drives stickiness, which drives retention - and that, in turn, drives growth. Therefore, systematically understanding and improving engagement will increase a product's health.

FRAMEWORK

In order to understand engagement across multiple types of consumer companies, it would be great to have a common framework. Many consumer technology companies can be thought of as two-sided marketplaces. These are organizations that create value primarily by enabling direct interactions between two (or more) distinct types of affiliated customers. American Express, PayPal, eBay, Uber, Facebook, iPhone, WhatsApp, Netflix, Amazon, and YouTube can all be considered as two-sided marketplaces.

These platforms exist because there is a need for an intermediary to match the supply and demand sides of the platform in a more efficient way. This intermediary will make possible exchanges that would not occur without them and create value for both sides. PayPal payment networks bring together cardholders and retailers; Amazon brings together shoppers and retailers; YouTube and Netflix connect video creators with viewers, and Facebook connects content providers with content consumers. In the case of YouTube, Netflix, and Amazon - the two-sided network broadly resembles the production/consumption model that we discuss at length.

While some strategies for generating engagement apply across multiple types of products, others are specific to certain types. For example, products that offer user-generated content (UGC) may benefit from different strategies and levers to affect engagement, compared to products that offer professionally generated content (PGC). These two types are discussed and contrasted in more detail below. Regardless of type, the health of most consumer products is driven by the production-consumption loop, which allows you to surface the right content to users and drive engagement. See the two figures below that depicts the scenarios for Facebook and Netflix.

NETWORK EFFECTS

A network effect is where an additional user of a good or service increases the value of that product to others. When a network effect is present, the value of a product or service (generally) increases according to the number of others using it. The classic example is the telephone, where a greater number of users increases the value to each. This is also true with product like Facebook or Snapchat where people are producing content on one side and consuming content on the other side.

With greater number of users, there are more people that you can connect with and there is a greater likelihood of you to be engaged and retained. So, in general, there is a positive network effect in the case of WhatsApp, Facebook Messenger, Instagram Direct and Snapchat.

In the case of Facebook, as the number of users increase, on one hand, when users provide feedback on the content they consume, that feedback influences ranking algorithms and can drive additional engagement. On the other hand, in a News Feed model, everyone can see explicit feedback which increases comparative pressure, which would discourage people to share more. So, this would create a negative network effect. I.e., as the number of users increases, the amount of content produced can drop which would eventually lead to lower engagement.

In contrast, a product like Snapchat, which uses a story model, makes that feedback visible only to the producer. The story model can thus reduce comparative pressure and, by extension, encourage users to post more.

In the case of a PGC product like Netflix, the network effect is more indirect. As there are more users, more original content gets produced; more content gets acquired and more inventory is available for all. As there is more content available, more users join the service.

PENETRATION

The percentage of the target market that are active users of the product is known as product penetration. In the case of social products, the target population is the country's population over 12. Penetration of a country has a strong impact on engagement in products where there are direct network effects,

Figure 4 shows Snapchat's retention vs. engagement and demonstrates a common trend among social apps: countries in which user penetration (represented by bubble size) is higher also tend to see much higher engagement, which in turn increases retention. Because a user in a high-penetration country tends to have more connections, they see many Snaps and stories every time they open the app, and that high inventory gives the user reason to return to Snapchat consistently. Thus, high user engagement drives long-term retention and product growth.

In contrast, products that do not have direct network effects are weakly impacted. This is largely because economies of scale take time to develop which ultimately drives greater engagement.

CONTENT PRODUCTION AND SHARING

In social apps like Snapchat, there are three phases of content production  - creation, capture and sharing - and four primary kinds of creators: friends, groups, pages, and news. As combined monthly active users for apps such as Snapchat, Instagram and Facebook have climbed to exceed 3.5 billion, social networks are seeing increases in creation and capture of UGC, and relatively high engagement levels follow.

Content creation is the engine that drives engagement for social apps. In the example of Snapchat, a useful metric is the number of Snaps a day. The most active Snapchat users sent around 150 snaps a day, while the average user sent 20-50 snaps a day. Sharing content is incredibly important for continued growth and engagement, but several factors impact sharing, including permanence of the content, size of a creator's audience, the perceived quality of other people's posts in a creator's feed etc.

In contrast, in the case of Netflix (or in general, professional content), none of the factors mentioned above matters for increased content production or engagement.

CONNECTIONS AND RANKING

In the case of a product like Facebook, once a larger number of posts are being generated, it's important to make the best connections between a user and other users or entities, given that user's available inventory. The more connections a given user has, the more inventory with which they can engage. As inventory is gated in these types of systems, "needy" users that have a low content inventory, generally have low engagement. Snapchat, for example, emphasizes the need for users to make connections early. Because 88 percent of Snaps are sent to just one other person, these connections are critical to building content inventory. Without connections, content created will not be viewed, nor will it receive feedback, which will ultimately discourage a user to continue creating content. Ranking the content in the right order is also very important for higher engagement.

In contrast, all the inventory is equally available to users in products like Netflix. Thus show the right recommendations in the right order and having a great search product that can infer the intent of the query is very important.

Given key signals, a machine learning model can determine with some degree of confidence whether a user will interact more with, for example, an article about the musician Shawn Mendes or one about an upcoming election. However, such signals are only a rough proxy for a user's true feelings, and blindly following some signals can lead you to optimize for virality over quality. All optimization functions have limitations - data will never completely model a person. Therefore, product teams should always fully understand the recommendations such systems make and ensure those recommendations comport with the broader goals of the team and company.

SUPPLY, DEMAND, LIQUIDITY, AND OTHER IMPLICATIONS

There are two sides to a marketplace - demand side and the supply side. When a home is listed on AirBnB, it creates supply. When you are traveling to Hawaii looking for a home to rent, it creates demand for the home. From a product perspective, early on in a product's lifecycle, seeding the right supply on a two-sided market requires an act of magic. It requires convincing the best supply possible that you will soon have demand and that the initial time-cost of the sign-up and onboarding process will be well worth it. Then within this tiny window when you have both interest and mindshare of quality supply, you need to convince some customers to engage. This is very hard and one of the main reasons why most marketplaces fail.

Having a unique supply creates a competitive moat. For example, the content shared by your closest friend on Facebook cannot be found anywhere else. So, this piece of high-quality content is very valuable. In contrast, when you are searching for a hotel room which is available across multiple travel websites (commodity supply), the most important criteria for the demand side is generally pricing, and due to competitive pressure, margins are generally reduced. Building brand and loyalty are some ways to overcome this pressure. Incentivizing the production of unique high-quality supply, creating a great brand and loyalty are very important considerations in building a strong two-sided marketplace.

To measure the health of the marketplace, a product owner needs to deeply understand the liquidity in the marketplace. Inventory, utilization, sell-through-rates, and demand and supply constraints at a granular level will help identify where the opportunities lie and help drive roadmap and strategy for the products. For example, in the case of 'needy users' (those with low inventory in UGC products) supply is low, and a potential product lever is to increase this user's supply by inducing them to add connections, thereby increasing their engagement.

The two sides of demand and supply need to be incentivized to use the platform to create strong network effects. As an example, a two-sided cleaning service marketplace that connects a cleaner to a home will work the first time. But over a period of time, once the cleaner has made enough connections, they are unlikely to use the platform. The platform needs to create unique and recurring value to both parties to retain them on the system.

Similarly, competition, user behavior, and other extrinsic considerations may shift the levels of supply and demand - the presence of WhatsApp, Instagram and Snapchat have definitely changed the value proposition of Facebook and the core product is used differently now than it was before. It is critical to understand the value proposition on both the supply and demand sides and how that evolves over time. This should inform future product roadmap and strategy.

NEXT STEPS

This will be our last Medium post before the New Year. We wish all of you a very happy holiday and the New Year. We will back in the middle of January with new content. Our next few areas of focus will be on building world-class data science teams; data platforms and experimentation and driving data-informed roadmap and strategy.

Two-sided marketplace is a good framework for understanding the engagement of consumer-facing internet products.

Surfacing relevant content via feed ranking or suggested content is a key driver of engagement in any consumer system.

TAKEAWAYS

# data[].attributes.attachment_link.description
For a product to achieve its full potential and create the most value for its users, it must grow sustainably and reach a high level of penetration with respect to the total addressable market. Ideally, the users engage deeply with the product forever and keep coming back to the product.

# data[].attributes.attachment_link.readable_text
This article is by Rahul Vohra, the founder and CEO of Superhuman - a startup building the fastest email experience in the world.

We've all heard that product/market fit drives startup success - and that the lack thereof is what's lurking behind almost every failure.

For founders, achieving product/market fit is an obsession from day one. It's both the hefty hurdle we're racing to clear and the festering fear keeping us up at night, worried that we'll never make it. But when it comes to understanding what product/market fit really is and how to get there, most of us quickly realize that there isn't a battle-tested approach.

In the summer of 2017, I was waist-deep in my search for a way to find product/market fit for my startup, Superhuman. Turning to the classic blog posts and seminal thought pieces, a few observations stuck out to me. Y Combinator founder Paul Graham described product/market fit as when you've made something that people want, while Sam Altman characterized it as when users spontaneously tell other people to use your product. But of course, the most cited description comes from this passage in Marc Andreessen's 2007 blog post:
"You can always feel when product/market fit is not happening. The customers aren't quite getting value out of the product, word of mouth isn't spreading, usage isn't growing that fast, press reviews are kind of 'blah,' the sales cycle takes too long, and lots of deals never close. And you can always feel product/market fit when it is happening. The customers are buying the product just as fast as you can make it - or usage is growing just as fast as you can add more servers. Money from customers is piling up in your company checking account. You're hiring sales and customer support staff as fast as you can. Reporters are calling because they've heard about your hot new thing and they want to talk to you about it. You start getting entrepreneur of the year awards from Harvard Business School. Investment bankers are staking out your house."
For me, this was the most vivid definition - and one that I stared at through tears.

We had set up shop and started coding Superhuman in 2015. A year later, our team had grown to seven and we were still furiously coding. By the summer of 2017, we had reached 14 people - and we were still coding. I felt intense pressure to launch, from the team and also from within myself. My previous startup, Rapportive, had launched, scaled and been acquired by LinkedIn in less time. Yet here we were, two years in, and we had not passed go.

But no matter how intense the pressure, I wasn't ready to launch. Common practice would be to "throw it out there and see what sticks," which may be fine after a few months of effort when the sunk cost is low. But the "launch and see what happens" method seemed irresponsible and reckless to me - especially given the years that that we had invested.

Further compounding the pressure, as a founder, I couldn't just tell the team how I felt. These super-ambitious engineers had poured their hearts and souls into the product. I had no way of telling the team we weren't ready, and worse yet, no strategy for getting out of the situation - which is not something they would want to hear. I wanted to find the right language or framework to articulate our current position and convey the next steps that would get us to product/market fit, but was struggling to do so.

That's because the descriptions of product/market fit I found were immensely helpful for companies post-launch. If, after launch, revenue isn't growing, raising money is tough, the press doesn't want to talk to you and user growth is anemic, then you can safely conclude you don't have product/market fit. But in practice, because of my previous success as a founder, we didn't have problems raising money. We could have gotten press, but we were actively avoiding it. And user growth wasn't happening because we deliberately choosing not to onboard more users. We were pre-launch - and we didn't have any indicators to clearly illustrate our situation.

The descriptions of product/market fit all seemed so post hoc, so unactionable. I had a clear understanding of where we stood, but I had no way of conveying that to others - and no plan for the part that should come next.

So I racked my brain for an answer on how to travel the distance between where Superhuman was and the high bar that we needed to hit. And I eventually started to wonder: what if you could measure product/market fit? Because if you could measure product/market fit, then maybe you could optimize it. And then maybe you could systematically increase product/market fit until you achieved it.

Reoriented around this purpose and reinvigorated by the new direction, I set out to reverse engineer a process for getting to product/market fit. Below, I outline the findings that followed, specifically unpacking the clarifying metric that made everything fall into place and the four-step process we used to build an engine that propelled Superhuman forward on the path to finding our fit.

ANCHORING AROUND A METRIC: A LEADING INDICATOR FOR PRODUCT/MARKET FIT

On my quest to understand product/market fit, I read all I could and spoke with every expert I could find. Everything changed when I found Sean Ellis, who ran early early growth in the early days of Dropbox, LogMeIn, and Eventbrite and later coined the term "growth hacker."

The product/market fit definitions I had found were vivid and compelling, but they were lagging indicators - by the time investment bankers are staking out your house, you already have product/market fit. Instead, Ellis had found a leading indicator: just ask users "how would you feel if you could no longer use the product?" and measure the percent who answer "very disappointed."

After benchmarking nearly a hundred startups with his customer development survey, Ellis found that the magic number was 40%. Companies that struggled to find growth almost always had less than 40% of users respond "very disappointed," whereas companies with strong traction almost always exceeded that threshold.

Ask your users how they'd feel if they could no longer use your product. The group that answers 'very disappointed' will unlock product/market fit.

A helpful example comes from Hiten Shah, who posed Ellis' question to 731 Slack users in a 2015 open research project. 51% of these users responded that they would be very disappointed without Slack, revealing that the product had indeed reached product/market fit when it had around half a million paying users. Today, this isn't too surprising, given Slack's legendary success story. Truly, this example shows just how hard it is to beat the 40% benchmark.

Inspired by this approach, we set out to measure what the responses would be for Superhuman. We identified users who recently experienced the core of our product, following Ellis' recommendation to focus on those who used the product at least twice in the last two weeks. (At the time we had between 100 and 200 users to poll, but smaller, earlier-stage startups shouldn't shy away from this tactic - you start to get directionally correct results around 40 respondents, which is much less than most people think.)

We then emailed these users a link to a Typeform survey asking the following four questions:

1. How would you feel if you could no longer use Superhuman?
A) Very disappointed
B) Somewhat disappointed
C) Not disappointed

2. What type of people do you think would most benefit from Superhuman?

3. What is the main benefit you receive from Superhuman?

4. How can we improve Superhuman for you?

With the responses collected, we analyzed the first question:

With only 22% opting for the "very disappointed" answer, it was clear that Superhuman had not reached product/market fit. And while this result may seem disheartening, I was instead energized. I had a tool to explain our situation to the team and - most excitingly - a plan to boost our product/market fit

FROM BENCHMARK TO ENGINE: THE FOUR-STEP MANUAL FOR OPTIMIZING PRODUCT/MARKET FIT

Determined to move the needle, I became singularly focused on ways to improve our product/market fit score. The responses to each survey question would be key ingredients in what became the framework for fulfilling our goal.

Here are the four components that comprised our product/market fit engine:

1) Segment to find your supporters and paint a picture of your high-expectation customers.

With your early marketing, you may have attracted all kinds of users - especially if you've had press and your product is free in some way. But many of those people won't be well-qualified; they don't have a real need for your product and its main benefit or use case might not be a great fit. You wouldn't have wanted these folks as users anyway.

As an early-stage team, you could just narrow the market with preconceived notions of who yo u think the product is for, but that won't teach you anything new. If you instead use the "very disappointed" group of survey respondents as a lens to narrow the market, the data can speak for itself - and you may even uncover different markets where your product resonates very strongly.

For me, the goal of segmenting was to find pockets in which Superhuman might have better product/market fit, those areas I may have overlooked or didn't think to scope down to.

To start, we grouped the survey responses by their answer to the first question ("How would you feel if you could no longer use Superhuman?"):

We then assigned a persona to each person who filled out a survey.

Next, we looked at the personas that appeared in the very disappointed group - the 22% that were our biggest supporters - and used those to narrow the market. In this simplified example, you can see we focused on founders, managers, executives and business development  - temporarily ignoring all other personas.

With this more segmented view of our data, the numbers shifted. By segmenting down to the very disappointed group that loved our product most, our product/market fit score jumped by 10%. We weren't quite at that coveted 40% yet, but we were a lot closer with minimal effort.

To go even deeper, I wanted to better understand these users who really loved our product. I hoped to paint as vivid a picture of them as possible, so I could galvanize the whole team to serve them better.

I turned to Julie Supan's high-expectation customer framework as a tool to do just that. Supan notes that the high-expectation customer (HXC) isn't an all encompassing persona, but rather the most discerning person within your target demographic. Most importantly, they will enjoy your product for its greatest benefit and help spread the word. For example, Airbnb's HXC doesn't simply want to visit new places, but wants to belong. For Dropbox, the HXC wants to stay organized, simplify their life, and keep their life's work safe.

With this in mind, I sought to pinpoint Superhuman's HXC. We took only users who would be very disappointed without our product and analyzed their responses to the second question in our survey: "What type of people do you think would most benefit from Superhuman?"

This is a very powerful question, as happy users will almost always describe themselves, not other people, using the words that matter most to them. This lets you know who the product is working for and the language that resonates with them (providing valuable kernels of insight for your marketing copy as well).

Using our customers' words and Supan's tips for building a profile, we crafted a rich and detailed vision of the Superhuman HXC:
Nicole is a hard-working professional who deals with many people. For example, she may be an executive, founder, manager, or in business development. Nicole works long hours, and often into the weekend. She considers herself very busy, and wishes she had more time. Nicole feels as though she's productive, but she's self-aware enough to realize she could be better and will occasionally investigate ways to improve. She spends much of her work day in her inbox, reading 100-200 emails and sending 15-40 on a typically day (and as many as 80 on a very busy one). Nicole considers it part of her job to be responsive, and she prides herself on being so. She knows that being unresponsive could block her team, damage her reputation, or cause missed opportunities. She aims to get to Inbox Zero, but gets there at most two or three times a week. Very occasionally  -  perhaps once a year  -  she'll declare email bankruptcy. She generally has a growth mindset. While she's open-minded about new products and keeps up to date with technology, she may have a fixed mindset about email. Whilst open to new clients, she's skeptical that one could make her faster.
With our HXC in mind, we had a tool to focus the entire company on serving that narrow segment better than anybody else. Some may find this approach too limiting, arguing that you shouldn't narrow in on such a specific customer base early on.

It's a commonly held view that tailoring the product too narrowly to a smaller target market means that growth will hit a ceiling - but I don't think that's the case.

These words of wisdom from Paul Graham explain why:
"When a startup launches, there have to be at least some users who really need what they're making - not just people who could see themselves using it one day, but who want it urgently. Usually this initial group of users is small, for the simple reason that if there were something that large numbers of people urgently needed and that could be built with the amount of effort a startup usually puts into a version one, it would probably already exist. Which means you have to compromise on one dimension: you can either build something a large number of people want a small amount, or something a small number of people want a large amount. Choose the latter. Not all ideas of that type are good startup ideas, but nearly all good startup ideas are of that type."
In a separate post, he drives this point home even further:
"In theory this sort of hill-climbing could get a startup into trouble. They could end up on a local maximum. But in practice that never happens. ... The maxima in the space of startup ideas are not spiky and isolated. Most fairly good ideas are adjacent to even better ones."
In essence, it's better to make something that a small number of people want a large amount, rather than a product that a large number of people want a small amount. In my view, the product/market fit engine process of narrowing the market massively optimizes for a product that a small number of people want a large amount.

2) Analyze feedback to convert on-the-fence users into fanatics.

However, just winnowing down to HXCs is not enough. We had gone narrow, but now needed to dig deeper. Since we were below the 40% threshold, we needed to figure out why this smaller subset really loved Superhuman - and how we could bump up more users into this segment.

To get to the root of how we were going to improve the product and expand the depth of its appeal, I found it helpful to focus my efforts on these key questions:

To understand why users loved Superhuman, we once again turned to the segment of those who would be very disappointed without our product. This time, we looked at their answers to the third question on our survey: "What is the main benefit you receive from Superhuman?"

Here's a sample of some answers that stood out:

"Processing email is much faster with Superhuman for two reasons: show one email at a time and overall speed is much better than gmail. I get through my inbox in half the time."

"Speed! The app is crazy fast, and the UX + keyboard shortcuts make me an actual superhuman."

"Using Superhuman is so much faster than using Gmail. Not even close. And it mirrors my favorite Gmail shortcuts, so there is zero learning curve for a power Gmailer."

"I can work through incoming email more quickly, sorting messages accordingly and streamlining my work process."

"Speed. Aesthetics. I can do everything from the keyboard."

"Speed and the great set of keyboard shortcuts. I rarely, if ever, have to use the trackpad."

After throwing the responses into a word cloud, some common themes emerged: the users who loved our product most appreciated Superhuman for its speed, focus and keyboard shortcuts.

With this deeper understanding of the product's appeal, we turned our attention to figuring out how we could help more people love Superhuman.

Our next step was somewhat counterintuitive: we decided to politely pass over the feedback from users who would not be disappointed if they could no longer use the product.

This batch of not disappointed users should not impact your product strategy in any way. They'll request distracting features, present ill-fitting use cases and probably be very vocal, all before they churn out and leave you with a mangled, muddled roadmap. As surprising or painful as it may seem, don't act on their feedback - it will lead you astray on your quest for product/market fit.

Politely disregard those who would not be disappointed without your product. They are so far from loving you that they are essentially a lost cause.

That leaves the users who would be somewhat disappointed without your product. On the one hand, the 'somewhat' indicates an opening. The seed of attraction is there; maybe with some tweaks you can convince them to fall in love with your product. But on the other hand, it's entirely probable that some of these folks will never be very disappointed without your product no matter what you do.

To fine-tune who we took our cues from, we segmented once again. From analyzing our third survey question, we knew that happy Superhuman users enjoyed speed as their main benefit, so we used this as a filter for the somewhat disappointed group:

After splitting the somewhat disappointed group into two new segments around speed, here's how we decided to act on their feedback:

Somewhat disappointed users for whom speed was not the main benefit: we opted to politely disregard them, as our main benefit did not resonate. Even if we built everything they wanted, they would be unlikely to fall in love with the product.

Somewhat disappointed users for whom speed was the main benefit: we paid very close attention to this group, because our main benefit did resonate. Something  -  probably something small  -  held them back.

Focusing on this last group, we looked more closely at their answers to the fourth question on our survey: "How can we improve Superhuman for you?"

This is what we saw:

After some analysis, we found that the main thing holding back our users was simple: our lack of a mobile app. In 2015, we had taken the contrarian approach of starting with the desktop. Most emails are sent from desktop, so that's where we thought we could add most value. We were always planning on building a mobile app, but at the beginning of our journey - like every startup - we had the chips for just one bet. In 2017, it was clear that we could no longer delay this, and that mobile had become critical for product/market fit.

Probing further, we found some less obvious and more interesting requests: integrations, attachment handling, calendaring, unified inbox, better search, read receipts and so on into the long tail. For example, as an early-stage company, internally we weren't making heavy use of our calendar and we wouldn't have prioritized calendaring much at all based on our own intuitions about email. Hence, this process of digging through feedback massively moved calendaring up on the product priorities list.

With a clear understanding of our main benefit and the missing features, all we had to do was funnel these insights back into how we were building Superhuman. Implementing this segmented feedback would help the somewhat disappointed users get off the fence and move into the territory of enthusiastic advocates.

3) Build your roadmap by doubling down on what users love and addressing what holds others back.

Even though we understood why users loved our product, and what held others back, it wasn't initially clear how we were going to navigate the tension between the two when it came to committing to a product roadmap.

I eventually came to this realization: If you only double down on what users love, your product/market fit score won't increase. If you only address what holds users back, your competition will likely overtake you. This insight guided our product planning process, effectively writing our roadmap for us.

To double down on what our very disappointed users loved, half of our roadmap was devoted to the following themes:

More speed. Superhuman was already extremely fast, but we worked to make it even faster. For example, the UI would respond within 100 ms, and search was faster than in Gmail. We pushed even further to response times of less than 50 ms, and worked to make search feel instantaneous.

More shortcuts. Users loved that they could do everything from the keyboard. So we made our shortcuts even more robust and comprehensive. We built shortcuts that no other email experience had and we started pipelining keystrokes, ensuring that everything still worked even if you typed faster than your machine could handle.

More automation. Users really valued the ability to be more efficient with their time. But we all hit the same limit: the sheer time it takes to type. So we built Snippets, a feature that lets users automatically type phrases, paragraphs, or whole emails. To save even more time, we made Snippets more robust, adding the ability to include attachments, automatically add people to CC, and even integrate with a CRM and ATS.

More design flourishes. In our feedback, we saw that users loved the design and its many small details, so we invested in hundreds of small touches to show that we care. For example, typing "-->" now automatically turns into a right arrow: →.

To gain ground with our speed-loving-yet-somewhat-disappointed users, the other half of our roadmap was focused here:

Developing a mobile app.

Adding integrations.

Improving attachment handling.

Introducing calendaring features.

Creating a unified inbox option.

Making search better.

Rolling out read receipts.

To stack-rank amongst these initiatives, we used a very simple cost-impact analysis: we labelled each potential project as low/medium/high cost, and similarly low/medium/high impact. For the second half of the roadmap, addressing what held people back, the impact was clear from the number of requests any given improvement had. For the first half of the roadmap, doubling down on what people love, we had to intuit the impact. This is where "product instinct" comes in, and that's a function of experience and deeply empathizing with users. (The HXC profile exercise from earlier helps a great deal with developing this muscle.)

With this plan of attack outlined, we got to work, starting with the lowest hanging fruit of low cost, high impact work so we could deliver improvements immediately.

To increase your product/market fit score, spend half your time doubling down on what users already love and the other half on addressing what's holding others back.

4) Repeat the process and make the product/market fit score the most important metric.

As time went on, we constantly surveyed new users to track how our product/market fit score was changing. (We were careful to ensure that we didn't survey users more than once, so as to not throw off the 40% benchmark.)

The percent of users who answered "very disappointed" quickly became our most important number. It was our most highly visible metric, and we tracked it on a weekly, monthly and quarterly basis. To make this easier to measure over time, we built some custom tooling to constantly survey new users and update our aggregate numbers for each timeframe. We also refocused the product team, creating an OKR where the only key result was the very disappointed percentage so we could ensure that we continually increased our product/market fit.

Reorienting Superhuman around this single metric paid off. When we started this journey in the summer of 2017, our product/market fit score was 22%. After segmenting to focus on the very disappointed set of users, we were at 33%. Within just three quarters of our work to improve the product, the score nearly doubled to 58%.

And we're not done - the product/market fit score is something that we're going to continue to track. I think it's always useful for startups to look at this metric, because as you grow you'll encounter different kinds of users. Early adopters are more forgiving, and will enjoy your product's primary benefit despite its inevitable shortcomings. But as you push beyond this group, users become much more demanding, requiring feature parity with their current products. Your product/market fit score may well drop as a result.

However, this shouldn't cause too much anxiety, as there are some ways around it. If your business has strong network effects (think Uber or Airbnb), then the core benefit will keep getting better as you grow. If you're a SaaS company like Superhuman, you simply have to keep on improving the product as the pool of users expands. To do that, we rebuild our roadmap every quarter using this process, ensuring that we're improving our product/market fit score fast enough.

BRACING FOR IMPACT

In the twists and turns of following this process, I found a way to define product/market fit and a metric to measure it. Our team had a single number to rally around instead of an abstract goal that left us feeling hopeless. By surveying our users, segmenting our supporters, learning what users loved and what held them back, and then dividing a roadmap between the two, we found a methodology to increase product/market fit.

It is hard to overstate the impact of this product/market fit engine on our company. Everything we do  at Superhuman -  from hiring to selling and marketing to raising capital  -  has become significantly easier. Our team has grown to 22 people and our NPS has increased right alongside our product/market fit score. Users became noticeably more vocal about how much they loved the product, both in our surveys and on social media. Current investors started asking if they could put in more money ahead of upcoming rounds while outside investors continually ask me if they can invest.

Taking a step back to reflect on what I've learned from building this product/market fit engine for Superhuman, I'm left with two final takeaways:

Investors advising early-stage teams should avoid pushing for growth ahead of product/market fit. As an industry, we all know that this ends in disaster, yet the pressure for premature growth is still all too common. Startups need time and space to find their fit and launch the right way.

For any founder looking to get out of the wilderness and on the path to the ever elusive product/market fit, I've been in your shoes - and I hope you'll consider retooling this engine in those proverbial startup garages to make it your own. And when you finally hit the product/market fit score you're targeting, my advice is to push the pedal all the way down and grow as fast as you can. It will feel uncomfortable, but you'll have the evidence you need to know that you'll succeed.

Image by DNY59 / Getty Images. Photo and charts courtesy of Rahul Vohra.

# data[].attributes.attachment_link.title
How Superhuman Built an Engine to Find Product/Market Fit

# data[].attributes.attachment_link.description
This article is by Rahul Vohra, the founder and CEO of Superhuman - a startup building the fastest email experience in the world. We've all heard that product/market fit drives startup success - and that the lack thereof is what's lurking behind almost every failure.

# data[].attributes.attachment_link.readable_text
Readers,

Starting from a blank canvas is the absolute hardest thing to do. It's insanely hard. And yet the brave entrepreneurs in the tech industry do it, time and time again, and every year, a few are successful. It's an awesome thing when it works though.

Anyone who's working on a new product has question: How do you get to product/market fit? If you've been wandering the desert and you can't retain your users, is there anything you can do about that? Is there a way to add structure to something that seems awfully random? How do you even know you're there?

Below is a short and sweet deck that I created and presented a few years back to Stanford students. It's focused on a simple idea: The path from zero to product/market fit can be a straight line, or windy, depending on what kind of idea you choose. I present the tradeoffs involved in creating a product that's a bit more incremental, versus something that's more breakthrough.

Although this deck is mostly focused on consumer and consumery enterprise products, it's also fairly general, drawing ideas from the book Blue Ocean Strategy and others.

Hope you enjoy!

Andrew
San Francisco, CA

📥 Get this deck as a PDF, plus new updates and essays in the future:

Above: It's difficult to product/market fit, but today we'll talk about some of the tradeoffs you can make for it to be easier!

Traction is everything - it's what investors ask for. It's what prospective employees and your team is looking for. But getting traction is really a reflection of your product "working" - where users are engaged and retaining, and there's some organic acquisition. Product/market fit is when people who know they want your product are happy with what you're offering - more on the topic here at the a16z blog.

Above: Important, once you get to product/market fit, you can shift your focus from the initial product to distribution and win the market. It's important to do it in this order! In the haste to get to market, there are teams that start spending too much time on partnerships, paid marketing, etc. - all in preparation for a big launch. But the problem is that if you have a leaky bucket, then all those users at the top of the funnel won't help.

Above: So what does product/market fit look like from a metrics standpoint? Really the best metric to think about is probably retention, which you'd analyze using cohorts. That means it's sticking, and you need to figure out acquisition. Once it's all figured out though, then you get the hockey stick. That's what we're all looking for, right?

Hitting the inflection point means that you can focus on scaling growth. The activities here are many of the things that I write about - funnel optimization, new user activation, creating notifications and email drip campaigns, etc., etc.

But it's important to think of these are distinct activities.

If your product isn't working, you won't solve that by doing growth activities.

There's a certain lightning-in-a-bottle aspect to building new consumer companies for that reason.

This is why 20-something year olds often build awesome new companies in Silicon Valley- they make lots of stuff, hit product/market fit, and the capital/talent in the Bay Area comes to help them scale

Entrepreneurs who are young are less encumbered by the notion of "this is how things have always been done" or "this kind of thing has never worked." They can try, fail, iterate, and make their way towards something pretty new.

Above: Here are some specific ways that I might quickly judge if a consumer product has p/m fit.

You want to see DAU/MAU at >25%. A world-class leading DAU/MAU would be over 50%.

There's a certain minimum for organic acquisition. You want to see hundreds if not thousands of signups per day, and a D1 of at least 30%. >70% would be world-class there.

And ultimately there's some scale to show it works - probably 100,000 DAU.

For a SaaS product, you'd like at other metrics that might focus on monetization rather than engagement. Although some products can be evaluated using both sets of metrics.

Above: If you think you have that, send me an email :)

If you compare the metrics above, most startups don't have product/market fit. It's usually not working.

So what do you do?

Above: These days, most startups fail because of lack of P/M fit, not technology risk. So if there's one thing that will kill you, it'll be the product never quite working, and thus, all the subsequent problems that come with that: Lack of investor interest, employees leaving, cofounder fighting, etc. It's a stressful time when it's not working

I find that product/market fit is usually reached either right away, or you have to be incredibly thoughtful about your iterations if you're almost there but not quite.

So how do you get there?

Above: I want to present to you one framework to think about this, and what kinds of tradeoffs you can make.

Above: Here's a coffee cup. Product/market fit is actually easy to get, for a coffee cup. It has to hold coffee. That's about it. There's not too many variables in designing a cup, and so there's not a lot to mess up.

Above: Contrast that to a digital product, where there are a million variables. It's so much more complex than a coffee cup! It's easy to mess something up.

(The above is a Facebook experimental project, called Paper, that never quite worked out)

It's easy to mess up because the software medium is so rich. You can build or create anything, and you often end up with something that users can't quite understand. Worst yet, once it's not working, adding features won't help! Sometimes the foundation is just not strong.

Keep these examples in your head, as two extremes on a spectrum. The coffee cup. The messy random digital product.

Above: The more I've thought about this problem, the more I've come to realize there's really a spectrum between new versus pre-existing product categories.

The new product category carries a lot more risk, which we'll discuss.

The pre-existing category carries less risk, but you face more competition, more expectations, etc.

What you want is a point in between those two extremes.

First, over time I've come to believe that you actually do want your product to be in a pre-existing product category. Google was not the first search engine, Facebook was not the first social network, and Microsoft was not the first OS.

However, those companies came to dominate those markets because they came in early, when the dynamics were still developing. And the markets grew and grew. But it was also clear what they offered, how they developed something killer.

Above: I want to be more precise than to say that the market must be big - more specifically than that, there must be a large number of users who are pulling on this category in the market. By pulling, I mean folks are actively searching for the product. You can see it in Google Trends. They are clicking on links that mention products in the category. There's already spend.

Let's contrast this to the big fake market sizes that entrepreneurs often trick themselves into. Yes there are a lot of college students, and maybe that's your target market. But how many of them actually want this? Already own a precursor version of your product, or are googling or searching or talking about the category? That's the real assessment of demand.

Above: Related to the above point, the best markets already have competitors. And maybe they are pretty decent. It's a good sign when folks can make a decent living building products in the space, but you just have to find a wedge to get in. Or maybe you have a clear thesis on why and how the market will get a lot bigger quickly.

The best case is where the market is large, but fragmented - perhaps because the competitors are not that competent or there's something structural, like regulatory issues, that keep things small.

Above: If you are in a pre-existing market with competitors and pre-existing demand, then you need a wedge. You need to analyze the market, understand the segments, and figure out how to break in.

My favorite book on this topic is Different: Escaping the Competitive Herd. Worth reading.

Above: There's a huge advantage to building for yourself because the intuitive advantage it gives you in iterating quickly is fantastic. This isn't always possible, but it can be a good thing.

Just as we discussed with the coffee cup, one way to get to product/market fit quickly is to just copy something.

There are a ton of problems with this strategy though - if you're the second player with the same product, you'll always remain the second player. Your team will hate this strategy. Investors won't get it.

Sometimes the market is big enough to make this work, but it's not often.

Above: The better approach is to do a twist. On the spectrum of pre-existing market to new market, pick a point in the middle. Build something that consumers fundamentally understand, but with a clear innovation that you can market around.

A lot of getting to product/market fit is being thoughtful about the category, the axis of differentiation, etc.

I want to run through some common mistakes that people make.

Above: We love these "X for Y" type companies. And it can be a way to generate cool ideas, but it doesn't say much about the underlying market size and existence of pull in the market.

For instance, "Pinterest for dogs" and "Pinterest for businesses" are both X for Y ideas, but only dogs are a segment of Pinterest. A business-targeted product is more a description of the mechanics of the product, not a smaller set of the broader Pinterest market. And as we mentioned, identifying a wedge in a pre-existing market is key.

📥 Get this deck as a PDF, plus new updates and essays in the future:

As a result, you can think about existing versus new markets using The Substitution Test. If I use this new product you're building, does it take away engagement or $ from something else. If so, then it's truly a segment and a heads-on competitor. It's counter-intuitive but I consider that a good thing, because they you are truly competing in a pre-existing framework- you just need the right functionality to win.

Above: Every existing market has a baseline of product quality, functionality, etc. We'd all love to build the minimum viable product, but sometimes it doesn't work because the category has evolved sufficiently that you need more than the bare bones.

Sometimes people love a certain kind of tech. Today it might be crypto, tomorrow it might be something else. And they are looking for some kind of product and category to apply to, rather than fundamentally solving peoples' core needs. It's an easy mistake to make for technologists.

As you know, I hate new consumer behavior :) Sometimes this can work but it's rare. At the very least, products have to tap into the same fundamental human motivations that have driven us for 100,000s of years.

Above: This is a San Francisco thing, but entrepreneurs are often building things that sound cool only to each other. No one else cares, but it's fun and captures some tech trend that's confined to the SOMA bubble. I call this "art for artists"

Above: This is common. You can execute everything correctly - iterate quickly, great tech, hire well, etc. - but ultimately still take huge risks on your product and market in such a way to not actually be successful.

Above: So let's say you get something that has P/M fit, and it starts to take off! It's a good problem to have. So what do you do?

Above: Luckily, you can summon the power of the Bay Area to help you solve this.

There's a large body of work and experts on scaling companies. You can raise money, hire operators who've scaled, and you're on your way.

I don't mean to downplay this, except to say that the zero to product/market fit part is so hard that this next stage will be much easier than the first. Ultimately, there's a few growth channels that work, and it's much more of an optimization problem.

Above: For the paid ads channel, you're ultimately looking at it as a LTV versus CAC problem. With some blended/organic traction too.

The ecommerce companies, OTAs, and marketplaces often grow like this.

Or you want to build a viral loop - if your product is social or collaborative in nature - and you can optimize for invites or content sharing or some other loop. This is a deep topic that could be a multi-part series of its own things.

This channel is used by the social networks, video sharing products, but also even productivity/collaboration products in the workplace.

If your growth loop involves a lot of user-generated content, then you can build on Google. For products that are really about reviews on local businesses, companies/products, or even real estate, we see them use this loop.

Above: If you're at product/market fit, I want to hear from you! Send me a note anytime. Thank you :) (For future reference, you can also download the PDF here).

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

# data[].attributes.attachment_link.title
Zero to Product/Market Fit (Presentation)

# data[].attributes.attachment_link.description
Readers, Starting from a blank canvas is the absolute hardest thing to do. It's insanely hard. And yet the brave entrepreneurs in the tech industry do it, time and time again, and every year, a few are successful. It's an awesome thing when it works though.

# data[].attributes.attachment_link.readable_text
A good product fulfills a deep-rooted, real human need. It is delightful, instills trust, and makes people want to be associated with it and become strong brand ambassadors. Healthy products exist on a wide spectrum: they can be enterprise or consumer, focus on specific verticals that provide a niche need, provide utility or improve efficiency, etc. This post explores multiple dimensions of healthy products with a particular focus on how consumer companies should measure aspects such as growth, retention, stickiness and engagement.

What is working in your product and what is not? Are you growing well? Are users being retained? Is your product sticky? Are your new cohorts doing as well as your old cohorts? Are people engaging with your product? Is there a core set of users who care about it deeply? Good products grow well as they penetrate their market, retain and engage their users, and bring users back regularly with a healthy content loop.

USER ADOPTION AND GROWTH

The total number of active users is the truest measure of your product's impact. Driving adoption and growth is important for your product to achieve its full potential and create the most value for the most users.

It is important to understand product and user growth in the context of the overall market. How many people are using your product? What is the total addressable market (TAM) of your product? How does the number of people using the product compare with the TAM? How much of the market share does your competitor have, and how is this changing with time? How is the TAM likely to increase over time, and how much share is your product likely to have? (See Figure 1.)

It's also important to understand specifics of growth for your product. How quickly is the product growing? Is it growing organically? Is growth primarily driven through new user acquisition? How is growth changing as market penetration increases? What is the balance between churn and resurrection within your product?

In a previous post, we discussed in detail the evolution of a product from a growth perspective. A healthy product has multiple phases of growth that resemble an S-curve (see Figure 2). Many products start with modest and shallow growth (Early phase), then growth accelerates to the point where the slope arches upward (Growth phase), followed by exponential growth (Hyper Growth phase) until it reaches maximum growth, after which growth tapers down and reaches maturity (Mature phase), where there is little to no growth.

If the market creating demand for the product is not large enough, it is very hard for the product to succeed. Furthermore, a healthy product requires a large market that is growing and not shrinking - otherwise, failure is inevitable. In a market that is still developing, timing the evolution of the product so you are growing along with demand is vital for success. This is often very hard to achieve, and many products fail because of timing.

How do you know if you are growing at the "right" rate? It's valuable to benchmark your product with other products at the same stage. Knowing how fast similar products grew at the same level of market penetration can give you a sense of whether your product is special.

As your product starts to grow, more and more people will install it, and with time, installs will become a larger and larger percentage of your addressable market. Therefore, it's very useful to understand how many installs or downloads your product has overall.

In a product's early stages, the number of installs is typically very small, all the net growth is driven by new users, and there is no churn or resurrection. On the other end of the spectrum, when a healthy product reaches maturity and has a high degree of TAM penetration, churn cancels out resurrection and the number of active users stays steady with time. This steadiness implies that each of the old retained cohorts have more or less flattened out.

For anything in between these two extremes, growth depends on the balance between new users, resurrection and churn. By the time your product reaches maturity, it is too late for major changes, so you want to ensure it is growing sustainably while you are in the Growth phase.

MARKET METRICS

MAU/installs To monitor growth, it is useful to track both your monthly active users (MAU) as a percentage of overall installs and the total number of installs as a percentage of your market.

If the total number of downloads for your product is beginning to reach high penetration of the TAM, then your MAU/installs should be relatively high. As installs approach maximum TAM penetration, new user acquisition is no longer a tactic for incremental net growth, as there are fewer users left to try your product. As a result, net growth must come from resurrecting previously churned users. In other words, retention of your older cohorts needs to improve over time in order to produce net growth. This is generally difficult. If the ratio of active users to installs is getting smaller over time, your product is losing steam. For very strong products, MAU/installs grows slowly over time as older cohorts begin to resurrect.

Keep in mind, different genres of apps have very different behaviors. For example, all games have a shelf life, regardless of how popular they may be at their peak. In the case of the app Empires & Allies, a huge number of people installed (see Figure 3), and many became active users. However, the game also churned a large number of users. Even though more new users joined, the churn was too large, and MAU declined rapidly. Only two percent of the downloaded population remained active at steady state, and very few new users were incrementally added. As such, the game is now almost dead, used only by the most die-hard fans.

GROWTH METRICS

MAU/WAU/DAU The top growth metric you should track is the number of active users. There are three primary metrics, all correlated with one another: the total number of people who actively use the product on any given day is daily active users (DAU), the number of unique active users in a given week is weekly active users (WAU), and the number of unique active users in a given month is MAU. Choose your top metric based on how frequently you expect the product to be used.

D/D, W/W, M/M, Y/Y changes Understanding how the number of active users changes day-over-day (D/D), week-over-week (W/W), month-over-month (M/M) and year-over-year (Y/Y) is also extremely valuable. While a product's DAU may be volatile, tracking its rolling seven-day average may be more meaningful. In general, it is best to use rolling metrics as opposed to calendar-based metrics. For example, comparing MAU between February and January is misleading, as the months are different lengths; a 28-day rolling MAU would be preferable. Comparing 28-day rolling windows has the additional advantage of avoiding a day-of-the-week effect that could skew results.

Quick Ratio The growth accounting framework helps us understand which factors contribute most to growth by splitting growth into new users, resurrected users, retained users and churned users. Between any two timestamps (t1 and t2), the change in growth = new users acquired within this time frame + users who were not active at time t1 but came back by time t2 - users who were present in time t2 but not there in time t1.

Δ growth = new users + resurrected users - churned users

When the Quick Ratio, which is the ratio of the sum of new and resurrected users over the churned users, is greater than one, there is positive net growth; when it is less than one, active users decrease month over month. For the most part, when a company has reached maturity, resurrection and churn roughly cancel each other out and much of the net growth happens through new user growth. For saturated markets, e.g., Facebook in the United States, new user acquisition is extremely small and growth remains relatively constant, with churn and resurrection balancing each other out.

New users/MAU For companies at a very early stage, all growth comes from new users, and there is no churn or resurrection. Soon after, new users continue to be the primary driver of net growth and contribute a large, but decreasing, percentage of overall MAU. As the company matures, new users' contribution to net growth shrinks, and the retained population becomes a larger percentage of overall MAU. For unhealthy companies, new users sometimes remain the largest driver of growth even after the product reaches a high degree of market penetration; this is a death spiral. Once a product reaches a high percentage of the TAM, a smaller percentage of growth should come from new user acquisitions for the product to be sustainable.

Sign-ups/installs Acquiring new users and retaining existing users are the two most important product focuses for consumer companies. Early in a company's life cycle, it's particularly important to acquire, retain and engage new users. If the product is retaining well and users are deeply engaged, it makes sense to focus on increasing new user growth. This acquisition can happen through multiple channels and strategies: contact importers, social media, ads (banner, mobile app install, video, etc.), incentivized installs and referrals, SEO, ASO (app store optimization), email marketing, PR and press, app cross-promotion, etc. Knowing where your users are coming from and understanding the effectiveness of each channel is important to prioritizing your efforts. Be sure you understand every part of the acquisition funnel and identify what is working and what is broken, including invites sent and received, clicks, downloads, installs, sign-ups, etc.

GROWTH CONSIDERATIONS

Possible segmentations It's useful to monitor growth by multiple dimensions depending on what is most meaningful to your business: country, device, age, gender, phone year class, connectivity class, platform, age in product, etc. Keeping track of the referrer source (paid, organic, SEO, App Store) is particularly important.

RETENTION

Retention is the best indicator of whether your product is valuable and whether you have product-market fit - because it tells you whether people who tried your product liked it enough to return and use it again. Generally, the first day or week is critical for a product with a new group of users. Do they have enough content to consume? Do they have a few close friends to chat with? Do they have a mental model of what the product is?

To enable healthy retention and build a successful product, you must have a core set of users who love and embrace and care deeply about the product. When you are starting out, always build your product for this set of users, focusing on the use case that matters most to them. This involves both creating a "magical moment" in which users first "get" the product and identifying the tipping point that establishes user retention.

For Facebook, the magical moment is when a new user sees the face of a friend for the first time. For WhatsApp, it's the first message a person receives. For Amazon, it could be when a buyer receives their first purchase or has their first interaction with the customer service team.

Once you have created the magical moment, study how users are retained. The retention tipping point is different for different products; in Facebook's case, it was connecting to seven friends in 10 days.

For your product to grow sustainably, you need to identify both its magical moment and its retention tipping point.

Retention is also the most important lever for your product's growth. A poorly retentive product is not sustainable in the long term - you will churn through the entirety of the total addressable market with no long-term users to fall back on. Before you invest in acquiring users through marketing or paid channels, be sure you understand any and all variables that can influence retention and help stabilize it for early cohorts. Ultimately, you want long-term retention, but early retention can be a strong predictor of long-term retention. While the absolute value of retention depends on the type of product (social, gaming, messaging, etc.) and the time frame you choose to measure (daily, weekly, monthly, etc.), it's always true that the higher the retention, the better.

There are three basic shapes of retention curves. In the first, worst-case scenario, long-term cohort retention drops to zero and the product eventually dies. In the second scenario, the retention curve flattens out at a number greater than zero. This ensures the product has active users - the higher it flattens out, the more users you have. The third scenario is best: when a product is very healthy and has reached strong product-market fit, there may be a Hyper Growth phase in which the older cohorts start to increase their retention. This is the case in Evernote's smile graph (Figure 4). (Eventually, at a higher rate, this will flatten out).

RETENTION METRICS

Dn/Mn/Wn metrics Retention is assessed in terms of cohorts - that is, by tracking a set of users who install a product on a given day, week or month and seeing what percentage of them return Ultimately, the goal is for the product to retain at high levels over a long period of time. However, for newer products, it is difficult to project what the long-term retention will be. Therefore, you need to identify an early indicator for long-term retention. With games, for example, the percentage of people who retain on day one (D1 retention) is indicative of the long-term retention.

A simple way to express long-term retention is to break down the number of people retained from a cohort after roughly a year (say, 364 days) into multiple ratios, as seen in the equation below. The D1 retention rate is D1/D0, the fraction of your cohort retained for one day. (D0 is the number of installers in a cohort, and D1 is the number in that cohort who still use the product after one day.)

If D7/D1 stays relatively constant for all cohorts but D1 retention appears to decline, focus your attention on improving D1 retention, as this will very likely be the biggest lever for long-term retention. Likewise, if D1 retention is flat but D7/D1 is declining, focus on finding new ways to engage users in the first week rather than the first day. Pick the metric that best aligns with your product.

Cohort curves In addition to Dn retention metrics, analyzing cohort curves is extremely important. Cohort curves will give you a much better idea of whether your retention is flattening, experiencing hyper growth or dropping to zero.

Some additional tips for healthy retention:

Compare new cohort retention rates to old cohort retention rates and make sure they remain healthy over time.

Benchmark retention rates against similar products.

Ensure that cohort retention flattens - preferably at a high rate.

STICKINESS

Even if your product is growing and retaining well, it may not be sticky. While retention is about getting people to come back to your product, stickiness is about getting people to come back of their own volition, thus helping reduce your dependency on tactics such as push notifications. When a product or service is sticky, the customer becomes tied to it and cannot easily leave. For example, Facebook's stickiness is a direct result of users' urge to share, and of their curiosity about other people's lives.

STICKINESS METRICS

DAU/MAU This is the most widely used metric for stickiness. The ratio of daily active users to monthly active users tells you how sticky a product is. It's valuable to understand benchmarks for these metrics for different products and to put your product in context; for example, SMS-type apps have a much higher DAU/MAU than social apps such as Facebook. A DAU/MAU ratio of 0.6 means 60 percent of people who come to the product on a monthly basis are also coming back on a daily basis. Generally, this means the product is very healthy. Social products in the early stages of growth tend to have a relatively small DAU/MAU, perhaps 30 percent. As more users join (and market penetration increases), DAU/MAU begins to grow. In addition, newer and older cohorts alike begin to retain at a better rate. The specific value of DAU/MAU is strongly dependent on the business or product and the expected usage. If your product has a low DAU/MAU, it does not necessarily mean it isn't doing well. While a core set of sub-products might have high product-market fit and high DAU/MAU, it's quite possible to find sub-products within that set that are less engaging. The ratio might be low due to a mix of casual and highly engaged users. In this scenario, you should investigate options to get more users highly engaged.

Open rate This is the percentage of people who have the product installed that use it in a given time frame. For a highly sticky product like Facebook, people open the app many times per day. For less-sticky products, the open rate may be 10 percent or less on a monthly basis. Stickier apps are generally correlated with higher open rates. (However, if the product is highly notification-driven, people may still open the app even if the product is not very sticky.)

Lness This is the number of days visited in a given time frame. For example, L5+/7 is the percentage of people who visit the product at least five times per week. Similarly, L21+/28 indicates how many people come back at least 21 times within a given month. The higher the percentage, the stickier the product is. Tracking Lness distribution over time will give you a very good sense of your product's stickiness.

Sticky or DoD/WoW/MoM retention While cohort-based retention helps us understand how each group of installers behaves, it does not tell us about the behavior of the most-engaged users. Remember, the goal is to build a product for the core users who engage with it the most. Sticky retention helps us understand how well a product is doing for the most-engaged people. It is generally measured using DoD/WoW/MoM retention, which refers to the number of people who come back to the product the following day, week or month (regardless of when they joined the product). For example, to determine week-over-week (WoW) retention, we simply take the ratio of users active in a given week to the total users active the previous week. (In order to eliminate the "new user effect," remove any user who first became active in the past 14 days.)

ENGAGEMENT

In most cases, if your app is growing well, users are retained at high levels and you have good indicators of stickiness, your product is in great shape. But it could also be the case that people are not returning often enough, aren't producing or consuming enough content, or aren't spending enough time using the product. These are indicators of low engagement. For social products (such as Facebook and Instagram) in particular, time spent in the product is the simplest indicator of whether your product is engaging. The higher the time spent, the higher the engagement. True product-market fit happens when a product is highly engaging (though it also manifests itself in retention).

What drives engagement? It is valuable to create a simple framework to understand what drives engagement in your product. Social products such as Facebook, Instagram, Reddit, Snapchat, etc. have a production/consumption framework to drive engagement. A simple version of this engagement model is seen in Figure 5. Users produce content which is then shown to others, creating delight. Consumers of the content provide feedback in terms of upvotes, comments, reactions, etc., which makes users want to produce content again. This production/consumption loop creates engagement, which can be measured by users coming back frequently (number of sessions), consuming more content (number of views) and spending more time on the site.

There are multiple approaches to increasing engagement: making it easier to create different types of content, helping connect users with the most relevant content, showing them the right content in the right order, making sure people are able to consume content easily on any device and any network condition, etc. In addition, as more individuals, influencers and businesses produce content, there is a greater likelihood that users will see engaging and relevant content they love. It's also imperative that you design your product to make it easier for people to interact with its content, as this strengthens the positive feedback loop.

Another engagement model is the e-commerce model. For example, eBay connects sellers to buyers. In this model, you can increase engagement by focusing on the quality of the inventory available, the quantity of unique listings, the relevance of items shown to the buyer, the value of the listings, connecting the right content to the right buyer at the right time, simplifying the buying process, building trust at every step of the experience, etc. Optimizing each part of the funnel should increase overall engagement.

ENGAGEMENT METRICS

Note that many of these metrics are specific to companies that fit the production/consumption paradigm.

Time spent/DAU This metric is a strong indicator of whether your product is engaging. Again, this depends heavily on the specific product and expected engagement. Market share of time spent is also a good indicator of whether you are doing well from a market perspective.

Number of sessions Number of sessions in a given day is another good indicator of whether your product is engaging. An increase in this metric is also the earliest indicator of whether you are achieving product-market fit. (Similarly, a decrease in number of sessions is the earliest warning sign that something is going wrong.)

Time spent/session Both number of sessions and time spent/session are levers to grow overall time spent. It is valuable to understand whether you can more easily increase overall time spent by increasing the number of sessions or by increasing time spent/session.

Inventory available For companies that have news feeds (ByteDance, Instagram, Facebook, LinkedIn, etc.), users can consume content only if there is inventory for them to see. Knowing how much inventory is available and how it is distributed helps you understand how content can be monetized.

Content consumption (number of views) The metric of time spent is often skewed by types of content that are inherently time-consuming, such as videos. Therefore, it is helpful to understand how many pieces of content people consume relative to the inventory. Are some people inventory-constrained and thus unable to consume content?

Production of content How much content does each user produce every day, week and month? How many of them produce content on a regular basis? The total production of a product is based on the number of users who produce content and the amount that each produces. For ecosystems in the production/consumption paradigm, the production of high-quality content is the most important metric for an engaging ecosystem.

Feedback on content Key feedback metrics to track are likes, comments and reactions. Likes are a weaker form of feedback than comments. Regardless, all feedback is important to the ecosystem. Without it, production will decrease, which will ultimately lead to a drop in consumption.

ENGAGEMENT CONSIDERATIONS

Possible segmentations Consider segmenting your metrics by country, device, age, gender, phone year class, connectivity class, platform, age in product, content format (video, picture, text, etc.) and content type (social, entertainment, informational, educational, etc.).

Leading and lagging indicators Imagine you start out using a product multiple times per day (multiple sessions) and spending a lot of time with it. Over time, you start getting bored with the product and your use drops off - but you are still an engaged user in many ways, visiting the product many times per week and more than twenty times per month. At this point, DAU, WAU and MAU have not yet changed.

Eventually, however, as you use the product fewer times per week, you will continue to register as a WAU and MAU but less as a DAU. Over time, you will no longer be a WAU, and then not a MAU, either. And chances are, you are not alone; many other users are likely getting bored, too. At an aggregate level, these types of behavioral changes first manifest in engagement metrics such as the number of sessions. It's likely that as your use of the product changes, first you will consume fewer items of content, then you will have fewer sessions and read fewer stories each time. A drop in number of sessions is the earliest leading indicator for a drop in DAU. Similarly, a drop in DAU is a leading indicator of a drop in WAU and ultimately, MAU.

Engaged/power users Your most-engaged users are the most important people to take care of. Keeping them happy is not really a growth concern, as they are not likely to churn. However, as mentioned in the discussion of retention above, it is ideal to target your product toward core users by making the experience easy for them and delighting them. Many of the metrics defined in this post can and should be segmented for the most-engaged users. Is the number of engaged users growing? Is their number of sessions and amount of time spent increasing? Are less-engaged users becoming more engaged? How many friends do these "power users" have? How much content do they produce? What features of the product do they use? Answers to questions like these can help build great experiences for your most-engaged group.

PRODUCT MYTHS

Good products don't have troubled times The truth is nearly all products have problems with churn, new user acquisition and/or engagement. But if your core users are still engaged, and a reasonable number of them are deeply engaged, you have achieved product-market fit. And when you have product-market fit, there are ways to get your product back on track and grow your base of engaged users.

Growth tactics can solve anything While growth tactics can be very helpful if you already have product-market fit, you cannot use them to solve problems with product-market fit itself. The product must deliver value to users. For example, BranchOut 's meteoric rise was due to growth tactics, and their subsequent fall was due to not having product-market fit.

Grow quickly at all costs The truth is growing sustainably is far more important than growing quickly. Growing with low retention will eventually kill your product.

Build a product for everyone It's a common misconception that you should try to satisfy all your users. But building a product for everyone will almost certainly lead to building a product for no one and will ultimately hurt the product. Build your product for your most-engaged users and ensure they have a great experience.

As long as the retention flattens, you are fine A high absolute retention is important for the long-term sustainability of your product. Yes, retention to a low number is better than nose diving to zero, but it still may not be sustainable.

TAKEAWAYS

Saturation of a market will slow down growth. Growth metrics should be considered in the context of the Total Addressable Market (TAM).

Retention is the best indicator of product-market fit and the most important lever for growth. "Magical moments" help your users understand the core value proposition of your product, and getting users to a "tipping point" will help you retain them.

Engagement is the most important driver of retention. Creating a framework for what drives engagement in your product will help you identify leading indicators for changes in user behavior and inform interdiction techniques.

This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email data-science@sequoiacap.com with questions, comments and other feedback.

# data[].attributes.attachment_link.description
A good product fulfills a deep-rooted, real human need. It is delightful, instills trust, and makes people want to be associated with it and become strong brand ambassadors. Healthy products exist on a wide spectrum: they can be enterprise or consumer, focus on specific verticals that provide a niche need, provide utility or improve efficiency, etc.

# data[].attributes.attachment_link.readable_text
For a product to succeed over a long period of time, several conditions must be present: product-market fit, positive unit economics, and the ability to scale and grow. Product-market fit requires a deeply engaging product to which people return because they find genuine value. Positive unit economics require careful attention to the fundamental financial building blocks of the business. To effectively scale requires, among other factors, a sustainable organization. In this post, we offer guidance on building both sustainable products and scalable organizations - specifically, by developing frameworks.

FRAMEWORKS: WHAT THEY ARE AND WHY WE NEED THEM

A framework is a way of thinking about a problem - a way to understand something complex. Frameworks help us simplify; most problems seem complex until we create frameworks to help us think through them. Frameworks make problem-solving tractable.

Frameworks are a constant presence not only in businesses but in our personal lives. Whether consciously or not, we use them to simplify and guide many decisions - for example, which school to attend and which major to choose. By determining key dimensions (such as schools, programs, earning potential and distance from home) and setting criteria (such as top five school, top five program, earning potential greater than $100,000 and West Coast location) we create a framework for decision-making. The same process can help us understand how well a product is performing, how to build an effective organization, how to retain users, how to activate more users, etc.

Without conceptual frameworks, the process of identifying and solving problems is difficult to scale, repeat, extend and reuse. Frameworks are necessary to effectively diagnose and understand the root causes and associated drivers of a problem, to obtain actionable insights and to communicate findings.

BUILDING FRAMEWORKS

Frameworks are developed by breaking a problem into parts, each of which can then be broken down further. The examples below are designed to help you understand this process.

The key takeaways below are not unique to each example; rather, you can apply them across many situations.
Example 1: Data-informed company building
At the company level, it is often valuable to establish a single framework that can connect your entire business. By helping employees understand what matters most to the company, such a framework can motivate the team, unifying individuals team around one metric and helping them connect the company's goals to its mission. A single company-level framework can also help you resolve conflict and decide how to structure the team.

As an example of how frameworks can help you build a data-informed company, consider Facebook. Facebook's mission is "to give people the power to build community and bring the world closer together." The company dreams of a world in which everyone is on its platform - where we are all plugged into and deeply engaged with the communities around us and therefore closer together.

From a metrics perspective, Facebook's mission implies a focus on user growth, meaning the company would like every single person on the planet to use the platform. The metric that best encapsulates this goal would be either monthly active users or daily active users (DAU), as building strong communities and bringing people closer requires users to deeply engage with the product. To encourage this behavior, Facebook recently began prioritizing "meaningful" (rather than total) time spent on the platform; users now see more posts from family and close friends and fewer viral videos.

To continue to grow its product and engage its users, Facebook must also monetize by running ads on the platform. These three factors - growth, engagement and monetization - are the primary levers by which Facebook can grow revenue, as shown in the simple framework in Figure 1.

By organizing its business units around these levers, a company can transform itself into a more data-informed and metrics-focused organization. In the case of Facebook, the Growth team could focus on growing DAU as its "north star" metric; such alignment could help resolve conflict about the value of specific initiatives. For example, we know that if everything else were equal, shipping products that increase DAU would be the right thing to do.

Similarly, the Engagement team could focus on increasing time spent per user, while the Monetization team could work to grow revenue per minute of time spent.

Effective pursuit of these goals also requires checks and balances to ensure improvement of these key metrics does not negatively affect other company-level metrics. For example, while attempting to increase DAU, the Growth team should also monitor revenue per unit of time spent.

While metrics such as those in Figure 1 are also useful at the team or business unit level, developing a simple company-level framework can significantly improve your ability to monitor your business at all levels. At the highest order of aggregation, executive dashboards shed light on how each business unit is performing overall.

Dashboards can also be tailored for the leaders of each business unit, to give them a deeper understanding of how their team is doing, what's driving their business and the strategies they should consider. These dashboards can then be further broken down into subunits of each business unit and monitored by mid-level executives to determine how they can affect a top-level business goal. In this way, a single company-level framework can be "broken up" into parts, empowering each business unit to monitor its own metrics and providing clarity, ownership and accountability at every level.
Example 2: Activating users
In this example, we examine how a growth team might build a framework for user activation. Growing users sustainably (a key goal for every product) requires retaining existing users, resurrecting churned users and attracting new users. Growth teams focus on getting new users at the top of the funnel and then converting them at each subsequent step.

New users can be reached through multiple organic and paid marketing channels; Figure 2 shows a funnel for outreach via email. "Users Reached" includes all users who receive the email, but only some will read it. Of those who do, a small percentage will click on a call to action, such as a link to a page where they can install an app. Among those who click on the link, only some will actually complete installation. Another drop-off occurs among people who install the app but do not open it and reach a landing page. From there, only a small fraction of users will reach the signup page, create a new account and successfully login. A simple formula for the number of successful logins is:

This framework can be used to set new goals and initiatives and a new product roadmap, with each term in the formula above as a potential lever to increase the overall number of logins. To improve activation, it is paramount that you understand every part of the funnel; as in the first example, monitoring each step of the process is critical. You should optimize at all levels, making the terms in the formula part of your monitoring dashboard; understanding them deeply and in the context of relevant benchmarks; "diagnosing with data" to hypothesize what would move them; and "treating with design" by setting your goals, initiatives and product roadmap against them.
Example 3: Retaining users
To improve overall growth, it is valuable to develop a framework for understanding retention. Below, we provide a formula that combines different levels of a funnel of user cohorts (similar to the framework in Example 2).

Retention is the best indicator of whether your product is valuable and whether you have found product-market fit, because retention tells you whether people who tried your product liked it enough to use it again. The best way to enable healthy retention is to build a product your core users - the ones who are most engaged - will love. This involves both creating a magical moment where users first "get" the product (see Example 4 below) and identifying the tipping point that establishes user retention.

Figure 3 shows the retention funnel for a cohort of users as they go through the signup flow and become activated or "new" users.

For many products, the first day or week is critical for user retention. Do users have enough content to consume? Do they have a few close friends to chat with? Do they have a clear understanding of what the product is?

Knowing the retention of a cohort after 1, 7, 28, 84 and 364 days can help your Product team prioritize and focus its efforts, as well as create specific strategies, roadmaps and initiatives. For example, if D364/ D84=1 (that is, the number of people who came to the product stayed constant between days 84 and 364), you know all net churn happened before day 84. If you also find that D84/ D28=1, you know all net churn occured in the first 28 days.

In this scenario, you should focus on what you can do in the first 0, 1, 7 and 28 days to create the best possible experience for a new user. To investigate options, you can perform an exploratory analysis - hypothesizing the reason or reasons for each drop-off, analyzing those potential causes, identifying issues and opportunities, and offering product roadmap recommendations based on the levers you have and the actions you can take. In a future post, we will describe details of how to conduct exploratory analysis.

As in the examples above, monitoring each level of the retention funnel is critical to improving your product. We strongly recommend creating a dashboard that captures retention at the highest level, as well as additional dashboards that address retention on various days of the user's journey.
Example 4: First experience and the magical moment
For many products, the first-day experience is the most important for engaging and retaining users. When a user first experiences a product, they don't know what to expect, don't understand what the product is about and don't know how to navigate it. Therefore, it is critical that you offer users a fantastic first-day experience and offer them a "magical moment" that makes them want to come back each day, leaving them with an understanding of the value that the product provides.

This is particularly important in gaming, as there is typically a sharp drop-off between the first and second day. The more time users spend with a gaming product on the first day, the more likely they are to return the following day; therefore, most games offer tutorials, levels, missions, bonuses or awards to new users.

For example, imagine the fictional game shown in Figure 4, in which a user who activates their account, continues to the home screen, completes a tutorial and reaches the first level then gets a bonus. This serves as the "magical moment"; users are encouraged to start Mission 1, and some will eventually complete it.

In this example, just five percent of users who do not reach the home screen come back the second day (D1), while users who finish the tutorial have a 40 percent chance of returning the next day and users who finish both the tutorial and their first mission return 80 percent of the time. This suggests the game's Product team should focus on encouraging users to complete Mission 1. However, if most people drop off at the tutorial stage and a very small percentage of people complete Mission 1, the team should instead focus on the user experience before the tutorial stage, where the largest drop-off occurs.

By understanding both user retention at each part of the funnel and the relationship between retention rates after one day, one week, etc., you will better understand where to focus efforts to increase engagement. If the drop-off at a given stage is negligible, that stage should not be prioritized. On the other hand, a "broken" experience that significantly affects the overall conversion rate should be addressed. In many cases, the fix is as simple as changing the design flow or fixing a bug.

Ultimately, the goal in this example would be to get most users to reach Mission 1 and to create magical moments along the way, ensuring those users are retained in the long term. By breaking this seemingly intractable problem into smaller parts and creating a framework, a Product team can divide and conquer.

TIPS FOR CREATING FRAMEWORKS
Understand the phenomenon and build the right model
The first step to problem solving is deeply understanding the phenomena. Once you understand the phenomena and know the outcome you desire then you should consider building multiple approaches to solving the problem. These approaches connect basic building blocks together simplifying the problem and providing your framework to solve this. The output of the framework is a model. There are multiple frameworks that may be created that fit the problem space but each are different. One needs to use human judgment to choose the right framework/model that optimizes the desired outcome. In Example 1, you need to understand Facebook's mission, its product offerings, what its user base looks like, and how it monetizes (phenomenology). From this, a series of frameworks will help you understand how to think about this expansive problem in terms of models. Here the model chosen best optimizes the outcome, forming a data-informed organizational structure.
Know the outcome
To create a framework, you must understand the outcome you are trying to achieve. In Example 1, the desired outcome is to develop a formula that logically connects different parts of the organization. In Example 2, the goal is to improve activation of users and get them to login. In Example 3, the desired outcome is to improve retention. And in Example 4, it is improving the first-day experience, which will eventually help achieve product-market fit. Understanding the outcome is pivotal to select the appropriate model and optimize it for the intended application.
Identify building blocks
To develop a useful product roadmap, you must carefully choose levels of your funnel. They should be actionable and should clearly lead to the desired outcome. In Example 3, this outcome is retaining as much of a cohort as possible over the long term. Because we have different levers to pull on a user's first day, first week and later on, it makes sense to break this retention problem into parts based on time. Hypothetically, one roadmap would be to improve retention in the first week by increasing the number of friends.
Create a formula
A formula can connect each step or stage of your process and help you understand which of them matter most. It can also help you move tractable metrics that align with your "north star metric." You can create such a formula by converting the funnel into a series of ratio metrics, thereby identifying levers to help you achieve a desired outcome.

CLOSING THOUGHTS

Frameworks are a valuable tool in building a data-informed company and can guide your roadmap to help you build both scalable organizations and sustainable products. As shown in the examples above, you can use frameworks to align your company on key metrics; to break down growth, activation and retention; and to help improve engagement by offering users magical moments and amazing first-day experiences.

However, despite their many benefits, frameworks are not silver bullets. Determining how to construct a funnel can be difficult, and measuring various parts of the funnel may be impossible. Ultimately, there is no substitute for human creativity, experience, discipline, diligence and judgement.

TAKEAWAYS

Design a framework by breaking your problem into parts (such as levels of a funnel) and deriving a formula that connects those parts.

Ensure your frameworks logically connect an overall desired outcome to its levers and the actions you can take.

Excellence in analysis, design, implementation, validation and optimization is necessary to reap the benefits of frameworks.

This story is published in The Startup, Medium's largest entrepreneurship publication followed by 343,876+ people.

# data[].attributes.attachment_link.description
For a product to succeed over a long period of time, several conditions must be present: product-market fit, positive unit economics, and the ability to scale and grow. Product-market fit requires a deeply engaging product to which people return because they find genuine value.

# data[].attributes.attachment_link.readable_text
Wouldn't it be nice if every new user continued to engage with your product forever? That's retention - a measure of the people who tried your product and liked it enough to return.

Many consumer-facing businesses define retention in terms of the percentage of users who take a given action (such as logging in, sending a message, etc.) within a certain period of time following sign-up. Subscription and SaaS businesses tend to define retention in terms of dollars spent within a certain period of time after the customer's first spend.

In any case, retention is a good way to measure product-market fit and by far the best lever for product growth. Without retention, a growing product will eventually be left with no users. Retention affects every significant metric and is discussed in the context of other product concepts in our blog post on product health.

While retention is framed primarily as the percentage of users returning to the product itself, it is also useful to understanding specific product features and subpopulations of users. For example, you can examine how retention varies by geography, gender or behavioral characteristics (e.g., daytime vs. nighttime use) to paint a clearer picture of your users. Similarly, at the feature level, you can examine how users interact with individual features and then use that knowledge to guide prioritization and the product roadmap.

Before you invest in acquiring new users through marketing or paid channels, you should understand how to strengthen and stabilize retention for early users. If retention is weak, few users will stay long-term, and you will churn through the total addressable market with little to show for it - at great expense.

UNDERSTANDING RETENTION

Retention is generally depicted by plotting the percentage of users who return to either the product or a core feature over time. There are three archetypal retention curves (see Figure 1).

Flattening curves This archetype suggests that a percentage of users who sampled the product found value in it and continued to return to it over time. Not all flattening curves are equal, however. The higher the level at which the curve flattens, the higher the long-term retention and the healthier the product. In Figure 2, the curve 2 flattens at a higher level, indicating a healthier product.

Declining curves When a product has not achieved product-market fit, the retention curve will continuously decline, eventually reaching very few or zero users. In this situation, it is important to focus on changing the product to find a value proposition for a core set of users and then expanding from that set. As shown in the case of BranchOut, applying growth-hacking techniques at the top of the funnel without fixing underlying weak retention will lead to a "leaky bucket" outcome, in which users simply pass through your product.

Smiling curves When a product is truly exceptional, its retention curve will actually rise as product development and network effects propel churned users to return during a Hyper Growth phase. In these situations (see Figure 3), users eventually return to the product eventually after initially churning away.

For most products, including those with smiling retention curves, retention will eventually trend to zero as the product is disrupted by competition, shifts in user behavior and other factors. For games in particular, the time frame of these shifts tends to be relatively short as users move on - for example, Angry Birds was extremely popular in the early part of this decade, amassing tens of millions of monthly users, but currently has only a few hundred thousand.

RETENTION MEASUREMENTS

Retention is measured relative to two factors: time frames and events.

Depending on the vertical of your product, a daily, weekly or monthly time frame may be most meaningful. An e-commerce or travel product might expect users to come back once per quarter, whereas a social app or game would expect daily usage. Measuring retention against these expected time frames is key; to look at monthly retention for a product with a natural weekly frequency, for example, would lead to an inaccurate account of product health.

When considering events as they relate to retention, it's also important to define what constitutes meaningful activity. Is a monthly active user anyone who simply logs in at least once per month? Anyone who has a five-minute session in a month? Purchases an item? Posts a message?

For example, a ridesharing app such as Uber or Lyft would define an event as a completed ride, rather than an app open. Content streaming services such as Spotify and Netflix might define an event as beginning to listen to, or view, a piece of content. For a messaging app, it might be when a user writes or reads a message.

In addition to product-level retention, you may also want to define feature-level retention, where the event is defined as using a particular feature of the product. For example, Amazon could measure how its Wishlist feature is used to help prioritize feature development and the product roadmap.

Triangle retention chart Retention by temporal cohort is a useful visualization to monitor frequently. The most common way to visualize temporal cohort retention is with a triangle retention chart (Figure 4). Each row of the chart corresponds to a temporal cohort, with the size of the cohort, in users or dollars, listed in the first column (0). Subsequent columns (1-22) show the percentage of that cohort remaining after a given period of time.

By assigning colors to certain percentage values, you can quickly identify changes in retention, which will typically appear as horizontal, diagonal or vertical features.

Horizontal features identify cohort-specific traits. For example, if you were to run an acquisition campaign or expand to a new market expansion in a given month, you might see a horizontal feature emerge showing improved or diminished retention particular to that cohort. In Figure 4, the February 2016 cohort is significantly larger than previous or subsequent cohorts (perhaps as a result of an experimental acquisition campaign or a referral feature), and retention for that cohort is relatively poor.

Diagonal features are usually a result of product feature releases, news or other events that affect overall usage. For example, the #DeleteUber campaign affected every Uber cohort, not just the ones who started using the service in January 2017 (when the campaign began to trend). Diagonal features may also indicate product outages or breakage: when AWS had service outages in early 2017, many companies' retention charts exhibited a diagonal feature for that day, as users couldn't access their products.

Vertical features are commonly seen in subscription businesses that have annual plans, as well as in those that offer trials. For example, a dollar or transaction retention chart for Amazon Prime would likely show a significant vertical feature every 12 months, when portions of cohorts renew their yearly memberships.

In addition to helping visualize retention, the first column of a triangle retention chart can give you a sense of your new user growth: Is it increasing? Do you see more users signing up on the weekends? You may also see the impact of exogenous events as a suppressed or increased new user sign-up in the first column.

Since total retention is the weighted average of your cohort retentions, it is especially important to track any cohorts that are dramatically larger than others. For example, if in January the product acquires 10 times the number of total users acquired in December, your overall retention is more or less determined by the January cohort.

Composing time frames As discussed above, it is important to choose the appropriate time frame when measuring retention. For monthly, quarterly or annual products, however, there may be a long delay between implementing a product change or updating an acquisition strategy and understanding the impact on retention.

In these instances, we can break down long-term retention of a cohort (364 days in the equation below) into multiple ratios. This equation can be understood from left to right as a funnel of users, where D1/D0, or the "D1 retention rate," is the fraction of your cohort retained for one day, and so on. (D0 is the number of installers in a cohort and D1 is the number in that cohort who still use the product after one day.)

If D7/D1 stays relatively constant for all cohorts, but the D1 retention rate appears to be declining, focus your attention on improving D1 retention, as this will likely be the biggest lever for long-term retention. Likewise, if D1 retention is flat but D7/D1 is declining, focus on finding new ways to engage users in the first week rather than the first day. Ultimately, your goal is to increase long-term retention, but it is important to monitor these fractions, as well, as they are early indicators of future retention trends.

HOW TO IMPROVE RETENTION

Tactics for improving retention vary widely based on the circumstances of the product, technology and user base. Once you have decided on the right retention metric, your goal will depend on your type of retention curve, as defined in Figure 1:

Declining curves: Flatten the retention curve of each cohort.

Flattening curves: Elevate the long-term retention of flattened curves.

Smiling curves: Celebrate!

The techniques below will help you improve retention for different segments of your population, but the unifying theme across all populations is to increase engagement. As we discuss in a previous post, engagement drives retention. (We will offer additional guidance on engagement in a future post.)

Engage retained users Retaining existing users is key to growth and is accomplished primarily through engaging them - by providing value in the product. Your most important users are the "super users" who are most engaged and thus best retained. Understanding how this group interacts with the product is the best way to improve overall retention.

To identify your super users, first hypothesize which features are core to the product's value proposition, which is frequently referred to as the product's "magical moment." PayPal's magical moment is a successful transaction; for Amazon, it is a seamless delivery experience. Once you've identified your "magical moment," segment your users based on how, and how frequently, they engage with it. (We plan to cover this topic in more detail in a future post on exploratory analysis.) The most frequent users of any particularly valuable feature are your super users: they watch the most videos, share the most content, create the most messages, etc.

Once you've identified these users, examine their prior behavior. What features did they use early on? Did they encounter the magical moment a certain number of times before they reached a "tipping point" and became truly hooked on the product? For example, Facebook famously discovered that connecting to seven friends in 10 days greatly improved retention - which led to a product strategy that encourages users to reach that milestone.

Early on, you should focus on building a product that will delight super users. Then, once you have identified their key behaviors and engagement tipping points, you should use those insights to create a playbook for motivating less-engaged users to behave similarly, which will ultimately drive better retention.

Remove friction for new users The most efficient way to improve retention is to bend the shape of the retention curve as early as possible. For most products, especially new ones, the curve is fairly steep for the first several days, weeks and months. Users churned during this period represent the vast majority of all churned users, so driving new user engagement (and thus retention) early on is especially valuable.

To improve retention among new users, it is important to understand the sign-up (or acquisition) funnel. For example, how many people viewed an advertisement for the product, and on which channels? How many then clicked on that ad? How many then created an account, and how many were retained 7 days later? 24 days later? 84 days later?

Studying the drop-offs in users at each stage will help you understand the efficacy of your acquisition channels (paid and unpaid) and identify any problems in sign-up and onboarding flow. For example, a significant drop-off between sign-up and account confirmation suggests an issue with confirmation email delivery. As another example, when the team at Poncho, a weather chatbot, studied and revised the number of questions in their sign-up flow, their seven-day retention increased from 60 percent to 80 percent. To further isolate points of significant drop-off, filter your data by channel, device, etc. Removing these sources of friction helps new users arrive at the "magical moment" as quickly as possible, which will dramatically improve retention in this group.

A well-designed onboarding guide that clearly communicates your product's value proposition and guides new users toward the tipping point of engagement will also strongly influence retention - not only for new users, but for all users.

GUIDING PRINCIPLES

Benchmark your product To establish the "correct" level of long-term retention for your application, set benchmarks based on similar products in your genre. For example, a game developer might compare their product to other games of the same type.

Table 1 shows the difference in 30-day retention rates between several successful casual puzzle games (7-10 percent) versus well-known strategy games (more than 30 percent). A casual game that has 20+ percent D30 retention and a 35 percent D30/D1 ratio would be well ahead of top casual games such as Angry Birds. But a social or strategy game with the same metrics would be significantly behind the leaders in its genre.

Engagement drives retention Because engaged users are less likely to churn, engagement metrics such as the ratio of daily active users (DAU) to monthly active users (MAU) correlate with and drive retention. Similarly, slipping engagement metrics are leading indicators of problems with downstream retention. If engagement declines, act quickly to remedy the situation before retention is affected.

Reduce churn early Monitoring and acting on changes in early retention (D1, D7, D28, etc.) are crucial to achieving strong long-term retention. For most products, users churn early; stanching this outflow will lead to better retention over longer time frames.

Newer cohorts retention depends strongly on the product type, phase of the product and its market penetration For most products, the earliest adopters of the product are generally the most excited about the product and retain the best. Until the product achieves product market fit, the later cohorts generally retain worse than the earliest adopters for whom the product fits a more immediate need. As the product starts to do better achieving strong product market fit and enters the growth and hyper-growth phases, the retention of users improves again as more people begin to see the value of the product. As such, the cohort retention of products may vary a lot depending on the phase of growth.

Moreover, products with network effects will likely see retention correlate across cohorts. Early on, when there are few users, users in cohorts may churn as there are few relevant connections (social products) or content available (professionally and user generated content products). In the growth and hyper growth phase for network-driven products, more users are drawn to the product and generate more content increasing retention and in turn resurrection from older cohorts. For non-social products, the retention of newer cohorts may not be correlated with how well older cohorts are doing.

For both network-driven and non-network-driven products, cohort retention will fall once penetration is very high - the marginal user at this point will be at the periphery of the core user.

Retention can be applied at a feature level The retention characteristics of your product may not mimic those of individual features. For example, a product might have excellent retention overall, while retention of its messaging and image-upload features is significantly lower. Wide variations are not uncommon between individual features. To ensure your product reaches its full potential, you should recognize this reality and analyze the retention of individual features as well as of the product itself.

CLOSING THOUGHTS

Retention can be used to understand how well your product is growing overall, to evaluate its use among subsets of users and to determine how specific features are performing. Ultimately, retention is driven by increased engagement and removing friction. Exposing users to your product's magical moment as early and as frequently as possible will reinforce your product's value proposition and drive them to return to the product, as well as to engage more deeply and more frequently. This will first increase short-term retention and later manifest in better long-term retention, putting your product on the road to sustainable growth.

TAKEAWAYS

Retention is a measure of the people who tried your product and liked it enough to return; as such, it underpins all significant metrics.

Think of retention as a funnel. Breaking down the long-term retention of a cohort into multiple ratios can help you identify the early indicators of long-term retention as well as the biggest levers for long-term retention.

Techniques for improving retention vary by user population. Retained users should be increasingly engaged, while new users should experience minimal onboarding friction. Exploratory data analysis will help identify precise levers for each population.

This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email data-science@sequoiacap.com with questions, comments and other feedback.
Previous Article: Data-informed product building playbook.

# data[].attributes.attachment_link.description
Wouldn't it be nice if every new user continued to engage with your product forever? That's retention - a measure of the people who tried your product and liked it enough to return. Many consumer-facing businesses define retention in terms of the percentage of users who take a given action (such as logging in, sending a message, etc.)

# data[].attributes.attachment_link.readable_text
All great products have an element of magic and provide true value. They deliver highly engaging experiences. You are alone and bored one Saturday morning and thinking about what you want to do. You ponder over your choices and eventually decide to go see a movie at the theater. It has your favorite actor in it which motivates you. The movie is gripping. As you are watching the movie, there is a constant feeling of suspense. You are staying upright and you have lost sight of time and space. This type of experience is engaging. It entertains our senses and leaves us with a special feeling!

Strong products deliver such engaging experiences multiple times a day, providing inherent value. When users truly love your product, they will come back to it more often, increasing number of sessions per week (L5+/L7 in Figure 1), spending more time with it (time spent, or TS), and eventually becoming daily active users (DAU).

Such users are also highly retained, which increases a product's retention (say, D1 and D7). And because DAUs continue to be weekly active users (WAU) and monthly active users (MAU), as well, they increase the intensity of engagement for each group (i.e., DAU/MAU and DAU/WAU). In other words, engagement drives stickiness, which drives retention - and that, in turn, drives growth.

Thus, engagement is necessary to achieve both sustainable growth and true product-market fit, and getting people to truly love your product should be the goal of your Product team.

ENGAGEMENT AND PRODUCT TYPE

The nuances of engagement vary based on the type of product. For example, the production-consumption framework of user-generated content (e.g., Facebook, Instagram, Snapchat, YouTube) is different than that of professionally generated content (e.g., Netflix, HBO). Similarly, engagement for marketplaces like eBay, Amazon and Airbnb differs from engagement for messaging products like iMessage, WhatsApp and Facebook Messenger. The consumption surface (like News Feed) and device (Mobile versus TV versus Desktop) also play a significant role in engagement.

Therefore, to provide thorough guidance on building highly engaging, data-informed products, we plan to provide frameworks for understanding engagement in the context of various product types: News Feeds, Professionally Generated Content, Marketplaces and Messaging. In the next several posts, we will delve deep into News Feed engagement, Content Production, Connections & Inventory, Activity Feed Ranking, & Consumption.
This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email

data-science@sequoiacap.com

with questions, comments and other feedback.
This story is published in The Startup, Medium's largest entrepreneurship publication followed by +367,690 people.

# data[].attributes.attachment_link.description
All great products have an element of magic and provide true value. They deliver highly engaging experiences. You are alone and bored one Saturday morning and thinking about what you want to do. You ponder over your choices and eventually decide to go see a movie at the theater.

# data[].attributes.attachment_link.readable_text
One question I often get asked by people starting out on growth is "what metrics should be in my growth dashboard?". I've written before about what metrics we value at Pinterest. In this post however, I'll give people a peek behind the scenes and share what our internal growth dashboard looks like.

We have organized our dashboard to reflect our user growth model. We start with our top line growth metric of MAUs. Then we follow the user lifecycle funnel; starting with acquisition metrics, followed by activation, engagement, and finally resurrection.

MAUs

1. Current progress to goal: Current number of MAUs & how much progress we've made towards our quarterly MAU goal.

2. MAU Forecast: Forecast of the number of MAUs we could expect to have extrapolated from our growth rate at the same time during previous years. We include this metric to help us anticipate the effect of seasonality on our growth numbers.
3. MAUs by app 4. MAUs by gender
5. MAUs by country: Tracking total number of MAUs in every single country would obviously be overwhelming to view on a chart, so instead we bucket countries together. The buckets we use are USA, Tier 1, Tier 2, Tier 3, and Rest of World. The tiers are based on size of Internet population, Internet ad spending, etc.

6. MAU Accounting: The MAU accounting helps us see what factors are contributing the most to our MAU growth. Specifically we split out total number of signups, resurrections, existing users churning out and new users churning out.

Acquisition
7. Total signups 8. Signups by app 9. Signups by referrer 10, 11, 12. Invites Sent, Unique Invite Senders, and Invite Signups
Activation

13. Overall Activation Rate: 1d7s is a term we use to refer to users who come back 1 or more times in the week following signup. We measure overall activation rate as 1d7s/signups, or in other words, the percentage of new signups that visit Pinterest again in the week following signup.

14. Activation by app: This is the same metric of 1d7s/signups split out by platform. We've seen that different platforms can actually have pretty dramatic differences in activation rates.
15. Activation by referrer source 16. Activation by gender
17. Overall signups to 1rc7: This metric is similar to the signups to 1d7 except it measures the percentage of new signups that repinned a pin or clicked on a pin in the week following signup. We use this metric to measure as a leading indicator of how well we are activating users into the highly engaged user buckets
18. 1rc7s by app 19. Signups to 1rc7 ratio by app
20. Signups to engagement funnel by app: This metric tracks the percentage of new signups that are still doing key actions during a one-week time window of 28-35 days after signup. Specifically, we track 35 days after a user signs up, what percentage of them are still an MAU, WARC (weekly active repinner or clicker), WAC (weekly active clicker), or WAR (weekly active repinner).
21. Signup engagement funnel by gender
22. Signups to WAU 35 days after signup: This is one of our key activation metrics. We track the total percentage of users who are still a WAU one month after signup. Specifically we look to see what percentage of signups were active between 28-35 days after signup.

Engagement

23. *AU ratios: We track the ratio of DAUs to MAUs, WAUs to MAUs, and DAUs to WAUs. The ratio between *AUs is a popular metric to gauge how engaged users are with your app.

24. Email Summary by type: Table of total number of emails sent, opened, & clicked-through split out by email type.

25. Push notification summary by type: Table of total number of push notifications sent & opened split out by type and by platform (iOS & Android).

Resurrections

26. Resurrections by platform: Total number of users that were dormant for 28+ days, but then came back to Pinterest, split out by which platform they came back on.
27. Resurrections by referrer
To wrap up, you can see we put a big emphasis on activation (the process of getting a new user to convert to a MAU). This is because we consider activation critical to long-term sustainable growth. Strong activation rates are necessary if you want to be able to scale a service to hundreds of millions of users. We also put an emphasis on segmentation by gender, country, referrer, etc., to more deeply understand how different segments of users interact with Pinterest and see which segments are underperforming. If you have any questions feel free to ping me on twitter or drop me a line.

# data[].attributes.attachment_link.title
The 27 Metrics in Pinterest's Internal Growth Dashboard

# data[].attributes.attachment_link.description
One question I often get asked by people starting out on growth is "what metrics should be in my growth dashboard?". I've written before about what metrics we value at Pinterest. In this post however, I'll give people a peek behind the scenes and share what our internal growth dashboard looks like.

# data[].attributes.attachment_link.readable_text
To succeed in the long term, a product must create real value. When it is deeply engaging and has high adoption, users will return to it frequently of their own volition. Most successful, healthy products can grow sustainably for long periods of time because they address fundamental problems.

Sustainable growth is growth that can be maintained without exhausting future resources. There is a trade-off between rapid growth today and growth in the future; rapid growth today may exhaust the addressable market and make it impossible to sustain your product over a longer period of time. For example, if your product is growing too quickly and new users are churning in large numbers, those users may never try your product again, and you will permanently lose the chance to make them active users. If instead your product grows slowly but retains a large percentage of users, it will likely grow more sustainably.

Most of the time, unsustainable growth is due to lack of product-market fit. A product that offers little to no unique value and grows purely through inorganic means is not likely to sustain itself over the long term. Furthermore, even when a product starts out growing sustainably, competition or market saturation may later dampen that momentum.

Understanding sustainable growth and how to create it is paramount to product success. In this post, we offer a framework you can use to determine whether your product is growing sustainably - as well as steps to take if it is not.

FRAMEWORK: KEY FACTORS

Fundamentally, sustainable growth depends on two key factors: product-market fit and positive net growth. The best indicator of product-market fit is retention; a great product not only gains new users at a healthy pace, but keeps those users coming back because they find value in the product.

The metric that best represents net growth is the " quick ratio," or the ratio of the sum of new users and resurrected users to churned users. A successful product must have both a high retention rate (compared to successful products in similar genres) and a quick ratio of more than one, which indicates positive net growth.

Based on these two key factors, the current growth status of your product may fall into one of four scenarios:

Leaky bucket: The product has positive net growth but does not have product-market fit.

Death spiral: The product has neither positive net growth nor product-market fit. Active users are declining with poor retention, and the product has little chance to succeed.

End of life: Growth is declining, but the product has a core set of users that continue to retain. Early-stage companies experience this scenario when they find product-market fit with a small group of users but are losing active users overall.

Sustainable growth: Active users are continuing to grow and users are retained well. This is the goal for all products.

In our exploration of these growth scenarios, we will review several case studies in which products evolved from one scenario to another - including one company that began as a leaky bucket and shifted to a death spiral, another that started out sustainable but has since transitioned into the end-of-life phase, and yet another that began as a sustainable product and remains so today.

LEAKY BUCKET
Characteristic metrics

High quick ratio (> 1)

Low retention rates

A leaky bucket product has positive net growth (meaning the overall number of active users is growing), but very few of those active users are retained. Generally, this happens when a product offers little or no unique value proposition for the majority of users - in other words, it has no product-market fit. Such a product may have acquired its users largely through viral, inorganic means or growth-hacking tactics. (Growth hacking aims to increase the user base as quickly as possible through content and product marketing using organic and paid channels.)

Case study: BranchOut BranchOut was a Facebook application launched in 2010 designed to help users leverage their social networks to build their professional networks. BranchOut used a growth-hacking tactic to attract new users; the app encouraged users to invite as many friends as possible by giving them only two options: send invitations to all friends or none. By manipulating Facebook's referrals system, BranchOut quickly reached nearly 400M of Facebook's monthly active users (MAU) - at the time, nearly half of the platform's overall MAU. But most of these users were not genuinely interested in using the product.

At the peak of its Hyper Growth phase, BranchOut had more than 12M MAU of its own and averaged more than 250,000 downloads per day for an entire month. However, as its invites penetrated more of Facebook's user base, they often reached people who had already been invited and churned, rather than those who were new to BranchOut. As such, top-line new user growth slowed.

Moreover, less than one percent returned to BranchOut the day after they joined the platform. The app's retention rates were extremely low because BranchOut provided no clear and differentiated value proposition and thus had no product-market fit; users came to the app, found little utility and left. BranchOut penetrated and churned through nearly all of its total addressable market and active users declined from 14M MAU to 3M MAU over a few months.

BranchOut started out as a leaky bucket with low retention and positive net growth, then rapidly moved to the death spiral scenario due to a combination of slowing new user growth and abysmally low retention rates (see Figure 4). This transition is reflected in the changes of the product's quick ratio (see Figure 3), which had an an extremely high value (~3) as BranchOut grew in March, but declined soon afterward as most of the users gained in March churned at a rate that outpaced growth from new and resurrected users.

What can you do if this is your product? If your product is a leaky bucket, start by slowing down acquisition of new users and focusing on retention. Divert your attention to achieving product-market fit and ensuring that you have a core audience that loves your product and is deeply engaged with it. Use exploratory analysis to understand what keeps those most engaged users coming back to your product, then figure out how to make more users like them.

DEATH SPIRAL
Characteristic metrics

Low quick ratio (< 1)

Low retention rates

When a product is in a death spiral, it has negative growth and no product-market fit. As such, the quick ratio for a product in this scenario is less than one; churn outpaces resurrection and new user growth. Moreover, because the product offers little unique value, it is not retentive. As discussed above, "death spiral" is an apt description of BranchOut's scenario months after its glory days.

What can you do if this is your product? If your product is in a death spiral, go back to the drawing board and rethink your product to determine how you can achieve product-market fit. Talk to customers and consider moving into different markets. Do not attempt to artificially stimulate top-of-funnel growth, as this will only lead you to churn through your addressable market and enter the unsustainable leaky bucket stage.

END OF LIFE
Characteristic metrics

Low to average quick ratio (< 1)

High retention rates

A product in an "end of life" scenario has a core set of highly-engaged users but is struggling to grow, perhaps due to high penetration of a target market or lackluster growth-hacking efforts. Two types of products generally fall into this scenario:

Early Phase: When an early-phase product has high retention but is losing active users and does not have high penetration, growth tactics may help it reach sustainability.

End of Life: This type of product likely started in a different scenario in its early phase before moving toward "end of life." When such a product continues to lose users and fails to grow, but a small core of highly retained users is keeping it alive, the death of the product is inevitable.

What can you do if this is your product? First, understand the reason behind your sluggish growth. Perhaps you have penetrated your target market, but perhaps you simply have not yet identified all of the users who may love your product. If you do have high penetration, look for opportunities to begin a "second act" by leveraging your existing user base to pivot into a new market (this happens frequently in gaming).

If, however, you feel you have not yet penetrated your target market, sustainable growth may be well within reach, and you should consider investing in marketing and growth campaigns. In such a case, you may be able to capitalize on your ability to retain a core group of users who love your product, in order to catalyze growth.

SUSTAINABLE GROWTH
Characteristic metrics

High quick ratio (> 1)

High retention rates

When you have a successful product - that is, a product that users love and are highly engaged with and which retains users well - you can grow sustainably for a long period of time. Two types of products generally fall under this scenario:

Single Act: A single-act company has found product-market fit and continues to grow sustainably until it achieves high penetration of the addressable market, when it naturally slows down. Eventually, churn and resurrection balance exactly, and net growth becomes zero. In many cases, such a company's failure to adapt to competition or changes in the ecosystem will push it into the "end of life" scenario. (Timehop is an example of this, as described below.)

Multiple Acts: A multiple-act company continuously evolves, building parallel products or expanding into new regions or verticals (see Figure 5). At such a company, the focus is on sustainable growth for each of its individual products, which allows sustainable growth for the company as a whole. Without multiple acts, a company will eventually move into an "end of life" scenario due to sluggish or negative growth. (For more on multiple acts, see the Netflix case study below.)

Case study: Timehop The Timehop app, which launched in 2011, is similar to Facebook's "On This Day" feature; each day, it shows users the social media photos and other posts they shared on that date in previous years. At its peak around May 2015, Timehop had more than three million MAU on Android, most of whom were using the app on a daily basis (70 percent DAU/MAU ratio) and half of whom continued to use it after a month (50 percent D30 retention rate). By any measure, these stickiness and retention metrics seemed to suggest Timehop had product-market fit. As seen in Figure 6, however, Timehop's MAU's subsequently declined.

While the app enjoyed strong retention and significant growth, it was entirely notification-driven and engagement was poor. Users spent an average of one minute in Timehop after receiving a notification, and most did not open the app again for the rest of the day. Users were not returning "voluntarily" and averaged just one session per day.

Furthermore, with the launch of Facebook's "On This Day" in 2015, Timehop no longer provided a distinctive value and downloads decreased dramatically (see Figure 7). New users never installed it, those who had been active failed to reinstall when they bought new phones, and the app saw a gradual drop in usage. What Timehop failed to do was capitalize on its existing customer base and launch a "second act"; instead, the app simply fell out of popularity and moved to an "end of life" scenario.

Case study: Netflix Netflix is classic example of a highly retentive product that successfully executed multiple acts, which enabled significant success and sustainable growth while the company entered new markets.

Netflix's first act was its U.S. launch in 1997; it achieved and maintained product-market fit and growth from a single product for more than a decade. The company's second act came when it expanded into new geographies: in 2012, new users in the United States accounted for eight percent of Netflix's new users overall; by 2018, U.S. new users were just one percent of the total. Expansion into international markets allowed the company to continue healthy growth overall as its growth in the U.S. slowed (Figure 8).

Recall the overlapping S-curves in Figure 5, where each curve represents a new product that individually exhibits sustainable growth. The same concept can be applied to Netflix's expansion into new geographies; overlapping curves allow for continued growth even as the individual markets become saturated. Despite decreases in new users as a percentage of overall users in the U.S. (Figure 9) and Spain (Figure 10), for example, Netflix's overall growth continued as the company launched in other new countries.

What can you do if this is your product? If your product is highly retentive and experiencing strong growth, make sure it continues by launching a "second act" during the growth phase of your product and capitalizing on your existing user base to fuel future growth. If you don't launch a second act, your product can eventually fail.

OTHER APPLICATIONS

While we have discussed the use of this framework primarily at the company and product levels, it can also be applied to sub-products, events and features.

For example, in the case of a product such as Facebook, Reddit or Nextdoor, in which hundreds of millions of users are part of millions of "groups," the product itself may have low retention and high growth (leaky bucket) while different groups map to all four scenarios, including sustainable growth. (The same applies to specific "tabs" within a product, such as "Explore" on Instagram, "Marketplace" on Facebook or "News Feed" on LinkedIn.) The product team's goal should be to identify and understand what's working well and attempt to make the overall product more like those successful groups or tabs.

The reverse may also be true: a product can grow sustainably overall while some individual features do not. Understanding growth at not only the product level, but also feature and event levels, can give you great clarity into overall success.

A NOTE ON ABSOLUTES

In this post, we did not use absolute numbers to define characteristics such "high retention rate" because the desired values of such metrics are highly dependent on the product and space. For example, a social media company would expect high daily retention, since people generally use such products daily. A travel app, however, might have a significantly lower daily or even monthly retention, since most people purchase plane tickets infrequently. To understand the success of your product, you must benchmark relative to similar products in the space.

TAKEAWAYS

A product's current growth status falls into one of four scenarios: "leaky bucket," "death spiral," "end of life" and "sustainable growth."

Retention (product-market fit) and net growth are the two key factors guiding sustainability of your product's growth.

Neither high growth without retention, nor high retention without growth, is sustainable in the long term.

If you have a high-growth, high-retention product, you should actively pursue multiple acts to ensure continued growth.

Within your company, individual product features may grow differently than your overall product and can affect your overall understanding of success.

This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email data-science@sequoiacap.com with questions, comments and other feedback.

# data[].attributes.attachment_link.description
To succeed in the long term, a product must create real value. When it is deeply engaging and has high adoption, users will return to it frequently of their own volition. Most successful, healthy products can grow sustainably for long periods of time because they address fundamental problems. Sustainable growth is growth that can be maintained without exhausting future resources.

# data[].attributes.attachment_link.readable_text
In Part 2 of this series, we examined the ways product factors can affect metrics. This post will explore seasonal factors, which often cause of dramatic fluctuations in key metrics. People behave differently based on time and circumstance, and seasonality is one of the most prominent influences that can be analyzed using data.

SEASONALITY

Seasonality is a broad label for periodic changes in user behavior. For example, people change their behaviors at the beginning of each year - at least for a short period of time - as people take on New Year's resolutions. In Figure 1, we see a significant increase after January 1 in the percentage of people exercising 30 minutes or more. Other examples of seasonality include behavioral changes based on time of day, day of the week, season of the year, etc. It is important to separate these from external factors such as the launch of Pokémon GO! which had a similar effect on physical activity (see Figure 1). We'll discuss these external shifts in next week's post.

ANALYZING SEASONALITY

Seasonality is often the root cause of changes in key metrics; as such, it is the first behavioral change factor to explore.

To properly analyze any behavioral change, you must understand your product's overall ecosystem. For example, it might be important to know what young people do during the summer, how middle-aged women shop, how people using Android behave compared to those using iOS or who is likely to be an early adopter of your product.

Understanding this ecosystem allows you to ask good questions and develop good hypotheses, which is how effective analysis of behavioral change almost always begins. Ask yourself: What is the likely root cause of the change in key metric? You might then hypothesize that the reason is seasonality, that users are getting bored or that there has been a change in perception - positive or negative - of the product. Once you have a comprehensive list of hypotheses for why the key metric might have changed, you're ready to investigate further.

If you are testing a hypothesis that seasonality is a factor in the change, you may take several approaches:
Week-over-week movement
Most products exhibit a very strong day-of-the-week effect, because people behave differently on weekdays than they do on weekends. To remove this effect, use seven-day (or rolling seven-day) averaging of the metric of interest. For example, Figure 2 shows strong weekly patterns in usage of Yelp, with increases on weekends. Without 7-day averaging, any week-to-week changes would be lost in day-to-day fluctuations.

(Note social apps often see similar intraday effects, because most people spend far more time on social apps in the evenings than they do during the day. If the lowest level of granularity you studied for such an app was daily use, you would miss that signal completely.)
Day-over-day movement
If you are interested in examining day-to-day changes, try overlaying each day of the week as seen in Figure 3. This approach will give you more granular information; for example, this shows us that week 3 has higher highs and lower lows compared to other weeks.

Year-over-year changes
Observing annual trends in your data is one of the most powerful ways to identify the effects of seasonality. This method will illuminate the effects of holidays, the back-to-school season, festivals, cold weather, etc. and help you determine whether those impacts on your key metrics are amplifying or diminishing over time.

For example, the chart on the left in Figure 4 shows a seven-day rolling average of revenue for an e-commerce company that is clearly rising. In some cases, where changes are less obvious, a longer rolling average such as 28 days (see the chart on the right in Figure 4) can help demonstrate how a metric is trending over time.

28-day rolling averages, however, tell us relatively little about how a key metric is increasing annually; we see a strong increase toward the end of the year (for the holiday season), but few other patterns are discernible. Instead, we should look at the year-over-year metric (see Figure 5, left), which will tell us much more about seasonal patterns.

The chart on the right in Figure 5 shows the same 28-day rolling average pattern year-over-year; the changes within the year are much more apparent. We can now clearly see that revenue between May and July is influenced by seasonality. Revenue decreases throughout the month of May, begins to sharply increase in early June, then decreases again until around July 4, at which point we see a large bump in both 2016 and 2017, likely due to a sale event. Understanding that these events happen every year will help you plan for the future.

Year-over-year changes can be even better understood by plotting the ratio of revenue from one year to the next (see Figure 6, left). Revenue growth declined year-over-year throughout 2016, but in 2017, things started to improve significantly. Indexing to 1 (see Figure 6, right) is another powerful method to see how things change from the beginning of a year. In this case, it shows 2017 was the best year from a year-over-year perspective, as the company managed to turn things around.

Comparison
You can also analyze seasonality by comparing your key metrics to other apps with similar demographics. For example, if your app targets younger populations and your key metrics shift in August and September, try to obtain data from another app with a similar demographic, such as Snapchat, to see if it experienced a similar shift. If so, the back-to-school effect likely caused your shift.

In this analysis, it is best to average over at least seven days and index to 1, in order to identify macro trends. Comparing your metric to those of apps in different locations (and with different school calendars) can also help identify this effect.

TAKEAWAYS

Behavioral changes based on seasonality often abruptly change key metrics.

Multiple techniques can be used to isolate the impact of seasonality.

This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email data-science@sequoiacap.com with questions, comments and other feedback. Originally published at www.sequoiacap.com

# data[].attributes.attachment_link.description
In Part 2 of this series, we examined the ways product factors can affect metrics. This post will explore seasonal factors, which often cause of dramatic fluctuations in key metrics. People behave differently based on time and circumstance, and seasonality is one of the most prominent influences that can be analyzed using data.

# data[].attributes.attachment_link.readable_text
Monitoring metrics is essential for determining the success of any product. Once a key metric has been selected - one that accurately represents an overarching goal for the product - moving that metric in the right direction becomes the top priority. This series focuses on how to determine the primary drivers of key metric changes and how to analyze them.

Metric changes are almost always due to one or more of the following factors: product changes, seasonal factors, competition, mix shift, and data quality. This seven part series will dive deep into how to think about each of these factors and analyze changes in your product's metrics:

Part 1: Introduction -  7/17

Part 2: Product Changes -  7/17

Part 3: Seasonal Factors -  7/24
Part 4: Competition and Other External Factors - 7/31
Part 6: Data Quality - 8/14

Part 7: Action Plan -  8/21

# data[].attributes.attachment_link.description
Monitoring metrics is essential for determining the success of any product. Once a key metric has been selected - one that accurately represents an overarching goal for the product - moving that metric in the right direction becomes the top priority. This series focuses on how to determine the primary drivers of key metric changes and how to analyze them.

# data[].attributes.attachment_link.readable_text
In Part 3 of this series, we covered how seasonality affects product usage. User behavior is also affected by competition and other external events, such as government actions, new products and social media campaigns. These factors can dramatically change how your users engage with your product. When Turkey blocked access to Facebook in 2016, the site's number of daily active users dropped. And Uber saw a dramatic drop in usage and overall market share due to the #DeleteUber campaign (see Figure 1).

Generally, the larger the change in a key metric and the shorter the time frame in which it changes, the easier it is to identify the cause. This is particularly true with many external events, although the effects of competition are often difficult to quantify.

ANALYZING COMPETITION

The impact of competition on your product can be subtle and difficult to identify - and even harder to fix. This is because competition generally manifests as churn; while we can identify that people are churning, it is difficult if not impossible to know whether people are leaving for a competitor without user experience (UX) research.

Consider the example from the music industry show in Figure 2. Pandora was growing steadily until Spotify's growth started to pick up speed.

It is likely that users began moving from Pandora to Spotify, further accelerating Spotify's growth and flattening Pandora's. However, it is difficult to quantify this shift precisely.

Leveraging third-party datasets that cover your industry (App Annie for usage, credit card panels for consumer spending, etc.) and frequently commissioning user surveys will help you understand how competition is cannibalizing your user base. But this is very hard to do well, and even the best analysis is usually directional. We will dedicate a future post to understanding competition, cannibalization and incrementality.

ANALYZING EVENTS

Understanding the impact of a singular external events is often relatively easy, as such events often trigger abrupt movement in key metrics that make correlations clear. The example of the #DeleteUber campaign (in Figure 1) is one such case.

ANALYZING MACRO TRENDS

Similar to behavioral changes due to competition, long-term behavioral trends are challenging to detect but can be guided by UX research.

These macro changes are often driven by external events - for example, the introduction of low-cost 4G connectivity in India (Reliance Jio) made internet access in that country cheaper and more reliable, which in turn dramatically shifted the time people in India spent on YouTube (see Figure 3).

Of course, the impacts of such macro changes are not all positive. In Figure 4, we see how increased access to broadband internet changed the way consumers obtained their news. While Google's ad sales dramatically improved, American newspapers saw print circulation decline quickly, which later manifested in significantly lower revenue from ad sales.

CLOSING THOUGHTS

External factors can dramatically affect the performance of your product, and most - aside from macro trends - are difficult to predict. However, if you are disciplined in monitoring key metrics and ensure your company has strong analytical DNA, you will be better positioned to take advantage of opportunities and mitigate the negative impacts of external events.

TAKEAWAYS

Effects from competition are difficult to measure and address but industry data sources can offer direction.

Singular external events usually result in abrupt changes to metrics.

Macro trends may dramatically affect metrics for your product, but will usually do so over an extended period of time.

# data[].attributes.attachment_link.description
In Part 3 of this series, we covered how seasonality affects product usage. User behavior is also affected by competition and other external events, such as government actions, new products and social media campaigns. These factors can dramatically change how your users engage with your product.

# data[].attributes.attachment_link.readable_text
The hardest thing about building a mobile company isn't gaining users; it's retaining them. And by this measure most social apps, including Snapchat, Vine and Twitter, are doing badly compared to Facebook and, surprisingly, Blackberry Messenger, particularly in Asia's emerging markets.

The Information asked a startup called Quettra for user-retention information for 19 well-known social apps from the first quarter of this year. The company is in the business of building tools to help app developers boost retention and has a vested interested in showing how apps lose users.

Quettra's code, in the form of a software-developer kit, is installed on more than 75 million Android phones around the world-roughly 6 percent of the total number of active Android phones-thanks to partnerships with certain app developers. That gives Quettra visibility into how many times and for how long a phone owner accessed various apps. Half the Android phones it analyzes are in Asia (except for China); the rest are spread relatively evenly across the rest of the world.

# data[].attributes.attachment_link.title
Which Apps Retain Their Users-And Which Ones Don't

# data[].attributes.attachment_link.description
The hardest thing about building a mobile company isn't gaining users; it's retaining them. And by this measure most social apps, including Snapchat, Vine and Twitter, are doing badly compared to Facebook and, surprisingly, Blackberry Messenger, particularly in Asia's emerging markets. The ...

# data[].attributes.attachment_link.readable_text
In the last decade, the combination of greater internet penetration, higher number of connected devices, and cheaper and more accessible storage has caused a huge spike in the volume of user interaction data. This fueled internal demand within companies to unlock actionable insights from the explosion of data. As a result, product analytics as a function began to emerge producing the virtuous cycle of more A/B testing and experimentation leading to faster product iteration leading to accelerated development releases leading to compounding product growth. Increasingly, a company's ability to compete and innovate on a product is driven by how successfully it can apply analytics to mine for insights across vast, unstructured data sets from disparate sources. In other words, the future belongs to data-informed companies.

In a series of documents over the next few months, we will provide deeper guidance on how to build great data-informed companies. There are several characteristics of great data-informed companies but it largely boils down to two - focus on impact and culture. In this document, we will provide guidance on the characteristics of data-informed product companies.

Achieving Full Potential of Product Analytics

Before the recent data explosion, products were primarily built using intuition. Intuition can be grounded in data but not necessarily in a structured form. Intuition comes from processing all of the available quantitative and qualitative data subconsciously. Because the data is limited to what you see and what you have learned and the fully scientific method of data science has not been followed, it generally has biases, and may result in flawed decision-making with significant risks.

This doesn't mean that intuition has no place in product development. It remains extremely valuable during the early stages of product development when data doesn't exist. At this stage, much of the approach to product development depends heavily on design and engineering and less so on data science. This was also true a decade ago when there was far less data captured and analyzed for even more advanced products. As a result, most early products were design and intuition driven.

With the explosion of data, companies began to find ways to effectively utilize data to drive impact and value. As Analytics rose as a function and started to show greater value, Data Science, Data Engineering and Data Infrastructure, became key functions within product organizations.

Early data work was all about counting numbers. (How many users do we have? What is our revenue?) There was really no science behind the data. The next phase of data analytics was automating the counting by building dashboards and visualizations. Most of this phase was about logging the right data, ensuring that it was of high quality and correctly defined, and creating the right pipelines and tools to automate all of these tasks in a scalable way. This was the essence of data infrastructure and data engineering functions.

The next phase in the evolution of data analytics was to ensure that the right products and features were being shipped. This is the advent of data science - applying the scientific method to product development. Experimentation was a big part of this evolution for two reasons: power of incrementality and failure of intuition. First, although we tend to seek big wins, success is more often about getting many small things right. Over time, incrementality and the power of compounding are what ultimately drive most product successes. Second, our intuition is very fallible, and in many cases, it is no better than random selection. In order for data-informed companies to see continuous improvement, a "test and learn" culture is necessary.

Finally, data analytics is invaluable not only for counting numbers, building dashboards and shipping products, but for helping to define goals, roadmaps, and strategies. Arguably this is the highest leverage provided by an analytics team. Facebook and Instagram provide good examples of this: Facebook Live, Instagram News Feed Ranking, and the "stories" feature would never have existed without the strategic insights driven by the analytics teams.

In a series of documents, we will shed light on how to build a world class data-informed company that can drive the greatest impact through data and thereby achieving the full potential of product analytics.

Characteristics of a Data-Informed Company

All companies have missions. Data-informed companies define metrics and goals to support their pursuit of the mission. They align the incentive systems (promotions, compensations, etc.) with their pursuit of these goals. In our experience, we have seen the strongest data-informed cultures exhibit the following characteristics - maniacal focus on impact and building a culture of measurement and truth-seeking.

Define Impact

It is hard to define impact without understanding and defining product success. Setting the right goals and metrics helps quantify success of a product. With this in place, one can objectively begin to quantify impact too.

Great data-informed companies are organized and propelled by a unifying metric. A unifying metric is a single, actionable, top-line metric that encapsulates the vision for a product. It should be easy to measure and clearly connected to the business's core drivers. Without a unifying metric and a related goal, it is very hard for a company to become truly data-informed and achieve its highest potential.

However, simply having a unifying metric is not enough. How it gets used matters just as much. It must be deployed by senior leadership to determine and inform what goals and metrics matter. Empowered with this clarity, each individual in the company is able to prioritize their work thus improving productivity and reducing needless debate.

After established what impact means with the help of a goal, the next step is to ensure that we are able to measure impact. Without the instrumentation for measuring impact, one would be flying blind and would not truly know if they are making progress to the goal.

Measure Impact

Impact can generally be broken down into three categories:

1. Moving a metric: Say your team has a monthly active user (MAU) goal. When the team identifies creative ways of increasing MAU and executes against those ideas, then they are creating impact by moving a metric.

2. Influencing a product or business change: Say your team has discovered, through deep exploratory analysis, that focusing on SMS notifications on Android in India would help increase MAU. Influencing that product change is a way of creating impact.

3. Influencing a process change: Say your team has identified a new way to automate or scale, doing more with less. This is creating impact by influencing a process change.

While these are not mutually exclusive or exhaustive, they do serve as good guidelines for measuring impact.

Measuring the movement of a metric requires an agile infrastructure and a culture of experimentation.

Create Agile Infrastructure

A great analytics team comprises of quantitatively minded people (analysts, data scientists), but also requires data engineers (DE) and data infrastructure engineers (DI). Engineering talent build scalable infrastructures ensuring speed, reliability, and useability. A good infrastructure is agile and adaptable for future demands. Companies whose infrastructures are only optimized for short-term needs generally fail to scale.

Most data infrastructure involves some amount of data acquisition (logged product events, sales, external data sources, etc.), transformation of the data into a usable format and loading the data into systems that power reporting, dashboards and ad hoc analysis. The visualizations in the reporting and dashboards are commonly the 'report cards' of the business and should be watched carefully. The purpose of ad hoc analyses undertaken by the analytics teams will frequently be to understand any deviation in the metrics against expected values.

DI groups own the underlying computing machinery and the DE group is responsible for moving data between these systems and working with the analysts to productionize repeated analyses.

Without well-thought and well-constructed infrastructure, teams will be, at best, delayed in their ability to understand phenomena impacting their business, or at worst, not be able to understand them at all.

Have a Culture of Experimentation

While intuition is incredibly valuable for building products, it does not scale. Data-informed companies need to establish a strong test and learn culture, codifying intuition into hypotheses and experimental design. The underpinning of this test and learn methodology is that small improvements every week compound in way that results in much greater impact than large gains infrequently. Not only is it harder to identify ideas that result in huge gain, it does not scale either. As a result, experimentation to drive impact has become common in all highly data-informed companies.

Amazon and Facebook are good examples of how a culture of experimentation allows for hard data to inform product decisions. The two statement below by Jeff Bezos and Mark Zuckerberg established the importance of experimentation to build products.

"Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day." - Jeff Bezos

"One of the things I'm most proud of that is really key to our success is this testing framework.... At any given point in time, there isn't just one version of Facebook running. There are probably 10,000." - Mark Zuckerberg

Have an Expansive View of the Analytics Team

In order to build a truly great data-informed company, analytics must be involved at every stage of product development and needs to be embedded within the product team. At the outset, the analytics team should help craft the relevant metrics to your product's success, measure progress continuously and help identify risks and growth areas for the business.

Moreover, their data-driven insights must be a constant input to product development. Generally the core product team consists of Engineering, Design, Product Management, Analytics and User Experience Research. Here is how a product team generally functions in many companies.

1. Engineering writes code;

2. Design makes the product easy to use and look good;

3. Product management is responsible for vision, strategy, and execution;

4. Analytics sets goals, creates a roadmap, and determines strategy; and

5. User experience research fosters understanding of user behavior.

It is important to use the analytics team for the highest leverage problems and have them produce the greatest impact - ensuring that they can help drive goals roadmap and strategy for the product.

Have a Rigorous Roadmap Process

Fast product development, testing and iteration require an efficient roadmap process. For example, the growth team at Facebook has a product building process that lasts 10 weeks. This entire cycle has three phases: understand, identify, and execute.

In the understand phase, the analytics team conducts deep exploratory analysis on a range of phenomena (e.g., advertiser churn) with the primary goal of creating a roadmap. The focus of this analysis is to identify the issues and opportunities that will lead to the greatest impact. Once the team understands the biggest areas for impact, the entire product team works together to identify a roadmap.

From the roadmap, a list of initiatives, tasks for specific functions, Finally, the team moves into the execution phase. This is the last,and longest, phase of the 10-week cycle.. This process is invaluable for fast iteration and optimization, but it's worth noting that the emphasis of the team is to move fast and not necessarily produce innovation.

Hire Well and Empower

Product analytics is a nascent function and is still evolving. Both data science and data engineering are continuing to find their feet and evolving their vision in even in the most advanced companies. As a result, building a world-class team is harder in analytics than in other functions as there are very few great leaders in this function.

Having said that, building world-class organizations is the only way companies can sustain themselves over decades. Most companies do not go under just because of a lack of ambition or vision; usually poor execution is a root cause. A huge part of sustainable execution is building amazing world class organizations.

Three key dimensions are chiefly valuable to consider when building a world class organization - people, culture and process. Driving impact by hiring A+ players, empowering them to do great work, and mentoring them as future leaders by showing them purpose, setting a bottom-up culture that empowers people to high levels of excellence, building a strong transparent organization that has accountability, ownership and trust as its core values and that focuses on company first, business unit second, team third and individual last.

Final Thoughts

Even with a great infrastructure, a culture of experimentation and having the right processes in place, if the leadership does not fully embrace building a data-informed culture, then it is very hard to fully utilize the value of analytics.

As an example, at Facebook, as the company evolved to be data-informed, "code wins arguments" became "data wins arguments". Product reviews evolved from whiteboarding user flows to debating the problem statement and agreeing on the metric that will define success. The "identify with data and treat with design" is how products improved and evolved. Analytics itself evolved from counting numbers to building dashboards to experimentation to setting goals, roadmap and strategy for the product. All this was only possible because the leadership fully embraced analytics as central to the product.

During this evolution, a scientific method of hypothesis testing was devised, and a scientific method for setting goals and creating roadmaps and strategies is continuously evolving. This continuous improvement is at the heart of building great products, and a data culture that supports this is imperative. Members of the product team need to know how to use data effectively, and to that end it may be worth training every individual on best practices for building data-informed products.

Common Pitfalls that Data-Informed Companies Face

Senior leadership does not appropriately embrace a data-informed culture. If a company's senior leadership does not believe or understand the power of data, it is nearly impossible to build an amazing data-informed company.

Analytics is misused as a service function. Analytics needs to be deeply integrated within the product team and should not be a consulting wing. Analytics teams' goals should be tightly aligned with the product goals and the success and failure of the team should align with that of the products. The most common mistakes include spreading analysts too thin by staffing them across multiple products, treating analytics as a service function to engineering or product management, and not embedding data scientists within product teams.

The role of an analytics team is too narrowly defined. In the worst cases, analytics is treated as a data-pulling team. In contrast, analytics teams should provide substantial leverage in helping to define goals, build roadmaps and develop strategy for products.

The data engineering team is weak. The best way to scale a data science team is to have an excellent data engineering team. When the ratio of data scientists to data engineers gets too high, data scientists often spend too much time pulling data rather than performing analyses that give valuable insights.

There is a lack of a strong tools team. In order to scale the analytics organization, strong tools are needed. This includes data cleaning tools, ETL tools, scientific tools, analysis tools, and visualization tools. Without access to and integration of these tools, it is hard for an analytics team to make significant progress, let alone to scale.

Takeaways

For a company to make tangible and repeatable progress towards its mission, the entire organization must perform at their full potential. Being impact focused and having a data-informed culture are important for leveraging the most from an analytics team and building a truly amazing product.
This work is a product of Sequoia Capital's Data Science team. Chandra Narayanan, Hem Wadhar and Ahry Jeon wrote this post. See the full data science series

. Please email

data-science@sequoiacap.com with questions, comments and other feedback.
This story is published in The Startup, Medium's largest entrepreneurship publication followed by +426,678 people.

# data[].attributes.attachment_link.title
The Building Blocks of a Data-Informed Company

# data[].attributes.attachment_link.description
In the last decade, the combination of greater internet penetration, higher number of connected devices, and cheaper and more accessible storage has caused a huge spike in the volume of user interaction data. This fueled internal demand within companies to unlock actionable insights from the explosion of data.

# data[].attributes.attachment_link.readable_text
They said, "If you build it, they will come." They lied. Pour your heart and soul into creating the most beautiful, user-engaging apps. Create content. Promote. Do everything you can, and still, you can't fill the stadium. Why is virality so tough on mobile? How can we aim for virality ourselves?

The elements of app virality: a breakdown

The old fashioned user-to-user sharing of an app gives us a good jumping-off point:

Do it right, and you get a chain. One weak link breaks it, though, so you need to make sure each link is as strong as the one before it.

1. What drives sharing?

INCENTIVES  - The most discussed driver of virality. For Airbnb and Dropbox, great referral programs and virality went hand-in-hand. We've designed hundreds of successful referrals for our clients. They're a reliable way to grow, but often overhyped.

Two-sided referrals are best, as we've learned from the hundreds of referral programs created at Branch. If you can only reward one party, incentivize the new user. This not only ensures that referrals become downloads, it also makes existing users more likely to refer: they feel they're providing value to their friends.

Make it easy for users to share -  one-step sharing is best. The more steps, the easier it will be for your users to drop off.

Never share without a user's explicit consent. Tricking users into sharing might work short term, but it will burn you eventually.

EGO  - Ego is often the biggest driver of virality. Think of test results that prove a high IQ, or surveys announcing how many countries someone's visited. People share when it makes them looks good and what is in line with the image of themselves they they try to project to the outside world.

Enable users to create things they are proud of, and prompt them to share those creations. Default sharing text should highlight how awesome the user is. Make sharing links rich deep links: the actual content should show up in Facebook, Twitter, SMS, and e-mail. Branch deep links do this automatically.

While some will say Uber and Dropbox were shared because of their incentives, Uber and Dropbox were shared because they were amazing products. Users wanted to associate themselves with those brands. The actual referrals were just the cherry on top. The lesson? Make your brand and your product something users want to share.

Creating a brand takes time, but the basics: hire a great designer, never spam users, and create allure by making your product stand out from the competition. Our friends at the League did this wellby creating an invite-only dating app and moving users higher in the waitlist when they invite their friends to the app.

EMOTION - Users click and share because something drives an emotion in them. Studies show this to be true, and what's especially interesting is the disconnect between what drives users to share and what drives them to click. In general according to Jonah Berger and his research in Contagious, high arousal emotions drive people share and positive emotions usually drive more shares (the happier the post, the more likely it is to be shared). However, happy posts aren't necessarily the most likely to be clicked on. More on that later.

PRACTICAL VALUE -  Users share things that drive value for them and their friends. The more value something can provide to their friends, the more likely it is to be shared because it increases the actual social currency of the user sharing. Moreover, if having more users using the app makes the product better for the use sharing, that adds an extra incentive to the user to share. Social networks are great examples of products that grow this way. The downside - getting enough critical mass to provide value to the user he needs to invite his friends.

2. What drives clicks on app links?

Strategy for clicks aligns with strategy for shares. Some key differences:

INCENTIVES  - New users need to know more than the incentive. Don't overemphasize the $25 users get for a download. Instead, show the app's intrinsic value, or even better, let users share something they discovered with their referred friend. Even if sharing isn't incentivized, more intrinsic value means more clicks.

EGO  - Show how an app will impact the clicker's lifestyle. If there's a feature that makes them look cool, highlight that feature. They can edit a photo just as well as their friend did, discover local restaurants like their friend, prove they are smarter than 97.6% of their friends, and so on.

EMOTION  - Strong emotions like anger can drive a lot more clicks than happy emotions.

How to easily test what can drive virality to your app? Facebook ads have made testing virality of content simple. 30 minutes and $10 is enough for a test on which associated emotion drives more clicks and more shares. We used cats, because let's face it, cats rule the internet. We can always learn from cats.

My quick experiment matched what psychologists say: positive and high arousal emotions are more likely to be shared and liked, and negative, controversial emotions drive more clicks.

3. What transforms clicks into app downloads?

App stores are black boxes when it comes to attribution. It's tough to look into what happens where. At Branch, we spend a ton of time making sure that links drive app installs. Check out my other article on what click to install conversions. Some tips:

Consider every device. A user might click links on Web, Android, iOS, Apple watch, etc. Our approach tailors the journey to the platform: the user might go to web content, the app store, a page that lets users text themselves the app, or a web preview of your app content, depending on what's right for each platform.

Track what happens on every device. Pay extra attention to where drop-offs occur.

Lead the user to content with crumbs and previews. If a user clicks on a news article, the next step should tease more of the article to keep interest up. Branch is in beta with Deepviews, a dynamic preview into app content, that makes it easy to do this.

4. How does a download become an activation?

Don't stop at downloads. The next step - activating users once they install - might seem insignificant, but my c o-founder Alex devoted an entire article to it.

A great first-time app experience translates value right away. Customize it. Our friends at Gogobot saw a 78% increase in signups when they made their app display the same content that the new user clicked on originally, instead of a generic welcome screen.

5. How does the double viral loop work?

When User B joins from an invite or share from User A, close the loop by notifying User A. It's a great excuse reengagement, and it can drive further shares. Josh Elman calls this the double viral loop: sharing results in new users, but it also results in more engagement and more sharing.

Lastly, how do you measure app virality?

K-factor is the industry standard. It's computed by multiplying three conversions together:

(download -> activation) * (activation -> share) * (share -> new viral-driven download)

Keep time as a constant, or you could see some strange time-sensitive responses.

Track the influencers that drive the most installs. Incentivize them with customized promotions. We struggled to identify these users when we built our first app - once we understood how important they were, we built a virality dashboard to help apps track just that.

Closing thoughts

Reaching virality is the holy grail for most product and the only way to get there is to keep trying, keep testing, learning and trying again. I'd love to hear your tests, what worked and what didn't. Leave any questions or comments below - we'd love to help any way we can.

# data[].attributes.attachment_link.description
They said, "If you build it, they will come." They lied. Pour your heart and soul into creating the most beautiful, user-engaging apps. Create content. Promote. Do everything you can, and still, you can't fill the stadium. Why is virality so tough on mobile? How can we aim for virality ourselves?

# data[].attributes.attachment_link.title
New data shows losing 80% of mobile users is normal, and why the best apps do better

# data[].attributes.attachment_link.description
Exclusive data on retention curves for mobile apps In a recent essay covering the Next Feature Fallacy, I explained why shipping "just one more feature" doesn't fix your product. The root cause is that the average app has pretty bad retention metrics. Today, I'm excited to share some real numbers on mobile retention.

# data[].attributes.attachment_link.readable_text
Goals are important in defining and monitoring success. When a goal is in place, the destination is clear - even though the route may change. Goals help connect your mission to your strategy, roadmaps, initiatives and tactics by tying the single metric you care about most with a target and a time frame during which it can be achieved. More than anything else, this process will help your team define success. In this post, we'll explain how to define success for your product by identifying the one key "metric that matters" and then, how to set the right goals.

DEFINING THE RIGHT METRIC

What is the one metric that matters most to the success of your company and that you can rally your team around? For Facebook, it is active users; for WhatsApp, it is number of sends; for eBay, it is gross merchandise; for PayPal, it is total payment volume. Once you identify this "top-line" metric, you can set success criteria around it, monitor it, understand what drives changes in it, obsessively push it in the right direction - and properly evaluate and manage the health of your product.

For growth, a simple top-line metric might be number of users: for engagement, time spent; for monetization, revenue or number of advertisers. Fundamentally, your choice of metric should be driven by your vision for your product and the mission of your company. If you dream years into the future and visualize your product and company, how would you qualitatively describe them?

A vision statement should be aspirational, inspiring and future-focused. For example, eBay's vision for commerce is "enabled by people, powered by technology and open to everyone." eBay's mission is "to be the world's favorite destination for discovering great value and unique selection." Taken together, these two statements point toward eBay's dream of a world where everyone can find whatever they want, however obscure, at a good price.

What top-line metric would best encapsulate this goal? It is not the number of active users shopping on the site; that metric doesn't measure whether buyers are actually finding what they want, and at the right price. How about the number of active buyers? While that and similar buyer-side metrics can tell us whether users are finding what they want at the right price, they cannot address the "unique selection" criteria of eBay's mission statement.

Maybe a seller-side metric would work - how about the number of sellers? This metric can be helpful, but it doesn't measure the inventory sellers are listing. How about the number of listings? That metric doesn't tell us whether the listings are unique or whether the inventory is selling.

Ideally, the metric we choose would measure the number of unique items sold. A large number would suggest buyers are likely getting what they want, and at the right price. Therefore, the "right metric" in this case would be the market share of the sale of unique selections. However, this metric is complex and difficult to define, measure and grow. Instead, eBay chose as their top-line metric gross merchandise volume (GMV), or the total value of merchandise sold over a period of time. While this metric measures value to users, it does not guarantee unique selection. Nevertheless, it captures much of the spirit of eBay's vision and mission. Picking the right metric can be as much art as it is science.

Some additional guidance on choosing a top-line metric for your company:

Do not pick more than one metric. A single "metric that matters" is unifying and will enable you to set priorities across your entire organization. While it may be tempting to track everything and choose multiple metrics, this isn't wise. Many metrics correlate with one other; they may help move the top-line metric but can become unimportant and distracting when measured on their own. The more metric goals you have, the more complicated it is to weigh them all and make trade-offs against them. Keep it simple.

Avoid vanity and non-actionable metrics. For example, the number of likes your company gets on social media generally isn't correlated with business results or customer success.

When choosing between multiple metrics, pick the simplest measurable metric you can move. For example, if your number of advertisers is correlated with your revenue, and the number of advertisers is easier to measure and move, choose the number of advertisers. You can always establish an exchange rate to determine the impact from one metric to another. Likewise, if you are ultimately interested in a metric that has a low sample size or takes a long time to measure, consider instead choosing a correlated metric to measure.

Pick the metric that most closely represents the usage of your product. For a company like Facebook and Instagram, for example, the single most important metric is active users. To measure growth at such a company, we could pick one of several active user metrics, such as daily, weekly or monthly active users (DAU, WAU or MAU), all of which are typically correlated. Choose based on the expected usage of the product. For example, if you expect the product to be used once per day or more, select DAU as your top-level metric. If instead you think the product will be used only on a weekly basis (e.g., when searching for specific restaurants, businesses, etc.), then choose WAU. One of these three - DAU, WAU or MAU - is a top-line metric for most consumer companies.

Do not be afraid to change the metric if you need to. This can lead to thrash, but it is better to change to a metric that accurately reflects your mission than to move the wrong metric. Ideally, you will put in the time and effort up front to make sure you begin with the right metric. But if you must change it, do so sooner rather than later.

Choose a simple metric that connects to your drivers. Let's say you are looking to increase new user acquisitions. You send emails to potential customers, and a fraction of them visit your landing page. A smaller group of users then sign up, and an even smaller group become active users. From these numbers, we can create a simple framework to think about the problem of user activation (as seen below). You can increase the total number of users who activate by increasing any of the four terms above. For example, if you see the largest drop-off among people who visit the site but do not sign up, it may make sense to set the percentage of site visitors who then sign up as your metric to move.

Avoid ratios. If click-through rate is what you really care about, see if you can instead measure the number of clicks. However, this isn't a hard-and-fast rule; there are many examples of companies that successfully used a ratio as the "metric that matters."

Consider counter-metrics if needed. In the eBay example above, useful counter-metrics include the number of unique items sold and unique inventory listed. If such counter-metrics remain flat or decrease, that may indicate you are drifting from your mission. To address this, you could set an explicit goal that these counter-metrics not decrease. In eBay's case, the primary GMV metric would remain in place, but the counter-metric would allow the company to maintain checks and balances in a complex environment.

Change the metric as your business evolves. Your top-line metric may need to change over time. For example, before the mobile age, users checked products such as Facebook less frequently because of lack of access and connectivity. As mobile use increased, these companies revised their primary metric from MAU to DAU. This evolution is also common when companies launch new products. For example, Amazon Video has likely increased overall visits to Amazon, which could warrant a change in the company's top-line metric.

SETTING GOALS

Teams often think about metrics and goals simultaneously, as they cannot easily be separated. Once you have identified the right metric and goal, you will be prepared to define a strategy and roadmap against which your product team can execute. Goals should highlight what you hope to accomplish and are often stepping stones to accelerating business growth. They can unify your company around a common objective and hold your team accountable for its promises.

1. Product or business aspirations: Most long-term goals are based on the company's mission. For example, if your company is in the video space, you might aspire to have the fastest-growing share of time spent on video. If you want to achieve that goal in x years, you can then break it into chunks to determine the growth you'll need over the next y months in order to stay on track.

2. Product metrics: If your product has been around for a while, you can do a "bottom-up" forecasting exercise to determine your goal for your top-line metric over a given period of time. For example, a forecast of MAUs could consider historical data on seasonality, platform, country, penetration and product changes. (Future blog posts will offer in-depth guidance on forecasting.)

3. New products: If your product is completely new, it will be useful to look at external benchmarks and set "top-down" goals. For example, if the product is a Messenger-style communication app, you may choose to study growth at similar companies and let that inform your goals. You may also consider postponing goal-setting for a completely new product for a couple of months, until you see how it performs.

As an example, let's assume you want to grow your number of active users. A goal statement could be "grow MAU to 10M by Q4 2018." This goal connects the metric (MAU) to a target (10M) and a time frame (Q4 2018), clearly describing what the product team wants to achieve and providing a purpose for the organization. Goals should be simple, actionable, achievable and most important, easy to measure and track.

Pick goals that are time-bound. Set a time frame, such as hitting the goal by the beginning of Q1 or the end of the year. You cannot hold people accountable to a goal that isn't time-bound.

Set different goals for different time frames. Choose a long-term, aspirational goal that aligns with your vision and mission for the company, as well as more tangible, short-term goals needed to reach that point. How you frame the short-term goals should depend on how your product team operates, as well as on the time frames over which you evaluate business results. For example, some companies operate on a quarterly cycle, while others run on a half-yearly or even yearly cadence. Your goals should reflect these different time frames, and your team should execute primarily on the short-term goals.

Set two goals: an 80-20 and a 50-50. 80-20 goals are the ones you have an 80 percent chance of achieving. These goals are attainable, and hitting them will motivate the team. However, they will not help the team stretch and perform at a higher level. Therefore, it's also important to set 50-50 goals, which you have only a 50 percent chance of achieving. While these goals are more challenging, they are also far more satisfying to reach. Whatever you do, don't "sandbag" by setting only goals you can easily achieve. Failing to set the bar high can lead to complacency on the team and a decline in the product. If you find you always reach or exceed your 80-20 or especially your 50-50 goals, it's likely you have set your sights too low.

Set specific goals. Make sure your goal is measurable - not something like "delight users." Combine a time frame with your top-line metric to define a specific, quantitative goal.

When writing your goals, don't include the "how." While specifics are important in planning your roadmap and initiatives and in setting goals from a bottom-up perspective, the goal statement itself should be simple and high-level.

Setting a goal is as much science as it is art. You'll often face many unknowns while setting a goal. For example, you may not understand how the market, competition, retention of your cohorts, expansion into new countries, etc., will affect your top-line metric. Accept the fact that, like choosing a metric, setting a goal will be part science and part art.

You can choose goals based on:

Some general guidance on choosing goals:

NEXT STEPS: STRATEGY AND BEYOND

Find a single actionable, top-line metric that encapsulates the vision for your product. This metric should be easy to measure and connected to your business drivers.

Goals define the success of your product and can be set by breaking down business aspirations into smaller chunks. For older products, try a bottom-up forecasting exercise to connect your goal to your top-line metric. Newer products should use a top-down approach that assesses performance relative to external benchmarks.

Your goals should be measurable, time-bound and staggered across time frames. To stretch the team, define 50-50 goal as well as 80-20 goals. Keep your goals sufficiently high-level and avoid the "how."

This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email data-science@sequoiacap.com with questions, comments and other feedback.
Imagine a consumer company with a 50-50 goal to increase monthly active users 10 percent by Q4. This goal meets many of the criteria mentioned above: it is specific, quantitative, time-bound and includes an appropriate top-line growth metric. The next step of the product-building process is to establish a product strategy, such as "resurrect users," that specifies how to achieve the goal. This strategy can then help drive a roadmap, such as "resurrect stale India users." The company could generate specific initiatives within this roadmap that would contribute most toward moving their top-line metric. Such initiatives might include "improve SMS notifications in India," and would in turn lead to engineering tasks such as "build SMS notifications for low-end Android phones." This example shows how a company's vision and mission can drive its goals, strategies, roadmaps, initiatives and tasks. Future blog posts will offer in-depth guidance on setting data-informed roadmaps and strategy.

TAKEAWAYS

Previous Article: Diagnose and analyze your product's health.

# data[].attributes.attachment_link.title
Defining Product Success: Metrics and Goals

# data[].attributes.attachment_link.description
Goals are important in defining and monitoring success. When a goal is in place, the destination is clear - even though the route may change. Goals help connect your mission to your strategy, roadmaps, initiatives and tactics by tying the single metric you care about most with a target and a time frame during which it can be achieved.

# data[].attributes.attachment_link.readable_text
Products evolve over time; the characteristics of an early-stage product are quite different from those of a mature one. For strong products, the phases of growth usually resemble an S-curve (see Figure 1): in the Early phase, growth is modest and shallow. Then it accelerates as the slope arches upward during the Growth phase. This is followed by a Hyper Growth phase - a period of exponential growth. After maximum growth is reached, growth tapers off and the product begins its Mature phase, during which there is little to no growth.

Below, we describe the dynamics of each phase to help you set expectations for your product. Note, consumer businesses tend to proceed through these phases far more quickly than enterprise businesses; for example, Google Search achieved its scale in a few years, while Google Apps for Domains continues to mature years later.

EARLY PHASE

When a new product launches, it is difficult to know whether people will use it in a way that aligns with your vision. The initial goal is to get the product adopted by a few thousand people who you have hypothesized will love it - it is better to have a few users who love a product than many who simply like it. For about a month after launch, many users will try the product for novelty; some will stick with it and become early adopters, while others will leave.

During this phase, there is a scarcity of data, and what data you do have likely isn't reliable or representative of the product's long-term future. All monthly active users (MAUs) are also new users, and there is no churn or resurrection. (Note: the total number of active users for a given time period is the sum of retention, resurrection and new users.)

This changes in the second month. While there is still no resurrection, you are beginning to see churn from first-month users. In the third month, a balance between new users, churn and resurrection begins to emerge. Resurrection is small, and new users remain the largest factor, but churn has started to contribute significantly. Your month-over-month net user growth depends entirely on the Quick Ratio: (new users + resurrected users) / churned users.

Figure 2 shows how retention of a cohort decreases and flattens out over time. Early on, churn far exceeds resurrection; this gap gradually narrows as the curves level out. (Note: for a weak product, the curves will not flatten out and will eventually reach zero, whereas for a strong product, the curves will begin to rise and will eventually flatten.)

In the Early phase, churn far exceeds resurrection and new users contribute significantly to MAU. The flattening of the retention curve indicates the product has achieved balance between churn and resurrection.
Healthy products during the Early phase

Cohort retention is strong compared to similar products in the space that have similar levels of penetration.

There is a group of users that absolutely love the product, which is reflected in very high retention.

New users are the largest share of active users.

GROWTH PHASE

By the end of the Early phase, the product may be adopted by tens of thousands of people who are beginning to use it regularly. However, this does not necessarily indicate product-market fit. It's important to think about who the next hundreds of thousands of users to adopt the product will be - and what changes should be made to accommodate them.

In the Early phase, penetration with respect to the total addressable market was low and the product grew slowly. Now, in the Growth phase (see Figure 3), growth accelerates (the size of the new cohorts increases) as you begin to achieve product-market fit. Something wonderful seems to be happening: users are finding value in the product and return to the product.

As the active retained population increases, new users become a smaller percentage of MAU. Slowly but surely, the product's market penetration begins to increase. In older cohorts, retention stabilizes as balance between resurrection and churn is achieved, while in newer cohorts, churn continues to far exceed resurrection. Because older cohorts are a larger percentage of overall users, overall resurrection thus grows relative to churn (though resurrection is still smaller) and begins to contribute significantly to MAU. During this phase, the biggest contributor by far to overall active users is retained users.
Healthy products during the Growth phase

Older cohorts' retention curves flatten and perhaps slightly improve.

New users account for a decreasing percentage of total users.

Overall churn remains higher than resurrection but is driven primarily by the newest cohorts.

Engagement metrics get stronger.

HYPER GROWTH PHASE

Many of the best products ever built - including Facebook, Instagram, WhatsApp, Amazon, Netflix, Uber and Evernote - have one thing in common: a Hyper Growth phase. During this phase, the one-month (M1) retention levels of newer cohorts begin to increase relative to those of the previous cohorts over the same time frame, thus improving the ratio of resurrection to churn. This typically begins when either 1) a certain level of penetration is reached with respect to the total addressable market or 2) the product becomes very well-known. In addition, older cohorts start to resurrect in greater numbers during this phase and churn begins to decrease. All users, new and old, find value from the product, as shown in Figure 4.

In this period, MAU grows dramatically as resurrection exceeds churn and the product continues adding new users. This may be the fastest year-over-year growth your company will ever experience. All engagement metrics, such as L5+/7, number of sessions and MAU/DAU (daily active users) are strong, and the product has achieved both product-market fit and scale.
Healthy products during the Hyper Growth phase

Older cohorts' retention curves improve.

Resurrection is greater than churn.

Engagement metrics are very strong.

MATURE PHASE

After a successful Hyper Growth phase, the product enters the Mature phase, during which it reaches a high level of penetration with respect to the addressable market and has few users left to add. In this phase, the product is no longer growing. Every cohort, old and new, has reached a steady balance between resurrection and churn, and, at the cohort level, the sum of retained users is equal to the overall number of active users. Overall retained users are also a large percentage of overall active users during this phase, and resurrection and churn contribute to a smaller percentage of active users.
Healthy products during the Mature phase

Resurrection and churn have balanced.

The number of new users is extremely small compared to the number of retained users.

Engagement metrics are retained at very high levels.

EXPANDING PRODUCT LINES

To continue to grow your product after hitting maturity, you must diversify your portfolio and increase your total addressable market. This could mean entering new markets, launching new products, etc.

Ask yourself: Who will the next few million users of the product be? Which countries could benefit from the product that are not using it now? What new features could make the product more valuable in those segments? You may need to rethink your approach to creating product-market fit in order to reach them.

As shown in Figure 5, growth of additional products or in new environments will again follow an S-curve: growing slowly at first, then accelerating, increasing rapidly and approaching an exponential growth rate, and ultimately reaching saturation. By diversifying, you overlap S-curves and allow for continued growth.

For example, consider Netflix's journey. The company started its streaming product in the United States in 2007 and went into its Hyper Growth phase in 2010 (see Figure 6). At this time, executives at Netflix had already started laying the infrastructure for international expansion, and the platform formally launched in other countries in late 2010. By the time U.S. growth hit maturity a few years later, the international business (which is itself a series of overlapping S-curves) was driving most of Netflix's subscriber growth. Most businesses should, like Netflix, think proactively about product and vertical adjacencies while in the Growth phase of their primary products. Delaying the launch of a second act - or failing to launch one at all - will stunt growth.

MEANINGFUL METRICS

A quick note: the previous sections assume we have chosen meaningful metrics and time frames, both of which are key to properly understanding growth. These factors will strongly influence your view of the balance between new users, churn and resurrection, the three contributors to growth.

As an example, consider two extremes: if we tracked the number of people using a product each minute (minute active users), our resulting churn and resurrection numbers would be huge, the number of retained users would be sizeable and the number of new users would be small. On the other hand, if we tracked the number of people using a product each year (yearly active users), churn and resurrection would be a much smaller fraction of active users. In contrast, retained and new users would be a larger fraction of the active users.

TAKEAWAYS

The phases of growth resemble an S-curve and include Early, Growth, Hyper Growth and Mature.

Early on, new users are the largest contributor to growth, with churn outstripping resurrection.

As your product grows and market penetration begins to increase, new users continue to be the biggest driver of net growth while resurrection becomes more significant, with older cohorts achieving a balance between resurrection and churn.

In the Hyper Growth stage, retention improves in older cohorts with resurrection outpacing churn.

As your product begins to completely penetrate the market, a balance emerges between resurrection and churn with new users contributing negligible growth.

Diversification in the form of multiple product lines is critical to achieve sustained long-term growth.

This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email data-science@sequoiacap.com with questions, comments and other feedback.

# data[].attributes.attachment_link.description
Products evolve over time; the characteristics of an early-stage product are quite different from those of a mature one. For strong products, the phases of growth usually resemble an S-curve (see Figure 1): in the Early phase, growth is modest and shallow. Then it accelerates as the slope arches upward during the Growth phase.

# data[].attributes.attachment_link.description
Editor's Note 1 : I have no editor. Editor's Note 2 : I would like to assure new subscribers to this blog that most my posts are not as long as this one. Or as long as my previous one .

# data[].attributes.attachment_link.readable_text
"HELL YEAH WE'RE USING @SlackHQ AT WORK I. LOVE. SLACK."

"Dear @SlackHQ, I love you. Yours, Dan"

"@SlackHQ YOU COMPLETE ME"

You've probably heard about Slack's exponential growth. And you may have read how the internal-communication platform - now just two years old - is already used by more than 30,000 teams and valued at over $1 billion. But have you visited its Twitter Wall of Love?

These tweets are real, and they're the stuff of founders' dreams. And yet Slack hasn't run any big integrated marketing campaigns - they don't have an elaborate email strategy or buy million-dollar billboards. (In fact, they hit those user numbers without a CMO.) So how did the company not only launch with enviable momentum, but so quickly win users' hearts? If there's one theme that emerges when founder Stewart Butterfield talks about Slack's success, it's that the company made customer feedback the epicenter of its efforts.

In this exclusive interview, Butterfield - previously one of the founders of Flickr- reveals how the company's go-to-market strategy worked like gangbusters. Here, he explains the importance of prioritizing your product's unique features (and why you can let go of the rest), and shares tips for becoming essential to your customers right away.

Slack started working on the app at the end of 2012. ("Never mind the part where we first tried to make a web-based massively multiplayer game and failed," Butterfield quips - another story for another article.) And by March 2013, he and his team had enough to work with that they were using the product themselves. Still, they knew that they represented just one team dynamic of a nearly infinite set; by May of that year, they were ready for more users.

"We begged and cajoled our friends at other companies to try it out and give us feedback," Butterfield recalls. There was Cozy, which sells rental management software for landlords and tenants, and the music service Rdio. "We had maybe six to ten companies to start with that we found this way."

Immediately, the Slack team learned that their product functioned very differently as team size increased. "Rdio, in particular, was much bigger than us. They used it with a small group of front-end developers for a while but then it spread to the whole engineering group and then to all 120 people in the company," Butterfield says.

Suddenly we saw what the product looked like from the perspective of a much larger team, and it was pretty gnarly.

Armed with these observations, the Slack squad made a number of changes to the product -then started the process all over again.

"The pattern was to share Slack with progressively larger groups. We would say, 'Oh, that great idea isn't so great after all.' We amplified the feedback we got at each stage by adding more teams," Butterfield says.

By summer, they had polished Slack into something they were ready to share more widely, and they announced their preview release in August 2013 (just seven months after they started).

"That was essentially our beta release, but we didn't want to call it a beta because then people would think that the service would be flaky or unreliable," Butterfield says. Instead, with help from an impressive press blitz (based largely on the team's prior experience - i.e. use whatever you've got going for you), they welcomed people to request an invitation to try Slack. On the first day, 8,000 people did just that; and two weeks later, that number had grown to 15,000.

The big lesson here:Don't underestimate the power of traditional media when you launch. Itmustbe your primary concern, starting months beforehand and continuing for weeks afterward. Pull the strings you have. Work closely with your PR firm to find your hook. It can be personalities on your team, impressive customers you already have in the bag, prestigious investors, etc. But don't leave it to two weeks beforehand and throw something together.

Most importantly, getting the story out doesn't end when an article is published. In fact, by Butterfield's estimation, that's only about 20% of the recipe for media success. "The other 80% is people posting about that article. I almost never go to news sites - it's overwhelming how much content is out there. But I will pay attention to what my friends are picking up and sharing."

Social media has leveled the playing field, so whatever coverage you earn, run with it - give it new life by sharing it with your immediate and extended networks again and again. Engage with interested parties in your networks (prioritizing those with lots of followers and known influence) to broaden your reach. Don't worry about repetition. It will only help you stay top of mind for prospective users.
Teach Users Why They Need Your Product
There's another key takeaway from Slack's early experience: Whatever you call your beta, however you announce and operate it, it's a crucial phase in your product's development. Wring every bit of feedback that you can from it.

Slack made the most of this time, sticking with their preview release for more than six months - a long time in the life of a startup. But there was no calculated strategy or predetermined timeline behind that, Butterfield explains. "By that August 2013 announcement, we had gotten a little bit of feedback. But we were still only seven months in, and it's a pretty complicated product."

For a company that has derived much of its success from its laser focus on quality and responsiveness, six months was simply how long it took to learn what they needed from that next wave of users. In many ways, Slack's private beta period was business as usual.

We started inviting teams in batches and watched what happened. Then we made some changes, watched what happened, made some more changes...

The biggest challenge was learning how to sell a product to teams, not individuals. "For most companies, the hard thing is making the product work well enough to convince a single person at a time to switch to it," Butterfield says. Take Dropbox, for example: A person tries it on a couple of devices, likes it, and commits to spending a few bucks a month for it. "We have to convince a team, and no two teams are alike."

From job functions to group sizes to whole companies, Slack's teams run the gamut. But there was one thing Butterfield ran into pretty consistently. When it comes to selecting a team-collaboration tool, every member has a veto - multiplying the product's risk of rejection. "If one engineer at a startup tries Slack and says, 'I hate it. I am not going to use this,' that's it for us. We won't get evaluated."

Given this pattern, much of Slack's beta period was spent minimizing that risk. "We created materials to explain Slack to individuals - what it was for, how it worked, what you're supposed to do - but we also built resources for team administrators. We wanted to give them ammunition to help convince the team," Butterfield says.

Slack is blazing trails in a relatively new arena, so that ammunition was equal parts product training and market education. "Somewhere between 20 to 30% of our users - and this is just an estimate - come from some other centralized group-messaging system like HipChat, Campfire, or IRC," Butterfield says. "When we asked the other 70 to 80% what they were using for internal communication, they said, 'Nothing.' But obviously they were using something. They just weren't thinking of this as a category of software."

When Butterfield digs into the "nothing" those companies are using, it's usually a smorgasbord ofsomething: "It's a lot of ad hoc emails and mailing lists. Some people on the team might use Hangouts, some use SMS. We see groups that use Skype chat, or even private Facebook groups and Google+ pages."

So Butterfield made it a goal to teach customers that thisisindeed a product category - one they're already filling poorly - priming Slack as a better solution. Helearned pretty quickly that the app's laundry list of benefits wasn't going to land sales on its own.

"If you're building a sales team for your startup, you know you will absolutely make a decision about what CRM to use. It's a no-brainer. If you're a software development team, you are absolutely going to choose a system for source control. That's a known category."If you're innovating in a nascent market, the push for recognition of your product category needs to be a major chunk of your go-to-market strategy.

Positioning a product for teams rather than entire companies does come with some positives though - which could be relevant for other enterprise startups.

"For small organizations, team and company may be one and the same. But if you look at an organization of 15,000 people, you end up with a situation like Adobe, with nine paid Slack teams," he says. That proved to be a helpful loophole. They didn't have to go through the long process of gaining buy-in from CIOs or other top management. "Mid-level managers could say, 'This thing sounds cool, let's try it out for our team.' If they liked it, it was affordable enough to just expense it."

In the end, that bottoms-up approach was a key factor in Slack's early enterprise success.

"We made it very simple to adopt Slack. We didn't have to convert the whole company and facilitate committee-level decisions," Butterfield says. A couple of years in, they've matured to the point that security audit reviews and marked-up terms of service are becoming the norm. "But at the beginning, we bypassed all that, and it was a big advantage."

Unencumbered, Butterfield and his co-founders were able to spend those six months in private beta educating existing customers so that an even wider audience would understand the need for their product once they launched. "From August 2013 to February 2014, we went through those initial 15,000 sign-ups - and more that we got over the course of time - and incrementally improved the new-user experience until we felt like we had gotten all the low-hanging fruit."
Make Active Listening Your Core Competency
As much information as Slack put out to customers, they learned even more themselves. Butterfield and his cofounders are voracious readers of user feedback, and they attribute much of the company's rapid traction to this skill. From the get-go, Slack made sure that users could respond to every email they received, and approached every help ticket as an opportunity to solidify loyalty and improve the service. As they listened to their ever-growing flock of users, the Slack team iterated accordingly.

"Sometimes you will get feedback that is contrary to your vision," Butterfield says. "You may be trying to drive in a particular direction that people don't necessarily understand at first. In our case, we knew the users we had in mind for this product. So in the early days, we looked at our customers, really just testers at that point, and we paid extra attention to the teams we knewshouldbe using Slack successfully."

When key users told us something wasn't working, we fixed it - immediately.

Take Rdio, for example, one of Butterfield's biggest beta-test companies. "In Slack, you create channels to discuss different topics. For a small group of people, those channels are relatively easy to manage and navigate. With a team that large, though, everyone was creating channels, and there was no way for people - particularly new hires - to figure out which ones they should join."

Once they understood that, the Slack team quickly identified small changes that had a big impact: Within the list of channels, they added fields for a description and the number of people using that channel. "In the grand scheme of things, that's a fairly trivial example, but those were things that would make Slack unworkable for certain teams. Beta-tester feedback is crucial to finding those little oversights in a product design."

Now, a year after Slack's public launch, that reverence for user feedback is part of the company's DNA. "We will take user feedback any way we can get it. In the app, we include a command that people can use to send us feedback. We have a help button that people can use to submit support tickets," says Butterfield. They've got eyes all over Twitter for comments good and bad. "If you put that all together, we probably get 8,000 Zendesk help tickets and 10,000 tweets per month, and we respond to all of them."

Where some people might see a huge customer-service burden, Butterfield sees one of Slack's greatest assets - so much so that he fielded half of these messages himself for a long time. "Especially in the beginning, I handled the lion's share of Twitter, and Ali Rayl, our Director of Quality and Support, handled the Zendesk tickets. Pretty early on, we combined quality assurance and customer support into one group that we called customer experience. They do everything from parsing customer feedback and routing it to the right people to fixing bugs themselves."

When Slack publicly launched, that group was already three people strong. And since then, it's remained a key area of investment for the company. Today, customer support alone has 18 people with an overlapping group of 6 working on Twitter 24/7. That latter number might sound high, but Twitter is increasingly a channel for customer feedback, and it was a game-changer for Slack.

"We bet heavily on Twitter. Even if someone is incredibly enthusiastic about a product, literal word of mouth will only get to a handful of people - but if someone tweets about us, it can be seen by hundreds, even thousands." Still, that fan mail on the Wall of Love? It didn't all start out so positive; in fact, many of those messages started out as problems to be solved.

Every customer interaction is a marketing opportunity. If you go above and beyond on the customer service side, people are much more likely to recommend you.

Whatever form it takes, incoming user feedback must be processed, stored and studied. "We're pretty fastidious about tagging all of these incoming messages, collating and entering and retaining the data that people are sending us," Butterfield says.

The company keeps track of how many people are asking for a certain feature, or how many want a new kind of integration. "Of course hard numbers tell an important story; user stats and sales numbers will always be key metrics. But every day, your users are sharing a huge amount of qualitative data, too - and a lot of companies either don't know how or forget to act on it."

Not surprising for a company working on communication challenges, Slack's leaders have also made sharing feedback with the right internal team at Slack mission critical. Tweets and help tickets are all stored in their own Slack channels and made searchable. And the support team knows that they have a direct line of communication to the product team.

"Whenever they hear something new that seems like it's actually a really good idea - or it's a pretty good idea but it's very easy for us to implement - it gets posted to a channel where we discuss new features. That's an ongoing, daily thing. There have already been 50 messages posted today," Butterfield says.

Know Your Magic Number and What Your Metrics Mean

Qualitative feedback is championed at Slack. But they also invest a lot of time making sure their metrics tell the same story as their Twitter feed. Butterfield affirmed a common founder sentiment: There are industry-standard numbers, no doubt. But at the end of the day, only you can really determine your company's magic numbers - the numbers that shed light on who is really using your product (and how you can get them to keep using it).

For Slack, the number is 2,000 - 2,000 messages. "Based on experience of which companies stuck with us and which didn't, we decided that any team that has exchanged 2,000 messages in its history has tried Slack - really tried it," Butterfield says. "For a team around 50 people that means about 10 hours' worth of messages. For a typical team of 10 people, that's maybe a week's worth of messages. But it hit us that, regardless of any other factor, after 2,000 messages, 93% of those customers are still using Slack today."

The mechanics of Slack work like this: Someone enters their email address and receives an email with a link. From there, they complete a simple form and hit submit to start their team. Of course, for Slack to work, users need to invite other team members and start using the software, and ideally even set up some integrations with other apps like Asana, Dropbox or MailChimp.

Most people who fill out the form and hit submit - more than 90% - never invite anyone or start using the software.

That figure sounds daunting. Of roughly 220,000 teams that have been created in Slack, upwards of 30,000 are actively using it. "However, because one active team has an average of eight or nine members, we have close to 250,000 daily active users. We have more daily active users than teams that were ever created. So we lose a bunch, but the ones that we get to really try it out stick with it."

Right now, you may not have an off-the-shelf metric that accurately captures your company's growth,but part of your go-to-market strategy absolutely has to be establishing this criteria. "You have to figure out what conversion means in your case. What does retention mean? What does activation mean? For every business, it's going to be slightly different because of the nature of the product and the kinds of people who use it," Butterfield says.

As soon as you have those numbers enshrined at your company, you can start working on innovative ways to move people toward those milestones - whether it's email reminders, or prompting them to take new actions in the product. Because Slack knows 2,000 is its golden number, it can iterate on ways to get customers across that line.

Amplify What Makes Your Company Special

At the root of all your qualitative and quantitative feedback is a product - and making it the best at what it does is all about knowing its core differentiators and unique opportunities. "All of the founders here are past the stage where we have a lot of ego about building something our way," Butterfield says. "We set ourselves an incredibly high quality bar, and we're just not going to be happy if we don't reach it."

When the product was first coming together, Butterfield and his co-founders returned again and again to Paul Buchheit's now-famous blog post, "If your product is Great, it doesn't need to be Good." Known as one of the creators of Gmail, Buchheit has a simple thesis: If you do a few things incredibly well, the rest doesn't really matter. And, if you look at the first generations of Gmail, Butterfield says, it was in fact missing a lot of features. Still, users were so impressed by the searchable interface, threaded conversations, and the then-unimaginable one gig of storage, that they weren't fazed by what the productdidn'thave.

Buchheit's words strongly resonated with Butterfield and his team.

We don't cut corners, and we try to focus on the few things that are most important to our product vision.

For Slack, those three most important features are:

Search: Much like Buchheit did with Gmail, the Slack team knew that the value of their product was in helping people find what they're looking for quickly. "People need to feel confident that when they read a document or conversation, they don't have to worry about labeling or storing it - that they'll be able to find it again later if and when they need it," Butterfield says. Google has set the standard so high in this category that people have certain expectations, and disappointing them can be fatal.

Synchronization: "One of the things that drove us nuts about every other internal platform was that it was very difficult to pick up in the same place when you switched devices - say, when you left your laptop and picked something back up on your phone," Butterfield notes. From the very beginning, Slack was built with what he's dubbed "leave-state synchronization." Slack knows where every person in every conversation leaves off, and it syncs to their cursor position in real time. This has given them real competitive bite in a market that already had well-known players.

Simple file-sharing: From the ability to quickly paste images to the ease of dragging-and-dropping files, Slack was built with attention paid to small shortcuts and intuitive UI actions - they add up quickly when it comes to software that users interact with sometimes for hours every day.

These may not be checkbox features, or buzzworthy new concepts, Butterfield notes; they may not even be things that users think they're looking for in a solution. But when it comes to a successful go-to-market strategy, perhaps the most important decision you can make is to build a product you believe is different from everything else out there, and an important change for the audience you're going after.

"We had a lot of conversations about choosing the three things we'd try to be extremely, surprisingly good at," Butterfield says. "And ultimately we developed Slack around really valuing those three things. It can sound simple, but narrowing the field can make big challenges and big gains for your company feel manageable. Suddenly you're ahead of the game because you're the best at the things that really impact your users."

# data[].attributes.attachment_link.title
From 0 to $1B - Slack's Founder Shares Their Epic Launch Strategy

# data[].attributes.attachment_link.description
"HELL YEAH WE'RE USING @SlackHQ AT WORK I. LOVE. SLACK." "Dear @SlackHQ, I love you. Yours, Dan" "@SlackHQ YOU COMPLETE ME" You've probably heard about Slack's exponential growth. And you may have read how the internal-communication platform - now just two years old - is already used by more than 30,000 teams and valued at over $1 billion.

# data[].attributes.attachment_link.readable_text
There has been an explosion in the development of new products in recent years with the rise of experimental product iteration. When a product changes, a reaction almost always follows causing a shift in the key metrics. Therefore, it is critical to understand the impact of shipping a product and ensure that the right outcome is achieved.

For example, in July 2014, when Facebook began notifying users that they would no longer have the option of messaging within its core mobile app, nearly 20 million iOS users in the United States downloaded the separate Messenger app within a one-month period (see Figure 1).

As another example, consider the metrics that might be influenced by changes to a product's notification algorithm: the number of people who see notifications, the percentage who act on notifications, the percentage of those who end up on the landing page, and the percentage who then actively interact with the product. All of these metrics and more may be affected when a product changes.

Of course, product changes aren't always intentional. Bugs, such as those that prevent users from downloading your app, updating it, sending messages or receiving notifications, can significantly influence key metrics. Therefore, carefully tracking your key metrics - particularly after a major product change or app store update - can help you detect, understand and limit the damage bugs cause.

The best way to evaluate the impact of a change to your product is A/B testing. In A/B testing, two or more variations are shown at random to users and statistical analysis is used to determine which variation performs better against a given goal. It is especially important to use A/B testing when a change is relatively small, which is most often the case: A large change, such as the Facebook example above, can sometimes be 'eyeballed'. Usually, though, it is nearly impossible to effectively detect and measure impact without an A/B test.

TAKEAWAY

A product change - whether a new product rollout, a changing notification strategy or an unintended bug - will invariably result in a shift in metrics.

# data[].attributes.attachment_link.description
There has been an explosion in the development of new products in recent years with the rise of experimental product iteration. When a product changes, a reaction almost always follows causing a shift in the key metrics. Therefore, it is critical to understand the impact of shipping a product and ensure that the right outcome is achieved.

# data[].attributes.attachment_link.readable_text
In the first post of our series on building data-informed companies, we shared the most important ingredients needed to build a data-informed company: singular focus on impact and building a data-informed culture. In this second post, we will focus on why data science matters and discuss the future of data science in the context of building products.

What, exactly, is data science? Data science is a scientific and truth-seeking discipline that uses data to extract knowledge and insights. Data science is one of the fastest growing functions and is already providing tremendous value across every industry and area of study. Nevertheless, data science is still in its infancy and like any developing field, it is often tempting to put boundaries around its definition. Rather than categorizing what does or does not count as data science or arguing about why we should be data-informed but not data-driven, we believe it is most important to leave room for the discipline to evolve organically.

WHY DOES DATA SCIENCE MATTER?

Starting a tech company, building a good product, and gaining traction have become easier thanks to improved connectivity, declining cloud storage and computing costs, and the accessibility of distribution platforms for reaching target audiences. As a result, the time it takes a product to reach 100 million monthly active users has shortened dramatically, and continues to shrink today. For example, it took about 100 months for iTunes to reach 100 million monthly active users in 2003 and mere days for Pokemon Go to do the same. The chart below has some more good examples, starting with the telephone.

The combination of more products built, more Internet-connected devices bought, and increased time spent online has caused a spike in the volume of user interaction data. There has been a tremendous interest in mining this data and deriving key insights to help build great products. A company's ability to compete is now measured by how successfully it applies analytics to vast, unstructured data sets across disparate sources to drive product innovation. Therefore, data scientists are in high demand, and that a team of smart data scientists can make or break a product.

This increasing interest in mining data for insights has led product teams to use data to focus on four specific outcomes.

1. Evaluate the health of the business: One of the key outcomes of product analysis is to evaluate the health of a product or a business. Once we have defined product success by the means of a goal and a metric, the next step is to monitor the metric to ensure that we are on track to hitting the goal. Tactically, analysts work on identifying outliers, understanding the drivers of changes in metrics, building dashboards / reports / visualizations etc.

2. Ship the right products and features: Another very important role of analytics is to ensure that the right products and features get built. Many companies run numerous experiments and ship products after evaluating the results of these experiments. Typically data scientists help design experiments, identify data-informed hypotheses on phenomena, and guide product team on constant optimization of the product through the data insights.

3. Forecast outcomes and Power production systems - Another role of data scientists is to build prototypes/models and power production systems using AI/ML. These data scientists train machine learning models of a phenomenon in order to forecast future expectations and trends.

4. Set roadmap and strategy for the product: Deeper exploration and analysis of the user journey and phenomena generate actionable insights that ultimately result in setting roadmap and strategy for the product. Data-driven roadmap and strategy is one of the most important outputs of a world class product analytics team.

These four outcomes have specifically led to two different types of data scientists in the industry - product analysts and algorithm developers.

WHAT DO DATA SCIENTISTS DO?

The title of data scientist encompasses multiple roles that vary significantly across companies and industries. That said, generally speaking there are two main camps of data scientists:

1. Product analysts, whose role is to deliver data-informed stories that advocate for a change in product or strategy. E.g., Our SMS notification delivery system is broken in India. As a result, we need to focus on improving SMS notifications in India, which will help reignite growth.

2. Algorithm developers, whose role is to incorporate data-driven features into products (e.g., optimizing recommendations or search results). E.g. The fraud levels in Indonesia has increased. Build a new model that focuses on the recent fraud trend in Indonesia.

Product analysts focus on setting goals and informing product roadmaps and strategies. They help improve products by evaluating and understanding their health and providing product decisions (largely via experimentation). The general deliverable from product analysts is a document to the product team narrating quantifiable issues, identified opportunities, and data-based recommendations and solutions.

An algorithm developer's primary job is to leverage data to improve product performance in pursuit of a specific end goal, typically forecasting outcomes or building production systems. Algorithm developers generally use machine learning and other complex algorithmic techniques to make predictions based on inputs from vast quantities of data. In general, algorithm developers prototype proposed solutions and work closely with engineering teams to implement them in production. The deliverable from algorithm developers is prototyped code and documentation that get provided to the engineering team.

While both types of data scientists require an analytical outlook, quantitative skills, and the ability to prioritize, it is rare to find a person who can fit both roles. Algorithm developers require more sophisticated technical knowledge, such as machine learning and artificial intelligence, and a level of software engineering skills closer to those of engineers. Product analysts are primarily problem solvers who are differentiated based on their business, product, and ability to communicate effectively to a wide variety of stakeholders.

While only some organizations require algorithm developers, all companies, especially those with a significant user base, benefit from product analysts who can help navigate product, competitive, and other strategic challenges. Later in this series of blog posts, we will provide additional guidance on hiring, training, coaching and managing product analysts so they can contribute at the highest levels.

So, do companies need to hire algorithm developers and be more data-driven or should they hire product analysts and be more data-informed? Outcomes are purely data-driven when data is the only signal required to make a decision. In contrast, in data-informed decisions, data is a strong input but not the only input. Generally speaking, Product Analysts are data-informed and algorithm developers are data-driven.

EVOLUTION OF DATA SCIENCE

Imagine a world in which a machine knows everything about you and can shop for you without even explicitly asking for it; knows the food you like and cooks for you; knows your choices and can make decisions for you and knows what is good for you and plans your life. This world is distant into the future and requires Artificial Intelligence to take over much of our lives. For us to make progress toward this dream, we need to become even more data-driven.

In a perfect world with perfect information and a complete understanding of all the drivers of your system and how they interact with each other, the two approaches would converge. In order to build a perfect model, the phenomenon under study needs to be completely understood; the relationship between the data and the phenomenon can be described by a perfect model (and associated rich feature set). In order to evolve to this level of perfection and also make progress in the interim, the world will need to continue to make progress on data-informed decision-making. i.e., we need to continue augmenting our decision making by other subjective measures that cannot easily be fully quantified yet. As we begin to have a deeper understanding of relationships between objects, more and more processes will get automated away and the future will be more data-driven than data-informed. However, data-informed decision making will continue to be extremely important for the next few decades and data-driven decision making will only improve with advancements from people who are data-informed.

It is most illustrative to understand the differences between data-informed decision making and data-driven decision making by means of examples.

Setting goals. Good goals are measurable and quantifiable. Being able to identify and track goals will become increasingly data-driven. For example, Facebook's tracking of its active users maybe completely automated. However, setting the right quarterly and annual goals for active users and revenue maybe only partly automated and will continue to be data-informed.

Defining a roadmap and strategy. Establishing a roadmap and strategy is not quantitative and hence requires data-informed approaches. For example, by using data, a roadmap can be developed for increasing daily active usage by focusing on SMS notification. A good roadmap considers the relevant goals, the drivers of these goals, the levers that the product team has, and all of the courses of action that can be taken. Much of this is qualitative, so the process of building a roadmap and defining strategy is primarily data-informed.

Forecasting outcomes. Forecasting outcomes is mostly data-driven. For example, figuring out whether or not to show a story to a user would require understanding multiple factors, including the probability of a user clicking or reading that story. Companies typically develop models, which are iterated on continuously, to forecast this specific outcome.

Powering production systems. For companies like PayPal to identify fraudulent activity of any transaction, it is prohibitively expensive to do this manually for all transactions. As a result, they largely rely on machine learning to power their production systems and automate the calculation of the probability of a transaction being bad. Much of the decision-making that follows the evaluation of the probability is also automated. However, in areas where there are lower levels of confidence in the probability evaluations, the decision process could be data-informed.

TAKEAWAYS

Improving products and monetizing through data has become a competitive advantage in recent years. A strong and well-organized data organization is a strong differentiator.

Data scientists are driving key product decisions across companies and building next-generation algorithms to improve decision making.

The world will continue to become increasingly data-driven, but data-informed decision making will remain relevant.

This work is a product of Sequoia Capital's Data Science team. Chandra Narayanan, Hem Wadhar and Ahry Jeon wrote this post. See the full data science series

. Please email

data-science@sequoiacap.com with questions, comments and other feedback.
This story is published in The Startup, Medium's largest entrepreneurship publication followed by +432,678 people.

# data[].attributes.attachment_link.description
In the first post of our series on building data-informed companies, we shared the most important ingredients needed to build a data-informed company: singular focus on impact and building a data-informed culture. In this second post, we will focus on why data science matters and discuss the future of data science in the context of building products.

# data[].attributes.attachment_link.readable_text
As the world changes and ecosystems evolve, companies are growing faster and products are easier to create than ever before. As a result, products are creating more data - and strategically collecting and analyzing that data has never been more important. Analytics and data science are now must-haves, not afterthoughts.

But despite the indispensable nature of data science, there is a scarcity of literature that explains how to conduct useful product analysis. We intend to help fill this knowledge gap with a series of posts on how to build both data-informed products and world-class data science teams (with particular attention to the consumer space). This introduction is a living document; the table of contents below will be updated as new posts are published.

We hope you find these articles useful, and we welcome your feedback: data-science@sequoiacap.com.
Table of Contents

1. Measuring Product Health: Metrics to diagnose and analyze product health.

2. Engagement: Engaging experiences provide value.

3. The Building Blocks of a Data-Informed Company: Successful data-informed companies do two things well - focus on impact and build a data-informed culture.

4. Why Data Science Matters: The world will become more data-driven over time but data-informed decision making will continue to be impactful.

Check back next week for more updates!

# data[].attributes.attachment_link.description
As the world changes and ecosystems evolve, companies are growing faster and products are easier to create than ever before. As a result, products are creating more data - and strategically collecting and analyzing that data has never been more important. Analytics and data science are now must-haves, not afterthoughts.

# data[].attributes.attachment_link.readable_text
You've decided you want to take a stab at building the next successful mobile app business. First off, spend some time and think about whether you're ready for this. I'd liken the process to running in a marathon followed by wrestling in a sumo match. If you have the energy to make it through the first part, you're most likely going to be eliminated in a fraction of a second by a professional.

Not fazed? Then let's get started. We at Branch spend the majority of our waking hours thinking about ways to help you get through the marathon and then be equipped to tackle the sumo. We first built apps and used all our learning to build the tools that empower every app to become a self-sustaining mobile growth engine. We've seen a lot of approaches: some that work and some that definitely don't. The rest of this post shares useful tactics through the use of examples that have worked for apps on our platform.

The mobile growth engine of your app

Before diving into the tools that I've seen work, let me first describe a framework for thinking about growth. A growth engine is a cyclical process that results in your existing users inviting their friends to join them. Breaking down the different components of this engine can help create a mental model for optimizations and improvements.

The four phases of the growth cycle are awareness, download, activation, and share.

The first step of the process is driving app awareness. How are people going to get to your app store page in order to press download and install? There are many, many different ways to drive awareness and the distribution of different channels will vary over time. It's important to try new initiatives and to measure every one of these channels so that you can improve and optimize your efforts.

The second step is the download. This download will hopefully come naturally with increased awareness, but there is always a drop-off from awareness to download. You'll find that some channels will drive much higher conversion from awareness to download.

The third and mostly overlooked step is activation. 20% of downloaded apps are used just once. To avoid this no-man's land, once a user commits to download and opening up your app, you want to get them to take some action to commit to your product.

The final step of the cycle is to drive users to share. Sharing can come in many different forms, from incentivized referrals to content sharing and everything in between. Word of mouth is the most effective form of marketing so the ultimate goal is to leverage your activated users to drive more awareness and therefore feed back into the cycle.

Phase 1 - Attributing downloads to the source

You have to send new potential users to the app stores in order to drive downloads but the app stores are a black box when it comes to measurement, which makes it extremely challenging to get a clear picture of where these new downloads are coming from. This is the first critical problem to resolve in order to grow.

In order to measure these various channels, you'll need to use a mobile attribution company like Branch, AppsFlyer, Adjust, or Kochava. Every referral link you create has the power to attribute downloads and even in-app user events back to the source. These can be easily visualized or exported to compare how different channels are performing.

The only growth channel that remains a mystery today is organic search traffic in the app store. It's currently impossible to know exactly how many users just typed your name into the different stores or searched for a keyword. After replacing every link that points to your app with a uniquely tagged Branch link, you can calculate the organic traffic by process of elimination for a high level estimate.

Example - Instacart & Yummly: Getting and Attributing Affiliate Traffic

Instacart, an on-demand grocery delivery service and Branch link user, found that a lot of their users were managing recipes with the recipe discovery app Yummly. They thought it would be a great idea to partner with Yummly to allow instant ingredient ordering through the Instacart platform. This affiliate relationship between the two companies would need to be tracked so that Yummly could be fairly compensated for driving downloads and orders through Instacart. Instacart decided to put Branch links to work here.

Instacart instructed the Yummly engineering team to create a Branch link in the Yummly app that points back to the Instacart app.

Since Branch also supports contextual deep linking, Yummly has the ability to include the exact ingredient list to pass to Instacart so that the new user's cart is immediately pre-populated on install. This gives the user a delightful experience upon fresh install.

Lastly, Instacart gets a report from Branch on the exact number of installs that occurred through this affiliate channel.

Example - 8tracks: Attributing Email Performance

In some situations, you might have collected a large web presence and are trying to drive new users to the mobile app. An awesome user-generated music playlist service named 8tracks was facing just this problem.

One tactic is to put an app download button in the frequent marketing emails that go out to the user base. Unfortunately, it's hard to know how many downloads are actually being generated by each email. To solve this problem, 8tracks started using Branch links to handle the button press in order to track the number of clicks, downloads and even signups per email. 8tracks is now able to measure the exact number of app downloads across both platforms for each email campaign.

Phase 2 - Converting downloads to activations

You've worked hard to raise enough awareness to drive a new download. Now we need to focus on the next phase of your mobile growth engine: converting this new download to an activation.

According to numbers we've seen across our network, if you require users to register in order to enjoy the features of your app, you'll lose over half of the users right away. This means that half of the work you've invested in convincing them to download is wasted and you need to work twice as hard for the same number of activated users.

Fortunately, there are a lot of things you can do to convince new users that your app is worth sticking around for. I bucket them into two categories depending on what's most applicable for the app: personalization and contextual deep links.

Personalization involves showing the user something relevant to their social context. This technique is best used by apps that are building an invite or referral feature for their app. For example, if a new user received an invite from their friend to install an app, that user could be greeted by their friend through a custom modal in the app upon install.

Contextual deep links bring the concept of a link router from the web era to mobile apps. It is the philosophy that if a user clicks a link that points to a page in your app, no matter if the app opens right away or the user has to install it first, you should route them to the appropriate page associated with that link. I believe the mobile ecosystem and its users benefit from the adoption of this philosophy, as it passes the user's context from outside of the app into the app, giving the user what they expressed interest in. From an implementation perspective, you simply create links for pages in your apps when users share them, then route new or existing users to that page when the link is clicked.

Example - Gogobot: Personalized Onboarding for High Activation

With 77% of app users downloading new apps based on recommendations, Gogobot decided to make this pathway easier for their existing users by allowing them to invite their friends to sign up and join through an SMS invite feature within the app. Additionally, they decided to use Branch links to personalize each download that came from an invite. In each message that went out to other users, they inserted a Branch link embedded with the original user's name, profile pic, and user ID.

When a new user receives the link and downloads the Gogobot app, Branch passes that original user's name, profile pic and user ID to the new app session and populates a personal welcome modal prompting a sign up. Users who saw a personalized welcome from their friend were 78% more likely to register in the app. This small change to the user experience powered nearly doubled the conversion from download to activation for referred users.

Example - Allthecooks: Contextual Deep Linking for High Activation

Allthecooks has built a community around sharing and discovering recipes. With this at the heart of their product, they want to be sure they are optimizing the sharing experience. Every time a user wants to share a recipe, Allthecooks automatically creates a Branch link that can easily be shared via text, email, Twitter, Facebook, or Pinterest.

What makes this link so special is that it directs the recipient to the recipe inside the app right after opening, regardless of whether or not they previously downloaded the app. It's a delightful experience. As a side note, users that originate from a Branch link are 2.1 times more likely to activate into a registered user.

Phase 3 - Driving more sharing

If you're reading this blog, chances are you don't have a large marketing budget to make paid a large component of your user acquisition efforts. Additionally, it's very rare that you'll be able to make paid a significant portion of your traffic in a sustainable way due to the astronomic prices of installs, but it can be a great method to buy an early batch of users to test the robustness of conversion from activated users to sharing.

Sharing as a label can be a little misleading, as it takes the form of many different viral implementations. For example, app sharing can take the form of an incentivized referral program, where users are rewarded with money for convincing their friends to download and use the app (ex. HotelTonight, Uber). Another example might be the basic content sharing feature, where a user is allowed to send a reference to some content in the app to their friends or social network (ex. Flipboard, Allthecooks).
(Read more: Which type of content is shared more?)
To do sharing well, it's important to build it into the natural flow of your app or to tap into the emotional ebb and flow of your users. If you just add a sharing or invite feature to check the box (as most apps do), don't expect more than 1% growth from this feature. If you build it into the product and it feels natural, you can see explosive growth from even the simplest referral feature.

'The League' is an exclusive, invite-only, application based dating app. To get access to the app, you must sit on a waitlist before being approved or be invited by a friend. The exclusivity factor has a couple natural companion sharing features that have driven astronomic growth to the core user base.

The first 'natural' sharing feature they built was the ability to move up the waitlist by inviting more people to the waitlist. Using a Branch link, they could track the number of users that each existing user generated. They would show your current position on the waitlist and reduce it with every person you invited, when Branch notified them of a new referred install.

The second sharing feature they rolled out leveraged their existing 'accepted' user base, allowing them to invite individual friends to give them exclusive access to bypass the waitlist. The League also used an expiring Branch link to ensure uniqueness of each invite that was sent out.

As a result of The League's clever sharing implementations, over 18% of the activated user base has shared the app via social media or other channels, resulting in a substantial portion of new traffic to be generated directly by invites.

Phase 4 - The Full Circle Growth Engine

This framework provides a mental lens for you to examine your growth problem in order to prioritize the work to improve it. For example, if you're seeing high awareness->download and activation->sharing but low download->activation conversion, you'll want to to work on optimizing that flow using some of the tricks I've recommended or other generic onboarding tweaks. Similarly, if it's activation->sharing, you can try to devise a sharing flow that is more inline with the user experience.

While I've broken down the circle into unique, bite-sized chunks, you must think about the growth engine as a system as it's likely that sometimes improvements to one section of the engine might have an impact down the funnel. One metric you can use over time is what I'll refer to as k-Factor, which is an effective measure for how viral your service or product is. K-Factor is simply just the total number of new activated users generated by a single existing activated user. You can use this to gauge the health of your engine, and the response to significant changes.

It's easy to your viral installs using Branch links. To calculate it, you just need to multiply three conversions together to get the result:
k-factor = (download -> activation) * (activation -> share) * (share -> new viral-driven download)
A note of caution: Keep time a constant while pulling the conversions, or you could see some strange time sensitive responses.

Closing Thoughts

There is no single answer to growth. There isn't a prescription solution and each successful initiative has to be perfectly tailored to every app's situation. The most important tenet to remember is to just keep building as you'll never get it right the first time.

Want to learn more? Check out our App Links integration guide to deep link on Facebook. This post was first published on our Mobile Growth publication on Medium

Branch is a mobile linking platform providing unified mobile experiences and measurement for more than 50,000 mobile apps, including Airbnb, Pinterest, BuzzFeed, Tinder, Foursquare, Yelp, and Sephora. Branch's linking platform can help you grow your mobile app through features like deep linking, sharing, referrals, mobile banners and interstitials, custom app onboarding, and unified attribution across platforms and channels. Learn more about Branch or contact sales today.

# data[].attributes.attachment_link.title
The Ultimate Guide to Unlocking Mobile Growth | Branch Blog

# data[].attributes.attachment_link.description
You've decided you want to take a stab at building the next successful mobile app business. First off, spend some time and think about whether you're ready for this. I'd liken the process to running in a marathon followed by wrestling in a sumo match.

# data[].attributes.body
https://medium.com/sequoia-capital/frameworks-for-product-success-aff3f29c2c57

# data[].attributes.attachment_link.readable_text
Fall 1996. A young Chris Fralic is selling software for Oracle. He's not sure what he wants to do next, but he's always been curious about venture capital. And then some unusual magic happens - a friend offers to introduce him to Kevin Compton, a vaunted name in VC. To his surprise, they talk on the phone for over an hour, and Fralic not only walks away with a comprehensive download on the industry, but a thesis on networking he's adhered to ever since: The best way to be highly influential is to be human to everyone you meet.

Fast forward to today, Fralic is a successful VC himself, responsible for First Round's investments in Warby Parker, Roblox, HotelTonight and Adaptly among others. When asked what's made his career possible, he'll tell you outright it's the relationships - built deliberately over many years. This might sound like a common response, but among his peers, he's acknowledged to be a world-class super-connector with rarefied expertise. Known for helping launch the famed TEDTalks (this is his 24th year attending TED), and a landmark Forbes piece on nailing email introductions, Fralic still responds thoughtfully to over 10,000 emails every year.

In this piece, he unpacks the strategies that have earned him this reputation - including how to become a genuine and highly-connective networker, how to propel your career forward with each interaction (while doing the same for others), do's and don'ts for getting responses from influencers in your industry, and how to regularly measure your performance in this area so it becomes a competitive advantage.

'Add value in conversations' is typical advice. This means making sure people walk away with a new idea, referral, intro, etc. But Fralic has found imparting energy to be even more important than sharing new information. To do this, follow these seven rules:

1. Convey genuine appreciation.
Actively project warmth and high energy. It's been observed people like you when they feel liked by you. So, do you greet them in a way that sounds like you're genuinely happy to see them? To make it clear you're interested in the other person, think about what they know that you don't. What do you actually want to learn in the interaction? Focus on that so that they can walk away knowing they added value too.

2. Listen with intent.
The focus you bring to asking specific questions about what's being said in real time makes others feel heard. This is a big one. Being a good listener is about two things: 1) Demonstrating that you've heard exactly what was said by the other person, and 2) encouraging them to continue. This breaks down into what's called "backchanneling" - offering short, enthusiastic responses as the other person talks (i.e. "yeah" "mm-hmm" "totally" "I can see that"), and asking follow up questions that reference the information you were just given.

You'd be surprised how often people flub on listening, says Fralic. People's minds wander, they'll be nodding but thinking about what's for dinner, they might look past the person speaking to see who else is in the room. All of this projects disinterest, a lack of value or prioritization for the person, and that can only hurt the relationship. If you're talking on the phone, asking specific follow-ups becomes even more important with no body language or eye contact to read.

3. Use humility markers.
What you say and how you say it can put others at ease and replace nerves with positive energy - even in tough situations. "I have relationships that have lasted over a decade that started with me meeting a founding team and not investing," says Fralic. "I'll often start that conversation saying, 'I'm wrong all the time and I very well may be here.'" Acknowledging your own fallibility and human imperfection can go a long way toward making yourself relatable. Especially if there's a power dynamic where someone is asking for your advice, attention or help, you want to put the other person at ease.

There's an unspoken distinction in the networking world between the Hunters and the Hunted.

When Fralic reached out to Kevin Compton all those years ago. He was approaching one of the 'hunted' - someone who had 1,000 other things to be thinking about. But he still took the time to engage and it was never forgotten.

You don't need to build yourself up any more or explain why you're important or going to be helpful. Your focus should be on building bridges between your experience and theirs so there are points of recognition, especially if you can organically work in shared struggles or challenges.

Taking the time to call or meet in person also expresses humility - which is paramount if you're about to reject someone. You want to emphasize that your time is no more important than theirs. "I like to call to explain opportunities I've passed on versus emailing. A rejection stands out among people's interactions. When you take the time to be conscientious and human, people are often appreciative and will respect you more."

4. Offer unvarnished honesty.
There are a lot of reasons why people don't share what they truly think in professional situations. They don't want to tarnish relationships or endure an uncomfortable exchange or risk being disliked. Even if you're one of the 'Hunted,' it's human nature to avoid these experiences. You can differentiate yourself by being as honest as you can. Just remember to root your honesty in what will actually have utility for the other party. This will set a good tone for all future conversations.

5. Blue-sky brainstorm.
Maybe you can't provide what someone is looking for. But, if you can change the angle or way they're thinking about something by openly brainstorming with them, you make them feel like they got something special and unexpected. It's key that you're brainstorming with them, not for them. So, in the example of passing on an investment, Fralic makes a point of listing others who might invest, or he spends time thinking through how they might pitch or message their business differently.

It's best when the conversation builds on itself. He'll suggest a few names or changes, and then provide a sounding board for any concerns or questions the founders might have. This way, he can help them find a new, albeit slightly different path forward, and that's what they'll remember - not just the no.

Give before thinking about what you get. Always offer something of value before expecting or asking for something in return. Key to this is not focusing on reciprocity.

"If you find yourself keeping score in your professional relationships, you're on the wrong track."

Instead make a list of everything you feel comfortable offering others (even if you get nothing back). Perhaps you provide connections or advice or office space or a next step in a process. That way, if you have to say no to one thing, there's still energy you can contribute.

6. End every meeting or conversation with the feeling and optimism you'd like to have at the start of your next conversation with the person.
"Assume you're going to run into everyone again - it usually happens either by plan or happenstance," says Fralic. "There are no closed connections. The world is too small." When you do meet again, you want the person to think, 'Oh great, it's so-and-so!' not 'I guess I'll get through this somehow.' If you envision running into this person again and how you want that to go, it'll undoubtedly influence how you navigate a present conversation - usually for the better.

For example, Fralic is always impressed by founders who - when turned down - send some variation of, "Thanks for looking even if it's not a fit. If you have other ideas for us or if anything changes, please let me know," or, "Chris, when we met, you had a question/issue about X. I just wanted to show you what we've done about it - no need to respond." "A person who says that shows she's savvy enough to not take bad news personally, or create obligation or awkwardness, or continue to argue their point after you've said no. I'll remember her for it," he says.

There's time beyond this fundraise and even this company. Relationships take years to build. Start now.

7. Don't fake it till you make it.
It may be common wisdom for finding confidence, but it has some negative byproducts. Namely, Fralic has seen it used to justify winging it in important meetings. Faking it in this context doesn't mean bluffing your way through interactions that make you feel insecure or intimidated. That leads to bad decision making.

"I've seen people overstate their credentials because they were put on the spot, or blindly target every executive in a room because they figured they should," he says. "This rarely leads to long-lasting relationships." If you want to connect with someone professionally to move your goals forward, you need to know exactly why you care about that person or their company. And you need to know how to articulate it succinctly. Everyone seems to have a story about a cold call miraculously turning into a career-making breakthrough. This doesn't happen by magic. It happens because your sincerity is clearly powered by diligent preparation.

If you know you're headed into a call or event and want to make a good impression on certain people, create mini dossiers for them. All it takes is a few bullet points:

What are the key milestones in their career?

What expertise do they seemingly love to provide? (Possible to suss out from any articles that quote them or talks they've given in the past.)

Are there any recent news stories or announcements about them?

What do you want to ask them or get out of the interaction if you get the chance?

Just knowing this much will give you a leg up in the moment as long as you're honest, straightforward, and have a clear objective.

"People approach me through email with 'faux familiarity' all the time, saying, 'Hey how have you been? It's been awhile!' Nope. It's been never. Just say you don't have a connection and make a compelling argument for why we should meet, i.e. 'You don't know me but you've done X and Y, would you be willing to tell me what you think of Z.' It might not work but it has a far better chance."

Build Long-Thriving (Not Just Lasting) Relationships - The Do's and Don'ts

Once you've made a strong first impression, it's up to you to turn that one touchpoint into a connection that has mutual purpose and positive impact. This is where a lot of people fumble: They get too busy, disorganized, nervous or pessimistic to follow up the right way. The result: too many high-potential relationships fizzle for no good reason. Here's a sure-footed way to make sure this doesn't happen to you.
DO: Keep your 'dream contact list' at the ready
When Fralic was in the computer reselling business, he kept stacks of trade journals that listed the names of top people in the industry. He would go through and mark those he wanted to learn more from - and, by hook or by crook (pre-LinkedIn), find a way to connect. Once he did, he'd ask if they could introduce him to the others on his list.

"What do you want your network to look like, and what are you trying to achieve? For example, I tell our founders that there should always be a slide in their board decks that shows the five most likely acquirers of their business and what they've done to further those relationships since the last board meeting " Fralic says.

"The same goes for relationships with individuals. If you know who your top 5 dream contacts are and what you want to talk to them about, you'll be ready when you run into someone who knows them. Likewise, it's good to have a forcing function for keeping those connections healthy - much like the board deck slide, consider creating a rolling reminder to get in touch. As a company, you want to have a relationship with potential acquirers before you call up asking, 'Do you want to buy us?' As a person, you want to have a relationship before you ask for a job, an intro, money."
DO: Craft low-lift requests
Make sure your asks are reasonable for the busiest of people. First, keep your emails short, simple, and to the point. Second, "If you send an email asking for something, do the first three steps of thinking for them. Make it really easy for the other person to say yes or no without creating an imposition. For example if you're asking for an intro, write a self-contained forwardable email," Fralic says. A good SCFE (as he sometimes calls it) has a subject line customized for the end recipient and quickly explains who you are, what you want and why - it's dead simple for your mutual contact to send along. Here's a real-life example:

---

Date: February 21, 2017 at 8:55:26 AM EST
Subject: Introduction to HUNTED PERSON

Hi Chris,
Thanks again for our discussion on Friday.

I appreciate your offer to introduce me to [HUNTED PERSON], to explore if there may be sales leadership roles within [COMPANY] where I can share the valuable experience I gained building [MY LAST COMPANY]'s saas business.

My CV is attached to this note. I'll keep you advised with my progress.

Best regards,
[HUNTER]

[FULL NAME
EMAIL ADDRESS
PHONE NUMBER]

----

Always include your phone number in case the person you're trying to get in touch with would find it easier to just call you directly. Few people do this. It's uniquely self aware.

Be really sure of the time commitment you ask from someone. Do you really need that much? "If your outreach is just, 'Can I take you to lunch?' that's a big ask for a lot of people versus, 'Hey, just thought I'd share with you this quick update about what I'm doing. No need to respond.' The 'no need to respond' is a powerful tool. It sounds counterintuitive, but if you give people an out, it makes them more likely to act," says Fralic. If you're aware and respectful of how busy a person must be, they'll assume positive data about your emotional intelligence and social skills, which could get you further than eating lunch with them in the long run.
DO: Follow up and follow through
It sounds so easy - just follow up after meetings and complete any deliverables you promised. But a shocking number of people don't, Fralic says.

Actually doing what you say you're going to do will put you in the top quartile.

"I recently met with Kenny Herman, who used to work for a First Round company and is one of the best business development people out there. In a meeting, he kept coming up with ideas for people to introduce me to and said he'd send over a list that I could choose from," Fralic says. "Afterwards, he actually did send me an email with LinkedIn links to all of the people he'd brainstormed and one-line details on who they were. With it came a short note offering to make introductions. That's the perfect follow up. Nine out of 10 people don't do that. Instead, most people just drop off or forget. Not only does the relationship stop there, but I'll never use the word reliable to describe them."

These days, there's tons of chatter about building one's personal brand. Ironically, your brand actually comes from you doing great, consistent work on time - not from investing in social media or a website or marketing your work, says Fralic. You have tools at your disposal to surprise and delight: speed and polish. If you can follow up fast, that'll get you noticed - even better if your content is perfectly proofread and detailed.
DO: Make your own system for keeping in touch
Fralic uses Contactually to organize his contacts and gets regular reminders to email the people he's marked as wanting to touch base with frequently. "A venture capitalist I admire used to recruit a friend of mine like clockwork - he would inevitably call every single year to see if he would join their firm. It worked. He eventually did. The takeaway: you're only human, you can't remember everyone, let a system do what you can't." Expensive software isn't required to create a CRM for your life. All you need is a system that will:

Store people's names and relevant data like email address, birthday, company, title, relevant personal or business facts that won't make you sound disingenuous. Don't ask about someone's kids if you don't know them that well.

Ping you when X days, weeks, months or years have gone by to remind you to reconnect however appropriate, depending on the nature of the relationship.

It could be as simple a creating a Reminder for each contact on your Mac that will send you a push notification every 3, 6 or 12 months. This also allows you to put their email address and any short details in the 'note' section and access contacts across devices.

"After a meeting, I'll quickly save a one-minute audio note with takeaways and next steps," says Fralic. "I then set aside an hour every Friday to go through these notes and write corresponding follow-ups if I haven't already." A good meeting recap email can inspire tremendous confidence. Consider sending along a few short bullets capturing the most important points discussed, along with a list of primary action items and who is responsible for them. This will make everyone feel more organized, clear-headed and certain progress is being made - essentially you're adding energy to an interaction over email.

Of course, while systems are important, don't let them prevent you from being spontaneous. If you stumble on an article that's highly relevant to something a valuable contact is doing or thinking about, send it along with a quick personalized note.

If you can add one 'Incremental Thoughtful Personal' email or call per day, do it. It can fundamentally impact your business and relationships in the long term.
DO: Special Ops-level recon
Fralic is meticulous about using tools and elbow grease to ready himself for meetings. Every appointment on his daily agenda includes a link to a person's LinkedIn, plus a few sentences on who made the introduction and the purpose of the meeting. On busy days, he'll list out key questions and takeaways for each meeting. He's also a fan of Clearbit, which makes a Gmail widget that surfaces context on people in your address book.

"There's so much information available, yet people don't take advantage of it. When I get an email or have a sit-down, you'd be surprised at how few people can give evidence that they've done any real work in advance. There's just no excuse for that," Fralic says. "I'll meet candidates who are interested in jobs at our startups and I'll ask, 'What companies are interesting to you?' Half the time they'll say, 'I haven't had a chance to look at your website yet.'"

This is a low bar. Set a higher standard. It's worth doing a Google search for tools to discover, capture, and make information about people accessible. This is the bread of butter of services like , and many others that make investigative legwork easy. You can set Google Alerts for your top 5 contacts, for instance, so you catch every news story they or their company is mentioned in. There are even IFTTT recipes devoted to keeping tabs on your contacts. Don't accept the standard tools. Go the extra mile to find out more, be better prepared, and keep in touch.
DO: Get to the 'Zone of Indifference'
When you approach meaningful outreach, or a sales pitch, or fundraising, or a new job opportunity, make sure you prepare as much as you can. You're aiming for the 'Zone of Indifference.' That's where you can look yourself in the eye and know you did everything you could to make it happen. When you're in the Zone, it's great if you achieve your goal, but if you don't, you spare yourself the regret, doubt or stress you might have otherwise.

On a related note, it's always better to get a quick 'no.' Leave it all on the field, so you make a good impression. If and when things don't go your way, learn from what happened. It's better to get that 'no' upfront than to get strung along waiting for a yes that never comes.
DON'T: Ambush people (always ask first)
As your network grows, you can start providing value in the form of introductions for others. But there's no easier way to burn a bridge than to send an email connecting two people without getting their permission first. Double opt-in emails are a must. If a contact asks to meet someone, shoot that person a quick email explaining who the requester is and why you think it could be a valuable interaction for them. Don't just paste their bio. If you do, you'll miss out appearing like the thoughtful strategist who put two and two together. You want both recipients to know you've got their back and you're on the lookout for opportunities to benefit them. And don't forget to always give everyone involved an out. Don't create an obligation.
DON'T: Reach out only when you need something
This can happen by accident or if you feel like you have no other choice. This is why the tools Fralic mentioned earlier are so handy. But let's say you need to make a decision on the fly. "Check your ''Sent folder to see when's the last time you reached out to someone and in what context before you fire off that note asking for help," he says. "It becomes obvious very quickly when someone just wants to use you. Don't be one of those people. This is a quick way to suffer single-digit response rates - if you're just asking people for things without thinking about what you've done for them recently, or without any context for why you're asking, people will feel imposed upon. And they won't respond. Your reputation takes a hit and everybody loses."
DON'T: Lose track of your response rate
You don't want your response rate be at 100%, because then you're not pushing far enough - you could probably ask for more assistance and still get it. "My goal is roughly an 80% hit rate for outreach," says Fralic. "For others in different roles it might be lower (if you're a sales development rep for instance) or maybe 50%. There's no magic number, but your response rate should make sense based on the strength of connection and amount of personalization and thought you've put into the outreach."

Very few people even pay attention to this metric, so they don't know where they stand. Learn from past emails to make future ones better. Let's say you send 20 emails out to close connections asking for candidates for an open role. Tally your response rate. Think about who didn't respond. Rephrase similar asks to them in the future that might appeal more to their specific interests or personalities. It's worth your time given the value that can be gained.

Email response rates are signals of your reputation. Under 10%? You've got a problem.

"You're only as good as the people you know," says Fralic. "That's why it's so worth it to make your reputation a priority. Your brand is what people think or say about you when you're not around. When they see your name in their inbox, think about what you want their first reaction to be," Fralic says. "If you follow these principles, your network will naturally widen into a solid foundation. It's not just about having another high-caliber LinkedIn connection or the email address of a VIP. Those are nice to have, sure, but the real win is knowing those people will respond when you need it."

Photography by Michael George. Chris Fralic tweets at @chrisfralic.

# data[].attributes.attachment_link.description
Fall 1996. A young Chris Fralic is selling software for Oracle. He's not sure what he wants to do next, but he's always been curious about venture capital. And then some unusual magic happens - a friend offers to introduce him to Kevin Compton, a vaunted name in VC.

# data[].attributes.attachment_link.readable_text
Every major social app, including Facebook, Instagram, LinkedIn, Reddit, and Snapchat, has a activity or News feed or equivalent feature. A user's subscriptions often include user-generated content such as photos, profile changes, and birthdays, as well as "professional" content such as news stories from media organizations and posts by celebrities. Many activity-feed-based products also rank this content, highlighting updates from the sources or types of content a user engages with most, each time they visit their feed.

The specific metric used to track and improve engagement with your product depends on the use case. Engagement in a activity feed environment, for example, is very different from engagement in an environment where content is consumed via a browse section, or through "stories." In this document, we will focus primarily on activity feeds, which highlight information based on users' subscriptions and, in some cases, on AI.

Engagement within a activity feed takes many forms. A user might simply scroll through multiple stories, or they might interact with a post by clicking on a link and reading the story, reacting with a "like" or comment, or perhaps even engaging in a conversation with other users. They might also take a picture of a piece of content, edit or comment on it, and share it with their friends.

WHY ENGAGEMENT MATTERS

Because engagement reflects whether users truly love a product and find value from it, it is the earliest indicator of product-market fit. It is also the most important driver of both retention and sustainable growth. Therefore, a deep understanding of the drivers of engagement is critical to product success.

In this series, we will explore and provide guidance on the various ways you can improve engagement: making content production easier for users, helping them connect with the content that is most relevant to them, showing them the right content in the right order, making sure they can consume content easily on any device and network, and enabling interaction with your product. Each post will also include "key metrics to track," designed to help you know whether product is truly successful.

PRODUCTION-CONSUMPTION SYSTEMS

Before we begin, we must understand the production-consumption systems that drive engagement with social products. In these systems, users produce content (or inventory) that others consume, and feedback from those consumers in the form of upvotes, comments, reactions, etc. motivates users to produce more content (see Figure 1).

Content created is inventory, how an individual user's inventory scales with their number of connections and the total content creation in the system. Open follow systems, like Twitter and Instagram tend to have more inventory per user than reciprocal follow systems, like Facebook. In most inventory-rich environment, like Facebook and Reddit, the order in which stories appear in users' activity feeds is determined by a ranking algorithm.

Engagement is fed by this production-consumption loop and, in most products, is ultimately measured by "time spent" metrics such as number of sessions, number of views, and total time spent with the product. Therefore, increasing one or more of these metrics will generally be a top-line goal for your Engagement team. In the next several posts, we will explore deeply the different parts of the engagement loop.
This work is a product of Sequoia Capital's Data Science team. Chandra Narayanan and Hem Wadhar wrote this post. We would like to thank Jamie Cuffe and Jenny Wang for their contributions to this post. Please email

data-science@sequoiacap.com

with questions, comments and other feedback
This story is published in The Startup, Medium's largest entrepreneurship publication followed by +369,832 people.

# data[].attributes.attachment_link.description
Every major social app, including Facebook, Instagram, LinkedIn, Reddit, and Snapchat, has a activity or News feed or equivalent feature. A user's subscriptions often include user-generated content such as photos, profile changes, and birthdays, as well as "professional" content such as news stories from media organizations and posts by celebrities.

# data[].attributes.attachment_link.readable_text
There are plenty of theories for why YouTube became one of the fastest growing startups in Silicon Valley history. Julie Supan says the reason is simple: "We sold the emotion."

When she became the company's first head of marketing and communications in December 2005, she immediately hunkered down with its cofounders to decide what-and who-they were about. Unlike other video-sharing sites at the time (and it's easy to forget now that there were hundreds), YouTube was built in Flash (every other product required people to download software), it didn't curate long-form professional content, or take the celebrity-driven approach that many advisors were suggesting. They weren't even particularly concerned about video quality.

Instead, they doubled down on ensuring YouTube was a "stage for everyone to participate"-specifically someone who was simply looking to have a human experience, to laugh, to share their talents, to learn something. "My team focused on showing not telling by promoting videos that people loved - whether it was a video of otters holding hands, a young woman singing in her bedroom, a man sharing his stories of WWII, or somebody teaching viewers how to fix a sink. What we cared most about was how it made them feel."

Celebrities and brands would follow, of course, as would that $1.65B acquisition by Google. But it was the everyday user who defined YouTube's strategy and introduced the world to a new kind of virality.

Since leaving the company in 2009, Supan has become one of the most sought-after branding experts in Silicon Valley, helping companies like Dropbox, Airbnb and Thumbtack craft their positioning prior to launch. Her first step? Identifying their target user-the high-expectation customer. In this exclusive interview, she explains who that is, how to find them, and why every startup needs to ditch generic platitudes and start speaking to the people who will really care.

Position with Precision

"Positioning is the big bet a company is making over the long term," says Supan. "It's the strategy for building your business, product, and brand. It's the clear action plan that will translate into positive revenue, happy customers and a great place to work." Your position isn't your messaging or your product roadmap-but it is the foundation of every decision you'll make along the way.

Before you can embrace where you're going, you need to understand who you're taking with you. "Identifying the one thing you do really well that is original and defensible is the first step. Then you have to align your business, product, operations and marketing to ensure that you are building precisely what customers need-even if they didn't realize they needed it until it came into their lives."

To kick off her positioning work, Supan walks founders through five key questions:

Who is the customer that needs/wants your service or product most?

Why does your product or service matter to them?

How do they feel about your product or service?

What is its true benefit to them?

Will your product exceed their expectations?

The name of the game here is precision. "If you're working with shared or autonomous vehicles, ask: if I don't have to drive my own car someday, then, what will I be able to accomplish now that I will get some time back in my day?" says Supan. "Or for that storage startup: If my life's work is in the cloud, how will that change what my workday looks like?" Think about the customer who'll get the most out of your product, so you can speak directly to them - and bet on their emotion."

What Supan sees more often than not, though, is the opposite approach: Early-stage founders, eager to look good for that next funding round or earn enough to make it another month, go after every possible user. They launch their products with a chorus of generalities-"We're going to change your life!" or "Revolutionize X!"-in an attempt to be something for everyone. After all, the goal is to rack up users, and a DAU is a DAU, right?

Wrong.

"Founders are told that they need to bring up their daily active users, their monthly active users. They need to make sure they have the right level of activity. They need to make sure they have the right level of retention. They need to have the right level of acquisition. It's a lot," says Supan. "And they believe that appealing to every customer is one way of solving these problems."

But Supan asks founders one more question-an easy one-to highlight the folly of that line of thinking: Could you do all of that on the product side?

"They will always say, 'Well, of course not,'" she says. "Resources are constrained. They have to force rank what they can build. Well, series A- and B-stage companies don't have the resources to target every customer, either. Be selective about which customers you speak to and which ones you don't.""

Moreover, a "see what sticks" approach in the early days can be a dangerous drain on your internal processes and staff morale. When a company lacks focus, that plays out in its day-to-day operations. "Meetings run longer, decisions get delayed. People get frustrated not having a clear strategy that they've bought into," says Supan.

"All of this means that you have every reason to do this work, to figure out your precise target customer for the short and long-term. Even if it means taking a couple of months to really nail this down. What's a couple of months in the roughly 5-10 years you'll spend betting on this customer to build and grow the company?"

Enter the High-Expectation Customer

The first and most crucial step to successful positioning, then, is to define your ideal user-what Supan has coined " the high-expectation customer."

"The high-expectation customer, or HXC, is the most discerning person within your target demographic. It's someone who will acknowledge-and enjoy-your product or service for its greatest benefit," says Supan. That discernment is key, because this customer is also someone who can help startups spread the word.

The HXC needs to be a person who others aspire to emulate because they see them as clever, judicious and insightful.

The high-expectation customer is a good consumer. They're someone who can be trusted to know the market and make good decisions. "They look things up. They research things. And they have ideas for new types of products or services that can help them save money, gain time, get healthier or make their team more productive," says Supan. "If your product exceeds their expectations, it can meet everyone else's."

Consider who this customer is for some well-established companies:

Airbnb: This HXC is invested in being a good global citizen, and doesn't want to simply visit new places but to belong. "It's the guest who wants to 'live like a local' and experience Paris as if they are living there," says Supan. "They're energized by the idea of staying in unique spaces and feeling welcomed, but are cost-conscious, too."

Startups like Airbnb that build a two-sided marketplace-guests and hosts, riders and drivers, buyers and sellers, etc.-may be wrangling twice the people, but their process for identifying the HXC doesn't actually look that different. "The company does have to take a position. While Airbnb will say consistently that the hosts are their business-and that is true-the position of the company is toward the guest. It's about belonging," says Supan.

But there's no rivalry there; hosts relate to that position. "They, too, want to open their homes to learn about different cultures, and increase their income to be more self-reliant. They understand who Airbnb's guest is and want to welcome them," says Supan. "And, many hosts are also Airbnb guests, which is typical of marketplaces where the top sellers can also be the top buyers. Messaging, in these cases, may change depending on which side of the marketplace you're talking to, but your positioning should not."

"There are more riders, joiners, attendees, viewers, customers, guests, and buyers in the world than there are drivers, organizers, creators, pros, hosts, sellers," says Supan. "If you bring in the largest audience in the world, you will bring in the best video creators. Same is true for Airbnb - if you bring in all the guests in the world, hosts are motivated to showcase their unique spaces and participate in this new sharing economy."

Dropbox: "This one may come as a surprise, because people used to think of them as just a file sharing and syncing technology company. But the Dropbox high-expectation customer is actually a person who wants to simplify his or her life," says Supan. "They're trusting, organized, tech-savvy, and looking for ways to get some time back in their day. They want to know they're covered, that someone has their back when it comes to their life's work, which is primarily in their computers-family photos, videos, work files, school docs."

That's not every Dropbox user; plenty of people just use their free account to share a file here and there. But that's the idea: your HXC should not be an all-encompassing persona. "The high-expectation customer will benefit from the product's greatest attributes and spread the word. Then others will say, 'If it works for my tech-savvy friend Lisa or if it works for Tishman Construction, which built the World Trade Center and was an early power user of Dropbox, then it will probably work for me,'" says Supan.

Lululemon: Of course, companies outside of tech benefit from well-defined HXCs, too. Chip Wilson, founder of Lululemon, described his company's two "muses" in a 2015 New York Times piece: Ocean, a 32-year-old single professional woman, inspired his initial target market and was later followed by Duke, her 35-year-old male counterpart, an "athletic opportunist."

These weren't just demographic sketches-Wilson understood his muses, his HXCs, down to their daily routines. Ocean, for example, owns a condo and likes to travel. She has about an hour and a half to work out each day. Of her, Wilson told the Times, "If you're 20 years old or you're graduating from university, you can't wait to be that woman. If you're 42 years old with a couple children, you wish you had that time back."

Why Your HXC Isn't Necessarily Your Early Adopter

It's tempting to assume your early adopters are also your HXCs, but that's not always the case-and failing to make that distinction can prove challenging to a young company.

Take FlightCar, the airport car-sharing startup that matched outgoing, car-owning travelers with incoming visitors. The company's earliest adopters were its monthly car-sharing customers-those car owners who traveled extensively and were wooed by the notion that they could earn a guaranteed monthly payment for cars that would otherwise sit idle, accruing parking fees. "Our early ads had dollar signs all over them," says Supan, who came on to help FlightCar shift its strategy and rebuild the product last year.

While those early customers did quickly drive supply and volume, they weren't invested in the company's underlying mission. "We built the entire model-the business model, the operations model, the marketing model-around a target who didn't embody the sharing economy. Those users didn't value their role or see themselves as part of a car sharing community helping people become more self-reliant," says Supan. "And when it came to helping FlightCar grow, they didn't care about that either. When there was a problem, they were the first to write angry Yelp reviews. They had no interest in helping the company improve its policies or being open to changes as the company began to evolve the business model and improve operational efficiencies. Because there was very little goodwill, the company was not given the benefit of the doubt as it expanded from one airport in the U.S. to 17 locations."

In parallel, another customer set was growing organically from the very beginning, which they called the standard customers, a very different type of user. Acquired by word of mouth, they were invested in the self-reliance of the sharing economy, and embraced their role as beta testers of a new concept. They were excited to save money on parking while they were away and earn a little extra money while renting their car to a visitor who came to their town. These were the high-expectation customers.

"They were the people that could have helped to create the right kind of growth, the customers that would promote a brand, spread the word over the years, and encourage others to follow and participate. But the model wasn't built for them," says Supan. "It had evolved with the paid customer as the North Star, and the company worked diligently for nearly a year to refocus and rebuild the FlightCar experience from the ground up. Ultimately, the team made a decision to sell its newly redesigned technology platform to Mercedes-Benz Research & Development North America, where it has become part of their innovation lab for global mobility services."

Find Your HXC

It's no exaggeration to say that correctly identifying the target customer can make or break a young company. Founders need to take the time to methodically define their HXC:

Do your research. This is not the time to guess or go with gut reactions. "A significant amount of customer research is required to deeply understand what people love, what frustrates them, what they believe your greatest challenges and true benefits are, and what they wish for in the long term," says Supan.

She advises startups to gather data by targeting the full range of customers - happy, unhappy, recent, early, active, inactive - across geographies and through no fewer than five channels, including:

general population or decision maker surveys

single-question and open-ended surveys

single questions embedded into sales or marketing calls

phone interviews

customer service channels

product intercept surveys

app store reviews

"You might even have to create new channels and build features to capture data as people are engaging with your product or service," she says.

Equally important at this stage is what not to do. "Don't ask customers questions about features or about your UI. Tools such as Usertesting.com won't work for this kind of research. The people who agree - and are paid - to test products tend to be much more polite and less passionate about your product," says Supan. "And if you can predict the answer to a question, don't ask it. You shouldn't cut your survey or interview short, but be mindful of people's time, too. I recommend letting the customer know that you are doing a survey to better understand how they feel about the product and that their thoughts will go directly into the company's strategy."

Calibrate the data. And don't rush it-this can take several weeks of your team's time. Looking at the insights gathered across channels, begin to identify the common themes in your data. Ask these questions to surface these trends:

What are you seeing that was expected and unexpected?

What direction are those results pointing you in?

How does that align with your current strategy, roadmap and team?

Define your customer. With your aggregated data in hand, it's time to start sketching out the profile of your high-expectation customer. This is where you'll identify the specific personality traits and characteristics that define the target audience.

If you've designed your surveys well, you can often look to respondents' own words for a head start. "One of my favorite questions is ' How would you define the kind of person that would benefit from this product most? '" says Supan. "When people answer, they usually describe themselves. It's a great way to ask the 'Who are you?' question while making people feel like they are describing someone else."

Calibrate again. Before wrapping up, Supan suggests running through some final questions to make sure you've hit on an HXC you can really live with:

Who looks up to this person?

Is this person someone people feel they can relate to or aspire to be?

Can you put yourself in their shoes and see your product from their point of view?

What would cause them to be disappointed by your product and what do you have to do to ensure this won't happen?

Consider Your Customer's Mindset

Once you've defined your HXC, you're almost there. To get the most mileage out of that definition, though, you need to put it in the context of the customer's past experiences-their consumer ecosystem and the biases they bring to your product. Otherwise put, you need to consider whether they're coming to you with a growth mindset or a fixed mindset.

If your target customer has a growth mindset, great. "In those cases, users are incredibly open. They've never tried this before, and they can't even imagine it can exist. Take the first time people discovered the most unique of spaces on Airbnb: 'Wow, I can stay the night in a windmill?' There's no context for that so they are excited and amazed," says Supan.

Other times, users will come to you with a fixed mindset, and it's important your product inspires and encourages them to reimagine what's possible and then delivers. "Take Thumbtack, the leading online marketplace for local service professionals. Many customers come to the site thinking, 'Contractors don't call back. They're hard to reach. I'm going to have to make 11 calls to find out if one is available,'" says Supan. "Thumbtack is aware of this mindset having done extensive customer research very early on. The team is positioning the company with a 'you can do it' approach, using statements like 'Customers use Thumbtack to get millions of projects done quickly and easily.'"

Having done that early research, Thumbtack has built and positioned its service to wow customers as soon as they request a pro to help them with their projects. "When the customer promptly gets a quote from an available, reputable pro without having to place calls and do the legwork, they've moved to a growth mindset and feel supported and valued by the company," says Supan. "This customer is very likely to come back."

"You can be more successful and generate goodwill speaking in an aspirational way to the customer rather than anti-positive messaging such as 'This was broken before, but now we've fixed it,'" says Supan. "The goal is simply to guide the customer toward a spirit of openness so they can benefit in a positive way from change."

To build anything transformative, you need a customer that's willing to give you the benefit of the doubt.

Get the Team on Board

If that sounds like a personal relationship, in many ways it is. After all, you're going to be spending an enormous amount of time with this "person." Simply put, you need to like them. And you need your team to buy into the relationship, too.

"Brian Chesky, the CEO of Airbnb, has said, 'You have to become the person you're designing for,'" says Supan. "The entire team needs to agree that this person is worthy of their time. In many ways, they need to aspire to be the HXC, too."

There's no way around it: startups are a grind. If you're not building something you want to build, it's next to impossible to stick it out. "I've seen companies retool the entire company for a customer that they're not excited about. Sometimes, I've talked to companies that don't like or can't relate to their high-expectation customer," says Supan. "That is really difficult. That affects the culture. That affects hiring. That affects the energy of people on the team-their focus and commitment."

On the flip side, nailing the high-expectation customer can be game-changing for a startup. "You'll actually see a physical response from your team-newfound energy-if they agreed with and were part of defining the target customer. You'll see more urgency and a sharper focus," says Supan.

A clearly articulated HXC can even help focus your hiring. "In many ways, if you look at everyone that works at Lululemon, they channel Ocean," says Supan. "When you get off the elevator for an interview at Airbnb, you're literally group-hugged and welcomed. They define their customer in terms of love, caring about people, being a global citizen, and you need to feel that as soon as you step foot in the office."

Hiring for shared sensibilities is not enough, though. You need to be deliberate about rolling out your HXC, and make this person a regular, familiar presence in the office. "Create a presentation or cheat sheet that can be shared with the whole team, offer a tutorial as part of onboarding new recruits or develop a scorecard of how well the product is doing in exceeding this customer's expectations. Mention the HXC in all-hands meetings. Consider convening a focus group or 'user council' that reflects a cohort of HXC customers and run strategies and new features by them."

Hit the Ground Running

Your work is not done, of course, once you've identified your high-expectation customer and brought the team on board. You need to incorporate that customer into the DNA of your company-your strategy, product roadmap, operations and marketing. "It can take 6-8 weeks to do this well and discuss how things need to be altered to exceed the expectations of your HXC," says Supan. "In almost every company I've been a part of, we were able to review the product roadmap with a sharper perspective, freezing some features and in most cases building new ones that better reflected our new position and HXC."

When it comes to marketing, the good news is that the work you've put in on the front end will quickly focus your strategy and messaging. Relieved of the burden of that "go after everyone" mindset, you can invest your sales and marketing dollars in acquiring and retaining a single customer target-they'll take care of the rest. "There's more objectivity in one customer sharing their passion and love for a product with another than there is in ads or promotion. It allows the company to spend less to achieve lasting growth," says Supan.

Don't be afraid to use your HXC's own words, too. You invested in customer data-gathering-now mine the results for the language that will resonate most. "Customers will hear their own words and say, 'Whoa. This product is for me.'"

Keep a close eye on nonverbal messaging, too, including your product's look and usability. "If Dropbox was going to say 'It just works,' for example, their product needed to work everywhere, across every platform, in order to exceed the expectations of their HXC," says Supan. "Their design vision must also line up with how the target customer feels about the product. One of the brand values is 'We have your back.' And there's barely any UI with that product. It's up in the right-hand corner. You almost never actually interact with it. It's simple and understated, but you know it has you covered."

Targeting your messaging doesn't mean single-mindedly sticking to the first thing you try, though. "You do need to identify ways you can do some split testing on how that position is received by customer," says Supan. "Look at every touch point. You're not going to overhaul it all in one day, but ultimately you want it all in total alignment by a certain point when you're ready to put a stake in the ground."

The high-expectation customer is the truest form of virality.

Check Back In

At the end of the day, your HXC represents people-real people, living in the world-and you need to regularly revisit your target to make sure you're still hitting the mark. Supan urges startup leaders to remain keenly attentive to shifts in their company or their market, and check in when their gut tells them to. "Maybe it's every two or three years, when you've seen a certain amount of growth. At some point in the future you need to ask yourself again, 'How has the world evolved, and what does my customer expect now? How might they have grown?'"

"I look at Twitter, and while I wouldn't say they got their positioning wrong, I think they skipped the step of checking back in," says Supan. "Early on, Twitter made the call to target celebrities and other media-savvy types eager for a new way to reach their fans. It's been a successful strategy from a PR perspective, and those celebrity superusers have certainly reaped serious branding rewards. But the company's slow growth suggests that while regular people enjoy a glimpse of celebrity, that doesn't mean they'll actively use the product themselves.

"No one can aspire to be Justin Bieber on Twitter. He has come a long way since he was a 12-year-old uploading homemade videos of himself singing on YouTube," says Supan. "But I look at the Twitter product and say, 'If it works for him it will definitely not be for me.' He's setting the bar so high you actually feel excluded. This HXC is not someone that people feel they can emulate or envision themselves to be."

It's easier said than done, but continually surveying and engaging with your customers is well worth the effort. "Employ someone creative and analytical who can manage ongoing multi-channel research and work closely with the product team," says Supan. "And make sure that you are continually calibrating those insights against the HXC. Rarely have I seen a company shift their HXC entirely over the years, but it's important to evolve with them and the world around you."

The goal the first time around was to achieve consensus, and that's true the second, tenth or hundredth time you check in on the HXC, too. "You're always working toward a state where the market, the company, and your customer are in broad agreement about what you are and why you matter," says Supan. As your company evolves, the HXC serves as a valuable touchstone to ensure that you're growing in the right direction and to validate - or invalidate - your action plan.

"The best thing about identifying the HXC is that everyone - from employees to customers - gains peace of mind. Your users will understand the product, and know that it's meant for them. You'll slash your cost of customer acquisition and enjoy the benefits of a clear roadmap. With bias removed, your team can be more creative," says Supan. "Keep checking in with your HXC and calibrating insights against them. Keep learning about them. If you do the work, they'll lead you to consensus about what your company is-and what it isn't."

Photography by Bonnie Rae Mills.

# data[].attributes.attachment_link.title
What I Learned From Developing Branding for Airbnb, Dropbox and Thumbtack

# data[].attributes.attachment_link.description
There are plenty of theories for why YouTube became one of the fastest growing startups in Silicon Valley history. Julie Supan says the reason is simple: "We sold the emotion." When she became the company's first head of marketing and communications in December 2005, she immediately hunkered down with its cofounders to decide what-and who-they were about.

# data[].attributes.attachment_link.readable_text
Here at Social Capital we spend a lot of time conducting diligence on potential investments. In the next few weeks we're going to discuss how we think about quantitatively assessing product-market fit. The approaches we discuss here are useful beyond diligence for investment. Many of our portfolio companies use these concepts as part of their operating process to keep tabs on their growth and evolving product-market fit. Hopefully you find it useful!

Diligence is designed to help us understand both what currently exists and what may come to be as well as to try and help the entrepreneur on their path. While we definitely talk to many entrepreneurs in the pre-product stage, most of our time ends up being spent with entrepreneurs who already have a product with users/customers. In this case, part of diligence involves developing an objective understanding of demonstrated product-market fit. While every company is different, we have a few standard ways of looking at core traction metrics and we'd like to share these with you in the hopes that it helps to give you insight into your own business. It should go without saying that there are many other aspects to diligence including, but not limited to, team, market, vision, competition, core technology, etc. and that we will only be discussing a particular subset of diligence in these posts.

A tentative outline for how we will present our approach is as follows:

1. Accounting for user growth

2. Accounting for revenue growth

3. Empirically observed cohort lifetime value (revenue)

4. Empirically observed cohort lifetime value (engagement)

5. Depth of engagement and quality of revenue

6. Epilogue: The 8-Ball and GAAP for Startups

These topics apply in slightly different ways for consumer businesses vs. enterprise SaaS businesses so we'll generally treat them separately, but the framework applies to both types of business.

Today I'm going to focus on the first of these topics.

Accounting for User Growth

To get started, let's pretend we have a consumer company that intends to get lots of users via some novel social/mobile/content product strategy. For these types of companies the most common graph that we see in pitches is a graph of users going up and to the right. Sometimes companies try to show us a graph of "cumulative registered users", which is clearly a vanity metric. A user who has registered and is not active in your product is probably not getting much value and is probably not a good indication of product-market fit.

Given that cumulative users is rarely shown in serious pitches, in the current era we typically see a graph of monthly active users (MAU) going up and to the right.

This is showing 16 months of roughly 12% m/m growth which is quite impressive. However we always go one level deeper to understand the nature of the growth. For the purposes of illustration, let's pretend that we have some sort of mobile application. For the definition of "active" you should use whatever definition best encapsulates active for your application. This can be as broad as "opened the app" or as specific as "carried out a particular action". Consider the following two accounting identities.
MAU(t) = new(t) + retained(t) + resurrected(t) MAU(t - 1 month) = retained(t) + churned(t)
The first one says that active users today (for the trailing 30 days) are either new users, retained from the previous month or resurrected from some prior time period. Note that this is a mutually exclusive and completely exhaustive classification of current users. The second identity says that the MAU from last month either came back and were retained or did not and thus churned.

Manipulating the above yields the following:
MAU(t) - MAU(t - 1 month) = new(t) + resurrected(t) - churned(t)
Which is to say that MAU growth receives positive contributions from new and resurrected users and receives a negative contribution from losing users to churn.

Here's how we prefer to look at the MAU growth accounting quantities for the above fictional company. If you're going to show us your MAU growth you'd do well to show it to us in this manner.

The bars show the three terms on the right of the equation above that contribute to MAU growth. I also overlaid two ratios that are useful. The retention rate is month-over-month so this says that our fictional company has a retention rate of ~40%. Note that because of the accounting identities above, this means that the churn rate is (100% - 40%) = 60%. The other ratio is (new + resurrected)/churned which is to say the ratio of the area above the line vs. below the line. This ratio needs to be greater than one if the app is to be growing, otherwise churn is overwhelming growth. We call this the "quick ratio".
Quick Ratio = (new + resurrected)/churned
This term was coined in a deck from Mamoon Hamid from earlier this year. We'll talk more about this concept in a later post when we discuss revenue. Also, this should not be confused with the normal finance quick ratio that measures the ability of cash and near cash assets to pay off liabilities . For this company the MAU quick ratio is fluctuating between 1 and 1.5. Which is to say that for every 3 new users the company adds it is also losing 2-3 users to churn.

Note that this gives much more information than the MAU graph above. This is telling us that there is some large month-to-month churn which is being overcome by large contributions from new and resurrected users. The retention rate has been stable and not showing any particular trend. On the plus side, it's not going down as the app is growing, but it's also not getting any better which you might assume if you only looked at the top-line number or at presumable feature/functionality changes in the past 12 months.

In terms of how the above looks, we'd classify this as a so-so situation for a consumer app. Most consumer applications don't have a very strong mechanism to bring users back month after month and so the quick ratio tends to be just above 1. The dynamic for each month in a consumer app is typically to add a bunch of users and to simultaneously lose a bunch of users with a small additive piece on top from resurrection yielding overall small positive growth.

To show you the power of this view, there are other ways that these components could conspire to produce the same top-line growth. For instance..

Note, these numbers would produce exactly the same MAU graph shown above. I kept the axes the same to emphasize the difference. In this case, the new component isn't growing as fast but it's also retaining much better. The quick ratio for this company is more in the 1.5-2.0 range which would be very good for a consumer company (for every 3 customers that you are gaining you are losing between 1.5-2 customers, much better than the first company). Note the spikes in resurrected possibly due to some resurrection campaigns which were not accompanied with a corresponding spike in churn.

All else being equal, the second example would be a more attractive company to us because it is starting from a better base. With such high retention it would be worth trying to push harder on the top of funnel with new users to drive growth (more aggressive sharing/referral mechanisms, paid acquisition, etc.). For the first example it's harder to justify pushing on new users as you would end up losing many of them. It's easier to fill the top of funnel than it is to fix some underlying churn problem.

It should also be clear that this accounting can be done on time-frames other than calendar months. Indeed, to make this operational, several of our portfolio companies implement it on a rolling 28 day basis (to remove day-of-week effects). Also, this approach works just as well with weekly active users as it does with monthly. Typically early stage consumer products already have trouble generating m/m retention, much less w/w retention so the w/w views will show very high churn and may not be useful. But if your product is extremely sticky and is already retaining at a very high level on a monthly basis then it might be time to explore generating the next level of engagement at a weekly level. For something as sticky as FB it even makes sense to get it down to the daily or even sub-daily level.

In the next post we'll apply this framework to subscription revenue which is useful both for enterprise SaaS as well as consumer subscription businesses.

Published in Startups, Wanderlust, and Life Hacking

# data[].attributes.attachment_link.title
Diligence at Social Capital Part 1: Accounting for User Growth

# data[].attributes.attachment_link.description
Here at Social Capital we spend a lot of time conducting diligence on potential investments. In the next few weeks we're going to discuss how we think about quantitatively assessing product-market fit. The approaches we discuss here are useful beyond diligence for investment.

# data[].attributes.attachment_link.title
The 20 Most Popular Android Apps in the Google Play Store

# data[].attributes.attachment_link.description
Ever wanted to know what are the most popular Android apps? Here are the most popular Google Play Store apps of all time.