Notes from Lean Analytics for Startups

I created this quick guide for understanding Lean Analytics as I was watching this talk by Alistair Croll, co-author of “Lean Analytics: Use Data to Build a Better Startup Faster” at a Google Ventures workshop.

The fundamental idea behind the Lean Startup movement is “Don’t sell what you can make, make what you can sell.”

This is the core process:

Lean Startup Process

Lean Startup Process

Most startups don’t know what they will become. Examples, Hotmail was a database company, Paypal was built for Palmpilots, Twitter was a podcasting company, etc.

Analytics is the measurement of movement towards your business goals. 

In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.

What is a good metric?

What is a good metric?

If a metric won’t change how you behave, it’s a bad metric. 

Screen Shot 2015-05-17 at 3.16.58 pm

Screen Shot 2015-05-17 at 3.19.37 pm

Basic Taxonomy: 

Cohort Analysis

Market Segmentation

A/B Testing vs Multivariate Testing

Lagging metrics are historical in nature. They describe the past.

Leading metrics are about the future. They predict the future.

Examples of Leading metrics

Examples of Leading metrics

The metrics in the chart suggest which users are more likely to become loyal users.

Remember : Correlation does not imply causation. For example, there is a strong correlation between ice-cream consumption and drowning deaths. But the cause of the drownings is “summer time” and not eating ice-cream. Therefore “summer time” is the causal factor.

Growth Hacking, demystified:

1. Pick a metric to change

2. Find correlation

3. Test causality

4. Optimize the causal factor

The Lean Analytics Framework

The Lean Analytics framework “steals” from models by Eric Ries and Dave McClure

1. Eric Ries’ Engines of Growth

Eric Ries' Three Engines of Growth

Eric Ries’ Three Engines of Growth

2. Dave McClure’s Pirate Metrics model (AARRR)

Dave McClure's Pirate Metrics model (AARRR!)

Dave McClure’s Pirate Metrics model (AARRR!)

The Five Stages of the Lean Analytics Framework:

The Five Stages of the Lean Analytics Framework

The Five Stages of the Lean Analytics Framework

1. Empathy stage: Run surveys and experiments before writing any code. Localmind used Twitter to ask location-specific questions to test whether users would answer these questions from strangers. Avoid leading the witness, phrase the words of the questions in an unbiased way. Use “ghost” features, empty links on the site to see how many users clicked-in. After clicking, tell them that the feature is not yet finished and ask them what they would like to see.

2. Stickiness stage: Take all that doesn’t encourage adoption and reuse out-of-the-way of the user. Example metric is Engagement, number of days since last engagement, number of users that are disengaged (haven’t come back for the last 10 days).

3. Virality stage: Content sharing, inviting other users, etc. Inherent virality (happens automatically while using the product) or manufactured virality (e.g. special offers).

How to calculate viral coefficient:

How to calculate viral coefficient

How to calculate viral coefficient

The viral coefficient was not the only metric which is important, also consider the virality cycle time.

4. Revenue stage: Can focus on metrics like MRR – Monthly Recurring Revenue, ARPU – Average Revenue Per User, CAC – Customer Acquisition Cost and CLV – Customer Lifetime Value.

5. Scale stage: Metrics like Incremental Order Cost, Unresolved Support Tickets, etc.

Types of Business Models for Internet Businesses:

1. E-commerce
2. 2-sided marketplaces
3. SaaS
4. Mobile Apps
5. User-Generated Content
6. Media

Whatever your business model is, you should draw a flowchart like this. The book has flowcharts for all six models.

User/Customer Flowchart

User/Customer Flowchart

The combination of the model and the stage you’re in gives you the “One Metric That Matters”.

Moz cut down on the metrics they were tracking and used Net Adds. Net Adds = New Users – Churn.

Screen Shot 2015-05-17 at 5.50.48 pm

Very useful Master table of Metrics based on Business Model and Stage is available here

The Lean Analytics Cycle

Screen Shot 2015-05-17 at 7.46.28 pm

You can only Draw a new line when there is some new information from the customer.

The “One Metric That Matters” changes with time, initially it could be all about sign-ups for the first couple of months, then it becomes about conversion rate. Here is an example of a traction graph, where the metric changes over time.

Example of a Traction Graph

Example of a Traction Graph

Managing a Lean Startup using the Three Threes Model:

The Three Threes Model

The Three Threes Model

Thank you Ben and Alistair for this great knowledge! Here is how you can reach them.

Screen Shot 2015-05-17 at 8.26.36 pm

Leave a Reply

Your email address will not be published. Required fields are marked *