LEAN STARTUP METRICS
     @STUECCLES

     4th Feb 2012
I NEVER FELT LIKE I
    WAS LEARNING
ANYTHING IMPORTANT
“An actionable metric is one
     that ties specific and
repeatable actions to observed
            results.”
      ASH MAURYA http://www.ashmaurya.com/
EVENT BASED TRACKING
F   UNNELS
SIMPLE SIGN-UP FUNNEL
EXTENDED SIGN-UP FUNNEL
A/B   TESTING
+152%
A/B TESTING USER MAP
 ON HOMEPAGE
GOOGLE BING
“THAT BLUE WAS WORTH $80M AT
           LEAST”
      PAUL RAY - PRODUCT MANAGER BING
AN
               INTUITIVE
               LEAP
LOCAL
MAXIMA

         PROPS @BOKARDO FOR LAST FEW SLIDES
@37SIGNALS
WHAT TO A/B TEST
Short Answer: Everything

Longer Answer: That you have
enough traffic to see significant
results that are not trival
optimisations (HINT you are not
Bing/Google)
WHAT TO A/B TEST
Every new feature should go
into A/B testing.

If has negative or no real effect,
pull it.

Optimise existing features/copy/
design (80% existing 20% new)
F   RAMEWORK
ACTIVATION                                       REFERRAL




                    RETENTION

                                                         REVENUE


ACQUISITION




         Dave McClure http://500hats.typepad.com/
DON’T TRY TO MEASURE TOO MUCH
E   XAMPLES
SAY HELLO TO OUR 3 FICTIONAL
          STARTUPS
G GIE CO LR
DO
 Social Network for Dog Lovers.

 Makes money through
 advertising and affiliate sales of
 lush dog collars.
G GIE CO LR
DO
 Key business drivers are:

 Advertising and affiliates need
 large traffic/page views so
 growth and retention are key
ACQUISITION              VISIT


ACQUISITION          SIGN-UP

                     CREATE
ACTIVATION           PROFILE

                POST
ACTIVATION                  COMMENT
              PICTURES

               SHARE          INVITE
 REFERRAL
              PICTURES       FRIENDS

 REVENUE           BUY COLLAR
IL ES AM BA
AG
Project management tool for Agile
startups.

Makes money through monthly
subscriptions. Has a free trial period.
IL ES AM BA
AG
Key business drivers are:

Conversion to paid usage and
cancelation rates
ACQUISITION     VISIT


ACQUISITION   SIGN-UP


ACTIVATION    CREATES
              PROJECT

              UPGRADES
 REVENUE        PAID

              CANCELS
 REVENUE      ACCOUNT
T MY DR ILL
R EN
Marketplace to rent/buy drills from
your neighbour (AirBnB for drills)

Takes a small percentage of the
transaction between two parties
T MY DR ILL
R EN
  Key business drivers are:

  Two sided business model
  means acquisition of both
  lenders and borrowers and
  transactions per user
ACQUISITION              VISIT

               LENDER       BORROWER
ACQUISITION    SIGN-UP       SIGN-UP

               SEARCHES     SEARCHES
ACTIVATION
              BORROWERS      LENDERS

                     BORROW
 REVENUE              DRILL
ONE KEY METRIC
There is generally only one key
metric you need to concentrate
on at a time. Such as referral or
sign-up conversion or
cancellation rate.
A   CQUISITION
TRACKING MARKETING
CHANNELS
Google Adwords
Banners
Social Media
Partnerships
PR
Blogging
Anything that links to your site!
TRACKING MARKETING
CHANNELS
Use unique urls (tracking
parameters) on every url you
create/give-out/pay for

Use bit.ly

Use HTTP referrers to group PR
MEASURING COST OF VISITS
Cost per click (site visit)

Cost of campaign / traffic
generated

(Social media) Cost of effort /
traffic generated
MEASURING COST OF
ACQUISITION
Cost of one visit / conversion
rate = Cost of an acquisition

Acquisition could be paid for
acquisition or free sign-up

Unfortunately not that simple
DIFFERENT CHANNELS CAN HAVE
DIFFERENT CONVERSION RATES
MARKETING CHANNELS
Per channel track:

Scale
Cost
Conversion

Optimise and scale channels
with best Cost of Acquisition
(CoA) and potential to Scale
ACQUISITION
Cost of Acquisition < Customer
Lifetime Profit

Then scale acquisition at a
similar or lower rate

A/B test acquisition channels to
optimise
ACQUISITION INCLUDING REFERRAL
 If it costs £3 to acquire a
 customer through advertising

 But that customer brought in 2
 other customers through referral

 The COA of that channel is £1
A   CTIVATION
SURE THEY SIGNED-UP BUT DID
       THEY LIKE IT?
ACTIVATION AT FACEBOOK
(i) which data points predict
whether a user will stay?

(ii) if they stay, which data points
predict how active they’ll be
after three months?
ACTIVATION AT FACEBOOK
(i) having more than one session as
a new user, and entering basic
profile information.

(ii) how often a user was reached
out to by others, frequency of third
party application use, and how
forthcoming a user was on the site.
ACTIVATION METRICS OFTEN DRIVE
  OTHER METRICS IN REFERRAL,
      RETENTION, REVENUE
R   ETENTION
RETURNING USERS
Tracking unregistered users that
return often (useful for content
sites with advertising)

Tracking registered users that
return and take some sort of
action.
COHORT METRICS - CREATED
PROJECT
COHORT METRICS - CAME BACK
R   EFERRAL
REFFERAL MECHANISMS
Social Media Sharing

Invite a friend type mechanics

App Reviews

Word of mouth
VIRAL COEFFICIENT
The average number of
customers each customer refers
VIRAL COEFFICIENT - EXAMPLE
Each Doggiecollr customer
invites 5 other customers by
email
20 % click on that email link

50 % of those users convert into
customers
VIRAL COEFFICIENT - EXAMPLE
  Viral coefficient =
Refferals x click through x conversion
    5 x 0.2 x 0.5 = 0.5
VIRAL COEFFICIENT
A viral co-effiecient greater than
1 means that every customer
gets more than one other
customer on average

Therefore your product will
grow virally
VIRAL COEFFICIENT
Measure automatically for all
your in built referral
mechanisms.

Give customers unique invite
URLs and track conversions
VIRAL CYCLE TIME
The average time taken for a
referral to turn into a customer

With the viral coeffiecent and
viral cycle time you can
determine your viral growth
NET PROMOTER SCORE
 Number of people who would
 recommend your product to a
 friend.

 A single question survey
NET PROMOTER SCORE
    How likely is it you would
    recommend AgileSamba to a
    friend or colleague?
0    1   2   3   4   5   6   7   8   9   10




     NPS = % Promoters - % Detractors
NET PROMOTER SCORE
 Survey a small percentage of
 your customers by email weekly

 Record who you sent to so you
 don’t repeat the same people
 every week
NET PROMOTER SCORE
 NPS is not a direct
 measurement that translates to
 key metrics

 Can act as a leading indicator of
 other metrics
R   EVENUE
MAKING MONEY
 Key for any business is to find
 out how much profit they make
 for every customer and scaling
 the number of customers
LIFETIME CUSTOMER VALUE
 How much money you make for
 every customer you acquire

 For some this is a single
 transaction

 For subscription businesses it
 depends on retention
CANCELLATION RATE
 Is the percentage number of
 customers who cancel in any
 given month compared to total
 (paying) customers

 [Cancellation rate] = [product
 utility] + [service quality] +
 [acceptable price]
LIFETIME CUSTOMER VALUE (LTV)
 LTV = monthly revenue x no
 months in lifetime

 No months = 1 / cancellation
 rate

 LTV = monthly revenue /
 cancellation rate
LIFETIME CUSTOMER VALUE (LTV)
 For example:

 If £20 a month and cancelation
 rate is 10% a month

 LTV = £200
LIFETIME CUSTOMER VALUE (LTV)
 Often not that simple. Short term
 cancellation rates are much
 higher than long term

 Use Cohort Analysis to
 determine
LIFETIME CUSTOMER VALUE (LTV)
 r = short-term cancellation rate
 (e.g. 0.15)
 p = long-term cancellation rate
 (e.g. 0.03)
 s = number of months in the
 “short-term” age group (e.g. 3)

 (1-r)^s × (s + 1/p) = expected
 months
T   IPS
TIPS
Concentrate on macro metrics

Everything you do should be an
attempt to change a metric

Measure if it did

Try to use as few tools as
possible
TIPS
Automate it and put it in a
dashboard

If it can’t be automated make a
simple spreadsheet and an easy
way to update it
TIPS
Don’t forget to measure
marketing

Make metrics a habit
T   HANKS

Lean startup metrics

  • 1.
    LEAN STARTUP METRICS @STUECCLES 4th Feb 2012
  • 2.
    I NEVER FELTLIKE I WAS LEARNING ANYTHING IMPORTANT
  • 3.
    “An actionable metricis one that ties specific and repeatable actions to observed results.” ASH MAURYA http://www.ashmaurya.com/
  • 4.
  • 7.
    F UNNELS
  • 8.
  • 9.
  • 10.
    A/B TESTING
  • 11.
  • 12.
    A/B TESTING USERMAP ON HOMEPAGE
  • 15.
  • 16.
    “THAT BLUE WASWORTH $80M AT LEAST” PAUL RAY - PRODUCT MANAGER BING
  • 17.
    AN INTUITIVE LEAP LOCAL MAXIMA PROPS @BOKARDO FOR LAST FEW SLIDES
  • 18.
  • 19.
    WHAT TO A/BTEST Short Answer: Everything Longer Answer: That you have enough traffic to see significant results that are not trival optimisations (HINT you are not Bing/Google)
  • 20.
    WHAT TO A/BTEST Every new feature should go into A/B testing. If has negative or no real effect, pull it. Optimise existing features/copy/ design (80% existing 20% new)
  • 21.
    F RAMEWORK
  • 22.
    ACTIVATION REFERRAL RETENTION REVENUE ACQUISITION Dave McClure http://500hats.typepad.com/
  • 23.
    DON’T TRY TOMEASURE TOO MUCH
  • 24.
    E XAMPLES
  • 25.
    SAY HELLO TOOUR 3 FICTIONAL STARTUPS
  • 26.
    G GIE COLR DO Social Network for Dog Lovers. Makes money through advertising and affiliate sales of lush dog collars.
  • 27.
    G GIE COLR DO Key business drivers are: Advertising and affiliates need large traffic/page views so growth and retention are key
  • 28.
    ACQUISITION VISIT ACQUISITION SIGN-UP CREATE ACTIVATION PROFILE POST ACTIVATION COMMENT PICTURES SHARE INVITE REFERRAL PICTURES FRIENDS REVENUE BUY COLLAR
  • 29.
    IL ES AMBA AG Project management tool for Agile startups. Makes money through monthly subscriptions. Has a free trial period.
  • 30.
    IL ES AMBA AG Key business drivers are: Conversion to paid usage and cancelation rates
  • 31.
    ACQUISITION VISIT ACQUISITION SIGN-UP ACTIVATION CREATES PROJECT UPGRADES REVENUE PAID CANCELS REVENUE ACCOUNT
  • 32.
    T MY DRILL R EN Marketplace to rent/buy drills from your neighbour (AirBnB for drills) Takes a small percentage of the transaction between two parties
  • 33.
    T MY DRILL R EN Key business drivers are: Two sided business model means acquisition of both lenders and borrowers and transactions per user
  • 34.
    ACQUISITION VISIT LENDER BORROWER ACQUISITION SIGN-UP SIGN-UP SEARCHES SEARCHES ACTIVATION BORROWERS LENDERS BORROW REVENUE DRILL
  • 35.
    ONE KEY METRIC Thereis generally only one key metric you need to concentrate on at a time. Such as referral or sign-up conversion or cancellation rate.
  • 36.
    A CQUISITION
  • 37.
    TRACKING MARKETING CHANNELS Google Adwords Banners SocialMedia Partnerships PR Blogging Anything that links to your site!
  • 38.
    TRACKING MARKETING CHANNELS Use uniqueurls (tracking parameters) on every url you create/give-out/pay for Use bit.ly Use HTTP referrers to group PR
  • 39.
    MEASURING COST OFVISITS Cost per click (site visit) Cost of campaign / traffic generated (Social media) Cost of effort / traffic generated
  • 40.
    MEASURING COST OF ACQUISITION Costof one visit / conversion rate = Cost of an acquisition Acquisition could be paid for acquisition or free sign-up Unfortunately not that simple
  • 41.
    DIFFERENT CHANNELS CANHAVE DIFFERENT CONVERSION RATES
  • 42.
    MARKETING CHANNELS Per channeltrack: Scale Cost Conversion Optimise and scale channels with best Cost of Acquisition (CoA) and potential to Scale
  • 43.
    ACQUISITION Cost of Acquisition< Customer Lifetime Profit Then scale acquisition at a similar or lower rate A/B test acquisition channels to optimise
  • 44.
    ACQUISITION INCLUDING REFERRAL If it costs £3 to acquire a customer through advertising But that customer brought in 2 other customers through referral The COA of that channel is £1
  • 45.
    A CTIVATION
  • 46.
    SURE THEY SIGNED-UPBUT DID THEY LIKE IT?
  • 47.
    ACTIVATION AT FACEBOOK (i)which data points predict whether a user will stay? (ii) if they stay, which data points predict how active they’ll be after three months?
  • 48.
    ACTIVATION AT FACEBOOK (i)having more than one session as a new user, and entering basic profile information. (ii) how often a user was reached out to by others, frequency of third party application use, and how forthcoming a user was on the site.
  • 49.
    ACTIVATION METRICS OFTENDRIVE OTHER METRICS IN REFERRAL, RETENTION, REVENUE
  • 50.
    R ETENTION
  • 51.
    RETURNING USERS Tracking unregisteredusers that return often (useful for content sites with advertising) Tracking registered users that return and take some sort of action.
  • 52.
    COHORT METRICS -CREATED PROJECT
  • 53.
  • 54.
    R EFERRAL
  • 55.
    REFFERAL MECHANISMS Social MediaSharing Invite a friend type mechanics App Reviews Word of mouth
  • 56.
    VIRAL COEFFICIENT The averagenumber of customers each customer refers
  • 57.
    VIRAL COEFFICIENT -EXAMPLE Each Doggiecollr customer invites 5 other customers by email 20 % click on that email link 50 % of those users convert into customers
  • 58.
    VIRAL COEFFICIENT -EXAMPLE Viral coefficient = Refferals x click through x conversion 5 x 0.2 x 0.5 = 0.5
  • 59.
    VIRAL COEFFICIENT A viralco-effiecient greater than 1 means that every customer gets more than one other customer on average Therefore your product will grow virally
  • 60.
    VIRAL COEFFICIENT Measure automaticallyfor all your in built referral mechanisms. Give customers unique invite URLs and track conversions
  • 61.
    VIRAL CYCLE TIME Theaverage time taken for a referral to turn into a customer With the viral coeffiecent and viral cycle time you can determine your viral growth
  • 62.
    NET PROMOTER SCORE Number of people who would recommend your product to a friend. A single question survey
  • 63.
    NET PROMOTER SCORE How likely is it you would recommend AgileSamba to a friend or colleague? 0 1 2 3 4 5 6 7 8 9 10 NPS = % Promoters - % Detractors
  • 64.
    NET PROMOTER SCORE Survey a small percentage of your customers by email weekly Record who you sent to so you don’t repeat the same people every week
  • 65.
    NET PROMOTER SCORE NPS is not a direct measurement that translates to key metrics Can act as a leading indicator of other metrics
  • 66.
    R EVENUE
  • 67.
    MAKING MONEY Keyfor any business is to find out how much profit they make for every customer and scaling the number of customers
  • 68.
    LIFETIME CUSTOMER VALUE How much money you make for every customer you acquire For some this is a single transaction For subscription businesses it depends on retention
  • 69.
    CANCELLATION RATE Isthe percentage number of customers who cancel in any given month compared to total (paying) customers [Cancellation rate] = [product utility] + [service quality] + [acceptable price]
  • 70.
    LIFETIME CUSTOMER VALUE(LTV) LTV = monthly revenue x no months in lifetime No months = 1 / cancellation rate LTV = monthly revenue / cancellation rate
  • 71.
    LIFETIME CUSTOMER VALUE(LTV) For example: If £20 a month and cancelation rate is 10% a month LTV = £200
  • 72.
    LIFETIME CUSTOMER VALUE(LTV) Often not that simple. Short term cancellation rates are much higher than long term Use Cohort Analysis to determine
  • 73.
    LIFETIME CUSTOMER VALUE(LTV) r = short-term cancellation rate (e.g. 0.15) p = long-term cancellation rate (e.g. 0.03) s = number of months in the “short-term” age group (e.g. 3) (1-r)^s × (s + 1/p) = expected months
  • 74.
    T IPS
  • 75.
    TIPS Concentrate on macrometrics Everything you do should be an attempt to change a metric Measure if it did Try to use as few tools as possible
  • 76.
    TIPS Automate it andput it in a dashboard If it can’t be automated make a simple spreadsheet and an easy way to update it
  • 77.
    TIPS Don’t forget tomeasure marketing Make metrics a habit
  • 78.
    T HANKS