Rajiv Srivatsa (Urban Ladder) / Rajan (Intuit)#PNCampDec 04 2013

Experiments never killed anybody…

1. Vision, Target, ProblemThese are more important to crystallize than the solution or the idea itself…

1. Vision, Target, Problem• The Urban Ladder Example• Vision: Beautiful homes for millions of Indians

Tip: The vision is always user-facing!

• Target: ‘Upper Middle-Class’ ‘Home-proud’ ‘Digitally-Savvy’ ‘Urban Indian’ ‘Earning > 1 lak family income’ ‘Well connected’ ’Travelled internationally’ Tip: The sharper it is defined, the better it is for engaging with a clear set of customers for each experiment

• Problem: Getting good quality, well-designed, trust-worthy furniture at reasonable pricesTip: Balance going too wide or too narrow

Bold. Specific. Solution-free. Customer-focused. Memorable.

1. Exercise• Vision, Target Audience & Problem Statement• Pair with 1 other person• 2-min to write the following on a post-it

• The vision• Target audience• Problem statement

• Your partner to introduce you from the post-it you have prepared!

2. Why Experiments

2. Experiments – the pre-work

Pick a broad decision / feature / initiative that you last did.

Please answer the following questions (write in a post – it )

• What was the decision? • Was it a successful one? (Y/N)• How long did it take to realize the learning about the decision? • What were some implicit assumptions made in the decision?

observe idea

analyze&

presentdecision design build

launch&

marketingusers

2. Why experiments

observe idea

analyze&

presentdecision design build

launch&

marketingusers

RsRs Rs Rs

Rs

RsRs

RsRs

Rs

Rs RsRs

Rs

Rs

RsRs Rs

2. Why experiments

observe idea

analyze&

present

DesignBuildTest

decision design build

launch&

marketingusers

2. Why experiments

To: Decisions by“ Experimentation & Learning”

From: Decisions by“Opinions and Powerpoint”

observe idea

analyze&

present

DesignBuildTest

decisionusers

2. Why experiments

In the age of rapid changes...

3. Designing the experimentsThere are no failed experiments, only negated hypotheses…

hypothesis[ Solutions/features that could support our leap of faith assumption . It can also be considered as a restatement of leap of faith in a numerical way]

experiment [ Conditions created to measure behavioral response to learn about the hypothesis ]

reflect [ Pivot the idea or persevere ]

need-gap

analysis

product-market concept test

leap of faithassumption

[ Testing if the need-gap is a big enough need; understanding priorities ]

[ A feasible brand and product concept that delivers on the vision: how much are users willing to pay for this ]

[ Key behavioral assumption about our idea that’s keeping us up at night - if this assumption is false, nothing else matters ]

repeat tests

• The Urban Ladder Examples• Confirm need-gap priority

Discovery: Quality and design were more important than priceTools: SurveyMonkey, Customer InterviewsDataset: 100 Responses; Over 40 in-depth interviews

• Is the brand promise exciting?Question: People should relate to the brand-name, logo and promiseTools: LaunchRock, FB, Customer Interviews Dataset: 350 sign-ups in 2.5 months; 25 FB shares; Over 100 conversations

• Validate product-market concept fitValidation: People should like the product at a feasible price-pointTools: Polls, PPT, Email, A/B TestsDataset: Over 40 responses; 25 in-depth interviews

3. Designing the experimentsneed-gap

analysis

product-market concept test

leap of faithassumption

• The Urban Ladder Example • Hypothesis: People get a sense of the furniture quality online and buy it • Dataset: Friends and family • MVP definition: Beta version of the site using outsourced technology,

design integrated with Google Analytics; Calls to check source and feedback; Basic range in 2 categories; Merchandised products with story

• Target Metrics: If 2% of the visitors buy items for at least Rs. 5k, then we can say that this experiment can be taken to the next level

• Data Gathering: Spread over a 2 week period to get to the first 25 transactions, largely from family and friends

• Results Analysis: Strong interest in buying product; ability to get a sense of colors and size from images; strong interest in other cities, categories

Experiment Repeats: Neutralize the audience, check with friends of friends who probably don’t have direct affinity to people or brand; Test with service / without service; test repeat rates

3. Designing the experimentsneed-gap

analysis

product-market fit

leap of faithassumption

3. Designing the experiments• Write down on a post – it ( 5 min) • Problem • Leap of Faith Assumption• Numerical Hypothesis• Experiment • Metric

• Pair up & review

4. Also important are… • A strong business plan with valid numbers and realistic targets• The right questions to ask during the probe phase• Messaging, Communication, Design, Brand• Focus on doing few things well• Clear milestones• A smile

Q&AWhat would you do differently in your work?

CreditsThanks to www.dilbert.com and Scott Adams for the awesomeness that is Dilbert!

MVP Fidelity