Transforming One Small Step At A Time: Optimisation & Testing
-
Upload
building-blocks -
Category
Internet
-
view
351 -
download
5
description
Transcript of Transforming One Small Step At A Time: Optimisation & Testing
![Page 1: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/1.jpg)
Building Blocks
Transforming One
Small Step At A Time
Building Blocks www.building-blocks.com
![Page 2: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/2.jpg)
Building Blocks
A talk about testing?!?
Jonathan WhitesidePrincipal Consultant
Kyle CassidyUX & Optimisation Consultant
![Page 3: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/3.jpg)
Building Blocks
It’s much cheaper to servethan acquire
Companies that embrace optimisation to make decisions increase conversion by 100%
Source: Adobe
![Page 4: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/4.jpg)
Building Blocks
A talk about testing?!?“Insanity: doing the
same thing over and over again and expecting
different results”- Albert Einstein
![Page 5: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/5.jpg)
Building Blocks
Supports improvement
Organisational Digital Maturity
Confidence
Business case justification
De-risk projects
Maximise campaign effectiveness
Incremental conversion increases
Support business-wide commercial decisions
![Page 6: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/6.jpg)
Building Blocks
What we’re going to talk about
1. What?
2. Why?
3. Examples
4. How?
![Page 7: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/7.jpg)
Building Blocks
So, what is site optimisation?
![Page 8: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/8.jpg)
Building Blocks
Optimisation includes:
• A/B & MVT testing• Analytics• CRO• User journeys• Usability• Information architecture• Traffic sources• Segmentation• User experience design• Personalisation
![Page 9: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/9.jpg)
Building Blocks
Optimisation is for everyone
Marketing Commercial
Web Operations/
IT
Optimisation
![Page 10: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/10.jpg)
Building Blocks
It’s much cheaper to servethan acquire
It’s much cheaper to serve, than acquire…
![Page 11: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/11.jpg)
Building Blocks
Clicks
PPC Cost per click
Total PPC spend
Conversion rate
Customers won
Cost per acquisition
Average profit per customer
Total profit generated
ROI
Simple numbers…
Non Optimised website Optimised website
100
£0.06
£66
1%
1
£66
£40
£40
-40%
100
£0.06
£66
5%
5
£13
£40
£200
200%
![Page 12: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/12.jpg)
Building Blocks
Experiments
![Page 13: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/13.jpg)
Building Blocks
A/B MVT
Examples
![Page 14: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/14.jpg)
Building Blocks
• 1 element on the page
• Multiple variants
• Tested against a ‘control’
• Evenly distributed traffic
• Measured over time
What is A/B Testing?
![Page 15: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/15.jpg)
Building Blocks
Examples of things to test
• Buttons• Form placement• Images• Copy changes• Icons• Video• Navigation• Banners
![Page 16: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/16.jpg)
Building Blocks
![Page 17: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/17.jpg)
Building Blocks
A/B testing: upgrades page
Conversion
20.47%
![Page 18: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/18.jpg)
Building Blocks
Testing variantsL
eve
l 2
-g
rey
ba
ckg
rou
nd
a
nd
up
gra
de
pro
mis
e
Lev
el
3 –
wh
ite
ba
ckg
rou
nd
a
nd
up
gra
de
pro
mis
e
Lev
el
4 –
gre
y b
ackgro
und
Lev
el
5 –
white b
ackgro
und
Co
ntr
ol
![Page 19: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/19.jpg)
Building Blocks
The results
ExperimentSubmitted form Conversion Rate
Lift Chance to beat control
Control20.47%
+/- 0.84%
Level 234.31%
+/- 0.98%+67.59%
100% chance to beat control
Level 334.94%
+/- 0.98%+70.64%
100% chance to beat control
Level 434.89%
+/- 1.00%+70.40%
100% chance to beat control
Level 534.21%
+/- 0.98%+67.07%
100% chance to beat control
Conversion
34.94%
![Page 20: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/20.jpg)
Building Blocks
20.47% 34.94% 70.69%
Additional RecordsCaptured each month
Controlconversion rate
Winnerconversion rate
Conversion rate uplift
17,800
The impact of the conversion uplift
(£20 per lead) Per month in upgrade leads£356,000
![Page 21: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/21.jpg)
Building Blocks
• Multiple elements
• Multiple variants
• Tested at the same time
• Tested against a ‘control’
• Traffic is distributed evenly
• Measured over time
What is MVT Testing? Opt. A
Opt. B
Opt. C
Opt. D
![Page 22: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/22.jpg)
Building Blocks
Co
ntr
ol
Lev
el3
Lev
el 2
Lev
el 4
Testing variants
![Page 23: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/23.jpg)
Building Blocks
ExperimentSubmitted form Conversion Rate
Lift Chance to beat control
Desktops
Control27.55%
+/- 0.69%-
Level 229.931%
+/- 0.71%+8.63%
High significance
Level 329.99%
+/- 0.71%+8.85%
High significance
Level 430.93%
+/- 0.72%+12.25%
High significance
iPads
Control 22.91% +/- 1.43%
-
Level 2 24.80%+/- 1.46%
+8.58%High significance
Level 3 25.63%+/- 1.48%
+12.36%High significance
Level 4 26.73%+/- 1.52%
+17.36%High significance
Desktops
30.93%iPads
26.73%
Results
![Page 24: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/24.jpg)
Building Blocks
Controlconversion rate
Winnerconversion rate
Conversion rate uplift
The impact of the conversion uplift
27.55%22.91%
30.93%26.73%
+12.25%+17.36%
4,920
Additional RecordsCaptured each month
(£20 per lead) Per month in upgrade leads£98,000
![Page 25: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/25.jpg)
Building Blocks
A/B vs. MVT
A/B MVTBest for Large scale re-designs Incremental tests
Complexity Low High
Variants Known (usually 2-4) Multiple (+25)
Served 1 at a time Random order
Output Result of the test Result of the elements
Time to execute Low High
Time to run (traffic/variant dependent)
Shorter Longer
![Page 26: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/26.jpg)
Building Blocks
Not always successful…
![Page 27: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/27.jpg)
Building Blocks
That all sounds great, where do I start?
![Page 28: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/28.jpg)
Building Blocks
A talk about testing?!?
Common excuses to ignore optimisation
52%
36%
28%
26%
25%
23%
22%
21%
10%
9%
3%
0% 15% 30% 45% 60%
Lack of resources
Lack of budget
Conflict of interest between departments
Lack of strategy
Poor technology
Poor intergration between systems
Siloed Organisation
Company culture
Lack of ownership
Reliance on third parties
Other
Source: eConsultancy
![Page 29: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/29.jpg)
Building Blocks
It’s much cheaper to servethan acquire90:10
![Page 30: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/30.jpg)
Building Blocks
90% 10% PlanningProcessesPeoplePresentation
Technology
Time investment
![Page 31: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/31.jpg)
Building Blocks
http://chiefmartec.com/
There are lots of tools…
![Page 32: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/32.jpg)
Building Blocks
Objectives
Campaignsupport
Mitigating risk
Developing a testing strategy
Justifying business cases
Improving site usability
![Page 33: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/33.jpg)
Building Blocks
Test Ideation & Prioritisation
Writing Test Plan and
performing QA
Test Live -Monitoring
Results
3 parallel tracks
![Page 34: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/34.jpg)
Building Blocks
What skills do I need?
10%
40% 40%
Presentation skills Technical
10%
Analytically minded
UX/Creative/Copywriting
![Page 35: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/35.jpg)
Building Blocks
How to secure buy-in
• Make sure data is clean, consistent and trusted.
• Use data to identify where to prioritise your optimisation roadmap.
• Define your main optimisation conversion event and how you will use it to calculate ROI.
• Ensure that it aligns to your organisation’s KPIs.
• Invite stakeholder sign-off to communicate that testing will help them reach their goals.
![Page 36: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/36.jpg)
Building Blocks
It’s much cheaper to servethan acquire
Companies with significant increases in sales are completing 6.45 A/B & MVT tests per month
Source: eConsultancy
![Page 37: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/37.jpg)
Building Blocks
• Valuable at every stage
• Test small, test often
• Build business/change cases based on known value
• Lean, agile with little waste
• Try new things, experiment and measure
• Use results as leverage to be bold
Transforming with optimisation
![Page 38: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/38.jpg)
Building Blocks
![Page 39: Transforming One Small Step At A Time: Optimisation & Testing](https://reader033.fdocument.pub/reader033/viewer/2022060201/559aa3df1a28abc4508b4663/html5/thumbnails/39.jpg)
Building Blocks
Thank you!
building-blocks.com