Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

12
More Bandits @lukasvermeer The World Needs

description

#ddtt #2 Keynote by Lukas Vermeer - the world needs more bandits.

Transcript of Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

Page 1: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

More  Bandits  @lukasvermeer  

The  World  Needs  

Page 2: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

we  are  all  in  the  business  of  running  experiments  

 we  run  experiments,  because  we  need  to  know  that  what  we  do  is  

good  for  our  business    

our  most  important  KPI  is  how  many  successful  experiments  we  run  

Page 3: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

MulC-­‐Armed  Bandits  geFng  the  most  out  of  your  experiments  

Page 4: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

A/B  Test  (Epsilon  First)  the  gold  standard  for  (business)  experiments  

Page 5: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

ExploraCon/ExploitaCon  experiments  cost  money  

Page 6: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

Regret  minimizing  the  cost    of  your  experiments  

Page 7: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

Greedy  Bandit  controlled  exploraCon  

Page 8: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

Thompson  Bandit  exploit  as  much  as  you  dare  

Page 9: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

Contextual  Bandits  far  beyond  basic  bandit  experiments  

Page 10: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

Start  Now  maximize  experimentaCon  –  minimize  costs  

Page 11: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

we  are  all  in  the  business  of  running  experiments  

 we  run  experiments,  because  we  need  to  know  that  what  we  do  is  

good  for  our  business    

our  most  important  KPI  is  how  many  successful  experiments  we  do  

Page 12: Digital Data Tips Tuesday #2 - A/B-testing: Lukas Vermeer - Booking.com

More  Bandits  @lukasvermeer  

The  World  Needs