Presentation is loading. Please wait.

Presentation is loading. Please wait.

15 A/B testing mistakes I see all the damn time (and what to do about it) by Peep Laja, ConversionXL Call to Action Conference 2014.

Similar presentations


Presentation on theme: "15 A/B testing mistakes I see all the damn time (and what to do about it) by Peep Laja, ConversionXL Call to Action Conference 2014."— Presentation transcript:

1

2 15 A/B testing mistakes I see all the damn time (and what to do about it) by Peep Laja, ConversionXL Call to Action Conference 2014

3 “You should test that!”

4

5 “Dude wtf?”

6 @peeplaja Typical scenario: A company runs 100 tests / year. After 12 months their conversion rate is still the same.

7 @peeplaja Testing mistake #1: Precious time wasted on stupid tests

8 @peeplaja Testing mistake #2: You don’t have a data- driven, learning- oriented hypothesis

9 @peeplaja Testing mistake #3: You think you know what will work

10 @peeplaja Testing mistake #4: You run tests when you have no traffic

11 @peeplaja Testing mistake #5: Your absolute conversions are way too low

12

13 @peeplaja You need *at least* conversions PER variation. Way more if you want to segment the data

14 @peeplaja Our "thumb rule" is: conversions per variation and 3-4 week test duration. That is enough traffic so we can even talk about valid data if we drill down into segments. Testing "sin" nr.1: search in segments for uplifts although you have no statistical validity, e.g. 85 versus 97 conversions. That’s bullshit.

15 @peeplaja Testing mistake #6: Your tests don’t run long enough

16

17 @peeplaja Testing mistake #7: You don’t test full weeks at a time

18

19 @peeplaja Testing mistake #8: Test data is not sent to third-party analytics

20

21 @peeplaja Pro tip: If you have over 500,000 pageviews / mo, you don’t (necessarily) need to spend $100,000 on GA Premium. Get unsampled data with Analytics Canvas

22 @peeplaja Testing mistake #9: You give up after your first test for a hypothesis fails.

23

24 Full case study: conversion/http://conversionxl.com/case-study-how-we-improved-landing-page- conversion/ 79.3% uplift in conversions

25 @peeplaja Testing mistake #11: You’re not aware of validity threats

26 @peeplaja History effect Instrumentation effect Selection effect

27 @peeplaja Testing mistake #12: You’re ignoring small gains

28 @peeplaja 5% monthly increase in your conversion rate will result in 80% uplift for the year. That’s how math works.

29 @peeplaja Testing mistake #13: You don’t know html/css/jquery - and thus only test the simple stuff

30 @peeplaja Testing mistake #14: You’re not running tests at all times

31 @peeplaja Testing mistake #15: You’re late for your lunch break!

32 Keep reading ConversionXL! Thank you

33


Download ppt "15 A/B testing mistakes I see all the damn time (and what to do about it) by Peep Laja, ConversionXL Call to Action Conference 2014."

Similar presentations


Ads by Google