Presentation is loading. Please wait.

Presentation is loading. Please wait.

Uplift Analysis with the Quadstone System Monday, 7 th January 2005 7.30am PST / 10.30am EST / 3.30pm GMT / 16.30 CET Any trouble getting into the conference.

Similar presentations


Presentation on theme: "Uplift Analysis with the Quadstone System Monday, 7 th January 2005 7.30am PST / 10.30am EST / 3.30pm GMT / 16.30 CET Any trouble getting into the conference."— Presentation transcript:

1 Uplift Analysis with the Quadstone System Monday, 7 th January am PST / 10.30am EST / 3.30pm GMT / CET Any trouble getting into the conference call: contact

2 © 2005 Quadstone How to ask questions Return to Meeting Manager Use Chat

3 Uplift Analysis Nicholas J. Radcliffe Chief Technology Officer Agenda MOTIVATION Demo 1: Up-sell example (binary outcome) When is Uplift Modelling important? Demo 2: Deep-sell example (continuous outcome) TECHNICAL CONSIDERATIONS Practical considerations and guidelines Small population issues and extensions The quality measure: Qini TRIAL How to get a trial copy & datasets

4 “We have to find a way of making the important measurable, instead of making the measurable important” — Robert McNamara “I know half the money I spend on advertising is wasted, but I can never find out which half” — John Wanamaker

5 Demo 1: Up-sell example Binary outcome SCENARIO Mobile phone company 3G MMS Video phone promotion Some mass advertising - non-targeted customers can purchase Direct calling campaign to drive further sales Random 250k chosen from 10m base for trial c. 75k actually targeted; c. 175k as control

6 When is Uplift Modelling Important?

7 © 2005 Quadstone Two Separate Benefits Not targeting people who are little affected Reponse: Don’t spend money targeting or offer discounts to people who will buy anyway Attrition: Don’t spend money trying to save people who will go anyway Targeting people with low probability but high responsiveness Response: Do spend money on people who aren’t very likely to buy if you do, but are very responsive to offers/contact Attrition: Do spend money trying to save people who aren’t at huge risk of attrition, but can be made much more likely to stay

8 © 2005 Quadstone When would a conventional model be misled? High pre-existing purchase rate Pre-existing knowledge of product coupon Many influences

9 © 2005 Quadstone When are negative effects likely? Sometimes, our actions actually drive customers away, especially when: dissatisfied / angry customers risqué / offensive communications attrition risk intrusive contact mechanisms forgotten standing charges

10 Demo 2: Deep-sell example Continuous outcome SCENARIO Grocery retailer Direct mail campaign to increase spend Weekly Spend measured in 12-week “pre”-period (AWS) Also in 6-week post period (AWSPostCampaign) Objective is difference: (PostMinusPreAWS) Random 250k chosen from 10m base for trial c. 75k actually targeted; c. 175k as control

11 © 2005 Quadstone Control group Must be representative: technique will give misleading results otherwise In practice, this means randomly select controls from target group There must be enough of them Control Group Structure All possible recipients Targets

12 © 2005 Quadstone Population Size Population size “Rule of 500”: to detect a x% difference (uplift), x% of the smaller population (usually controls) should ideally be at least 500 people So if looking for 1% difference, control group needs to have at least 50,000 people So consider longitudinal controls – contact half now, half later

13 © 2005 Quadstone Pruning and Validation Pruning Autopruning is implemented, based on qini variance In practice, fairly unaggressive, so recommend manual pruning Validation Ordinary test-training fine if there is enough data If not, consider k-way crossvalidation

14 © 2005 Quadstone Small Population Extensions Bagging (oversampling method) and k-way cross- validation Analysis candidate selection useful if there are “too many” analysis candidates Stronger pruning (variance-based) Stratification Not part of product, but potentially available as an extension if purchased

15 © 2005 Quadstone Return on Investment Key thing is that Campaign ROI depends on the net effect (i.e. uplift) of action, not apparent response ( reduction in churn) × (value of saved people) – (cost of action) (increase in purchase rate) × (value of purchase) – cost (increase in spend) – (cost of action) etc. Quadstone System has many suitable ROI FDL functions ( f x ) built in (even without uplift license)

16 So how do you measure what’s important?

17 © 2005 Quadstone Quality Measure Considerations Can only estimate uplift by segment This is what we are used to with control groups One person does not have a (knowable, measurable) uplift Generalizing measures like classification error/accuracy or R 2 doesn’t look promising Rank statistics do seem more promising because they can sometimes be computed on a segmented basis

18 © 2005 Quadstone Can we use/modify the Gini for Uplift? % of customers targeted 0% x% 100% uplift Overall uplift: x% Possibility of negative effects x% Positively affected by action Negatively affected by action

19 © 2005 Quadstone Summary: When to use Uplift Uplift modelling is just a better way of modelling the true effect of an action Particularly relevant to: Retention (where it’s the number/value of people you save that’s important Up-sell, cross-sell, deep-sell (where it’s the incremental revenue or profit that’s important) Risk management actions (where it’s the reduction in risk achieved that’s important)

20 © 2005 Quadstone Where to find out more For more in-depth training: our Uplift Analysis course. Contact

21 © 2005 Quadstone Questions and answers

22 © 2005 Quadstone After the webinar These slides, the data and a four-week trial license are available via Any problems or questions, contact

23 © 2005 Quadstone Uplift: Quick Reference Building uplift models Ensure random control group exists Set partition field with P interpretation (1 for treated, 0 for control) Set objective (binary, continuous/discrete) Hit go Pruning Switch to test dataset Hit Autoprune Creating results field Use “Uplift as difference” Using difference viewers Crossdistribution Viewer places partition field on “  axis” automatically For view shown, drag count to depth, duplicate mean (ObjectiveField) and drag on to height Can configure which population is viewed by right-clicking on functions Using ROI Functions These are available under f x in Table Viewer when deriving new field.

24 © 2005 Quadstone Upcoming webinars If there’s a webinar topic you’d like to see, please let us know via Thursday, 17 th February 2005 Data Preparation in the Quadstone System Version am PST / 10.30am EST / 3.30pm GMT / CET

25 © 2005 Quadstone Your feedback Suggestions or feedback? Please enter them in the feedback form or send them to

26 © 2005 Quadstone Modifying the Gini for Uplift? % of customers targeted 0% x% 100% uplift x% Positively affected by action Negatively affected by action Unaffected by action

27 © 2005 Quadstone The Shape of the Qini Curve % of customers targeted 0% x% 100% uplift x% +ve –ve neutral Can’t do better than 100% sales: if 90% of control group purchases then maximum uplift = 10% 10% Can’t do worse than 0% sales: if 5% of control group purchases then maximum negative uplift = 5% ? Why is this flat?


Download ppt "Uplift Analysis with the Quadstone System Monday, 7 th January 2005 7.30am PST / 10.30am EST / 3.30pm GMT / 16.30 CET Any trouble getting into the conference."

Similar presentations


Ads by Google