1. © 2014 Fair Isaac Corporation. Confidential. This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent. Addressing Attrition: Ultra-Dynamic Multi-Dimensional Attrition Analytics with Tree Ensemble Models Dr. Gerald Fahner Senior Director Analytic Science FICO

2. © 2014 Fair Isaac Corporation. Confidential. Retaining Your Best Customers Vs. Acquiring Your Competitors’ Best Customers Easier Identification Targeted Actions Lower Cost Unknown Risk/Gain Generic Offers Higher Cost Retention Acquisition 2

3. © 2014 Fair Isaac Corporation. Confidential. Machine Learning and Daily Scoring Improve Key Requirements Easier Identification Targeted Actions Lower Cost Precision Speed Insight Retention 3

4. © 2014 Fair Isaac Corporation. Confidential. Putting Attrition Scoring On Steroids Traditional Attrition Scores Ultra-dynamic Attrition Scores Score monthly behavior rollups Score daily transaction behavior Velocity Predict and preempt attrition before it happens Detect, intervene, re-engage as attrition happens Timing Direct mail campaigns Email, SMS, mobile Channel Customer attritionCustomer + category attrition Entity Manually developed models Machine learning Algorithm 4

5. © 2014 Fair Isaac Corporation. Confidential. ► Daily score predicts attrition risk ► Based on daily activity, or lack thereof ► Prolonged inactivity signals higher attrition risk ► Engage customer when attrition score exceeds some threshold Daily Scoring Approach … Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We … Customer’s daily attrition risk score Customer uses card 5

6. © 2014 Fair Isaac Corporation. Confidential. ► Given information at time of scoring, who is more likely to attrite? ► Which measures are most informative? ► How to combine Recency and Frequency into predicting attrition risk? Transaction Dynamics Hold Key Information Attila Spence Attrite? Time of Scoring Recency Frequency Recency Days Since Last Card Use Frequency Fraction of Days Card Used 6

7. © 2014 Fair Isaac Corporation. Confidential. How Machine Learning Complements Domain Expertise Product Showcase: 4:45–5:30: “A Power Tool for Prediction Exploration”: Also at Solution Centers, 12:30 and 4:30 Domain Experts Great at intuiting key predictors Intuition doesn’t scale to many variables Poor at combining multiple predictors Difficulties in quantifying uncertainty Machine Learning Lacks intuition Excels at combining many derived features into accurate probability predictions Tell the “story” behind the numbers Diagnose/visualize models to make sense 1 2 3 4 # Optimal path 7

8. © 2014 Fair Isaac Corporation. Confidential. Key Analytic Elements of Our Approach Powerful machine learning tools ► Stochastic Gradient Boosting ► Black-box model visualization Rich set of relevant variables/features ► High-dimensional feature space of complex events ► Based on Recency and Frequency Problem-oriented performance evaluation ► Lift, portfolio profit gain ► Out-of-sample/Out-of-time 8

9. © 2014 Fair Isaac Corporation. Confidential. Jerome Friedman [1] Stochastic Gradient Boosting Training Data Prediction Function Predictors Outcomes Tree 1 Tree 2 Tree M Weighted average Score aggregates predictions from many shallow trees Predictors ? Scored New case Score 9

10. © 2014 Fair Isaac Corporation. Confidential. Simulated Data Demonstration Problem Noisy training samples Predictive relationship from which data were generated (“ground truth”) Outcomes Predictors 10

11. © 2014 Fair Isaac Corporation. Confidential. Stochastic Gradient Boosting: 1 Shallow Tree Tree 1 Weighted Average 11

12. © 2014 Fair Isaac Corporation. Confidential. Tree 1 Tree 5 Weighted Average Stochastic Gradient Boosting: 5 Shallow Trees 12

13. © 2014 Fair Isaac Corporation. Confidential. Tree 1 Tree 200 Weighted Average Stochastic Gradient Boosting: 200 Shallow Trees 13

14. © 2014 Fair Isaac Corporation. Confidential. Addressing Customer Attrition ► Machine Learning for Maximal Profit 14

15. © 2014 Fair Isaac Corporation. Confidential. Project Design ► ~5 million accounts generated: ~1 billion transactions over 3 years ► Transaction information: Date, Merchant Code, Amount, Authorized Flag Credit Card Case Study Observation Period Time of Scoring Performance Period Attrition DefinitionExclusions 0/1 indicator of card activity during performance period Less than 3 transactions during observation period or Card not used within 3 months prior to Time of Scoring Observation Period Development Out of Time Validation Performance Period 2 years6 months 15

16. © 2014 Fair Isaac Corporation. Confidential. Lift and Precision Statistical Measures of Model Performance Low Scoring Accounts High Scoring Accounts Attriters Non-attriters Target Top % with retention offer Lift( %): 16

17. © 2014 Fair Isaac Corporation. Confidential. Relating Attrition Model Performance To Profit Actual Behaviors of Targeted Customers Would-be attriters we persuade to stay Unpersuadable attriters Non-attriters we target erroneously Fraction of Targeted Customers with this Behavior Precision * Persuasion Rate Precision * (1 – Persuasion Rate) 1 – Precision Profit Contribution per Customer (CLV Gain – Contact Cost – Incentive Cost) (No CLV Gain – Contact Cost) (No CLV Gain – Contact Cost – Incentive Cost) 17

18. © 2014 Fair Isaac Corporation. Confidential. Lift from model A Lift from model B Targeting Fraction5% Base Attrition Rate8% Portfolio Size5 million Customer Lifetime Value$1,000 Incentive Cost$100 Persuasion Rate20% Scott Neslin et al. [3] Relating Model Improvement to Portfolio Profit Gain Portfolio specific parameters and assumptions We will benchmark alternative models 18

19. © 2014 Fair Isaac Corporation. Confidential. Benchmarking Models of Increasing Complexity Dimensionality of Feature Space Flexibility of Score Formula Model 1: Additive model in R and F of card use Model 2: Interaction model in R and F of card use Model 3: Interaction model in R and F of complex events How much can we improve Lift and Profit by making models more complex? Are more complex models robust over time? Examples: ► Recent restaurant visit and frequent hotels ► More than $1,000 spent on airline last week ► Recent car deal and frequently at the pump R: Recency, F: Frequency 19

20. © 2014 Fair Isaac Corporation. Confidential. Do Recency and Frequency Interact in their Effect on Attrition? ► Predictors: Recency and Frequency of card use ► Model 1: Additive, nonlinear in R and F ► Model 2: Capture interaction between R and F Experiment A Out-of-sample / Out-of-time validation Capturing (R x F) interaction is profitable! ► Interaction effect is in agreement with research by Fader and Hardie [4] in the context of stochastic models for CLV 20

21. © 2014 Fair Isaac Corporation. Confidential. Spence Attila Probability to use card during the next 6 months = 1 – Pr(Attrition) Frequency Recency Who is more likely to attrite? Spence: R = 20, F = 0.05 Attila: R = 20, F = 0.55 Time of Scoring Two-dimensional Partial Dependency Function [2] Interaction Visualization Tells Story 21

22. © 2014 Fair Isaac Corporation. Confidential. Do Complex Event Features Boost Model Performance? ► Define R and F features for complex events ► Model 3: Candidate predictors include: Card use events + Hundreds of merchant category events + Monetary events (defined by hitting spending bands) + No-authorization events Experiment B Predictors based on complex events are very profitable! Out-of-sample / Out-of-time validation 22

23. © 2014 Fair Isaac Corporation. Confidential. Experiment C Simpler model doesn’t benefit from more than 60,000 training samples Complex model keeps improving at least until 500,000 training samples 23 Effects of Training Sample Size To Train Complex Machine Learning Models, Use as Much Data as You Can!

24. © 2014 Fair Isaac Corporation. Confidential. Addressing Category Attrition 24

25. © 2014 Fair Isaac Corporation. Confidential. Overall Customer Status Food Travel Gas Station ► Category attrition may signal early belt-tightening or competitive influences— before total attrition occurs ► Early detection informs relevant actions/interventions: ► Customer dialogue ► Category incentives ► Product switch ► Terms adjustment Marketing Benefit Trigger customer dialogue. Perhaps offer incentives at service stations. 25

26. © 2014 Fair Isaac Corporation. Confidential. Category A Attriters Category B Attriters Attriting completely as customers ► Cards: stop buying from a merchant category while continuing card use ► Retailers: stop buying from a department while continuing other store purchases Defining Category Attrition No Yes Category A Other categories Category A attriter? Time of Scoring 123123 26

27. © 2014 Fair Isaac Corporation. Confidential. Gas Station Attrition Example 27

28. © 2014 Fair Isaac Corporation. Confidential. What Have We Learned? 1.Daily scoring quickly detects emergent attrition ► Retain most valued customers with rapid contact/offer ► Category attrition models inform offers 2.Machine learning enhances scale and insight ► Automate development of multiple models ► Visualize models to gain understanding 3.With “Big Data”, complex models beat simpler ► Portfolio profit gains are substantial ► Attrition model performance remains robust over time Let us know if you’re interested in a Proof of Concept! 28

29. © 2014 Fair Isaac Corporation. Confidential. References [1] Greedy Function Approximation: A Gradient Boosting Machine, by Jerome Friedman, The Annals of Statistics, 29(5), 2001, 1189-1232. [2] Predictive learning via rule ensembles, by Jerome Friedman et al., The Annals of Applied Statistics, 2(3), 2008, 916-954. [3] Defection Detection: Measuring and Understanding the Predictive Accuracy of Customer Churn Models, by Scott Neslin et al., Journal of Marketing Research, 43(2), 2006, 204-211. [4] RFM and CLV: Using Iso-Value Curves for Customer Base Analysis, by Peter Fader, Bruce Hardie, and Ka Lok Lee, Journal of Marketing Research, 42(4), 2005, 415-430. 29

30. © 2014 Fair Isaac Corporation. Confidential. This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent. Dr. Gerald Fahner geraldfahner@fico.com ++1 512 698 0609 Thank You!

31. © 2014 Fair Isaac Corporation. Confidential. Learn More at FICO World Related Sessions ► Research Showcase: A Power Tool for Prediction Exploration ► Applying Sequential Decisions for Customer Management Products in Solution Center ► FICO ® Model Builder ► FICO ® Analytic Modeler Experts at FICO World ► Michelle Davis ► Shafi Rahman White Papers Online ► Big Data: Overhyped or Underexploited? ► Does AI + Big Data = Business Gain? Blogs ► http://www.fico.com/en/blogs/category/analytics-optimization/ 31

32. © 2014 Fair Isaac Corporation. Confidential. Please rate this session online! 32 Dr. Gerald Fahner geraldfahner@fico.com ++1 512 698 0609