© 2014 Fair Isaac Corporation. Confidential. This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent. New Reasoning on Applying Sequential Decisions for Customer Management Dr. Gerald Fahner Senior Director Analytic Science FICO

© 2014 Fair Isaac Corporation. Confidential. You Can Increase Customer Value by Optimizing Sequential Decisions 2 What are the Analytic Foundations for Moving Beyond Single-Shot Decisions?

© 2014 Fair Isaac Corporation. Confidential. Single-Shot Perspective on Decision Optimization Rationale: Optimize expected outcomes through t = 2 Customer state Business decision A 1 (t = 1) Potential actions Y(t = 2) Potential outcomes (response, revenue, loss, profit) X(t = 1) A 2 (t = 1) A 3 (t = 1) 3

© 2014 Fair Isaac Corporation. Confidential. Customer Lifetime Value (CLV) Kotler and Armstrong (1996)—A profitable customer is a person whose “revenues over time exceed, by an acceptable amount, the company costs of attracting, selling and servicing that customer.” “The present value of the future cash flows attributed to the customer during his/her entire relationship with the company.” Wikipedia: CLV is the excess, defined as 4

© 2014 Fair Isaac Corporation. Confidential. Sequential View of Customer Relationship Decision sequence has cumulative effect on all future outcomes and on CLV New rationale: Optimize decision sequence to maximize CLV X(t=1) Y(t=2) X(t=2) Y(t=3) X(t=3) Y(t=4) X(t=4) A(t=1)A(t=2)A(t=3) 5

© 2014 Fair Isaac Corporation. Confidential. Sequential Problems Are Ubiquitous, Impact Bottom Line 6 ► Customer acquisition ► Targeting and offer design ► Customer/account management ► Changing card limits and pricing ► Influencing migration to profitable states ► Managing inactive customers ► Orchestrating customer dialogue ► Collections ► Optimizing treatment sequences/scenarios

© 2014 Fair Isaac Corporation. Confidential. When to End Marketing to Inactive Customers? 7

© 2014 Fair Isaac Corporation. Confidential. Inactive Customer Problem 8 Periodic marketing … makes a few more purchases …may have lost interest End marketing, or try a different offer New customer makes first purchase $$$$$$ $ Q6Q5Q4Q3Q2Q1

© 2014 Fair Isaac Corporation. Confidential. Parameters to Consider When to End Marketing 1.From business economics: ► Profit contributions from purchases, costs for marketing, operational costs 2.From customer behavior: ► Probabilities of customers making future purchases Predictors of Purchase ProbabilitiesPossible Measures Periods since last purchaseRecency (R) Depth of relationshipCumulative Frequency of purchases (CF) Other customer characteristicsTime as customer, monetary, demographics,... ► How to compute optimal marketing policy based on these parameters? ► Develop and solve a Markov Decision Process (MDP) model 9

© 2014 Fair Isaac Corporation. Confidential. ► Guidelines: ► Current state should suffice to predict next state (Markov property) ► States should inform actions (Marketing / No Marketing ) and associated rewards ► We chose State = (Recency, Cumulative Frequency) = (R, CF) ► New customer: State = (1, 1) ► Customer who made 3 purchases, then did not buy for 4 periods: State = (4, 3) ► We add a special terminal State = (END) for customers no longer marketed to ► Who goes there, stays there 1. Define a “State Space” Representing Dynamic Customer Relationship 10 $

© 2014 Fair Isaac Corporation. Confidential. ► Customers experience state transitions between discrete time periods ► Empirical purchase probabilities that customers will buy in next period: P buy (R,CF) ► Marketed customer makes purchase: (R, CF)  (1, CF+1), with probability P buy ► Marketed customer makes no purchase: (R, CF)  (R+1, CF), with probability 1-P buy ► Customer is no longer marketed to: (R, CF)  (END), (END)  (END) with probability 1 (R+1, CF) 1-P buy (R, CF) (1,CF+1) P buy $ (END) 1 1 (R, CF) Structure of transition graph 2. Estimate State Transition Probabilities 11

© 2014 Fair Isaac Corporation. Confidential. ► Business collects (or loses) reward points during each time period ► Profit Contribution PC, Marketing (operational) Expense ME are reward parameters ► Marketed customer makes purchase: PC – ME, with probability P buy ► Marketed customer makes no purchase: – ME, with probability 1-P buy ► Customer is no longer marketed to: 0, with probability 1 (R+1, CF) 1-P buy, -ME (R, CF) (1,CF+1) P buy, PC-ME $ (END) 1,01,0 1,01,0 (R, CF) Structure of transition graph 3. Specify Reward Structure 12

© 2014 Fair Isaac Corporation. Confidential. Estimating and Optimizing CLV 13

© 2014 Fair Isaac Corporation. Confidential. ► CLV is a concept based on an indefinite time horizon: ► “The present value of the future cash flows attributed to the customer during his/her entire relationship with the company.” How to Estimate CLV From a Finite Data Window? ► Markov model allows CLV estimation from finite data window ► State transition probabilities and reward structure can be learned from a finite data window and/or augmented by judgment ► Model entails assumptions ► Transition probabilities, rewards depend only on state variables, are independent of time George E. P. Box (1919–2013) “Essentially, all models are wrong, but some are useful” 14

© 2014 Fair Isaac Corporation. Confidential. Parameters and Assumptions Hypothetical Portfolio Empirical purchase probabilities for 50 nonterminal states 15 ► Discretize R and CF into 51 states: ► R = {1, …, 10} CF = {1, …, 5+}, (END) ► Estimate P buy for each state ► New customers’ data over 3 years ► Discretize activities into quarterly intervals ► Use Recency, Cumulative Frequency to define states ► Historic marketing policy: ► All customers marketed until Recency = 10 ► P buy = 0 after marketing ends ► Few customers make more than 5 purchases ► Reward parameters: PC = $100, ME = $10 ► Discount Rate: 12% p.a.

© 2014 Fair Isaac Corporation. Confidential. CLV Estimation Via Random Walks 16 (Similarly, can estimate CLV for all other states, by starting random walks from each state) Simulate all future marketing, purchases, rewards, until END state is reached Perform 10,000 random walks. CLV(1,1) is the average discounted cumulative reward. Example: CLV for New Customers Subject to Historic Policy. PC = $100, MC = -10 is the associated discounted cumulative reward ( : Discount Rate) (1, 1) 100 Buy -10 $ (2, 1) NoBuy -10 $ (3, 1) NoBuy 90 $ (1, 2) Buy -10 $ (2, 2) NoBuy New customer $ (11, 2) Several Periods NoBuy … $ -10 (END) 0 0 NoMarketing/NoBuy

© 2014 Fair Isaac Corporation. Confidential. Easier: Calculating CLV Using Matrix Algebra Purchase probabilities (R, CF) (1,CF+1) (R+1, CF) P buy, PC-ME 1-P buy, -ME (END) 1,01,0 1,01,0 (R, CF) $ Dynamic model Reward parameters Matrix algebra [1] 17

© 2014 Fair Isaac Corporation. Confidential. Historic Policy and Associated CLV 18 All customers marketed ( ) until Recency = 10 $

© 2014 Fair Isaac Corporation. Confidential. ► CLV is negative for 17 out of 50 states ► Might improve CLV by not marketing to customers in those 17 states ► But it’s not as simple... Computing Optimal Policy Policy Iteration Algorithm [2] ► Initialize with an arbitrary policy (e.g. historic policy) ► Alternate between two steps: ► Policy evaluation:Given current policy, calculate CLV for all states ► Policy improvement:For each state, find action that maximizes one-step look-ahead estimate of CLV. If any action changes, replace current by new policy ► Stop when CLV no longer improves 19

© 2014 Fair Isaac Corporation. Confidential. Historic Policy and Associated CLV 20 All customers marketed ( ) until Recency = 10 $

© 2014 Fair Isaac Corporation. Confidential. $ : Market : Don’t market Improvements From Historic Policy After First Policy Iteration 21

© 2014 Fair Isaac Corporation. Confidential. Marketing to (R = 4, CF = 1) ( )Yields Further Improvements $ : Market : Don’t market 22

© 2014 Fair Isaac Corporation. Confidential. Optimal Policy and Associated CLV After Convergence 23 $ : Market : Don’t market

© 2014 Fair Isaac Corporation. Confidential. Value of Optimizing Sequential Marketing Decisions for a New Customer Policy HistoricOptimal CLV(1,1)$131.03$ Profit Contribution from initial purchase$ Expected present value from future cash flows$31.03$34.92 % Improvement of future cash flow % 24

© 2014 Fair Isaac Corporation. Confidential. ► CLV is not a passive metric—it can and should be managed ► Markov Decision Processes offer framework to optimize sequential decisions, CLV ► Model formulation (states, transition probabilities, rewards) ► Data acquisition and parameter estimation ► Software for matrix algebra and optimization Summary of CLV Optimization 25

© 2014 Fair Isaac Corporation. Confidential. Extensions and Challenges 26

© 2014 Fair Isaac Corporation. Confidential. ► Marketing example could be extended in many directions Extensions ► Inclusion of additional customer attributes ( monetary, product holdings, time as customer, demographics,...) ► Customer dialogue—asking for customer preferences ahead of offers ► Alternative marketing offers and their costs ► Drop assumption that customers don’t buy unless marketed to ► Transition probabilities could depend on macro-economic parameters (such as consumer confidence) 27 ► Varying profit contributions per customer per period

© 2014 Fair Isaac Corporation. Confidential. ► “Curse of dimensionality”: ► As more variables are added to define the state ► State space will explode ► Data will be too sparse to estimate transition probabilities directly  Be parsimonious, leverage dimension reduction/machine learning techniques ► “Curse of consistency”: ► For a business that does not experiment with alternative actions ► Only a tiny sliver of state-action combinations will be observed ► Data will give little guidance on alternative policies  Implement a learning strategy that tests reasonable actions for each state Challenges and How to Address Them 28

© 2014 Fair Isaac Corporation. Confidential. ► Sequential decisions can substantially increase customer value ► Markov Decision Models provide analytic framework ► “Art and science” to develop model ► Need customer transactions, business treatments, cash flow information ► What, when, how much? ► Advanced analytic methodologies address curses of dimensionality, consistency Discussion 29 Let us know if you’re interested in a Proof of Concept!

© 2014 Fair Isaac Corporation. Confidential. References [1] Modeling customer relationships as Markov chains. Phillip E. Pfeifer and Robert L. Carraway, John Wiley & Sons, Inc. and Direct Marketing Educational Foundation, Inc. Journal of Interactive Marketing Volume 14, Issue 2 (2000), pp. 43–55. [2] Dynamic Programming and Markov Processes. Ronald A. Howard, The M.I.T. Press,

© 2014 Fair Isaac Corporation. Confidential. This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent. Dr. Gerald Fahner Thank You! 31

© 2014 Fair Isaac Corporation. Confidential. Learn More at FICO World Related Sessions ► Addressing Attrition: Ultra-Dynamic Multi-Dimensional Attrition Analytics with Tree Ensemble Models Experts at FICO World ► Larry Rosenberger ► Oliver Bastert White Papers Online ► Closing the Loop and Eliminating Costly Delays in Analytic Development and Deployment ► Customer Centricity: Four Bank Success Stories Blogs ► 32

© 2014 Fair Isaac Corporation. Confidential. Please rate this session online! 33 Dr. Gerald Fahner