1. Data Mining in Industry: Putting Theory into Practice Bhavani Raskutti

2. Agenda What do analysts in industry actually do? Analytics in Australian Industry Case studies – Telecommunications Telecommunications – Wholesale Wholesale Take-home Points

3. What do analysts in industry actually do? Business understanding of complex trends To make strategic & operational decisions Business Problem Data Acquisition & Preparation DAP Problem Definition PD D Deployment Presentation P Mathematical Modelling (Algorithms) Data Matrix MM Initial Development Iterative 90% DAP Decision-making by users Insights via GUI Automation Training Documentation IT Support

4. Agenda What do analysts in industry actually do? Analytics in Australian Industry Case studies – Telecommunications Telecommunications – Wholesale Wholesale Take-home Points

5. Analytics in Australian Industry IndustryClustering / Segmentation Classification / Scoring Other Customer/market segmentation Survey analysis Sentiment analysis … Upsell/Cross-sell Fraud detection Credit scoring Location services Churn modelling … Marketing effectiveness Market share understanding Next best offer Asset management … Telecom Finance Wholesale Retail Bio-informatics

6. Analytics in Australian Industry IndustryClustering / Segmentation Classification / Scoring Other Customer/market segmentation Survey analysis Sentiment analysis … Upsell/Cross-sell Fraud detection Credit scoring Location services Churn modelling … Marketing effectiveness Market share understanding Next best offer Asset management … Telecom Finance Wholesale Retail Bio-informatics

7. Agenda What do analysts in industry actually do? Analytics in Australian Industry Case studies – Telecommunications Telecommunications – Wholesale Wholesale Take-home Points

8. Win-back? Stop churn? Upsell? DAP PD D P MM - Winning back customers is hard - Churn is hard to identify and harder to prevent - Upsell to existing customers increases retention & revenue Increasing Revenue for Telstra Business Customers Increase revenue from business customers Imbalanced data – too few examples of take-up for most products - Data aggregation & Interleaving Comparable predictors from revenue - Raw, change from previous, projected - Use values as is & normalised - Binarise using 10 equi-size bins - Satisfaction survey - Service assurance - Demographics - Quarterly revenue from different products for each customer - SVMs to score with likelihood of take-up - Weighting by value of take-up to find high value take-up Excel spread sheet with potential customer list - Take-up likelihood for all modelled products - Last quarter revenue for all products - Implementation in Matlab & C - Different predictive models for over 50 products in 4 segments - Automatic updates every quarter - Used by sales consultants to re- negotiate contracts Create models to predict customers likely to take up a product soon i-5 i-4 i-3 i-2 i-4 i-3 i-2 i-1 i-3 i-2 i-1 i i-1 i i+1 i+2 Predictors Prediction Labels TRAINTRAIN

9. Increasing Revenue for Telstra Business Customers (Cont’d) Evaluation: Piloted predictive modelling in 2 different regions – Region 1: 9 new opportunities from just 5 products with an increase in revenue of ~400K A$ – Region 2: Opportunities identified were already being processed by sales consultants Conclusion: Predictive modelling better than previous manual process – Identifies more opportunities – Spreads techniques of good sales teams across the whole organisation Deployed in 2004 & still operational For more details, refer to “Predicting Product Purchase Patterns for Corporate Customers” by Bhavani Raskutti & Alan Herschtal in Proceedings of KDD’05, Chicago, Illinois, USA

10. Agenda What do analysts in industry actually do? Analytics in Australian Industry Case studies – Telecommunications Telecommunications – Wholesale Wholesale Take-home Points

11. DAP PD D P MM - Sales  demand - Similar products @ similar outlets have similar demand to sales relationship - Anomaly may be due to lack of stock Wholesale Sales Opportunities at Retailers Increase wholesale sales into major retailers - Quantify demand - Define normalised sell-rate - Define a long term in-stock measure - Define products & outlets that are similar - Weekly SOH & sales for each store & SKU - SKU master - Store master Simple univariate regression in SQL Perform comparisons & find anomalies with stock issues - Self-serve report in Cognos for each sales rep - Presents list of products with opportunities - Opportunities click through to detailed graphs showing demand, sales & stock position of the two products compared

12. Wholesale Sales Opportunities at Retailers (Cont’d) Demand In-stock % ·R1 ·R2 Demand Sell Rate

13. DAP PD D P MM - Sales  demand - Similar products @ similar outlets have similar demand to sales relationship - Anomaly may be due to lack of stock Wholesale Sales Opportunities at Retailers Increase wholesale sales into major retailers - Quantify demand - Define normalised sell-rate - Define a long term in-stock measure - Define products & outlets that are similar - Weekly SOH & sales for each store & SKU - SKU master - Store master Simple univariate regression in SQL - Self-serve report in Cognos for each sales rep - Presents list of products with opportunities - Opportunities click through to detailed graphs showing demand, sales & stock position of the two products compared - Implementation in SQL & Cognos - DataMarts for reports updated weekly - Documentation on intranet wiki - Training by corporate training team - Support from IT helpdesk Perform comparisons & find anomalies with stock issues

14. Take-home points Data acquisition & processing phase forms 80-90% of any analytics project Business users are tool agnostic – R, SAS, Matlab, SPSS, … for statistical analysis – Tableau, Cognos, Excel, VB, … for presentation Business adoption of analytics driven by – Utility of application – Ease of decision-making from insights – Ability to explain insights

15. Questions?