1. Predicting Consumer Choice Using Supermarket Scanner Data: Combining Parametric and Non-parametric Methods Elena Eneva eneva@cs.cmu.edu April 20 2001 CALD Lab

2. Problem and Motivation Profit optimization for stores Knowthycustomer Micro-marketing Predicting Consumer Choice Use scanner data Previous research: profit increase Gross profit margin 4% to 10% Operating profit margin 33% to 83% a classic business rule that is taught in every 100-level course An under-utilized gold mine

3. Retail Store Scanner Data Chilled Juice Category 14 products 2 years 100 stores Store-level aggregation Weekly reports

4. Goal Build an accurate predictor of consumer choice (that knows the customer). In-prices, out-quantities Category: Chilled Orange Juice Price of Product 1 Price of Product 2 Price of Product 3 Price of Product 14... “I know your customers” Predictor Quantity bought of Product 1... Quantity bought of Product 2 Quantity bought of Product 3 Quantity bought of Product 14

5. Previous Work on Retail Data Traditionally – using parametric models (linear regression) Recently – using non-parametric models (neural networks) NN outperforms LR in accuracy, although LR performs adequately NN are mistrusted

6. Our Niche Advantage of LR: known functional form (linear in log space), extrapolation ability Advantage of NN: flexibility, accuracy extrapolation ability accuracy NN new LR Take Advantage: use the assumed prior to bias the accurate learner Higher level model from simpler models

7. Related Research In Other Fields Patrice Simard et al. “Tangent Prop” Possible to directly learn the invariance of the data independently from the “real” learning task Michael Perrone “Improving Regression Estimation: Averaging Methods for Variance Reduction with Extensions to General Convex Measure Optimization ” generalized ensemble method estimator

8. Combination Approaches Train Separately, then Combine Outputs as Inputs Jumping Connections

9. Train separately, then combine train a NN and a LR separately, and calculate the weighted average for the final prediction NN Input Prices Output Quantities... LR Input Prices Output Quantities... Final Prediction

10. Outputs as Inputs adding to a learner the prediction of the other learner (over the same 14 input prices) as an extra input NN Output Quantities Input Prices... LR... Input Prices... Output Quantities NN Output Quantities Input Prices... LR... Input Prices... Output Quantities

11. Jumping Connections Combining two types of NN connections in one NN Gives the effect of simulating a LR and NN all together...

12. Results – RMS Errors Linear Regression Neural Net Weighted Average Jump Connections Outputs as Inputs 0.192 0.075 0.074 0.072 0.070

13. Results - % Error in Predicted Q Comparison with IO method: percent error in predicted output

14. Summary Combining NN and LR gives a more accurate and robust model Better in terms of understandability for the marketing community Learns a better consumer model Improves pricing strategies

15. Future Work Include: demographics data promotional data competitor data Apply Multitask Learning Another data set with more density End