Modelling Market Dynamics and Consumer Behaviour From The Bottom Up Iqbal Adjali Mathematics & Informatics Unilever R&D, Colworth Science Park Bedford,
Published byModified over 2 years ago
Presentation on theme: "Modelling Market Dynamics and Consumer Behaviour From The Bottom Up Iqbal Adjali Mathematics & Informatics Unilever R&D, Colworth Science Park Bedford,"— Presentation transcript:
Modelling Market Dynamics and Consumer Behaviour From The Bottom Up Iqbal Adjali Mathematics & Informatics Unilever R&D, Colworth Science Park Bedford, UK First ESSA Summer School Brescia 15 September 2010
Leading Categories & Brands Foods Home and Personal Care World Number 1 World Number 2 Local Strength Savoury & Dressings Spreads Weight Management Tea Ice Cream Skin Deodorants Laundry #1 in D&E Daily Hair Care # 1 in D&E Household Care Oral Care Our 12 €1 billion brands
Mathematics & Informatics Group Personalisation Algorithms Agent Based Methods Network Science Graphical Models Economics/Game Theory Behaviour Change Theory Interactive Systems Persuasive Technology On-line Therapy Natural Language Processing Ubiquitous Computing Connected Sensors Consumer Complexity Mathematics Psychology Computer Science
Agenda Background and Motivation ●Marketing science – the legacy ●The social simulation approach to Marketing ●Challenges in applying social simulation to market and consumer behaviour Application Examples ●Modelling the diffusion dynamics of the trial patterns of an online grocery store ●Break after 1h30mn? ●Modelling the volatility in market shares – A Micro-simulation approach ●Social network interventions – Experimental study ●Social network interventions – ABM of the take-up of physical exercise General Discussion
Why is Understanding Consumer Behaviour Important? As an applied field, it’s relevant to: ●Business and market strategists ●Marketing managers/practitioners ●Market regulatory bodies ●Consumer groups As a research discipline, it’s relevant to: ●Economics, sociology, psychology, anthropology, biology ●Key to understanding human behaviour
Consumer Behaviour Complexity and Emergence ●Unpredictability of market behaviour (e.g. path dependence): Stochastic behaviour of consumers ●Consumer-consumer interactions: e.g. Word-of-Mouth dynamics ●Consumer lock-in phenomena: what makes a new product successful ●Understanding and measuring the effect of advertising / marketing interventions
Consumer Behaviour Literature: Two Schools of Thought ●Economic theory –Consumer behaviour is based on Demand Theory and utility maximisation –Purchase behaviour is a direct function of external economic factors (price, income,...) ●Behavioural Sciences –Internal and non-economic factors are considered –Based on statistical models –Fragmented field –Data driven
Consumer Behaviour: Economic Approach Rule for choice selection Logit Model U i utility of product i has for the consumer ●Traditional Theory of Demand culminates in Lancaster’s Consumption Technology Matrix Kelvin Lancaster, “Consumer Demand: A New Approach”, Columbia University Press - NY, 1971
Consumer Behaviour Economic Approach ●Advantages –Mature theory - existence of consensus –Amenable to quantitative modelling & analysis ●Disadvantages –Focuses on external economic factors –Unrealistic assumptions (e.g. perfect information, rationality) –views consumer behaviour as the outcome of the consumer decision process
Behavioural Sciences: Some Approaches Search and evaluation, innovative behaviour, change behaviour, e.g. new product launch Habitual behaviour, e.g. influence of advertising Post-purchase behaviour, e.g. consumer satisfaction/loyalty Influence of society on decision process, e.g. Word-of-mouth Theory of Planned Behaviour Stimulus-Response Models Cognitive Dissonance Theory Social Network Theory
Theory of Planned Behaviour Outcome beliefs Reference beliefs Control beliefs Attitudes Perceived control Norms IntentionBehaviour Predispositions Attitudes MotivationsBehaviour F.M. Nicosia, Consumer Decision Processes, Prentice-Hall, 1966
Behavioural Sciences Approach ●Advantages –takes into account internal (psychological) and social and cultural factors –more accurate in describing observed behaviour ●Disadvantages –No unified approach; disparate and often conflicting models –No formal models, statistical inference only technique that can often be used –Data difficult to get
Social Simulation and Consumer Behaviour Bottom-up modelling approach that allows one to ●deal with consumers as individuals ●rules and mechanisms of arbitrary complexity ●take into account consumer-consumer interactions ●consider time evolution/dynamics Useful for both theory building and practical applications Forces researcher to develop formal frameworks
Social Simulation vs. Traditional Methods Agent-Based ModellingTraditional Methods Bottom-up (disaggregate) Consumers as individuals Top-down (aggregate) Representative Consumer Market discontinuities (emerging structure) Stable Markets (given structure) Explicit Modelling of Social NetworksNo easy way to model SN Evidence-based, general (non-Normal) distribution of variables Single-valued or Normally distributed variables Relative Merits Integrated Framework Separate Models
Key Challenges for Consumer Behaviour Modelling ●Data availability/gathering ●Standardised model building approaches – for comparison/duplication ●Model testing/validation methodologies
Model Validation ●Model validity is crucial for our purposes, as a robust agent based simulation will be one based on a validated MODEL, giving GOOD PREDICTIONS, and providing USEFUL INSIGHTS P R E D I C T I O N I N S I G H T Model Validation refers to the process of selecting the right model to match observations with predictions, given model complexity. Validation increases both insight and prediction. Model complexity acts as the constraint on the whole exercise.
Validation Approaches ●K-I-S-S (Keep It Simple Stupid) ●Quantitative (Fagiolo, 2005; Werker, 2004; Windrum, 2007) –Numerical techniques to search parameter space efficiently –Model calibration with macro AND micro level observations –Use of optimization techniques like GA/GP/ANT/SWARM ●Qualitative (Garcia, 2007) –History friendly approach –Based more on expert opinions and stylized facts –Draws heavily from established theory ●Experimental (Richetin et. al. 2009 and our ongoing collaboration) –Uses in-situ human players to run experiments –Simulation designed to mimic experiment OR experiments designed to test simulation –Hybrid approach using quantitative, qualitative and experimental styles –Cutting edge interdisciplinary
Advancing Consumer Behaviour Research ●Investigate impact of advanced psychological theories on consumer behaviour ●Investigate role of social networks and context dependency in determining consumer behaviour ●Design controlled experiments (e.g. to test word- of-mouth dynamics, effect of advertising,…) ●Closer collaboration across disciplines and between academia and industry
A Spatio-temporal Agent Based Model of Consumer Trials of a Swiss Online Grocery Store
Modelling the diffusion of a new consumer service ●Context & Motivation ●Diffusion dynamics –Bass, ABM ●Data –The LeShop database ●Model Specification ●Simulation results ●Conclusion
Introduction ●Diffusion dynamics is an important concept in Marketing and the modelling of markets ●Different approaches have been used in the literature (aggregate, disaggregate, statistical,…) ●Difficulty in getting the right data for the right approach ●Data available: individual-based customer transactions from the online grocery supermarket (LeShop) and geo-statistical data ●Develop an empirically motivated ABM ●Opportunity to integrate an ABM in a spatial (geographical) context ●Research question*: Do neighbourhood effects play an important role in the spatio-temporal dynamics of information diffusion? Background Motivation *Bell, D. and Song, S. (2004). Social contagion and trial on the internet: Evidence from online grocery retailing. Working paper, Wharton School Marketing Department.
●Bass Diffusion Model* –Top-down (aggregate) model –Global variables/parameters –Has been successful in predicting market take-up of innovations in consumer durables –Not easily generalisable (to many products and many competitors) –Restrictive assumptions (e.g. homogeneous population, perfect mixing…) *Bass, F., “A new product growth model for consumer durables”, Management Science 15(5), 1969 Modelling Diffusion: Bass
Shopper and Market Data Transaction data GIS/demo data Quality data Advertising data Customer transaction data Marketing Agencies Office of Swiss Stats Marketing Agencies ABM Platform
Data Description Three Linked Datasets: ●Customers, Transactions, Products (1998-2003) Customer Table (~4.5k): ●UserID,ZIP_Code,Age_Range,Gender,Language Transaction Table (~2m) : ●User_ID,Basket_ID,Purchase_Date_Time,SKU_Number,Pri ce,Quantity,Discount Product Table (~10k): ●SKU_Number$Category$Sub_Category$Brand_ID$Quality Customer data GIS/demo data
Social Networks Recommendation diffusion ●We looked at the diffusion of the recommended membership (Only components with at least 9 members are taken into account) ●2002-2004 remarkable increase in “horizontal” connectivity – people started to react earlier, recommendations are taken during the same year 2001 2000 2002 2003 2004 2005
Recommendation network Age distribution ●Again, we analyse only components with more than 9 members! ●Notable absence of older generations early on in the recommendation patterns!! 15-17y 18-24y 25-34y 35-44y 55-64y
Recommendation network: Gender and geographical distribution Families have a big role in recommendation scheme! VS ZH GE AG VD NE FR OTHER Intercantonal connections which surpass language barriers are due to Lausanne – Zurich connection and to “small” cantons.
Agent Behaviour The only endogenous variable in agents is the trial state: A ij. It evolves according to the following rule: If A ij =0 then invoke the external influence rule: If A ij still =0 then invoke the internal influence rule: random number drawn from uniform distribution [0,1] Implement a small world network for social interactions
Simulation Results ●We performed several hundred simulation runs, sweeping the parameter space ( F, G, F, G ). ●We calibrate the model on the real aggregate trial data by maximising Goodness-of-fit according the LOESS algorithm. ●This procedure led to the following calibrated parameter values:
Goodness of fit – LOESS Statistics R2R2 0.630.520.64 Degree18.104.22.168
Spatio-Temporal Diffusion Each customer agent is unique and reacts to: Global Communication and Word of Mouth interactions Actual ABM Merger with Migros (Janurary 2004) Promotional Campaigns Monthly Customer Trials Cumulative Total Customers I. Adjali et al, (2007) “An Agent Based Model of Customer Tirials in an Online Grocery Store", presented at: ABM Marketing Conference, Groningen, 22 August 2007
Conclusion (Innovation Dynamics) ●A social simulation approach to marketing does make sense: it is a very effective way to capture: –Market and consumer heterogeneity –Consumer interactions ●Comparison with real data is a crucial step in validating the model (aggregate and individual levels) ●Neighbourhood (spatial) effects do exist and seem to matter even in online markets –Useful to consider the spatial dimension explicitly when appropriate ●Challenges: explore parameter space, macro vs. micro validation, look at different diffusion processes e.g. product launches, opinion dynamics…
●Computational modelling of individual consumer behaviour, social networks and brand relationships leads to better insight in consumer behaviour and market dynamics ●Enhance predictive power of marketing mix models ●develop robust marketing strategies through detailed scenario analyses What Social Simulation Means For The Marketing Discipline
Volatility in the Consumer Packaged Goods Industry – A Micro-Simulation Based Study In collaboration with Nigel Gilbert and Alan Roach, Surrey University Frank Smith and Stephen Glavin, UCL Partly published in: Sengupta et al., Advances in Complex Systems 2010
Introduction ●Consumer goods markets –Characterised by noisy market share dynamics and instability (Jager: 2007) –Heterogeneous consumers with wide variety of tastes/preferences (Allenby, Rossi: 1998) –Intense competitive interventions from multiple firms using pricing, promotions, advertising etc. (Ailawadi et. al.: 2001, Blattenberg, Wisniewski: 1989). ●Additionally, social networks, WoM may play important roles
Consumer Strategies ●Each strategy filters the list of available products. ●The final choice is selected randomly from any remaining. ●Available strategies –Cheapest – with a noise parameter on perceived difference. –Brand Loyalty – choose the brand it has bought most in the past. –Threshold – selects products below its predefined maximum price. –Change of pace – will swap brand after a number of weeks, defined by a parameter.
Assessment ●To evaluate runs, –The difference between the actual monthly sales for each brand and the simulated sales data is calculated. –Each brand’s overall fitness is the sum of all the monthly differences. –And the model’s fitness is the sum of all the brands’ overall fitnesses. ●Quite crude, but we are looking for a rough fit. ●This is then used to assess the fit of models under different parameters settings.
Parameter Exploration ●Two aspects of tuning, parameters and strategies. ●Even with this simple model there are a lot of possible variations – 61,440! ●A subgroup of 17 strategy combinations was identified. ●So two sweeps were done. –A broad sweep to identify best strategy combinations and parameters. –A fine tuning sweep to identify the best parameter settings for these strategies.
Charts of best runs Target SalesThreshold Fitness 307
Charts of best runs Target SalesThreshold, Change of pace, Brand Loyalty, Cheapest. Fitness 343
Conclusions ●The search process identified models which roughly replicated the data. ●The best model for these data is the simplest one, Threshold. ●But other combinations were fairly successful. ●Overall the main weakness was not modelling the late sales peak of the red product. ●This is a prototype process for assessing the success of consumer strategies for modelling our data.
Fruit Juice Category dynamics ●Traditional approaches – Logit and Probit models (Frances, Paap:2001; Anderson et. al.: 1992). ●General focus is on the effects of promotions on quantity, incidence and market structure over a short term (Chan: 2008). ●They are generally independent of time and ignore the dynamics. ●Fail to examine the effects of heterogeneity at the individual level on market dynamics. ●Lack non-linearities such as social interactions (Jager: 2007). ●Predictions at the individual level not possible. ●Poor out-of-sample performance.
DATA Transactions from Swiss online supermarket from January 2006 to December 2006 (52 weeks) Fresh fruit juice category Each transaction included –Household ID –Individual Product (SKU) ID –Week number –Net price paid –Discount received –Quantity of product purchased
DATA FILTERING ●Eliminate households who purchase infrequently 55 SKUs, 2435 households, 28179 transactions. ●Data is divided into three temporal partitions –Week 1-24: To be used in initialisation. –Week 25-38: To be used for calibration. –Week 38-52: To be used for out of sample prediction. ●Three dominant characteristics in products chosen – brand, flavour, pack size.
PRODUCT FEATURES BrandNo. of ProductsFlavourNo. of Products Isola2Kid’s Drink3 Danone2Grape Fruit2 Mickey’s Adventures2Nectar7 Oasis1Pineapple2 Capri Sonne2Apple11 Granini5Multi-fruit12 Michel5Orange12 Obi2Other fruit6 Nectar8 Hohes C4 Ramseier5 Max Havelaar1 Actilife3 M8 Gold5 Total55
●Have industry with K distinct products and a consumer base of size N. ●Each product has M attributes that make it unique for a consumer – defining an address vector for the product in characteristic space. ●Each consumer has a preference – a vector of characteristics that they would like to see in a product; their ideal point. ●The closer the ideal point is to the address of a product, the higher the subjective utility for the consumer from purchasing it. MODEL
The utility from product k to agent i : Normalized relative distance of product from ideal point Net price of k Individual specific single parameter MODEL
–Initialisation Product characteristic vectors and agent ideal points are initialised within 3-d characteristics space –Calibration Agent level calibration carried out at both micro and macro level. –Testing Carried out at both micro and macro level. Parameterised agents used to make out of sample predictions Monte-Carlo type simulation using multiple runs where each corresponds to a random draw from the optimised parameter set of each agent. (100 runs per agent) A market share based random model used as benchmark for comparison of our model VALIDATION
MODEL TEST RESULTS ●Macro Validation –Comparison of predicted and actual market shares of brands and flavours with predicted and actual percentages. –Two measure used: Mean weekly relative difference between actual and simulated market shares per brand/flavour Correlation coefficient. –Volatility in market shares is captured very well. –Small price changes (promotions in flavours) result in high levels of volatility which is captured in our model –Significantly outperforms the benchmark
PREDICTED AND ACTUAL WEEKLY MARKET SHARES BY BRAND
●Micro Validation –Measure accuracy of predictions for household specific choice of products AND characteristics in each dimension separately –Measure joint predictions along subsets of dimensions. –The model works very well at the individual level as well as aggregate level. –Individual level prediction of characteristic choice accuracy is superior over product choice accuracy –Results significantly outperforms the benchmark MODEL TEST RESULTS
SUMMARY ●Bottom-up simulation developed to model fruit juice market using loyalty card checkout data for empirical validation. ●Agents using simple behavioural model initialised and calibrated using part of the data with predictions tested against remaining data ●Heterogeneity within consumers’ tastes and preferences and price responsiveness ●High levels of out-of-sample prediction achieved at both aggregate and disaggregate household level.
●Weekly evolution of product groups with medium to large market shares overall predicted to high degree –Slightly better in terms of direction of change over magnitude ●Micro-level: model predicted accurately every SKU purchase for 35% of transactions. ●Brands and flavour choice captured with high degree of accuracy. ●Volatility in market shares captured without the addition of “noise” factors. ●The benchmark model without heterogeneity fails to achieve same levels of accuracy SUMMARY
ON GOING AND FUTURE RESEARCH ●Test our model on other data sets. ●Restrict rationality in agents decision making. Add cognitive elements. ●Introduce changing preferences, changing attitudes, changing social norms. ●Incorporate social feedbacks and peer pressure effects. ●Adapt modelling framework for longer term analysis, introducing technology changes, entry exit etc.
Social Network Interventions In Consumer Markets In Collaboration with: Marco Perugini & Juliette Richetin, University of Milan Bicocca Alex Linley, Centre for Applied Positive Psychology, UK To be part published in Greetham et al. Proceedings of Applications of Social Network Analysis Conference, 2010
Background 1 Research on the spread of obesity, smoking (Christakis& Fowler 2007, 2008) and happiness (Fowler & Christakis 2009) based on Framingham heart study data shows evidence of social network influence on long term dynamics.
Background 2 Network Spread of Obesity? Clusters of obese people (1971 - 2003) Chances of becoming obese increased by 57% if had friend who became obese. 40% if adult sibling became obese 37% if spouse No effect of geographic neighbour Greater influence from same sex Network Spread of Happiness? Clusters of happy and unhappy people Influence up to 3 degrees of separation Chances of becoming happy increases by 25% if near-by friend becomes happy 8% for spouse 14% sibling 34% next door neighbour No effect between co-workers
Richetin et al (2009). A Micro Level Simulation for the Prediction of Intention and Behavior, Cognitive Systems Research Compared to standard statistical approach the agent-based simulation generally improved the prediction of behaviour from intention The improvement in prediction is inversely proportional to the complexity of the underlying theoretical model The introduction of varying degrees of deviation from rationality in agents’ behaviour can lead to an improvement in the goodness of fit of the simulations Future work could focus on conflicting goals (CG), discrepancy (goal – actual) & Effort Advantage of a Network/ABM Approach – Modelling Irrationality
Pilot Study to Map Transfer of ‘Positivity’ with CAPP 20 Participants in second year university course. Plus a minimum of 5 people in their network Equals 20 egos and 100 alters [120 nodes and the ties between] Complete daily measures for 14 days Agent based modelling of transfer within network over time Baseline Measures Specify others in network: reciprocity of friendship Personality: (Gosling et al, 2003) Trait Gratitude: GQ-6 (Emmons & McCullough, 2003) Trait Emotional Intelligence: (Petridis & Furnham, 2001) Trait Positive and Negative Affect: PANAS (Watson, Clark, & Tellegen, 1988) Satisfaction with Life: (Diener, Emmons, Larsen, & Griffin, 1985). Daily Measures Daily frequency of interaction with others in network Type of interaction with others in network: Proximal - Distal Valence of interaction Daily levels of positive and negative affect Daily influences on well-being Network Mechanisms Study Design
Design ●Baseline: gender, age, ethnicity, big 5 personality traits, grattitude score ●Daily measures: PANAS, sleep, general health ●Contacts – mode, valence, duration ●Post-hoc social network
Objectives I.To analyse short term dynamics of happiness contagion II.To identify causal mechanisms by which happiness spreads Positive and negative affect (independent constructs) PANAS – Positive Affect Negative Affect Schedule
Social Network Snapshots NA PA and NA http://www.stanford.edu/group/sonia/
Results - PA, NA Selection process: Actors with higher value of NA would increase no. contacts more rapidly. Influence process: Different shapes: PA - push toward the midpoint of range, negative feedback NA - push to extremes
Results - personality traits ●Emotional stability has positive effect on sleep and health and negative on NA ●Extraversion has positive effect on PA ●Openness to experience similarity: actors prefer ties to others with similar value ●Actors with higher values of openness tend to increase their out-degree more rapidly ●But opposite for extraversion!
Results related to gratitude and sleep ●Grattitude has positive effect on NA rate, and negative on sleep rate ●Sleep effect from ethnicity negative and from emotional stability positive ●Sleep influence – push to midpoint range
Summary ●PA&NA have different mechanisms of influence ●Personality traits (esp. emotional stability and openness to experience) have effect on PA, NA, general health and sleep ●Grattitude scores have effects on NA and sleep rate
Effect of peer pressure on individual behaviour – Simulating the case of physical activity with an Agent Based Model (with a view to applying the approach to behaviour change interventions) 1 Presented by A. Sengupta at the WCSS conference in Kassel, Sep 2010
What this talk is about… Goal Formation Intention Behavior What neighbours are doing… Goal Directed Behavior Ajzen, 1991 Perugini & Conner, 2000 Perugini & Bagozzi, 2001 Physical Activity Recycling Preparing for exam… ●Can my friends cure me of my laziness?
Tasks, Influence and Behaviour ●Role of the environment on individual motivation –Stahl, Rutten, Nutbeam, Soc. Sc. Med, 2001 –Giles-Corti, Donovan, Soc. Sc. Med., 2002 –Giles-Corti, Donovan, Am. J. of Soc. Health, 2003 –McNeill, Wyrwich, Brownson, An. of Beh. Med., 2006 –Zoethout, Jager, Molleman, JASS, 2006
Questions ●What are the emergent patterns of behavior within groups under influence of both internal and external factors? ●What is the nature of interaction between internal and external factors? ●Does the nature of external influence have an effect on behavioral outcomes? Generate hypotheses using ABM simulations which can be tested experimentally. Extend the framework to incorporate other goal directed behaviors as well.
Models ●Model 1 –Goal directed motivation (actual vs. ideal state) –Goals constant over time –Observance of positive and zero behavior in neighbours (dual threshold model) ●Model 2 –Shifting goals for some (optimizers vs. satisfiers) –Opportunity cost of behavior –Habit formation from repeated behavior
External Influence ●Network topologies –Erdos-Renyi Random Network ER(n,m) –Barabasi Scale Free B(n,d) –Watts-Strogatz Small World WS(s,d,p) –Empirical: Newman PNAS, 2006 ●Network influence –“Dual” threshold model –Upper threshold –Lower threshold
Simulation Setup ●Netlogo environment ●379 agents (empirical network driven) ●Ideal state ~ U[80, 120], Actual state ~ U[50, 150] ●B ~ # of hours per week ●Time step ~ 1 week ●Theoretical networks –ER (379, 914), B(379, 3), WS(7, 3, 0.4) ●Parameters tested in Model 1
Results (Model 1) No Networks 1.There exists a critical value of rho around which the average properties of the system change 2.Increasing rate of physical decay leads to more physical activity on average No Networks: Benchmark
Results (Model 1) With Networks 1.Average Behavior fails to reach the maximum. 2.A critical value of rho exists around which the average steady state behavior of the system changes. 3.Low rho: expected behavior 4.High rho: unexpected behavior in (ideal – actual) Frequency
Model 2 ●Introducing Optimizers and Satisfiers –Optimizers: Readjusts goal –Satisfiers: Constant goal ●Behavior depends on ideal-actual discrepancy and opportunity cost ●Opportunity cost of positive behavior ●Habit formation due to repeated behavior Behavior Opportunity Cost Repeated Behavior
Results (Model 2) No Networks Small WorldRandomScale Free
Results (Model 2) ●No Network case: –Avg behavior is strictly decreasing in prop. of satisfiers and strictly increasing in B. –Avg opp. cost is strictly increasing in prop. of satisfiers and strictly decreasing in B. –Avg motivation is weakly decreasing in prop. of satisfiers.
Results (Model 2) ●With Networks –Avg behavior and motivation energy fall, and avg opp. cost rise significantly in presence of networks. –Patterns of change of avg behavior and opp. cost is preserved when compared to benchmark. –But avg motivation is now a strictly decreasing function of prop of satisfiers.
Conclusion ●In a closed system with individual goals, observations from peers is more likely reduce behavior – increase laziness. ●Amount of behavior can decrease but frequency may increase if internal motivation is high enough. ●Introducing habit formation does not improve the situation if networks are present. ●Type of network does not make a significant qualitative difference.