© dunnhumby 2008 | confidential Deep Data Diving Bringing Online and Offline Inline

© dunnhumby 2008 | public 2 An Apology We are here to talk about how best to work with the data available to us Working with good data can be a competitive advantage Some people do not like to give away what they are doing Today all the data I can publish comes from the UK Even UK data has been anonymised Sorry! Don’t worry, the learnings from the data are universally applicable

© dunnhumby 2008 | public 3 Agenda ● Online ● Offline ● Integration ●eMetrics –where we were –where we are –where we are going –What we need to watch for on the way ●Accountability and ROI –Why ROI & accountability are important –How we can measure them –A comparable currency and how it might be used –A real example off-line ●Bringing online and offline inline –Types of media, how they differ and how they are the same –What we have learned –Questions

© dunnhumby 2008 | public 4 The Promise ●Online promised perfect information ●It delivered data ●We needed insight Data Information Insight

© dunnhumby 2008 | public 5 eMetrics Then Justification Usability Inform strategy

© dunnhumby 2008 | public 6 eMetrics Now Tactical changes E.G. search engine terms matching current content Personalisation & Targeting Targeted content through web, mobile, , etc. Inform strategic decisions Some things never change, but these are now better defined

© dunnhumby 2008 | public 7 What to watch for on the way… The McNamara Fallacy ●“The first step is to measure whatever can be easily measured. This is OK as far as it goes. ●The second step is to disregard that which can’t be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. ●The third step is to presume that what can’t be measured easily really isn’t important. This is blindness. ●The fourth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.”

© dunnhumby 2008 | public 8 The holy grail 1.Linking ad exposure to customer purchase behaviour (google) 2.Understanding the effects of all your advertising (retailmedia.org) 3.Before-the-event information about what somebody is planning to buy (Alan Mitchell, Marketing Week)

© dunnhumby 2008 | public 9 Recap ●We have evolved our ability to use eMetrics ●We used to think we just needed behavioural data ●Online tracking gave us that data and we used it ●We now want more information but can still be caged in by what we can easily measure. ●We need to watch out for assuming what we have is all we need

© dunnhumby 2008 | public 10 How do advertisers work out if their ads work? $38,000,000,000 spent in UK in 2007 Ask some people if they’re aware of their brand? Can they recall the advertisement? Work out how many people were watching/reading/surfing/passing-by the medium in which their ad appears ….& how often c. 10% will use aggregated sales data

© dunnhumby 2008 | public 11 ….& they base this kind of data on….. Viewing behaviour of 5100 households The claimed consumption of 3000 products by 25k TGI respondents Claimed reading behaviour of 15k NRS respondents 2500 radio listeners filling out a ‘listening’ diary each week How many times someone clicks on a webpage

© dunnhumby 2008 | public 12 It’s no wonder…… “Half our advertising budget is wasted. Trouble is, we don’t know which half” Lord Leverhulme

© dunnhumby 2008 | public 13 Offline can teach Online ●There never was perfect information ●Offline learned to test as well as infer ●Insight comes from analysis of data in context Data Information Insight

© dunnhumby 2008 | public 14 What is the data offline? Three modes of measurement ●Geographic ●Claimed ●Behavioural

© dunnhumby 2008 | public 15 How understanding can work offline Three modes of measurement ●You are where you live ●You are what you say you are ●You are what you do

© dunnhumby 2008 | public 16 You are where you live ●Demographic targeting can be useful –Good areas and bad areas –Social groupings and cultural targeting ●It is easy (but remember Mcnamara) ●In the same zip code, within 300 yds are places to live with market values between $3m and $100k ●IP address targeting is no better or worse than zip codes

© dunnhumby 2008 | public 17 How understanding can work offline Three modes of measurement ●You are where you live ●You are what you say you are ●You are what you do

© dunnhumby 2008 | public 18 Copyright, dunnhumby confidential Proportion of H/M/L organic purchasers by answers to "I've never tried organic and have no intention to" 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Strongly disagree Somewhat disagree Neither agree nor disagree Somewhat agree Strongly agree high medium low You are what you say you are

© dunnhumby 2008 | public 19 How understanding can work offline Three modes of measurement ●You are where you live ●You are what you say you are ●You are what you do

© dunnhumby 2008 | public 20 You are what you do ●Behaviour does not define a person (unless you are a philosopher) ●We do not have access to all of anyone's behaviour and to try would be intrusive ●We do have the one bit that matters to us

© dunnhumby 2008 | public 21 You are what you do has been changed by loyalty cards How loyalty has step changed evaluations Was ●Store-relevant promotion –Putting a six sheet near a store –Floor Graphics in store ●Check uplift in store Is ●You buy nappies regularly ●We send you baby food vouchers for a new launch What we can do with this insight ●Ranging, Promotions, Wallet Share, New Customers

© dunnhumby 2008 | public 22 What insight comes from our data? “Not everything that counts can be counted and not everything that can be counted counts” Albert Einstein

© dunnhumby 2008 | public 23 One common currency across all advertising: Where does our financial data come from? Looking at actual shopping behaviour in store and online 53 million people in the US 13 million people in the UK $

© dunnhumby 2008 | public 24 How do we use this data? Comparisons show changes and trends Day-by-day Week-by-week Month-by-month Nationally Regionally By zipcode, by individual store Year-by-year

© dunnhumby 2008 | public 25 This allows you to establish Which elements of your media mix helped & in what combination…. …& which hindered …& where your sales came from

© dunnhumby 2008 | public 26 An online or TV advertising campaign… How does this work in practice? An amusing or thought provoking TV advertisement can inspire groups on the social networking website Facebook and postings on the video sharing site YouTube. This particular advertisement had millions of downloads

© dunnhumby 2008 | public 27 1)Product X is a seasonal category 2007 TV campaignEquivalent period The campaign Duration

© dunnhumby 2008 | public 28 Week Commencing 2)The uplift was supported by promotions TV campaign panel sales uplift (£) for Product X 250g bars 2 for £2

© dunnhumby 2008 | public 29 3)Uplifts were comparable with the press campaign from earlier in the year TV Campaign (cost $7m) Press Campaign 13% uplift 9% uplift From current customersFrom new customers (cost $1.7m)

© dunnhumby 2008 | public 30 So what does this tell us?? In this instance the TV campaign was not as effective as people had thought. It drove people online in millions and promoted an awful lot of awareness but had little impact on trade sales.

© dunnhumby 2008 | public 31 Recap ●There is a lot of data, but still not enough ●We need to be careful to understand what we do not know ●A lot of money spent so returns are important ●A lot of the current off-line practices are outdated ●There is a difference between data and information ●Loyalty cards help us to change data into information ●We can look at when a campaign is active and examine the uplift, but we need to take externalities into account to ensure it is meaningful ●The only reasonable currency for a trade driving advert is sales and this is true regardless of the media channel ●This still leaves a potential data gap between who sees the advertisement and who purchases the product

© dunnhumby 2008 | public 32 Addressing the gap ●Privacy considerations ●Technical considerations ●Analytical considerations

© dunnhumby 2008 | public 33 Media and the ‘new’ metric Steps to measure a medium 1.Identify who has seen the medium 2.Ensure you have a control who are similar but have not seen the medium 3.Compare the two groups taking care to examine both the expected behaviour and the actual behaviour

© dunnhumby 2008 | public 34 Types of media ●Retail Media –Car park posters –Digital signage –For a full list see whi have a fairly complete sumarrywww.retailmedia.org ●Online media –Display advertising (banners, pop-ups, etc) –Search –Micro-sites and advertorial –For full list see the eMetrics summit speakers ●Traditional media –TV –Press (newspapers, magazines, etc) –Radio –Other

© dunnhumby 2008 | public 35 Retail Media ●Purchases measurable ●Opportunity to see defined ●In store research to quantify proportions viewing Result is a good measurable medium whether purchase is online or in store

© dunnhumby 2008 | public 36 Online Media ●Full data available ●Incorporating data must be done in a sensitive and trustworthy way ●Potential to be fully measurable Result can be a completely measurable and defined medium whether purchase takes place in store or online

© dunnhumby 2008 | public 37 Traditional Media ●Ability to determine opportunity to see varies amongst media –TV vs digital TV –Press, newspapers and magazines –Fleet media and roadside posters Results vary amongst the different media but are not affected by whether the actual purchases are online or in store

© dunnhumby 2008 | public 38 Using the new metric ●It is essential to have good customer centric purchasing information ●It doesn’t matter whether a customer prefers to buy in- store or online ●The different media can be hard or easy to measure (online is generally the easiest) It is finally becoming possible to measure the effectiveness of different media in a comparable way

© dunnhumby 2008 | public 39 Overview ●Online media tell us the things we need to know to make them the most measurable media in terms of sales uplift ●Offline media are susceptible to analysis in terms of sales uplift where individual (opt in) shopping data is available ●Online and offline can therefore be brought into alignment if a metric of sales uplift is applied

© dunnhumby 2008 | public 40 Conclusion ●Having good (or even perfect) data is not sufficient ●Real analysis must be done in terms of the effect on the business ●The fewer levels of proxy between cause and effect the more robust the insight produced Only when online and offline data are measured in an integrated way, in terms of their effect on the business can a real evaluation of effectiveness be made. The only measure of business success is financial return

© dunnhumby 2008 | public 41 Questions?

© dunnhumby 2008 | public 42 thank you (Copies of the slides for this presentation will be Available to download via