1. Catalog Matchbacks and Advertising Attribution How to Best Invest Advertising Dollars in a Confusing Environment Kevin Hillstrom President, MineThatData

2. Multichannel Forensics Most of my projects are called “Multichannel Forensics” projects. I analyze customer data to see how customers migrate across channels and interact across channels … I’ve conducted about 50 projects for catalog/retail/online clients in the past 3+ years. What we’ll learn today is an outcome of 50+ Multichannel Forensics projects! 2

3. Let's Talk Attribution!!! Quiz: An existing customer receives a catalog on September 1, receives e-mail marketing campaigns on September 7 and September 9, and purchases on your website on September 10, buying merchandise featured in your catalog and merchandise available only online. What % of the order do you allocate to catalogs, e-mail, and to organic brand loyalty? 3

4. Let's Talk Attribution!!! Quiz: An existing customer receives a catalog on September 1. On September 2, the customer uses Google to search for merchandise, visits your website, and purchases an item. What % of the order do you allocate to catalogs, to e-mail, to search, and to organic brand loyalty? 4

5. Let's Talk Attribution!!! Quiz: An existing customer receives a catalog on August 1, and receives 17 e-mail marketing messages between 8/1 and 10/1. On 10/4, the customer uses Google to search for merchandise, visits, and buys an item. What % of the order do you allocate to catalogs, to e-mail, to search, and to organic brand loyalty? 5

6. Can Anybody Agree? If you poll 100 marketers, asking them to attribute orders to the marketing channel that caused the order to happen, you’re likely to get as many as 100 different methodologies!! And if you ask a computer model to attribute orders to the marketing channel that drove the order, you’re likely to obtain an answer you don’t agree with! 6

7. The Impact on Catalogs is Significant A simple 10% change in catalog demand due to attribution changes causes a significant change in circulation strategy. Base Strategy: Circ = 1,500, Demand = $3,237. New Attribution: Circ = 2,050, Demand = $4,430 (or demand really is $3,987, you just over-attributed orders & over-circulated). 7

8. Why Guess? The beautiful thing about catalog marketing and e-mail marketing is that you can test not sending contacts to customers. In other words, you can see what happens to other channels when you do not mail a catalog. You can see what happens to other channels when you do not send an e-mail campaign! 8

9. Test Design! Sample 80,000 twelve-month buyers with a valid e-mail address. Group 1 = 20,000: Catalogs = Yes, E-Mail = Yes. Group 2 = 20,000: Catalogs = Yes, E-Mail = No. Group 3 = 20,000: Catalogs = No, E-Mail = Yes. Group 4 = 20,000: Catalogs = No, E-Mail = No. Execute for a month, quarter, season, or year! 9

10. Test Results = Attribution Rules In a controlled experiment, the results of your test tell you what impact catalog marketing and e-mail marketing have on other channels (search, mobile, social, display ads, affiliates). When you know, via a controlled experiment, what impact catalogs and e-mail truly have, you can set up reasonable attribution rules! 10

11. Test: Overall Results, One Month Test We have four test panels in this test. Our company sent one catalog and nine e-mail campaigns during this one- month timeframe. Wow! 11

12. Test: Compare Catalog Mail/No-Mail Via classic matchback or attribution rules, the catalog would be credited with generating $10.70 per customer. Via mail/no-mail tests, the catalog is truly credited with $3.59. Oh oh. 12

13. Catalog Attribution Rules Attribute $1.00 of every $4.75 online to the catalog (21% of online dollars). Attribute ($0.50) of every $0.80 e-mail demand to the catalog (-63% of e-mail dollars). Attribute $0.78 of every $1.58 in search to the catalog (48% of search dollars) --- and, attribute search expense, too! 13

14. Catalog Attribution Rules Attribute $0.00 of every $0.28 social media demand to the catalog (0%). Attribute $0.00 of every $0.38 mobile demand to the catalog (0%). Attribute $0.04 of every $0.11 (36%) affiliate demand to the catalog, and ($0.07) of every $0.17 display ad demand to the catalog (-41%). 14

15. Catalog And Search Secret So often, I find that catalogs cause search to happen. Two problems: First, catalogers tend to match the search order to a catalog, when in reality, without search, the catalog order doesn’t happen! Second, catalogers need to fully allocate all search expenses (converted & non- converted clicks) back to catalogs. 15

16. Catalog Attribution Rules The results are very different from classic matchback algorithms, and for good reason! This is a controlled experiment. In this case, customers continued to spend money, when catalogs were not mailed. This means that we are dramatically over- circulating catalogs to customers. Oh oh. 16

17. Test: Compare E-Mail Mail/No-Mail We can run the very same analysis with e- mail marketing. Average the results of the two e-mail ‘mail’ segments, and average the results of the two e-mail ‘no-mail’ segments. What do you observe? 17

18. E-Mail Attribution Rules Attribute $0.00 of every $1.48 telephone demand to e-mail marketing (0%). Attribute ($0.50) of every $4.00 online demand to e-mail marketing (-13%). Attribute $0.28 of every $1.33 in search to the e-mail marketing (21%). Remember to allocate search expense to e-mail as well. 18

19. E-Mail Attribution Rules Attribute $0.05 of every $0.30 social media demand to e-mail marketing (17%). Attribute $0.25 of every $0.50 (50%) mobile demand to e-mail marketing (e-mail drives mobile). Attribute $0.04 of every $0.11 affiliate demand to e-mail (36%), and ($0.03) of every $0.19 display ad demand e-mail (-16%). 19

20. E-Mail Attribution Rules The results are very different from classic open/click/conversion metrics. The end result is nearly identical. The results by channel, however, are very different. E-Mail caused Search, Mobile, and Affiliate orders to happen … without e-mail, those channels wouldn’t perform as well. 20

21. Profit: Which Strategy Works Best? Once catalog expenses, e-mail expenses, and search expenses are allocated across test groups, we can identify the strategy that works best. Take a look!!!!! 21

22. Which Strategy Works Best? In this test, the most profitable strategy is essentially a tie … catalogs + e-mail, or just e- mail. When a business has a high organic percentage, matchbacks and attribution algorithms consistently allocate too many orders to marketing activities. Mail/Holdout strategies yield different results. 22

23. What Is The "Organic Percentage"? The organic percentage is possibly the most important metric a direct marketer / catalog brand can track. The organic percentage is the percentage of demand that will be generated if no marketing exists. What about our example? 23

24. The Organic Percentage: 51%???? Take the $5.80 generated in the no catalogs / no e-mail test panel, and divide it by the $11.37 generated in the catalogs + e- mail test panel. The result is 51%. 24

25. A 51% Organic Percentage We must execute catalog and e-mail mail/holdout test panels, in order to properly estimate what our organic percentage is. When the organic percentage is 40%, matchbacks and allocation programs become increasingly inaccurate. 25

26. Does The Organic Percentage Vary? If you follow my blog, you know that I am an advocate of “Digital Profiles”, segments of customers with unique buying behavior. BIG SECRET: Some customers are “highly organic”, while other customers require “large amounts of marketing”. There are HUGE profit opportunities in knowing this difference! 26

27. Digital Profile Secret! 27

28. Digital Profile Secret! Customers who mail orders to a company or use the telephone to order require advertising. Customers who combine catalogs and online channels are a “hybrid”, requiring much less advertising. Customers who order online or in stores are highly organic, you can reduce advertising! 28

29. Optimal # Of Contacts I work with many catalogers that execute 3- month and 12-month holdout tests. This is where all of the magic happens! You don’t lose demand … you learn the right number of annual contacts!!!! You test four combinations, and you can literally simulate dozens or hundreds of combinations!! 29

30. The Optimal Number Of Contacts! My clients often test four combinations (just like in our earlier example). In this case, eight catalogs a year, not twelve, yield optimal profitability. E-mail contacts appear to be optimized at around 100 per year. This strategy is different than the 12 catalogs / 100 e- mails currently executed. 30

31. Key Takeaways If you have a low organic percentage, you can probably trust your matchback/attribution algorithms. If you have a medium or high organic percentage, your matchback/attribution algorithms are allocating too many orders to your marketing activities, causing you to spend too much $$$ on marketing. 31

32. Key Takeaways Have the courage to execute both catalog mail/holdout tests and e-mail mail/holdout tests. Test four panels, test for a quarter or season or year if you can. The results are going to be breathtaking! The results will be different than what your matchback/attribution algorithms suggest. 32

33. Key Takeaways: Catalogs Most matchbacks/allocation/attribution models over-estimate the ability of catalogs to drive incremental volume. In many cases, orders that would have happened anyway are attributed to catalogs, causing us to spend way too much money mailing catalogs. 33

34. Key Takeaways: E-Mail Most matchbacks/allocation/attribution models fail to properly measure what happens when e-mail campaigns are delivered to a customer. E-Mail is frequently cannibalized by catalogs. E-Mail frequently causes Search/Mobile orders to happen. 34

35. Key Takeaways: Search Search is often the outcome of catalog marketing or e-mail marketing. Our job is to allocate search orders driven by catalog marketing and e-mail marketing (and search expense, converted and non-converted clicks) back to the catalog segment that caused search to happen. This may cause us to actually not mail certain segments!!!!! 35

36. Key Takeaways: Search Search is often the outcome of catalog marketing or e-mail marketing. Our job is to allocate search orders driven by catalog marketing and e-mail marketing (and search expense, converted and non-converted clicks) back to the catalog segment that caused search to happen. This may cause us to actually not mail certain segments!!!!! 36

37. Key Takeaways: Social/Mobile In the early stages of a channel, sales are frequently cannibalized from existing channels, or the existing channels cause the sale to happen in the new channel. Over time, new channels become “organic”, and do not require old-school channels in order to create sales on their own. Tests/Holdouts can validate this for you! 37

38. Key Conclusion It is my opinion that most of our industry is incorrectly using matchbacks/allocation/attribution models, and as a result, we spend way too much money on marketing, attributing orders that would have happened anyway to marketing channels. Run a test, and learn how you, too, can save yourself money and increase profit! 38

39. Questions? Kevin Hillstrom President, MineThatData Website: http://minethatdata.comhttp://minethatdata.com Blog: http://blog.minethatdata.comhttp://blog.minethatdata.com Twitter: http://twitter.com/minethatdatahttp://twitter.com/minethatdata E-Mail: kevinh@minethatdata.comkevinh@minethatdata.com Phone: 1-206-853-8278 39