# Business Intelligence and Decision Modeling Week 12 Campaign Management: Testing & Performance.

## Presentation on theme: "Business Intelligence and Decision Modeling Week 12 Campaign Management: Testing & Performance."— Presentation transcript:

Business Intelligence and Decision Modeling Week 12 Campaign Management: Testing & Performance

Outline Testing and Control Groups (Chap. 14) Campaign Performance Measurement (Chap. 12) Analytics and Modeling Revisited (Chap. 13)

Testing for Effectiveness Copy (communications) Execution (creative) Package bundles and offers Clickstreams (email and websites) Cross-channels

Topics in Testing Methodology Sample size Control group or benchmark Test groups set aside for time length Half-life …

Testing Methodology Half-life 100% 50% 33% 25% Response Time

Testing Methodology: CI for a given sample P +/- CI or P +/- Z x SE p or P +/- Z √(pq/n)

Testing Methodology: CI for a given sample CI = +/- Z x √(pq/n) If p =.04, n = 10,200 and Z = 1.96 CI = 1.96 √(.04 x.96/ 10,200) CI =.0038 Thus UCL =.0438; LCL =.0362 Two-tail test! Note: CL confidence level (Z), CI confidence interval (+/-), UCL and LCL Upper and Lower confidence limits

Two-tail Test for CL.025 Z = 1.96 for 95% CL.04.0438.0362 -.00368.00368 +/- 2 SD

Testing Methodology: Sample size for a given CL If CI = Z x √(pq/n) Then CI = Z x √(pq/ √ n √n = Z x √(pq) / CI If p =.04, Z = 1.96 and CI =.0038 √n = 1.96 √(.04 x.96)/.0038 √n = 101 or n = 10,215 Two-tail test!

One-tail test for CL

Testing Methodology: CI for a given sample CI = +/- Z x √(pq/n) If p =.04, n = 10,200 and Z = 1.64 CI = 1.64 √(.04 x.96/ 10,200) CI =.003182 Thus UCL =.043182 One-tail test! Note: CL confidence level (Z), CI confidence interval (+/-), UCL and LCL Upper and Lower confidence limits

One-tail Test for Sample.05 Z = 1.64 for 95% CL.04.043182.00318

Testing Methodology: Sample size for a given CL If CI = Z x √(pq/n) Then CI = Z x √(pq/ √ n √n = Z x √(pq) / CI If p =.04, Z = 1.64 and CI =.0038 √n = 1.64 √(.04 x.96)/.0038 √n = 84.57 or n = 7,152 One-tail test!

Test against Control One single item variable: 2 Groups SPSS DM Control Package Test SPSS Independent Sample T-test Also available on Excel One single item variable: 3+ groups SPSS Compare means Oneway ANOVA

Testing Interactive Marketing Treatments Two or more variables: experimental design Two-way ANOVA (Excel and SPSS)

Interactive Marketing Campaign Effectiveness CPM, RPM, OPM (Cost, Revenue and Orders per M or thousand) CPO (Cost per order) CPR (Cost per Response)

Cost Measures CPM = (Cost / Volume) * 1000 (19 000\$ / 50 000) * 1000 = 380\$ CPM media ((Media + Insertion)/Circulation) * 1000 ((40 000\$ + (10 * 1 000))/ 1 000 000) * 1000 = 60\$ (Insertions are charged in CPM)

Response Measures Response in % = (Response / Quant.) * 100 RPM = (Response / Quant.)*1000 OPM = (Orders/Quant.)*1000)

Email and Web Analytics

e-commerce/emails Campaign Effectiveness Delivered Click to open (CTO) Unique click / Click through rate (CTR) Cost per impression (CPI) Paid per click (PPC) Cost per action (CPA) Cost per lead (CPL)

Clickstream Analysis SearchPreference Edit profile Welcome Main message Download Offer Discount Offer Second subject/ offer Links/ DirectoryViral information

Clickstream Analysis

Email Performance NameEmail 1Email 2 Mailed 1,172,839 Delivered 1,129,444 1,138,827 Del%96.30%97.10% Opened 242,830 268,763 Open%21.50%23.60% Unique Click 44,681 54,559 CTR18.40%20.30% Conversion 1,385 1,582 Conv. Rate3.10%2.90% Tot. Sales\$169,551\$190,498 \$ per Del\$0.15\$0.17 Avg. Order\$122.42\$120.42 Mailing Costs\$10,555.55 Cost per Open\$0.043\$0.039 Cost per Click\$0.236\$0.193 Cost per Sale\$7.621\$6.672