# Analyzing your Survey Data: The Impact of the Campaign.

## Presentation on theme: "Analyzing your Survey Data: The Impact of the Campaign."— Presentation transcript:

Analyzing your Survey Data: The Impact of the Campaign

KAP Data analysis Part 2: BR, BC, TR and CR

By the end of this lesson you will be able to: Analyze your data according to your analytical plan (BR, BC, TR, CR) Analyze your data based on your reporting needs Complete Sections 3.4 B – D of your Campaign Learning Report that identify the impact of your campaign along the Theory of Change

Structure for the Day Check in9:30 – 10:00 AM Individual Work time10:00 – 12:00 Check in1:30 – 2:00 PM Individual Work time2:00 – 3:00 Check in/Assignment3:00 – 3:30 4

Assessing Impact Provides an overview of the results of the campaign along the: – BR SMART Objectives – BC SMART Objectives and – TR SMART Objectives – CR SMART Objectives

Flow/Guidelines 1.Review of your KAP Monitoring Plan (BR, BC, TR, CR) and Hypothesis 2.Complete the tables in your section 3.4 per Target Audience 3.Write a short narrative after each table describing how well your campaign achieved its SMART Objectives 4.Assess the impact of your campaign along TOC

Analyze Data: Statistics clarify truth Show critical data in body of report (SMART Objectives) –Tables –Charts Show all your data in tables in Appendix: –Honest thing to do –May show you where your campaign failed so you can fix it in future!

Analyze Data: Comparability of Surveys Best: –All Chi-Square tests < 95% OK: –A few Chi-Square tests ≥ 95% –Differences in frequencies small (5 to 10 percentage points) So-so: –Several Chi-Square tests ≥ 95% –Differences in frequencies large (10 to 20 percentage points) Unacceptable: –Many Chi-Square tests ≥ 95% –Differences in frequencies large (> 20 percentage points)

Worse Case Scenario Assume you have large differences in gender: –Baseline = 35% male –Post-campaign = 56% male –Difference of 21 percentage points Problem: If a dependent variable increases by 15 pp, it could be due to 2 things: –Pride campaign impact –More men in 2 nd sample Solution is to “control” for gender using filters: –Filter for men, run analysis –Filter for women, run analysis

Reminder # 1 For BR, BC, TR and CR, KAP SMART objectives are based on self-reported data. Triangulate results with non-KAP metrics

Analyzing BR Example: BR KAP Question (36) I am going to read you a number of statements about the management of the local no-take area. For each statement, I would like you to tell me if you strongly agree, agree, disagree, or strongly disagree with it. (E) There is enough money and other resources to fully manage and enforce the rules of the no-take area [ ] SA [ ] A [ ] D [ ] SD [ ] NS/DK (G) The rules of the no-take area are unclear and local fishers don't understand them [ ] SA [ ] A [ ] D [ ] SD [ ] NS/DK BR: Non-KAP Measure?

Example: BC Question (43)During the past 6 months, would you say that you have been regularly involved, occasionally involved, or not involved with the creation and/or the management of a no- take fishing area in your local area (A)[ ] Regularly involved [ ] Occasionally involved [ ] Never involved [ ] Don't know / not applicable Analyzing BC BC: Non-KAP Measure?

Example: TR Question (41)I am going to read you a list of different types of fishers, and for each one, I would like you to tell me whether you remember seeing someone like that fishing in this area in the past 6 months (show the NTZ on a map of the area but don't mention whether it is NTZ or not) (A) Subsistence fishers from your village [ ] Seen [ ] Not seen [ ] Not sure / Don't remember (B) Subsistence fishers from nearby villages [ ] Seen [ ] Not seen [ ] Not sure / Don't remember Analyzing TR TR: Non-KAP Measure?

Example: CR Question Has your catch increased, decreased or stayed the same as a result of the Lola Marine Sanctuary? (If the person does not fish or glean mark as NA) [ ] Decreased [ ] Increased [ ] Stayed the Same [ ] N/A Analyzing CR CR: Non-KAP Measure?

Structured Time: KAP monitoring Plan data survey template Proceed to section 3.4A in your campaign learning report SMART Objective ToC Category Metric (KAP or non- KAP) Pre- campai gn result Pre- campaign frequency error (if KAP) Tar get Post- campaig n result Post- campaig n frequenc y error (if KAP) Chi- squared significan ce (if KAP) Change (in pp if applicabl e) Knowledge Attitude Interpersonal Communicatio n

Reminder # 2 ToCSMART ObjectivesMetricMethod Baseline (Pre- campaign) Result (Post- campaign ) Change (in percent age points) Chi- Square (X 2 ) Significan ce SMART Objective Attainme nt Interpersonal communication GENERAL COMMUNITY By September 2010, 25 % of Southern Residents who do not hunt will have spoken with someone about wildland fires in the past 6 months (a 10 percentage point increase from 15%; N=237). Q28: In the past 6 months, have you talked to anyone about wildland fires? KAP Survey Analysis 153419 >95 significant 190% 1.GENERAL COMMUNITY (NON-HUNTERS)

Reminder # 2 Use the exact SMART objectives language Ex. Strongly agree is not the same with agree SMART Objective TA1 - Fishers The number of Burgos/Uba local fishers who says strongly agree that the Burgos Marine Sanctuary regulations need to be followed will increase from 86.9% measured in February, 2011 to 89.9% in July, 2012 ( an increase of 3pp, Q30F in the KAP survey) SurveyPro result

Scenario There is an increase in reported BC but the SMART Objective target are not met. WHY?

Could it be how you set your SMART target? Historic data Baseline - Diffusion of innovation Type of audience  Increase/Decrease  Maintain

The potential for change for different types of objectives across the Theory of Change 22 CriteriaKAICBC Average Percent of Target Audience Changed 22pp13pp28pp14pp Sample Size (n of campaigns) 2131394245 Historic Data Pride campaign average results summary till 2010

Baseline and diffusion of innovation According to Diffusions of Innovation the Rate of Change Depends on the Starting Point Innovators – 2.5% Early Adopters – 13.5% The Late Majority – 34% Laggards – 16% The Early Majority – 34% Source: Everett Rogers, graph from Wikipedia.org

The potential of change for different types of audiences 24 Baseline\Target Audience (data based on median of knowledge) General Public Influencer Resource User <20%20pp24pp1.7pp 20% to 40%23pp37pp25pp 40% to 60%31pp33pp21pp >60%17pp8pp16pp Selective perception(Hassinger) – people who don’t “want” to know don’t seem to learn. The critical mass phenomenon / social norms

3. Using right words for accuracy The SMART objective should use the same words as the question & as the answer option used

Reminder # 3 Check if you use the right filter. The latest appears in the dropdown menu. Take note of the code of the filters you use.

Stages of behavior question Can it be used to identify the current stage of behavior?

Reminder # 5 Try other figures beside using tables When you are done with your analysis, do a publish report

Reminder # 6 When you are done with your analysis, do a publish report