Presentation is loading. Please wait.

Presentation is loading. Please wait.

Western Region Evaluation Framework Update Andy Riesenberg, MSPH Team Lead, Food Security and Obesity Prevention Supplemental Nutrition Assistance Program.

Similar presentations

Presentation on theme: "Western Region Evaluation Framework Update Andy Riesenberg, MSPH Team Lead, Food Security and Obesity Prevention Supplemental Nutrition Assistance Program."— Presentation transcript:

1 Western Region Evaluation Framework Update Andy Riesenberg, MSPH Team Lead, Food Security and Obesity Prevention Supplemental Nutrition Assistance Program Western Region

2 Feb 20152

3 Introduction

4 Community Example: Linda Vista Neighborhood, San Diego Feb 20154

5 What sets SNAP-Ed apart? 1.Focus on low-income population 2.Comprehensive strategies (nutrition education, marketing, environmental changes) 3.Community and stakeholder engagement 4.Evidence-based prevention strategies Feb 20155

6 Evaluation Framework – At a Glance Feb 20156

7 7

8 Evaluation Questions Individual ‐ level To what extent does SNAP ‐ Ed programming improve participants’ diet, physical activity, and health? Environmental ‐ level To what extent does SNAP-Ed programming facilitate access and create appeal for improved dietary and physical activity choices in the settings where nutrition education is provided? Sectors of Influence To what extent is the SNAP-Ed grant program integrated into comprehensive strategies that collectively impact lifelong healthy eating and active living in low- income communities? Social and Cultural Norms and Values To what extent do community-level obesity prevention strategies impact the public’s priorities, lifestyle choices, and values for healthy living? Feb 20158

9 Healthy People in Healthy Places in Healthy Communities Individual: People Environmental: Places Sectors of Influence: Communities Unit of Analysis Youth and AdultsSettings and NetworksLow-income Communities ReachParticipantsOrganizations and Places Populations and Jurisdictions What Gets Measured? Behavioral changes in nutrition, physical activity Adoption, Implementation, Effectiveness, Maintenance Collective Impact/ Nutrition in all policies How to Measure? Surveys Focus Groups/ Interviews Direct Observation Assessments/ Environmental Scans Direct Observation Document Review Network Analysis Data warehouses CDC/DHHS/RWJF Indicators Health registries Feb 20159

10 10 Example

11 FY ‘15 Priority (Preferred) Indicators Feb 201511

12 FY 15 Priority Indicators – Basic Individual LevelEnvironmental LevelSectors of Influence MT1: MyPlate BehaviorsST4: Identification of Opportunities MT9: Agriculture MT2: Shopping BehaviorsST6: Partnerships MT3: Physical Activity Behaviors LT2: Fruits and Vegetables LT4: Dairy LT6: Food Security Feb 201512

13 Environmental LevelSectors of Influence MT4: Nutrition Supports Adopted MT8: Local Government MT5: Physical Activity Supports Adopted MT12: Health Care MT6: Marketing and Messaging FY 15 Priority Indicators – Enhanced (States > $2 mil) Feb 201513

14 Measuring Individual Changes Feb 201514

15 MT2 Shopping Behaviors Logic Model Component Medium-Term Outcome – Changes in individual and family behaviors that reflect smarter shopping and food resource management strategies, enabling participants to stretch their food resource dollars. What to measure: Number, or %, of SNAP-Ed participants who increased targeted shopping behaviors during the period assessed. For each participant, compare responses to survey administered before the first lesson and after the last lesson. Numerator:Number of SNAP-Ed participants with a matched pre- and post-test who performed the behavior at post-test for the first time or with greater intensity or frequency. Denominator:Number of SNAP-Ed participants with a matched pre- and post-test. Examples of behaviors Read nutrition facts or nutrition ingredients lists Compare prices before buying foods Identify foods on sale or use coupons Shop with a list How to Measure Visually-Enhanced Food Behavior Checklist  Do you use this food label when shopping? Plan, Shop, Save, and Cook Survey  How often do you shop with a grocery list?  How often do you compare unit prices before buying food? Cooking Matters for Adults Survey  How often do you compare prices before you buy food?  How often do you use the “nutrition facts” on food labels? Feb 201515

16 Feb 201516 Example

17 Measuring Policy, Systems, and Environmental Changes Feb 201517

18 RE-AIM –Reach, effectiveness, adoption, implementation, maintenance –Reach, adoption, effectiveness, implementation, maintenance Feb 201518

19 REACH (MT4-5) Reach: Number of SNAP-Ed eligibles that benefitted from the change(s) during the period assessed (e.g., number of persons < 185% of Federal Poverty Level reached by the change in SNAP-Ed eligible settings). Feb 201519

20 ADOPTION (MT4-5) Adoption: Aggregate number of SNAP-Ed settings where at least one organizational or environmental change is made in writing or practice to improve or strengthen access or appeal for healthy eating or physical activity during the period assessed Feb 201520

21 Interpreting Reach and Adoption MT4: Nutrition Supports Adopted Logic Model Component Medium-Term Outcome – Reach and adoption of nutrition policies or environmental changes What to measure: The documentation of change(s) adopted in the low-income setting and associated reach Numerator: Number of settings where at least one change is made in writing or practice to expand access or improve appeal for healthy eating Denominator: Number of settings in ST4, by category How to Measure ADOPTION Documentation (direct observation, photograph) or interviews with key informants to confirm the uptake of the policy or environmental change in the low-income setting. REACH Means-tested setting The number of SNAP-Ed eligibles, based on qualification for free or reduced price meals or federal poverty level, who benefit from the change(s).  For example, if the school cafeteria adopts changes in meal service, and there are 750 students who qualify for free or reduced price meals, the reach is 750. Census tract, or census designated place The number of SNAP-Ed eligibles, based on census data and direct observation, who benefit from the change(s). For example, if there are 6,000 (out of 10,000) individuals within 185 percent of FPL in a given census tract where a new farmers market opens, and on average in the past month, there have been 350 customers at the market per shopping day, multiply 350 x.60, which equals 210. Feb 201521

22 Feb 201522 Example

23 IMPLEMENTATION (LT9) Implementation: Aggregate number of SNAP-Ed settings that report a multi-component initiative with one or more organizational or environmental changes adopted AND at least one of the following: 1) evidence-based education, 2) marketing, 3) parent/community involvement, 4) staff training on continuous program and policy implementation. Feb 201523

24 EFFECTIVENESS (LT9) Effectiveness: Number of settings with improved food or physical activity environment assessment scores using a reliable and consistent environmental assessment tool. Feb 201524

25 MAINTENANCE (I3) Maintenance: Average percentage increase, or number of institutional or community resources invested in nutrition and physical activity supports or standards at SNAP-Ed settings in terms of staff (number of full time equivalents), cash, or in-kind supports Feb 201525

26 Channels: Eat, Live, Learn, Work, Play, Shop Eat: Restaurants, mobile vending/food trucks, congregate meal sites (or, other places where people primarily go to “eat”) Live: Public housing, shelters, places of worship, community organizations, residential treatment centers, adult or senior services (or, other community or neighborhood settings where people “live” or live nearby) Learn: Child care, head start, early care and education, adult education, schools, after-school, Cooperative Extension offices (or, other places where people go to “learn”) Work: Worksites with low-wage workers5, job training programs, TANF worksites (or, other places where people go to “work”) Play: Parks and recreation, YMCA, county fairs, Boys and Girls clubs, bicycling and walking paths (or, other places where people go to “play”) Shop: Farmers markets, grocery stores, food retailers, food pantries (or other places where people “shop” for food) Feb 201526

27 Feb 201527 Number of Sites by Category of Channel NEOPB PSE Strategy EatLiveLearnWorkPlayShop Early Childcare 55 schools/ childcare centers School Wellness Policies 29 schools 23 district offices School Wellness Policy- Water Stations 1 city 23 schools Farm to School 9 district offices Joint Use Agreements 1 school Healthy Retail 70 stores Restaurant and Mobile Vending 2 vendors Structured Physical Activity 1 public housing 5 churches 2 schools Community/School Gardens 1 rehab center 4 apt complexes 18 churches 12 community sites 4 community centers 25 schools 1 park Worksite 52 worksites Safe Routes to Schools 2 neighborhoods 1 city 1 community center 14 schools 3 parks/ public land Farmers' Markets 16 markets 4 produce stands 1 business facility 1 public land Healthy Food and Beverage Standards 5 emergency food28 churches 2 community centers 4 community-based organizations 3 WIC programs 2 cities 1 school 1 park 1 store

28 Next Steps Selecting priority indicators for FY 16 Developing instructions and reporting book for priority indicators Developing interpretive guide for 51 indicators Ongoing collaboration with ASNNA Eval Sub-committee, Regional Centers of Excellence, NIFA, CDC, & HRSA for alignment of indicators Feb 201528

29 Evaluation References CDC. State indicator report on fruits and vegetables. ‐ Indicator ‐ Report ‐ Fruits ‐ Vegetables ‐ 2013.pdf CDC. State indicator report on physical activity. CDC. Recommended community strategies and measurements to prevent obesity in the United States. (COCOMO) Leeman et al. An evaluation framework for obesity prevention policy interventions. Prev Chronic Dis 2012;9:110322. Available at: NIFA. Community Nutrition Education Logic Model. Center for Training and Research Translation Web site. Available at: Altarum Institute and RTI International. Cates, S., Blitstein, J., Hersey, J., Kosa, K., Flicker, L., Morgan, K., and Bell, L. Addressing the Challenges of Conducting Effective Supplemental Nutrition Assistance Program Education (SNAP-Ed) Evaluations: A Step-by-Step Guide. Prepared by Altarum Institute and RTI International for the U.S. Department of Agriculture, Food and Nutrition Service, March 2014. Available at: RE-AIM Framework. Healthy People 2020. Feb 201529

30 Thank you! (415)645-1927 Feb 201530

Download ppt "Western Region Evaluation Framework Update Andy Riesenberg, MSPH Team Lead, Food Security and Obesity Prevention Supplemental Nutrition Assistance Program."

Similar presentations

Ads by Google