Why Indicators Matter Using Outcome Indicators to Plan and Evaluate Comprehensive Tobacco Control Programs Todd Rogers, PhD California Tobacco Control Program, CX Webinar July 9, 2009

Plan for this Presentation Demonstrate the importance of indicators for: – Planning and evaluating tobacco control programs – Monitoring the national tobacco control effort – Advocating for policy support and additional resources

Specific, observable, and measurable characteristics that shows progress a program is making toward achieving a specified outcome What is an indicator? Measuring Program Outcomes: A Practical Approach. Alexandria, VA: United Way of America; 1996

Characteristics of a “Good” Indicator Relevant – important/useful information Affordable – requires reasonable resources Valid – Appropriate – meaningful for intended use – Understandable – makes sense Reliable – minimal measurement error

Inputs Activities Outputs Outcomes Context Simplified Program Logic Model

Focus on Program Outcomes Outcomes Completed efforts to disseminate anti-tobacco/pro-health messages Completed efforts to reduce and counteract pro-tobacco messages Inputs Activities Completed efforts to increase tobacco curricula and tobacco free policies Outputs Reduced initiation among youth Reduced youth smoking prevalence School- based prevention Policy and Regulatory Action Targeted to Disparate Populations 3 Decreased tobacco- related disparity Reduced tobacco- related morbidity and mortality Completed efforts to increase restriction and enforcement of tobacco sales to minors Completed efforts to increase cigarette excise tax 5 4 Community mobilization Counter marketing Long-term State Health Department and Partners Reduced susceptibility to experimentation Decreased access Increased price on cigarettes through tax Increased anti-tobacco policies and programs in schools Increased in anti- tobacco attitudes, support for policies to reduce youth initiation 7 6 IntermediateShort-term Reduced tobacco industry influences Increased restriction, enforcement on restrictions on tobacco sales to minors

Outcome Component 9 Reduced tobacco industry influences Indicator Extent of tobacco advertising outside stores

Indicators Drive Program Planning Develop program outcome objectives in the context of logical model “pathways” and associated outcome indicators Integrate program and evaluation planning

Indicators Shape Program Objectives From: Local Program Evaluation Planning Guide (December 2004) California Department of Health Services, Tobacco Control Section

Local Evaluation of a “Pathway” Does passage of SHS policies lead to: – Increased enforcement of policies? – Increased compliance with policies? – Reductions in reported exposure to SHS? Weber MD, Bagwell DAS, Fielding JE, Glantz SA. Long term compliance with California’s Smoke-Free Workplace Law among bars and restaurants in Los Angeles County. Tobacco Control 2003;12:

Although CX was an Early Innovation in State Tobacco Control Planning and Evaluation …

… There are Other National Efforts to Track Key Outcome Indicators for Tobacco Control …

… and Indicators are Being Developed for Many Other Public Health Issues

Indicators Are Widely Used to Track State and National Progress in Tobacco Control

CDC OSH Indicator Utilization by Funded State Tobacco Control Programs ( ) InitiationSHSCessationTotal AvailUsedAvailUsedAvailUsedAvailUsed Short Inter Long Total TM

CDC OSH Indicators Not Being Utilized by Funded State Programs ( ) IndicatorType Level of awareness among parents about the importance of discussing tobacco use with their children Init ST Proportion of jurisdictions with policies that control the location, number and density of retail outlets Init ST Proportion of jurisdictions with policies that regulate the extent of tobacco advertising outside of stores Init ST Extent of tobacco advertising in print mediaInit ST Number and type of MSA violations by tobacco companiesInit ST Amount of tobacco industry campaign contributions to local and state politicians Init ST Proportion of young people reporting that they purchased cigarettes from a vending machine Init IT TMTM

Of 120 KOIs named by CDC, CTCP lacks data on only 22 indicators (18%): – Youth initiation – 6/54 missing (11%) – Secondhand smoke – 4/32 missing (13%) – Cessation – 12/34 missing (35%), especially on health care provider and insurance-related indicators CTCP Evaluation Indicators Overlap Substantially with CDC OSH Indicators

State Indicator Tracking Motivates Policy and Media Advocacy Actions

Indicator Data Presented by CTFK in State-Specific Press Releases

Summary Indicators help your planning and evaluation efforts Indicators are used to monitor state-specific and national progress Indicator data are used to advocate for poicy change and program support Indicators matter