1. Enhanced Implementation of CDC Guidelines for School Tobacco Programs: Results of a Statewide Evaluation Shelly A. Greller, MS Wisconsin Department of Public Instruction D. Paul Moberg, Ph.D University of Wisconsin – Madison

2. Acknowledgements Wisconsin Department of Public Instruction –John Hisgen –Douglas White University of WI – Monitoring & Evaluation Program (www.medsch.wisc.edu/mep) –David Ahrens –Ann Christiansen –Amy Anderson –Patrick Remington, P.I. Funded by the Wisconsin Tobacco Control Board

3. Participants should be able to: Describe WI statewide school tobacco grant program Describe CDC Guidelines for School Health Programs to Prevent Tobacco Use and Addiction Identify effective program resources and strategies Understand & apply results of WI school tobacco program to enhance implementation of CDC Guidelines Apply assessment & evaluation tool to operationalize CDC Guidelines

4. Background Master Settlement Agreement led to creation of WI Tobacco Control Board (WTCB) WTCB developed comprehensive strategic plan for tobacco control –One focal area was youth tobacco prevalence –Youth goal: By 2005, tobacco use among middle & high school age youth will decline by 20% School-based programs funded for $1,250,000 by competitive process through DPI

5. Background DPI stated purpose for school-based tobacco control programs –“Create or expand upon strategies identified as most promising or effective in reducing or eliminating youth tobacco use” Operationally, discussion focused on increased implementation of CDC Guidelines

6. Monitoring and Evaluation Monitoring and Evaluation Program (MEP) contracted to perform monitoring/evaluation functions for WTCB MEP comprised of –UW – Comprehensive Cancer Center –UW – Extension –Center for Health Policy and Program Evaluation MEP collaborated with DPI on evaluation of school-based tobacco control programs

7. CDC Guidelines Guidelines for School Health Programs to Prevent Tobacco Use and Addiction (MMWR, 1994) –Summarizes school based strategies “most likely to be effective in preventing tobacco use by youth” –Developed by CDC in collaboration with experts –In-depth review of research, theory & current practice

8. Impact of Guidelines Rohde et al. (2001 in MMWR) –High or medium implementation of Guidelines in Oregon associated with significantly greater decline in 8 th graders’ 30-day smoking prevalence

9. Impact of Guidelines Rohrbach et al. (2002 Presentation at Society for Prevention Research-SPR) –2 waves of student data from sample of schools –Indexed Guideline Implementation based on teacher surveys –Most schools did not fully implement comprehensive program of evidence-based strategies –Guideline implementation significantly related to smoking prevalence - quit attempts - negative expectations/attitudes regarding tobacco

10. Impact of Guidelines Hallfors & Godette (2002, HER) –Dept. of Education has recent statement of “Principles of Effectiveness” for prevention programs –Surveyed 104 School Districts in 12 states –Only 19% of school districts are implementing research-based prevention curricula with “fidelity”

11. Goal of Our Evaluation Focus on institutional change in implementation of the guidelines in schools with infusion of targeted funding Not on impact of implementation of Guidelines on student outcomes

12. Methods of Our Evaluation 2001-’022002-’03 0$$$$0$$0 00[0?] 0 = Self Assessment Form –School Building $$$$ = Funded District Design Sample: Applicant School Buildings/Districts

13. Measures – Self Assessment Each building in applicant district completed checklist operationalizing CDC Guidelines –Operationalizes CDC Guidelines with 58 questions –Original purpose was self assessment and planning –Program evaluation secondary Repeated at end of year 1 and year 2 –Funded districts submitted as part of year end report –Unfunded districts mailed as survey at end of year 1

14. Scaling/Data Reduction Exploratory Principal Components factor analysis indicated most items within areas of the guidelines were correlated We created additive indices for each of the 7 areas Scaling 0 = No 1 = Somewhat 2 = Yes

15. Scale Validation 2001-2002 School Health Education Profile (SHEP) oversampled to include all funded schools Reports from SHEP will be correlated with reports from schools Compared baseline data of funded schools to full SHEP sample –How representative are they?

16. Scale Construction Scale# ItemsMeanS.D.Alpha School Policies171.550.33.82 Curriculum171.290.48.95 Instruction60.980.42.59 Training40.540.58.77 Family & community involvement 70.660.47.79 Cessation Services30.410.52.62 Evaluation40.410.49.84 Note: Reported for baseline data, n=320. Means in terms of original metric: 0=No, 1=Somewhat, 2=Yes

17. Sample Size & Response Rate AssessmentNot funded (Comparison) FundedTotal Baseline154180334 Follow-Up*61 (40%)155 (86%)216 (65%) *27 additional buildings submitted follow-ups without baselines Buildings represent 98 Districts (42 funded) and 11 Consortia (4 funded)

18. Mean Change During Year 1 VariableBaselineOne YearDifferenceT-Statistic Policy Funded1.541.72.1777.15** Unfunded1.661.78.1253.29* Curriculum Funded1.191.49.2967.87** Unfunded1.491.65.1613.03* Instruction Funded0.991.28.2878.58** Unfunded1.491.65.1342.32* *P <.05**P <.001

19. Mean Change During Year 1 VariableBaselineOne YearDifferenceT-Statistic Training Funded0.600.99.3907.23** Unfunded0.630.85.2152.49* Family / Community Involvement Funded0.681.04.3659.38** Unfunded0.700.89.1912.42* Cessation Funded0.420.90.4819.08** Unfunded0.450.67.2182.68* *P <.05**P <.001

20. Mean Change During Year 1 VariableBaselineOne YearDifferenceT-Statistic Evaluation Funded0.450.88.4227.55** Unfunded0.500.79.2892.74* Overall Funded1.081.38.29411.5** Unfunded1.231.41.179 4.5** *P <.05**P <.001

21. Regression Results Regression analysis covarying baseline value found: Baseline covariate always a sig. predictor of one year implementation measure Significant effects of funding on –Training: B =.155 (.084) –Family / community involvement: B =.162 (.072) –Cessation: B =.25 (.09)

22. Percentage of Funded Schools Reporting Positive Change Any Positive Change +.25 SD Pos. Change # Schools Reporting Policy66.9%55.0%151 Instruction66.264.1142 Curriculum66.753.9141 Training61.9 134 Parent/community74.661.2126 Cessation63.3 147 Evaluation59.7 134 Changed on any90.389.0155

23. Funded Schools Prohibit Tobacco Use…

24. Funded School Policies…

25. Funded Schools Tobacco Instruction…

26. Rather Than Punitive – Funded Schools…

27. Funded Schools Teach Developmentally Appropriate

29. Does Funded School..

31. Variation by Level – Funded Schools Elementary schools (57 schools) –High on baseline family & community involvement –Lowest on evaluation Middle schools (52 schools) –Highest overall score at baseline, especially in training and curriculum High schools (30 schools) –More change (than elementary & middle schools) on policy, training, parent & community involvement & cessation –Highest initially on cessation Combinations (16 schools) –Highest level of change overall: high change on policy, curriculum, and instruction

32. Limitations and Issues Tool originally intended as self assessment –Not a research tool –Validity/reliability to be established SHEP link will help – in progress Self Report from school staff (who want funding to continue) without validation Low response rate from comparison group No student outcomes

33. Conclusions Baseline implementation of Guidelines highest in areas of –Policy –Curriculum Implementation of Guidelines lowest in –Cessation –Evaluation –Training –Family / community involvement

34. Conclusions Most gain with funding in –Cessation –Evaluation –Training –Family / community involvement Unfunded schools also report gain in all areas-- but significantly less in several areas The tool is promising to assess school tobacco control programs both for planning & evaluation