1. Youth Tobacco Evaluation Project Virginia Commonwealth University

2. The YTEP Team Elizabeth Fries, PhD, Principal Investigator
Ilene Speizer, PhD
Diane Baer Wilson, EdD
Brian Smith, PhD
Donna McClish, PhD
Samy Uguy, PhD
Patricia Obando, PhD
Karen Mitchell
Melanie Bean
Joyce Phillips
Panumas Assavarakpreecha

3. Youth Tobacco Evaluation Project Virginia Commonwealth University
Regional Training
Part 1: Progress & Report on Data from Round 1

4. Youth Tobacco Evaluation Project: Our Mission
Funded by VTSF, our mission has been to design and implement a comprehensive evaluation of state-wide tobacco prevention programming
Determine impact of VTSF funding on youth participants
Compare various compendium programs by setting and delivery
Our mission from the beginning
Specifically, we are responsible for
Evaluating overall funding of VTSF programs, that is, determining the utility of this statewide funding. Have the youth participating in the programs been affected.
Evaluate which program are more or less successful in varying settings at affecting youth knowledge, attitudes, skills, and behaviors. When possible, we will compare the same programs in different settings
Examining the impacts of programs by program fidelity.

5. Progress to Date For the first round of data collection
Designed outcome and process evaluations
Developed all necessary tools for evaluation
Designed, organized, and trained all grantees funded by VTSF to implement the evaluation
Worked with grantees to ensure that all necessary tools for the evaluation are provided
Receiving completed tools
Data are logged, entered, and analyzed
First round of data collection (year 1 of evaluation tools)
Second round of data collection, beginning July 1, 2003

6. What We Have Received From You…
Instructor surveys: 500
Logs (session and program): 634
Parent surveys: 596
Parent & Family Participant Survey: 76
Brief Participant Survey (4th-5th): 2981
Outcome surveys (6th-12th)
Baseline: 14787
Follow-up: 9232

7. Presentation Today … Presentation of process information
Instructor surveys
Presentation of outcome information
Baseline surveys
Comments from participants

8. Instructor Surveys Obtains information from instructors on:
Fidelity
Ease of delivery
Perceptions of curriculum by instructor
Post-implementation survey completed by participating instructors
Using 287 surveys entered and cleaned
Presentation focuses on compendium programs that have at least 14 instructor surveys returned

9. Instructor Surveys by Compendium Program (n=287)

10. Locations where Programs are Delivered (Instructor Surveys, N=287)
Added scriptural texts to each lesson as appropriate (faith-based organization)

11. Delivery Setting by Compendium Program
Program is made for single-grade, classroom setting rather than a multi-grade after-school setting (Here’s Looking at You)

12. Fidelity of Program Delivery – Number of Sessions

13. Fidelity of Program Delivery – Minutes per Session
I deleted some of the movement activities, not enough space in the classroom (Positive Action)

14. Target Grade of Participating Youth (reported by instructor)

15. Program was Age-Appropriate

16. Program was Well Received
I loved it!! And I think the kids really enjoyed it too (TATU)

17. Youth Understood Program

18. Participants Were Enthusiastic

19. Youth were Attentive
Stories are too long. Some words and stories students can’t relate with. Some are not entertaining and don’t keep students’ attention (Positive Action)

20. Adequacy of Materials
LST is not “active” enough. Writing in a workbook is not “active” for a 6th grader

21. A Lot of Teaching Effort was Required

22. Presented as Prescribed by Curriculum

23. Program was Effective
It’s (the program) is exactly what my students need at this age. I think that by building up their self-concept, they make better life choices. (Positive Action)

24. Instructor Would Recommend Program

25. Preliminary Data from Baseline Surveys
Based on 14,557 baseline surveys entered and cleaned
Caveats:
Most data from 6-8th graders – emphasis of presentation
Bigger programs contribute more information
Not representative of VA, Regions, or sites
Data presented here are based on the first 12,800 surveys that were entered and cleaned. We currently have another 2000 pre-test surveys being entered and cleaned.
No post-test data are being presented yet as only a small proportion of programs have sent this in thus far.

26. Thoughts from Participating Youth
40% of youth gave a comment on the survey
The majority of the comments were positive or neutral (55%)
A third (35%) were negative
Some (9%) were both positive and negative
Collected thoughts from sample of participants, includes all thoughts from selected folders (all regions, all grades)

27. Thoughts from Participating Youth
It was Kool
I felt good about it
I felt this survey might help to figure out a quick way to not have people smoke
It was very interesting and it asks a lot of questions. But it prevents you from smoking
I felt that I needed to do this where you get answers to help kids quit smoking
I think it was very good to get opinions from the student
It helped me get some stuff out
I really felt confident taking it

28. I think it is a very good program to help kids to stop smoking and there should be other programs like it in school
It’s an okay idea
I think it is a good way to help people prevent smoking
I think this was a nice survey it makes you think a lot
I feel the questions are very appropriate for all ages
I think it was alright. I mean you all need to know how we feel about drugs
I think it was a little too much information but it could help certain things
It will show how people do stuff so we can improve
I don’t understand what hair/eye color have to do with smoking but I feel good about the fact that things are being done

29. I thought the survey was boring, some of the questions do not apply to kids my age
It was personal and I didn’t like it
It was hard and a lot of questions
I felt it was a little embarrassing answering the questions
Didn’t like it!!
Dumb, no interests to my knowledge. You don’t need to know our eye color
It’s too long
It was long and boring
Is too long
This was stupid, if kids want to smoke or dip let them
I thought the smoking questions were crazy too much
It was stupid and too personal for an anonymous survey. I hate this survey it was very dumb. It took up too much of my time and it asked the same questions I answer every single time. It wasted too many trees. Trees are cool.

30. I thought it was way too long and too many personal questions about our parents and ourselves
You repeat the same things too many times
I didn’t feel very good because I just want my family to know and not people I don’t know
It was kind of personal to me
I like it in a way but I do not like private part about it
I think you shouldn’t ask dumb questions like this most
people I know don’t do drugs and I stay away from the ones that do
Was okay (but boring)
It was alright but it was long. It seemed repetitive
I feel that a company that sells a legal product shouldn’t have to pay money to trash its own product
It was ok. I don’t smoke so I really didn’t care

31. Baseline Data from the YTEP Evaluation: Ethnicity (N=10,950, 6th – 8th grades only)

32. Baseline Data from the YTEP Evaluation: Gender (N=10,950, 6th – 8th grades only)
I felt at times it was a little confusing, but it really made me think. I don’t think I’ve ever worked harder

33. Participation in Tobacco Prevention Programs in Last Year

34. Number of Grantees Implementing Compendium Programs (not complete list)

35. Smoking Experience (all grades, N’s small for 10th - 12th grades)

36. Snuff Use by Grade (male youth only, all grades, N’s small for 10th - 12th grades)

37. Try Smoking in Next 6 Months (N=10,950, 6th – 8th grades only)

38. I Could Avoid Smoking (N=10,950, 6th – 8th grades only)

39. Cigarettes are Addictive (N=10,950, 6th – 8th grades only)
“I felt very good because knowing that sometimes smoking was on my mind that I knew [what] the consequences of smoking was”

40. Report Cigarettes Easy to Get (N=10,950, 6th – 8th grades only)
… its my body and business what I do. If tobacco is out there I can get it if I want to no matter what age

41. People Smoke Because Family Members Smoke (N=10,950, 6th – 8th grades only)

42. People Smoke Because Friends Smoke (N=10,950, 6th – 8th grades only)

43. If I Stay Tobacco Free, I Will be Healthier (N=10,950, 6th – 8th grades only)

44. Thoughts about Your Data
Provided baseline data to programs participating with 20 or more youth in baseline
Representative of your program, assuming reasonable participation
The data are not meant to be representative of youth in the region, community, or school surveyed

45. What Will We Do With Data
Entering follow-up survey
Match baseline and follow-up samples
Provide back to programs – tables on follow-up (with some impact vars.)
Compare responses from baseline and follow-up
Link instructor information on program delivery to youth responses
Determine impact of programs on outcomes
Examine process results for K-5th grade programs
. . .