Cost Effective Evaluation Strategies Robin Kipke Tobacco Control Evaluation Center November 19, 2009

Topics We’ll Cover Adapting existing instruments Saving time in data collection Tapping into volunteer labor Using TCEC resources

Caveat TCEC is not advocating for cutting evaluation corners that result in a decline of rigor or quality of data! Cost-cutting strategies should only be used as the context of other factors allow Always think first about what data will be convincing to your target audience

Adapt Existing Instruments Not always necessary to reinvent the wheel Adapt other data collection instruments for your project’s purposes Start with high quality instrument Add questions you need data on Delete unnecessary questions Make sure it will collect what you need

Instrument Sources TCEC website has model instruments on limited number of objectives Contact TCEC to access repository of instruments created/used by other projects Ask other projects working on similar objectives to share their instruments

Statewide Sites Download instruments from statewide sites CA Health Interview Survey – CA Healthy Kids Survey – layout=surveys layout=surveys CDC – YTPS on STORE Website –

Saving Time on Data Collection Ask to have your questions added to data collection activities being conducted by other departments  Pay attention to what’s going on within your county or agency  Find out when other projects/divisions will be collecting data  Learn which population groups will be targeted by data collection and see how that fits your needs

Adding Your Questions  Obtain a copy of the instrument to see what questions are already being asked  Insert your most important questions  Add transitions as needed to preface your question topics  Examples: County’s annual Healthy Kids survey Maternal & Child Health survey (every 5 years) Needs assessments (use of Prop. 63 funds, adult health survey)

Some Limitations Think about data sources – who will be reached/excluded? Will not be a pre-/post measure of same people, so cannot measure intervention effect Will be limited to space allowed Access to analysis – may need raw data

Saving Time on Data Collection Distribute questionnaires/materials via high traffic locations  Back-to-school packets  Church service programs  Grocery store flyers  Classroom teachers  Entertainment tickets (e.g. movie theaters)  Organizational newsletters

Other Distribution Avenues Online surveys (Survey Monkey) Invite participation via , social networking media (Facebook, Twitter) Your organization’s website (and those of coalition members, agencies that serve target population)

Other Distribution – Limitations With each distribution outlet, think about who will it reach/not reach? How to address representativeness Use screening questions to avoid duplication Loss of control over administration Potentially lower response rate

Saving Time on Data Collection When low population density makes collecting a sufficient POS sample difficult, a series of focus groups (with purposive samples to ensure representativeness) may suffice Caveat: First find out if the data will be convincing enough for your purposes

Tapping into Volunteer Labor Develop relationships with local universities, high schools, programs or clubs Recruit students looking for research experience to do data collection Offer training, internships, course credit, other incentives

Tapping into Volunteer Labor Make use of community volunteers and coalition members  To field test instruments/materials for appropriateness to data sources  To translate instruments/materials into other languages  To collect data  To provide access to other distribution points

Utilize TCEC Resources Online coalition satisfaction survey administration and analysis service Additional model data collection instruments on a variety of topics will be developed in coming months Model instruments will be translated into other languages (Spanish, Vietnamese, Hmong and Russian) depending on need We’re working with other statewide projects to develop tools that will help you use and share your data more effectively for your policy work

Contact TCEC For individualized assistance with:  evaluation planning  development of data collection instruments  advice on analyzing & interpreting data  guidelines for reporting & sharing info