Creating Observation Forms for Collecting Data An evaluation webinar hosted by the Tobacco Control Evaluation Center Presenter: Robin Kipke, Evaluation Associate April 30, 2009

Agenda  Questions to ask  Factors to consider  Examples  Sample forms  Training tips  Resources

End Use Strategizing Think about:  What you are trying to achieve  Why it’s important to do so  Who the data should convince/inform  What you want to observe  Where to observe  How many resources will be available  What logistical factors to consider  What other conditions could influence data

What are you trying to achieve?  What opinion, belief or action do you want to influence, change or prompt?  Why is this important?

 Document the extent of the problem (assess the need for a policy)  Assess change over time (using a baseline and follow up measure)  Measure the implementation of, compliance with, or enforcement of an existing policy How will observation data contribute to that effort?

Who should the data convince/inform?  Policymakers  Public officials  Owners/managers  The public  The media  Other stakeholders  Your program staff

What type and quantity of data will be convincing/informative to them?  Types of evidence  Diversity of observation sites  Adequate sample size  Documented and valid data collection

What do you want to observe?  Smoking behavior (number of people smoking)  Physical evidence of smoking (tobacco litter, smell of smoke, ashtrays)  Signage  Tobacco advertising  Illegal youth purchase

Where should the observation be conducted?  How large is the site?  What areas can reasonably be covered?  What specific locations/settings at the site make the most sense to observe?  Where will observers have access?  Map the site and create a grid Identify and code sections Include fields for codes, descriptions, drawings which describe the precise location

When should observations be conducted?  Timing—when is the best time to observe? What time of year, month, week, day?  At intervals or a specific point in time?  Frequency—how often?  Duration—how long should the area be observed? As a scan or for a specific period of time?

How many resources can be allocated?  How many trained data collectors will be available to conduct observations?  How much time do you have to collect data (overall, and at each observation)?  How large of a sample will you need?

How should the sample be determined?  Total number of observations OR a specific number of observations from a particular location/time?  How to determine which sites and/or sectors of the grid to observe?

Other factors to consider  Time of year  Weather/temperature  Access to the location  Use of the site  Competing events

Documenting your logic  Create a data cover sheet Who is conducting the observations  Create codebook of organizations, individuals Record date, day and time Identify the site  Describe how to get to sites (cross streets, landmarks, other visual clues) Make a map of the site divided into grids  Identify probable observation points  Create codebook of grids, observation points Record date/time of last clean up/maintenance Document logic behind timing, site, grids

Creating data fields on your form  Make it easy on data collectors  Use data cover sheets, codes  Cluster information on form w headings  Pre-determine likely pieces of data  Use check boxes, circle choices  Include general observation field for comments

Training data collectors  Define all terms and parameters (e.g., does litter include cigarette pack wrapper or not; is someone smoking if they’re holding an unlit cigarette?  Demonstrate how to define the grid area, set observation locations, describe & record locations, scan an area, count and record data, take measurements, etc.  Have all team members practice each component of observation  Assess competency of ea data collector

Observation resources from TCEC Contact TCEC’s evaluation associates for:  Help w end use strategizing about your observation activity  Individualized feedback on drafts of your observation forms and protocols  Sample forms used by other projects  On our website, view a mini training on conducting observations  Review a recording of this webinar  Download end use strategizing tool

Contact us Robin Kipke ― Main office ―