1. Agenda: Surveys –In-person, Paper, telephone, and web –Computer assisted –Instrument Design Administrative data WorkFirst Longitudinal Study Hands-on surveys

2. Constructing qualitative quantitative instruments: Identify research question you are addressing Design sample and timing of collection Borrow or craft questions and protocols –Previous surveys for organization –Previous evaluations –Academic literature Get input on instrument from diverse set of stakeholders Pre-test instrument on diverse group of subjects; perhaps usability study

3. Survey Basics: Surveys are hard to do well Mode will depend on how you can best get to people –In-person, paper, telephone, computer assisted, web –How can you best access sample? How are they most likely to respond? –Computer assisted helps with complex questions or skip patterns or sensitive questions (in-person) Resources required for modes are dramatically different Know what information the respondents will have (and don’t ask about info they don’t have)! Collection, analysis, and communication are critical processes and can be collaborative

4. Instrument construction: Start with topic outline—tie to research questions Question order and grouping matters (interest, sensitivity, answers) Skip patterns can shorten survey and keep relevant Question wording must be clear : –Time reference period (last 12 months vs last calendar year) –Specific definitions (e.g., family income vs. individual earnings) –Ask only about one issue per question Response options are critical: –Match options to how you need to use data –Label categories clearly for respondents (not data coders!)

5. Using Administrative Data: Know what you want to measure Know what is in the administrative data: –Who is included? –Over what time period? –How is information collected? –How have the collection or definitions changed over time? How to find out: –Is there a codebook or data input manual? –Can you talk to someone who inputs or analyzes the data? [Watch them work!] –Do you know how the data are currently used (gives clues)? Understand and work with the limitations of the data!

6. WorkFirst Study: 2000-2004 Longitudinal study to describe family dynamics Multiple cohorts: 1999, 2000, 2001, 2002 Telephone surveys: family and individual characteristics and outcomes Administrative records: TANF, food stamps, support services, UI earnings records Topics: Employment, Public Benefits, Education, Health, Child outcomes, Childcare, Food and Housing

7. WFS : CATI telephone survey –Initial outline –Instrument Administrative data –Could check selection bias in survey response –Patterns in services (process evaluation) –Patterns in employment and TANF (outcomes) Merged survey and administrative data to understand and explain patterns in outcomes –Connect employment to services, while controlling for individual characteristics (impact eval)

8. Survey instruments: –What is the purpose? –What is your sample? –How will you implement the survey? –What is your question design?