1 Understanding Event History Calendars ( Update on Reengineering SIPP) COPAFS March 5, 2010 David Johnson Jason Fields US Census Bureau

2 The Evolution of a Phoenix

3 Congresswomen Maloney hails 25th Anniversary of SIPP, Census Bureau’s Survey of Income and Participation "October, 2008 marks 25 years of SIPP data collection. The vital data collected by career professionals at the Census Bureau allows for the evaluation of the efficiency and effectiveness of government programs and gives us a more robust picture of how well we are doing as a nation in helping families progress through tough economic challenges" “The SIPP allows Congress to allocate scarce government resources and save tax dollars. It’s fitting that during this national economic crisis we draw attention to this important diagnostic tool which helps us understand how we can best provide assistance to families in need.”

4 The Unique Value of SIPP To provide a nationally representative sample for evaluating: –annual and sub-annual dynamics of income –movements into and out of government transfer programs –family and social context of individuals and households –interactions between these items

5 National Academy of Sciences National Research Council Committee on National Statistics Panel Report Reengineering the Survey of Income and Program Participation Constance F. Citro and John Karl Scholz, Editors July 2009

6 CNSTAT Report – Importance of SIPP Conclusion 2-1: The Survey of Income and Program Participation is a unique source of information for a representative sample of household members on the intrayear dynamics of income, employment, and program eligibility and participation, together with related demographic and socioeconomic characteristics. This information remains as vital today for evaluating and improving government programs addressed to social and economic needs of the U.S. population as it did when the survey began 25 years ago.

7 Our work on SIPP Improvements Improve Processing System and Collection Instrument Develop Event History Calendar (EHC) Instrument Examine use of administrative records data to supplement and evaluate survey data Continue meetings with stakeholders, development of survey content, and use of reimbursable supplements

8 Statements from the CNSTAT Report: On EHC methodology As discussed in Belli (1998), in an event history calendar, “respondents are encouraged to consider various events that constitute their personal pasts as contained within broader thematic streams of events. Not only can respondents not the interrelationship of events within the same themes (top-down and sequential retrieval) but, depending on which themes are represented by the calendar, respondents can also not the interrelationships among events that exist with different themes (parallel retrieval).”

9 Timeline for SIPP Development 2008 paper EHC Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan SIPP Re-Engineering Instrument Dev. Processing and Evaluation SIPP 2008 Panel – Waves 1 – 10 collection Eval. Analysis Systems Tests - Preparation 2009 Re-engineered SIPP automated Prototype Reference Period Field Activities 2012/13 SIPP Re-Engineering Instrument Refinement Systems Tests - Preparation 2013 Reengineered SIPP Reference Period Field Activities SIPP 2008 Panel – Waves 1 – 13 data release Data Gap SIPP nd automated prototype Reference Period Field Act. SIPP 2004 Panel data release Waves 11 – 13

Re-engineered SIPP – Progress Update – COPAFS March 5, 2010

Current SIPP Basics National panel survey – Since 1984 with sample size between about 11,000 and 45,000 interviewed households The duration of each panel varies from 2½ yrs to 4 yrs The SIPP sample is a multistage-stratified sample of the U.S. civilian noninstitutionalized population The survey uses a 4-month recall period – 3 interviews / year The sample is divided into 4 rotation groups for monthly interviewing Interviews are conducted by personal visit and by decentralized telephone

EHC Interviewing Human Memory - structured/organized - links and associations EHC Exploits Memory Structure - links between the occurrence and timing of events EHC Encourages Active Assistance to Rs - flexible approach to help elicit an autobiographical “story”

Evaluations of EHC Methods Many EHC vs. “Q-List” Comparisons - various methods - in general: positive data quality results BUT, Important Research Gaps - data quality for need-based programs? - comparison to 4-month reference period?

SIPP Re-engineering Field Test Plans -Proof of concept test paper and pencil reinterview test -EHC CAPI test Integrated Blaise and C# instrument prototype -CAPI Revised test Test improvements to the wave 1 instrument, training, and expand sample to all regional offices Test wave 2 concepts and instrument, examine movers and attrition issues, and refine training procedures.

2008 Paper Field Test Goals & Design Basic Goal: Can an EHC interview collect data of comparable quality to standard SIPP? - month-level data - one 12-month ref pd interview vs. three 4-month ref pd interviews - especially for need-based programs Basic Design: EHC re-interview of SIPP sample HHs Approximately 2000 HHs in IL and TX

Results Summary 3 Patterns: 1. EHC = SIPP All Year equivalent data quality

SSI -- % Participation for Each Month of CY2007 According to the SIPP and EHC Reports Analysis Summary - no “main effect” for method (SIPP = EHC) - no significant method difference in any month

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) Results Summary

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Results Summary

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) Results Summary

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year reduced EHC data quality, but not due to longer recall period Results Summary

SOCIAL SECURITY -- % Covered in Each Month of CY2007 According to the SIPP and EHC Reports Analysis Summary - significant “main effect” for method (SIPP > EHC) - method difference is constant across months

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) Results Summary

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) 3. EHC < SIPP, Early in the Year Only Results Summary

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) 3. EHC < SIPP, Early in the Year Only Food Stamps (TX); TANF (TX); employment; school enrollment Results Summary

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) 3. EHC < SIPP, Early in the Year Only EHC data quality may suffer due –to longer recall period Results Summary

FOOD STAMPS (Texas Only) -- % Participation for Each Month of CY2007 According to the SIPP and EHC Reports Analysis Summary - no significant “main effect” for method - BUT significant variation by month -- JAN-MAY: SIPP > EHC later months: no difference (reversal?)

FOOD STAMPS (Texas Only) -- % Participation for Each Month of CY2007 According to the SIPP and EHC Reports and ADRECS

3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) 3. EHC < SIPP, Early in the Year Only Food Stamps (TX); TANF (TX); employment; school enrollment Results Summary

2008 Paper Field Test Overall Summary Successful “Proof of Concept” Overwhelming Finding: SIPP-EHC Agreement Valuable Lessons to Inform Next Test - larger, broader sample - “correct” timing of field period - automated questionnaire Specific Data Comparisons are Instructive

2010 EHC CAPI Test Goals & Design Basic Goal: Can a CAPI Event History Calendar interview be: - developed in-house (new/unknown requirements) - integrated with survey management systems - administered by regular field staff interviewers Develop and test new processing system Determine the comparability of data collected: - month-level data - (1) 12-month ref pd intvw vs. (3) 4-month ref pd intvws - especially for need-based programs Basic Design: 8000 HHs interviewed in 10 states limited to high poverty strata that can be matched to areas in 2008 Panel SIPP data. Recording of the completed interviews

2010 EHC CAPI Instrument

Current Status Interviewing in 6 of 12 regional office areas (10 States) Interviews continue through March 13. Recording of the completed interviews – transcription will begin shortly. Reviewing and making changes to content and design. Reviewing and revising training materials and methods Planning for 2013 Production implementation

LANDMARKS RESIDENCES PRESENCE OF PARENT - MOM PRESENCE OF PARENT - DAD REFERENCE YEAR 20XXINTERVIEW YEAR 20XX+1 INTV SCHOOL ENROLLMENT – (CURRENTLY ENROLLED fill) JOBS/BUSINESSES – (fill name of emp/bus from ehc as entered) + Job/Bus 2 – (fill name of emp/bus from ehc as entered) + Job/Bus 3 – (fill name of emp/bus from ehc as entered) + Job/Bus 4 – (fill name of emp/bus from ehc as entered) + Job/Bus 5 – (fill name of emp/bus from ehc as entered) ALL OTHER WORK FOR PAY TIME NOT WORKING MARITAL STATUS – (CURRENT MARITAL STATUS fill) SSI - SUPPLEMENTAL SECURITY INCOME SNAP - FOOD STAMPS/SUPPLEMENTAL NUTRITION ASSISTANCE PRGM TANF - TEMPORARY ASSISTANCE FOR NEEDY FAMILIES GA – GENERAL ASSISTANCE PROGRAM WIC – WOMEN, INFANTS AND CHILDREN PROGRAM HEALTH INSURANCE – PRIVATE COVERAGE + ADDITIONAL PRIVATE COVERAGE MEDICARE COVERAGE MEDICAID COVERAGE – (fill interview state program name) MILITARY COVERAGE – (VA, CHAMPUS, CHAMPVA) OTHER HEALTH INSURANCE COVERAGE TIME WITHOUT HEALTH INSURANCE COVERAGE Topic – (FILL TOPIC LABEL) * INTRO TO TOPIC QUESTION FILL – ASKS ABOUT STATUS NOW OR DIRECTS FR TO PROCEED[ ] (1) YES/PROCEED (2) NO (3) REFUSED * FOLLOW-UP ASKS ABOUT EVER DURING REFERENCE YEAR OR DIRECTS FR TO PROCEED[ ] (1) YES/PROCEED (2) NO (3) REFUSED * IDENTIFY NEW PERIOD OF TIME[ ] 0-NEW[F4 – EDIT SPELL] [CTRL-X DELETE SPELL] * WHEN DID THIS PERIOD OF (TOPIC FILL) END?[ ] CHOOSE MONTH * WHEN DID THIS PERIOD OF (TOPIC FILL) BEGIN?[ ] CHOOSE MONTH ANY MORE PERIODS OF (TOPIC FILL) DURING REFERENCE YEAR?[ ] (1) YES (2) NO RECORDING ‘NO’ ADDS STRIKEOUT TO ANY UNASSIGNED MONTHS IN THIS TOPIC – ANSWERING ‘YES’ CYCLES BACK TO NEW PERIOD – UPON SPELL COMPLETION YOU EXIT THE DETAILED QUESTIONS TO THE ‘ANY MORE’ ITEM. DEPENDENT DATA FILL UP TO LAST WAVES ‘INTV’ MONTH FILL’S ARE USED TO LABEL INTERVIEW MONTH STATUS OR NAMES FROM JOBS/PROGR AM NAMES (FILL NAME OF PERSON FOR THIS EHC) Mockup (example of possible changes) for 2011 EHC CAPI Instrument DATA ARE RECORDED UP TO THE MONTH OF INTERVIEW, WE ALLOW UP TO 6 MONTHS FROM THE END OF THE CALENDAR YEAR TO CONDUCT INTERVIEWS – MONTHS BEYOND THE INTERVIEW ARE GREY’ED OUT. IF YES TO ‘NOW’ INITIALIZE FIRST PERIOD AND FILL INTERVIEW MONTH IN THE ‘TO’ BOX AND SET FOCUS ON THE ‘FROM’ BOX

Assessing Users’ Needs URL:

Comments: Jason Fields –