1. Documentation Tools in the Survey Lifecycle

2. Outline What is NSFG Webdoc? Instrument documentation != Survey documentation Data Cleaning/Processing in the Data Lifecycle Unexpected dividends: Documentation Tool as Management Tool

3. National Survey of Family Growth, Cycle 6 Cases  7643 female cases (age 15-44)  4928 male cases  13593 pregnancy cases 5300 variables  Intended and unintended births  Sexual intercourse  Contraceptive use  Infertility  HIV testing …… Conducted by the U.S. National Center for Health Statistics of the Department of Health and Human Services in 2002

4. NSFG WebDoc Previously, for NSFG Cycle 5  Codebook: 6000 pages  Questionnaires: 600 pages  User Guide: 450 pages Features of WebDoc for NSFG Cycle 6  Web browsable codebook  PDFs for printable codebooks  Search functionality (based on Oracle Text Indexing)

9. WebDoc Components Document Assembly Component DDI from Blaise Word documents Excel spreadsheets Text documents SAS data definition statements SAS output Web Display Component Cocoon XSLT Oracle 80%20%

10. Instrument Documentation Blaise instrument documentation Female questionnaire Male questionnaire 800+ questions Final public release 3 data files (female, male, pregnancy) 5300+ variables  Raw  Computed  Recodes  Imputation flags  Weights  intermediate Inhouse data files full data files Interviewer observations Audio CASI files Survey Documentation ≠

11. Documentation in the Lifecycle Study Design Data Collection Data Processing Data Distribution Data Discovery Data Analysis Data Archiving Instrument DocSurvey Doc

12. How do you manage this process? Perturbation Confidentiality Review Consistency Checks Recodes (10 levels) Consistent Codes for DK, RF, NA Multiple regression & Logical Imputation A Documentation Tool can help!!

13. Unexpected Dividend: Documentation Tool as Management/Diagnostic Aid WebDocs was started near the end of the Data Collection Phase of the NSFG project. A working version of WebDocs was finished early in the Data Processing/Cleaning Phase. Small unplanned modifications to the working prototype was used to produce diagnostics/metrics of the data cleaning process.

14. Documentation in the Lifecycle Study Design Data Collection Data Processing Data Distribution Data Discovery Data Analysis Data Archiving Survey Doc

15. WebDoc as diagnostic tool Features  Identifies problems: Missing Documentation Pieces: question text, description, Editing errors: missing labels, misspelled variable names, Data errors: missing values, out-of-range values (incorrect computation or recode logic), inconsistent missing codes, SYS-MIS …  Wiki used to track comments on each variable (almost zero programming necessary)

20. Do not treat documentation as an afterthought. Have the documentation tool available early at the beginning of the phase in order to use it as a diagnostic/management tool to manage the process Metadata in a database can be repurposed. Another example: SRC’s BlaiseDoc  Can be used to produce HTML Questionnaire with programmed skip patterns  HTML Questionnaire can be used by non-programmers to quickly check the work of instrument programming. Documentation Tool as a Diagnostic/Management Tool

22. Documentation in the Lifecycle Study Design Data Collection Data Processing Data Distribution Data Discovery Data Analysis Data Archiving Instrument Doc Survey Doc Instrument Doc

23. Summary Make Documentation Tools available early. Use a database so documentation metadata can be easily repurposed. Documentation tools can be used by non- programmers to inspect/manage the product quality and the process. NSFG WebDoc  http://www.icpsr.umich.edu/NSFG