1. Use of NDACAN Data to Inform Training Evaluation Elliott G. Smith National Data Archive on Child Abuse and Neglect National Human Services Training Evaluation Symposium June 14, 2011

2. The plan for today’s session Who are you? What is NDACAN? How are secondary data relevant to your work? Brainstorm ideas and opportunities 2

3. NDACAN’s Mission Mission Promote secondary analysis of existing data related to child abuse and neglect Funding provided by the Children’s Bureau, U.S. Department of Health and Human Services 3

4. Core Functions of NDACAN Acquire child maltreatment data Prepare the data for secondary analysis Protect the confidentiality of respondents License the data to eligible researchers Encourage data use and collaboration Support data users 4

5. www.ndacan.cornell.edu

7. The Spirit of the Times More data are being collected Expectations for accountability have grown In addition to process data, outcome data are being collected in administrative systems Data are easier to obtain The Internet age and online search make data widely available Federal initiatives encourage data sharing Data are easier to use Statistical software has gotten easier to use Sophisticated analytic techniques are available to a wider research audience 7

8. Advantages of Secondary Data Inexpensive Often exempt from human subjects IRB review Administrative data are comprehensive Large samples allow subgroup analysis Administrative data are collected over the long term and are repeated at regular intervals 8

9. Difficulties of Secondary Data It takes longer than you expect to understand a secondary dataset Respondents may not fit researcher’s target group Constructs may not be measured adequately or in a way that fits the researcher’s goals For administrative data, the definition of community is problematic (e.g., county level data) For administrative data, only service recipients are included in the system 9

10. Children’s Bureau Data Collection Administrative Data National Child Abuse and Neglect Data System (NCANDS) Adoption and Foster Care Analysis and Reporting System (AFCARS) National and Cross-Site Surveys National Survey of Child and Adolescent Well-Being (NSCAW) Longitudinal Studies of Child Abuse and Neglect (LONGSCAN) National Incidence Studies of Child Abuse and Neglect (NIS) 10

11. NCANDS CHILD FILE 11

12. NCANDS Description National effort to collect detailed data about all maltreatment investigations conducted by CPS Case-level data collected since 1993 12

13. NCANDS is the data source for Child Maltreatment 13

14. NCANDS: Data File Structure The dataset is one LARGE rectangular file (3.6 million records) Each record is a report-child pair of reports disposed in the Federal fiscal year For some states, it is possible to link adjacent years by the encrypted Child ID 14

15. NCANDS: Child File Content Report Data Child Data Maltreatment Data Child Risk Factors Caregiver Risk Factors Services Provided Staff Data Perpetrators Data

16. Child File: Report Data Encrypted IDs for report and child State County of report (limited) Report date, investigation start date Report source Report disposition, disposition date Agency notifications

17. Child File: Child Data Age at report Sex Race and Hispanic origin Living arrangement Family member in the military Prior victim

18. Child File: Maltreatment Data Maltreatment type (up to 4 types) Physical Abuse Sexual Abuse Neglect Medical Neglect Psychological Maltreatment Disposition of each allegation Substantiated Indicated Alternative response (victim or not victim) Unsubstantiated Closed – No finding

19. Child File: Child Risk Factors Alcohol and drug abuse Mental retardation Emotional disturbance Visually or hearing impaired Learning disabled Physically disabled Behavior problem Other medical condition

20. Child File: Caregiver Risk Factors Alcohol and drug abuse Mental retardation Emotional disturbance Visually or hearing impaired Learning disabled Physically disabled Behavior problem Other medical condition Domestic violence Inadequate housing Financial problem Public assistance

21. Child File: Services Provided Any post investigation services Family support services Family preservation services Foster care services, removal date Juvenile court petition, petition date Court-appointed representative

22. Services Provided (cont.) Availability varies by state Adoption Case management Counseling Daycare Educational & training Employment Family planning Health-related & home health Home-based Housing Independent & transitional living Information & referral Legal Mental health Pregnancy & parenting Respite care Special-disabled Special-juvenile delinquent Substance abuse Transportation

23. Child File: Perpetrator Data Up to 3 perpetrators Relationship Parental status Caregiver status Age at report Sex Race and Hispanic origin Military member Prior abuser Maltreatment outcome

24. AFCARS 24

25. AFCARS: Description National effort to collect child level information regarding every child in the U.S. foster care system and every child adopted through public child welfare agencies Detailed data regarding the child’s most recent removal Significant state participation since 1998

26. AFCARS Data File Structure Each record corresponds to an individual child NDACAN distributes an annual file Each annual file consists of about 800,000 records For some states, it is possible to link adjacent years by the encrypted Record Number

27. Is AFCARS a longitudinal database? Not really Detailed case information for the most recent removal However, 95% of foster care children have 1 or 2 removals, allowing a complete timeline

28. AFCARS: Foster Care File Content Child demographics (Age, Sex, Race) Child disabilities Total number of removals Case relevant dates (first removal, latest removal, discharge, latest placement, TPR) Removal reasons Placement setting Foster parent demographics Discharge reason Various subsidies and support

29. AFCARS: Adoption File Content Child demographics (Age, Sex, Race) Child disabilities Date of adoption, dates parental rights terminated Adoptive parent demographics Relation to child prior to adoption Adoption subsidy

30. NSCAW 30

31. NSCAW: Description Federally mandated national survey of children investigated following a maltreatment allegation First national study of child welfare services to collect data from children and caregivers and detail about the home environment First study able to produce national estimates of well- being, safety and permanency for children involved with CPS 31 Adapted from Webb et al. (2007)

32. Two-Stage Sampling Design Stage 1: Selection of PSUs (counties, mainly), with probability of selection proportionate to size, based on estimates of size of child welfare population in each U.S. county. Public child welfare agencies in selected counties were recruited for participation Stage 2: Selection of children within PSUs o children were ages 0-14 at time of sampling o All children who were investigated during 16 month baseline period were eligible for sample Adapted from Webb et al. (2007)

33. NSCAW Child Sampling Oversampled on the basis of: o Children/Families Receiving Services o Infants o Sexually Abused Children Not Sampled on the basis of: o Substantiated Reports (cases are included whether substantiated or not) Adapted from Webb et al. (2007)

34. NSCAW Cohort Total 6,228 Enter through investigation 5,501 No services 1,421 Ongoing services 4,080 In home 2,803 Out-of-home 1,277 Longer-term foster care 727 Adapted from Webb et al. (2007)

35. Data Collection Timeframe Target population: Children involved in investigations closed between October 1, 1999 and December 31, 2000 Wave 1: Baseline Nov, 1999 – Apr, 2001 Wave 2: 12 Month Follow-up Oct, 2000 – Apr, 2002 Wave 3: 18 Month Follow-up Apr, 2001 – Sept, 2002 1999 ‘ 2000 ‘ ‘ ‘ ‘ 2001 ‘ ‘ ‘ ‘ 2002 ‘ ‘ ‘ ‘ 2003 ‘ ‘ ‘ ‘ 2004 ‘ ‘ ‘ ‘ 2005 ‘ ‘ ‘ ‘ 2006 Wave 4: 36 Month Follow-up Oct, 2002 – Apr 30, 2004 Wave 5: ~6-7 Year Follow-up Sept, 2005 – Feb. 2007 Adapted from Webb et al. (2007)

36. Data Sources Children o Assessments by Field Representatives (all children) o Interviews (children 7 and older) Caregiver (parent) interviews Caseworker interviews Teacher questionnaires (children 5 or older) Agency and PSU data for context Adapted from Webb et al. (2007)

37. Measurement Domains Child functioning and well-being Services to children and families Family and caregiver functioning Community environment Agency and system level factors Adapted from Webb et al. (2007)

38. NSCAW Child Interview BD. Battelle Developmental Inventory (BDI) NI. Bayley Neurodevelopmental Infant Screener (BINS) CO. Preschool Language Scales KB. Cognitive Status (K-BIT) AH Mini-Battery of Achievement SE. School engagement RP. Relationship with peers PF. Protective factors RC. Relationship with caregiver(s) CL. Closeness to caregiver(s) PM. Parental monitoring FE. Future expectations CD. Depression TR. Trauma Symptom Checklist EV. Exposure to violence (VEX-R) YA. Youth Activities YB. Child Behavior Checklist Self Report SA. Substance abuse SX. Sexual activity DE. Delinquency IJ. Injuries CM. Child maltreatment Adapted from Webb et al. (2007)

39. HH. Household roster LE/LV. Child living environment CE. Community environment HS. Child Health & Services VI/VN/VE. Adaptive Behavior PS/PT/PU. Prosocial skills TE. Emotional regulation - Temperament TB. Emotional regulation - Toddler behavior TC/BC - Child Behavior Checklist Parent Report IN. Income SH. Services - baseline SR. Services - post-baseline SS. Social support PH. Physical health DP. Depression Audio-CASI AD. Alcohol dependence DD. Drug Dependence IL. Involvement with law DS. Discipline and child maltreatment SF. Satisfaction w/caseworker HM. HOME Scales NSCAW Caregiver Interview Adapted from Webb et al. (2007)

40. HH. Household roster LE/LV. Child living environment CE. Community environment HS. Child Health & Services VI/VN/VE. Vineland Adaptive Behavior PS/PT/PU. Prosocial skills TE. Emotional regulation - Temperament TB. Emotional regulation - Toddler behavior TC/BC - Child Behavior Checklist Parent Report IN. Income Foster caregiver services HM. HOME Scales AM. Adoption module NSCAW Foster Caregiver Interview Adapted from Webb et al. (2007)

41. LN. Living environments IV. Caseworker involvement CP. Family compliance and progress DM. Decision-making CT. Court hearings CB. Caseworker background CI. Case investigation AA. Alleged abuse RA. Risk assessment SP. Services to parents SC. Services to child HB. History before case report HR. History since case report NSCAW Caseworker Interview Adapted from Webb et al. (2007)

42. LONGSCAN 42

43. LONGSCAN: Description A consortium of five different studies of the causes and consequences of child maltreatment While the sites have different samples, they have common measures, data collection methods and schedules Data are collected longitudinally at ages 4, 6, 8, 12, 14, 16, and 18. Wide array of measures at multiple levels of the child’s ecology

44. LONGSCAN: Measurement Guided by Social-Developmental-Ecological Theory (NRC, 1993; Bronfenbrenner, 1989; Hawkins & Catalano, 1996). Domains assessed: Child/Youth Caregiver Family microsystem: Home environment, functioning Macrosystem: Neighborhood, school, support Review of official CPS case reports 44 Slide provided by Desmond Runyan

45. LONGSCAN: Domains Child/Youth Demographics Physical health and medical history Child care and separations from caregiver Social competence Temperament Cognitive function and academic achievement Socioemotional function Developmental status, adaptive behavior, living skills Problem behavior and risky behavior Exposure to violence, maltreatment, alcohol and drugs 45

46. LONGSCAN: Domains Caregiver Demographics Physical health History of loss and victimization Parenting attitudes Substance use Mental health 46

47. LONGSCAN: Domains Family Household composition Income and poverty Service use Family function and satisfaction Daily stress Quality of partner relationship, domestic violence Quality of relationship with the child, physical discipline, parental monitoring Home environment 47

48. LONGSCAN: Domains Macrosystem Community income and employment Neighborhood characteristics Social support for the parents and the child School safety 48

49. Knowledge creation is iterative Collect ConfirmInform Inspire 49

50. The Evaluation Pyramid 50 Assess Need for the Program Assess the Program’s Theory of Change Assess Program Process Assess Outcomes Assess Efficiency Adapted from Rossi, Lipsey, & Freeman (2004)

51. For more information Elliott G. Smith egs1@cornell.edu