1. The Survey of Well-being of Young Children (SWYC)
Ellen C. Perrin, M.D.
R. Christopher Sheldrick, Ph.D.
So I’m going to tell you today about an instrument for surveillance that we have been working on for the past several years.

2. We have no relevant financial relationships with the manufacturer(s) of any commercial product(s) and/or provider(s) of commercial services discussed in this CME activity. We do not intend to discuss an unapproved/investigative use of a commercial product/device in this presentation.

3. Historical perspective
Rosie D. lawsuit
CBHI
Available Instruments
Lengthy
Proprietary
Scoring complex
Fragmented
So most of you remember the lawsuit brought by a few families of mentally ill young adults, attesting that the state had fallen down in its obligation to screen, detect, and treat their psychiatric illnesses while they were younger. When the state lost that lawsuit, the solution involved the creation of the CBHI. It had many charges, but the first one it took on was developing a program for behavioral screening of all children from birth onwards. As you know, it studied the available screening tools and propogated a list of 8 that the committee flet were the best validated, and asked all pedatricians an family practitioners to adminsiter one of these instruments to every child they saw for well child care.
When we looked at that list we worried about the practical aspects of the measures that were available for children below 5. They were lengthy, many were proprietary, all cost money, scroing was somewhat complex, and they only addressed behavior, not other aspects of development.

4. Our goals Surveillance Instrument (birth to 5) First level screening
Integrate “behavior” and “development”
Include family context/risks
Quick to fill out, score & interpret
“Developmental growth curve”
Free and in public domain
Amenable to electronic format
Computer
Telephone
Internet
So we were just naïve enough to think we could build a better mousetrap. We wanted to create an instrument that oculd be used at every well child visit, essentially a surveillance instrument. This would be a first level screen, it would integrate cognitive, language, motor, and socail-emotional development, as well as assessing family risks and also autism at the appropriate age. We knew it would have to be short, and easy to adminsiter and to score, and that it would have to be free and easily available. We also wanted to make sure that it would be easy eventially to transfer it onto a computer platform.

5. Survey of Well-being of Young Children
SWYC
Development
Behavior
We called our new instrument the “Survey of Well-being of Young Children” or SWYC,
Family Context 5

6. Development Behavior Family Risk
Preschool Pediatric Symptom Checklist (PPSC)
Development
Behavior
Internalizing
Externalizing
Baby Pediatric Symptom Checklist (BPSC)
For behavior, we considered the Pediatric Symptom Checklist, or PSC, to be a model. The PSC is short, easy to score with only one total score, and it is freely available. Based on a host of validation studies, the PSC has recently been provisionally accepted by the National Quality Center to assess medical outcomes—it is one of only two psychiatric measures to do so (the other is the PHQ9). Unfortunately, the PSC is only valid for children 5 and up, but fortunately, Drs. Mike Murphy and Mike Jellinik, the creators of the PSC, agreed to collaborate on a downward extension. Thus, we created the Preschool PSC for children 18 months – 5 years, and the Baby PSC for children under 18 months.
For the domain of emotional/behavioral development, we took the Pediatric Symptom Checklist as the model, and worked with Michale Murphy and Michael Jellinek to create a downward extension of that instrument. As you probably know, the PSC is free and in the public domain, and is widely respected and used as a screening tool in pediatric and other settings. Se we are calling our instruments the Baby PSC, for children from 1 to 18 months, and the Preschool PSC, for children from 18 months to 5 years.
Family Risk 6

7. Development Behavior Family Risk Milestones
Preschool Pediatric Symptom Checklist (PPSC)
Parent’s Observations of Social Interactions (POSI)
Development
Delays
Autism
Behavior
Internalizing
Externalizing
Baby Pediatric Symptom Checklist (BPSC)
For cognitive, language and motor development, we planned two measures. The Milestones was conceptualized as a way to help structure physicians’ surveillance of key milestones across developmental domains at each well-child visit. The POSI was created as a brief autism-specific screening tool STOP to be administered between 18 and 36 months,
Family Risk 7

8. Development Behavior Family Risk FAMILY RISK FACTORS Depression PHQ-2
Substance Abuse
Alcohol
Tobacco
Other drugs
Parental Discord
Hunger
For family risk factors, we were able to identify a number of 1-2 questions assessments of problems that affect child well-being and are potentially amenable to intervention by the pediatrician, STOP including the PHQ2 for parent depresseion, screening questions for parental substance abuse, 2 questons about marital discord, and a single-item screen for hunger
Family Risk
FAMILY RISK FACTORS 8

9. Four Domains Social-emotional-behavioral Cognitive, Language and Motor
BPSC (Birth to 18 months) – 12 items
PPSC (18 months to 5 years) – 18 items
Cognitive, Language and Motor
Milestones (each WCV) – 10 items
Autism
POSI (18 to 36 months) – 7 items
Family Risks
Each WCV - 11 items
So, just to reiterate, the whole instrument includes a social-emotional-behavioral screen, a developmental screen, an autism screen, and a family risk screen. Obviously the specific questions are different for different age children, so this either needs to be converted to a computer or telephone format, or you have to have a whole bunch of stacks of papers for the receptionist to hand out to families. One practice has put this up on a patient protal, and asks families to fill it out from home the night before their appointment.
So what we have is a set of instruments that together, we hope, describe a child’s overall well-being. We have completed an iniital validation of the measures and I’m going to let my colleague Chris Sheldrick describe that process to you.

10. SWYC Psychometrics! Latent variable modeling Comparative effectiveness
I know that nothing gets people excited like psychometrics.
The CDC actually invited us to give a talk about the SWYC, and after regaling them for about 45 minutes with all of the ins and outs of the SWYC’s psychometrics, several audience members politely suggested that I try to be a bit more succinct the next time. So here it goes.
There are two main types of analyses we have done with the SWYC that are worth knowing about:
Latent variable modeling and
Comparative effectiveness analyses

11. Latent variable modeling
item clusters
PPSC bifactor model
Reliability
Item 1 f1
Item 2
Differential Item Functioning
Item 3
Item 4
general factor
Item 5
Item 6
Item 7
Social/ emotional problems f2
Item 8
Item 9
Item 10
Item 11
For example, here is a depiction of the bifactor model we developed for the BPSC. It fit the data well, and it is important for a couple reasons. The first is reliability.
The model tells us that the items function well together, and that it makes sense to combine them to score a single total scale. Replicating these findings across multiple samples gives us greater faith in its reliability.
The second is differential item functioning, which is a way of testing whether the items perform as well across different populations.
For example, we tested items by SES (i.e., family income and parent education).
We know expected that social/emotional problems would be more common among low-SES children than higher SES children, as depicted here.
The question was whether parents from different SES groups answered PPSC questions differently, even after controlling for their child’s overall level of social/emotional problems. Through latent variable modeling, we were able to eliminate items that displayed high levels of DIF with respect to a range of factors, including SES and race/ethnicity. This increases our confidence that the SWYC will work well across diverse populations.
Item 12
Item 13 f3
Item 14
Item 15
Item 16 ?
Item 17
SES
Item 18 11

12. PRIMARY CARE SAMPLE Sensitivity Specificity 1.0 1.0 0.8 0.8 0.6 0.6
PPSC
ASQ-SE
1.0
PPSC
ASQ-SE
0.8
0.8
0.6
0.6
0.4
0.4
In a separate set of analyes, we compared the how well the SWYC predicted key outcomes to how well other validated screenign instruments predicted key outcomes. On the left, the blue lines depict the PPSC’s sensitivity in predicting CBCL total scores, internalizing scores, and externalizing scores. The red lines are for the ASQ-SE, an instrument that is already approved by CBHI. You can see that the confidence intervals overlap, suggesting no significant difference. Specificity is on the right.
0.2
0.2
0.0
0.0
Total Internalizing Externalizing
CBCL 1.5-5
Total Internalizing Externalizing
CBCL 1.5-5

13. CLINIC SAMPLE Sensitivity Specificity 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4
PPSC
ASQ-SE
1.0
PPSC
ASQ-SE
0.8
0.8
0.6
0.6
0.4
0.4
Here you can see the same set of analyses in a clinical sample, enrolled mostly from DBP and psychiatry clinics. Again, the PPSC performs at least as well as the ASQ-SE.
0.2
0.2
0.0
0.0
Total Internalizing Externalizing
CBCL 1.5-5
Total Internalizing Externalizing
CBCL 1.5-5 13

14. Sensitivity Specificity
1.0
0.8
0.6
0.4
In the combined sample, we looked at how well the PPSC predicted parent reports of diagnoses, including ADHD, anxiety, and other conduct problems. Again, the PPSC performed about as well as the ASQ-SE
PPSC
ASQ-SE
CBCL 1.5-5
0.2
0.0
ADHD Anxiety Behavior or Conduct Any diagnosis
Parent report of existing diagnosis
Specificity 14

15. Other SWYC studies POSI v MCHAT Milestones v ASQ-3 BPSC v ???
We have done similar studies for other parents of the SWYC. For example, the POSI has displayed higher sensitivity than the MCHAT in two studies. In a DBP clinic, the POSI detected 89% of children with confirmed diagnoses of autism, while the MCHAT only detected 71%. In a separate study, the difference in sensitivities was even greater, although the POSI’s specificity of 75% was somewhat lower than the MCHAT’s at 84%.
In an initial study, the Milestone’s 10 items detected developmental delays as well as the ASQ-3, and a replication study is nearing completion.
The BPSC has been the toughest one to validate because there are no well accepted criteria for behavioral disorders for kids under 18 months. Therefore, we know that the BPSC has good factor structure and that its items don’t display strong DIF, and we also know that it correlates well with the three forms of the ASQ-SE that are available for this age group.

16. Social/ Emotional Behavior
Example SWYC Forms
Milestones
12-Month
24-Month
Social/ Emotional Behavior

17. Page 2 (on the back)
Autism
12-Month
24-Month
Family Risk Questions

18. Summary Surveillance/first level screening instrument Includes
Cognitive, language, motor, and social-emotional development
Autism spectrum disorders
Family context/risks
Compares well with older instruments
Free, public domain
English and Spanish
Convertible to computer platform
Tailored for child’s age
Link to follow-up questionnaires