1. Sample Design and Weighting of the 2017 Mental Health of Children and Young People Survey
Salah Merad
Methodology Division, ONS
Special thanks to Robynne Davies and Alexandra Pop
RSS Social Statistics Section Event
12 February 2019

2. Outline
Survey design
Weighting Strategy
Achieved sample

3. Sample Design (1) Population: Resident England population aged 2 to19
Sampling Frame: NHS Patient Register (NHS Digital)
Child benefits records used in 2004 (5 to 16 years old)
Two-stage stratified design
Stage 1: selection of postcode sectors (explicit stratification by Region, implicit stratification by census factors (Social Renting, NSSEC, Unemployment)
Stage 2: selection of children from Patient register

4. Sample Design (2) – Sample requirements
Achieve similar numbers to 2004 for 5 to 16 age group for England
Required achieved age distribution
Oversample children

5. Sample Design (3) – Regional Boost
Reduce sample size variation across regions
Boost sample size of North East

6. Sample Design (4) – Main and reserve samples
Assumed a response rate of 62% for 5 to 16 age group
In total, needed to select about 16,000 children from Patient Register
Selected a total of 379 postcode sectors, using probability proportional to size (PPS) within each region
Selected 42 children in each postcode sector using systematic PPS
Selected reserve sample using same design: 80 postcode sectors

7. Sample Design (5) – Halls of residence
Initially intended to sample from university halls of residence to achieve about 100 interviews
Difficulty in accessing halls, so decided not to include them in sample

8. Sample selection Age was calculated as of August 31st 2017.
The target population is children born between and on the 1st September 1997 and 31st March 2015.
This was to ensure only those aged 2 before 1st April 2017 were included in the sample, and also to exclude those who turned 20 before the end of August 2017.

9. Weighting strategy – Design weights
Design weights for main sample
PSU selection probabilities 𝑃 𝑠𝑡𝑎𝑔𝑒1,𝑖,𝑔 = 𝑛 𝑔 𝑐 𝑖 𝐶 𝑔 ,
2nd stage selection probabilities
1/(product of selection probabilities)
Design weight for reserve sample

10. Weighting strategy – Combining samples
Main and reserve samples combined together by assigning factors proportional to the respective samples sizes over the design effects stemming from weight variation
Design effect is given by Deff=1+ 𝐶𝑉 𝑑𝑤𝑒𝑖𝑔ℎ𝑡 2
The combined weight was adjusted so that the distribution by Single Year of Age (age at interview) matches the distribution according to the population estimates.

11. Weighting strategy – Non-response
Non-response factors were obtained by fitting a logistic regression model on the response indicator variable
Model covariates: the age used for sample selection(grouped), region (at selection), IMD and Urban/Rural indicators, and the source of the data (main or reserve sample)
Pre-calibration weight =
Combined design weight/predicted probability

12. Weighting strategy – Calibration to ONS population estimates
Separate calibration in each age group: (2-4, 5-9,10-15,16,17-19)
Calibration by sex and region within each group
Didn’t use data about the number of people in Higher Education, Further Education, Secondary education
Not easy to obtain an accurate estimate of HE students under 20 using data from HESA

13. Distribution of final weights in age group 5 to 16

14. Weighted sample: Distribution by ethnic group

15. Design factor for Any Hyper Activity Disorder

16. Weighting strategy – Teacher non-response adjustment
Teacher interview for 5 to 16 age group
Teacher’s score is combined with parents’ score to obtain clinical score
A proportion of children didn’t have a teacher interview
If teacher’s score is unavailable, clinical score is based on parents’ score only
Sampling weights were adjusted to compensate for lack of teacher’s scores

17. Teacher non-response adjustment - Method
Calculate revised number of children with each clinically-assessed type of mental disorder in children with no teacher data
Adjustment factor given by

18. Teacher adjustment factors for 5 to 16 year olds