1. Canadian Community Health Survey A new program for collecting health information Interuniversity Research Data Seminar University of British Columbia Béland Yves Household Survey Methods Division Statistics Canada February 19, 2002

2. Presentation Outline &Health Information Roadmap –Origin of the CCHS –Objectives / Content –CCHS two-year plan &CCHS Cycle 1.1 - Sample Design –Allocation, frame –Selection - Oversampling –Data Collection –Imputation –Weighting, sampling error –Bootstrap Variance Estimation –Data Quality –Data Dissemination &CCHS Cycle 1.2 - Overview &Future Cycles of CCHS

3. Health Information Roadmap &Four-year action plan to strengthen Canada’s health information system &Earmarks funds for specific priorities/activities based on national vision and provincial/regional consultations &Partners: Health Canada, Canadian Institute on Health Information (CIHI) and Statistics Canada &Key elements: –fill critical data gaps in health services and address population health data gaps at a sub-provincial level –foster common data and technical standards –develop indicators and conduct special studies

4. Canadian Community Health Survey Results of the Consultation Process &Assess health measure variations at many levels of geography &Collect data on issues unique to a health region or province &Respond quickly to emerging issues &Explore certain key health issues in-depth &Analyse the effects of shocks including policy changes

5. Canadian Community Health Survey Two-year Plan &Cycle 1.1 - Health region-level survey –Produce reliable estimates for sub-provincial areas –Continuous monthly collection : Sept. 2000 - Nov. 2001 –Sample size : 133,300 respondents –Questionnaire content health determinantshealth determinants health statushealth status utilization of health servicesutilization of health services socio-demographic / socio-economic characteristicssocio-demographic / socio-economic characteristics &Cycle 1.2 - Provincial-level survey –Produce reliable provincial estimates from a sample of 30,000 respondents –Monthly collection : May 2002 - Dec. 2002 –In-depth focus content: 90-100 minute interviews on mental health and well-being

6. CCHS and NPHS A More Robust Health Survey Program &CCHS –cross-sectional –sample of 160,000 respondents over two years –national, provincial and regional level estimates –customized questionnaires at regional level –built-in flexibility for buy- in sample and/or content –continuous development of in-depth health content &NPHS - Household –« goes longitudinal » only, starting in wave 4 –sample of 20,000 persons –national and provincial level estimates &NPHS - Health Care Institutions –longitudinal and cross-sectional –sample of 2,500 –national level estimates

7. CCHS - Cycle 1.1 Health Region-level survey &Produce timely cross-sectional estimates for 136 health regions &Target population –individuals living in private occupied dwellings aged 12 years old or over –Exclusions: those living on Indian Reserves and Crown Lands, residents of institutions, full-time members of the Canadian Armed Forces and residents of some remote areas &CCHS 1.1 covers ~98% of the Canadian population

8. CCHS - Questionnaire content &45-minute interview questionnaire –30 minutes of common modules common to all health regions –10 minutes of optional items selected by health regions from a predefined list of modules –5 minutes of standard socio-economic items & 27 different versions of the questionnaire The complete questionnaire can be found at www.statcan.ca/health_surveys

9. CCHS - Sample Allocation to Provinces ProvPop # of 1st Step2nd StepTotal Size HRs500/HRX-propSample NFLD551K 6*2,7801,2304,010 PEI135K 21,0001,0002,000 NS909K 63,0002,0405,040 NB738K 73,5001,6505,150 QUE7,139K 168,00016,28024,280 ONT10,714K 3718,50023,76042,260 MAN1,114K 115,5002,5008,000 SASK990K 11*5,4002,3207,720 ALB2,697K 17*8,1506,05014,200 BC3,725K 2010,0008,09018,090 CAN29,000K 13365,83064,920130,750 * The sampling fraction in some small HRs was capped at 1 in 20 households

10. CCHS - Sample Allocation to Health Regions Pop. Size# of Mean RangeHRsSample Size Smallless than 75,00041 525 Medium75,000 - 240,00060 900 Large240,000 - 640,00025 1,500 X-Large640,000 and more7 2,500

11. CCHS - Sample Allocation to Territories Population Sample Yukon25,000850 NWT36,000900 Nunavut22,000800

12. CCHS - Sample Frame &CCHS sample selected from three frames: Area frame (Labour Force Survey structure)Area frame (Labour Force Survey structure) RDD frame of telephone numbers (Random Digit Dialling)RDD frame of telephone numbers (Random Digit Dialling) List frame of telephone numbersList frame of telephone numbers Three frames are needed for CCHS for the following reasons: 1. To yield the desired sample sizes in all health regions 2. Have a telephone data collection structure in place to quickly address provincial/regional requests for buy-in sample and/or content at any point in time 3. Optimize collection costs

13. Area frame - Sampling of households &83% of CCHS sampled households &Stratified multistage sample design Stratum #1 Stratum #2 #1: Each health region is divided into strata #2: Clusters selected within strata (PPS sampling) (1st stage)        #3: Dwellings selected within clusters (2nd stage)  

14. RDD frame of telephone numbers Sampling of households &Elimination of non-working banks method –7% of CCHS sampled households –Telephone bank: area code + first 5 digits of a 7-digit phone # 1- Keep the banks with at least one valid phone # 2- Group the banks to encompass as closely as possible the health region areas - RDD strata 3- Within each RDD stratum, first select one bank at random and then generate at random one number between 00 and 99 4- Repeat the process until the required number of telephone numbers within the RDD stratum is reached

15. List frame of telephone numbers Sampling of households &Simple random sample of telephone numbers –10% of CCHS sampled households –Telephone companies’ billing address files and Telephone Infobase (repository of phone directories) 1- Create a list of phone numbers 2- Stratify the phone numbers by health region using the residential postal codes 3- Select phone numbers at random within a health region 4- Repeat the process until the required number of telephone numbers is reached

16. CCHS - Sampling of persons &Area frame  SRS of one person aged 12 years of age or older (82% of households)  SRS of two persons aged 12 years of age or older (18%) &RDD / List frames  SRS of one person aged 12 years of age or older

17. CCHS - Sampling of persons Age1996LFS* CCHS groupCensussamplesimulated (all persons) sample ( only 1 person) 12-1913.213.7 8.5 20-2916.414.4 14.3 30-4430.828.7 29.1 45-6425.828.0 27.9 65 +13.815.2 20.2 * averaged distribution over 100 repetitions using the May 99 LFS sample

18. CCHS - Representativity of sub-populations To address users’ needs, two sub-population groups needed larger effective sample sizes: & &Youths (12-19 years old) – –Decision > Oversample youths by selecting a second person (12-19) in some households based on their composition & &Elderlies (65 years old and +) – –Decision > Do not oversample - let the general sample selection process address the issue by itself

19. Sampling strategy based on household composition Number of persons aged 20 or over Number of persons aged 20 or over Number0 1 2 3 4 5+ of 12-19 Number 0 1 2 3 4 5+ of 12-19 0 0-AAAAB 1 1AACCCB 2 2ACCCCC 3+ 3+ACCCCC A: SRS of one person aged 12+ B: SRS of two persons aged 12+ C: SRS of one person in the age group 12-19 and SRS of one person 20+

20. CCHS - Sample Distribution after Oversampling Age1996* CCHS* CCHS groupCensussimulated simulated samplesample ( only 1 person)( some 2 persons) 12-1913.2 8.5 14.9 20-2916.4 14.3 13.1 30-4430.8 29.1 28.1 45-6425.8 27.9 26.3 65 +13.8 20.2 17.6 * averaged distribution over 100 repetitions using the May 99 LFS sample

21. CCHS - Initial data collection plan &12 monthly samples &12 collection months + 1 Area frame !CAPI !STC field interviewers !targeted response rate: 90% !anticipated vacancy rate: 13% (09 / 2000 - 08 / 2001) + 09 / 2001 RDD / List frames !CATI !STC call centres !targeted response rate: 85% !telephone hit rate: 15-60%

22. CCHS data collection - Observed situation &Field interviewers –workload exceeded field staff capacity &Call centres –new collection infrastructure –unequal allocation of work among call centres

23. CCHS - Final response rates Field Call centres Total Field Call centres Total NFLD86.689.386.8 PEI87.782.684.7 NS88.889.388.8 NB88.492.488.5 QUE85.784.885.6 ONT82.879.582.0 MAN90.085.089.5 SASK87.085.486.8 ALB85.284.985.1 BC83.986.784.7 YUK79.395.682.7 NWT89.685.489.2 NUN*66.334.662.5 CAN85.183.184.7

24. CCHS - Proxy interviews &Higher number of proxy interviews than expected –~ 6% instead of 2-3% &Major consequence: one third of the questionnaire is missing which could be proble- matic for small health regions &Solution : Imputation

25. CCHS - Imputation & 3-step strategy –common modules / mental health related optional modules / other optional modules & more than 2,000 imputation classes (region, age, sex, questionnaire type, skip patterns, etc…) & hot-deck imputation using nearest neighbour approach according to 12-16 key characteristics

26. CCHS - Weighting and Estimation & &Three separate weighting systems: – –Area frame design – –RDD frame design – –List frame design & &Several adjustments – – non-response (household and person) – – seasonal factor – – etc... & &Integration of the two weighting systems based on Deffs & &Calibration using a one-dimensional poststratification adjustment of ten age/sex poststrata within each health region & &Variance estimation : bootstrap re-sampling approach – –set of 500 bootstrap weights for each individual

27. CCHS Weighting Strategy

28. Weighting & Estimation

29. CCHS - Special Weights & &For various reasons, many other weights are produced – – Quarter 4 special weight – – PEI special weight – – Share weights (master, Q4 and PEI special) – – Link weights (master, Q4 and PEI special)

30. Sampling Error &Difference in estimates obtained from a sample as compared to a census &The extent of this error depends on four factors: –sample size –variability of the characteristic of interest –sample design –estimation method &Generally, the sampling error decreases as the size of the sample increases

31. Sampling Error &Measure of precision, reliability of the estimates –Variance (standard deviation) –Coefficient of variation Standard deviation of estimate x 100% / estimate itselfStandard deviation of estimate x 100% / estimate itself CV allows comparison of precision of estimates with different scalesCV allows comparison of precision of estimates with different scales –Example: 24% of population are daily smokers, std dev. = 0.003 24% of population are daily smokers, std dev. = 0.003 CV=0.003/0.24 x 100%=1.25%CV=0.003/0.24 x 100%=1.25%

32. Sampling Variability Guidelines Type of estimate CVGuidelines Acceptable 0.0-16.5 General unrestricted release Marginal 16.6-33.3 General unrestricted release but with warningcautioning users of the high sampling variablitity. Should be identified by letter M. Unacceptable> 33.3 No release. Should be flagged with letter U. Should be flagged with letter U.

33. Sampling Error &Measuring sampling error for complex sample designs: –Simple formulas not available –Most software packages do not incorporate design effect (and weights adjustments) appropriately for calculations –Solution for CCHS: the Bootstrap method

34. Bootstrap method &Principle: –You want to estimate how precise is your estimation of the number of smokers in Canada –You could draw 500 totally new CCHS samples, and compare the 500 estimations you would get from these samples. The variance of these 500 estimations would indicate the precision. –Problem: drawing 500 new samples is $$$ –Solution: Use your sample as a population, and take many smaller subsamples from it.

35. Bootstrap method T = 40 Var =  (B i - B) 2 / 499 &How CCHS Bootstrap weights are created (the secret is now revealed!!!)

36. Bootstrap Method &How Bootstrap replicates are built (cont’d) lThe “real” recipe 1- Subsampling of clusters (SRS) within strata 2- Apply (initial design) weight 3- Adjust weight for selection of n-1 among n 4- Apply all standard weight adjustments (nonresponse, share, etc.) 5- Post-stratification to population counts lThe bootstrap method intends to mimic the same approach used for the sampling and weighting processes

37. Bootstrap Method &Sampling weight vs. Bootstrap weights –Sampling weight used to compute the estimation of a parameter (e.g.: number of smokers) –Bootstrap weights used to compute the precision of the estimation (e.g.: the CV of the number of smokers estimation)

38. CCHS - Data Dissemination Strategy & &Wide range of users and capacity – –136 health regions – –13 provincial/territorial Ministries of Health – –Health Canada and CIHI – –Internal STC analysts – –Academics – –Others & &Data products – –Microdata – –Analytical products (Health Reports, How Healthy are Canadians, etc…) – –Tabular statistics (ePubs, Cansim II, community profiles, etc…) – –Client support (head and regional offices, CCHS website, workshops, etc…)

39. CCHS - Access to microdata & &Master file – –all records, all variables Statistics Canada university research data centres remote access & &Share / Link files – –respondents who agreed to share / link provincial/territorial Ministries of Health health regions (through the STC third-party share agreement) & &Public Use Microdata File (PUMF) – –all records, subset of variables with collapsed response categories free for 136 health regions cost recovery for others

40. CCHS - Overview of Cycle 1.2 & &Produce provincial cross-sectional estimates from a sample of 30,000 respondents & &Area frame sample only / one person per household & &CAPI only & &90-100 minute in-depth interviews on mental health and well- being based on WMH2000 questionnaire & &Scheduled to begin collection in May 2002

41. CCHS - Future Plans & &Same two-year cycle approach: – –health region level survey starting in January 2003 – –provincial level survey starting in January 2004 & &New consultation process with provincial and regional authorities & &Flexible sample designs (adaptable to regional needs) & &Development of an in-depth nutrition focus content (Cycle 2.2)

42. CCHS Web site www.statcan.ca/health_surveyswww.statcan.ca/enquetes_santé

43. Contacts in Methodology &Yves Béland: yves.beland@statcan.ca &François Brisebois: francois.brisebois@statcan.ca