Statistics Canada Statistique Canada Cost-Efficient Framework for Data Collection for CATI Surveys Social Surveys Collection Research Steering Committee (SSCRSC) François Laflamme December 2010

17/11/2015 Statistics Canada Statistique Canada 2 Past and current research  Paradata research has mainly directed its effort in improving the current data collection process and practices  Paradata research has also suggested that the interviewer staffing levels are not always optimally allocated with respect to the workload sample and the expected productivity

17/11/2015 Statistics Canada Statistique Canada 3 Challenges and objective  One of the main challenges is to collect cost- effective data while maintaining a high level of quality  According to the forecasted budget reductions, the pressure has increased and will continue to grow  The objective of the project is to propose a cost- efficient framework for data collection for CATI surveys

17/11/2015 Statistics Canada Statistique Canada 4 Cost-efficient framework Five 5 dimensions: 1.Metrics used for costing and budgeting surveys 2.Resources allocation within surveys 3.Resources allocation between surveys 4.Collection process and practices 5.Operational constraints

17/11/2015 Statistics Canada Statistique Canada 5 1. Metrics used for costing and budgeting surveys  Should be based on objective, measurable and comprehensible assumptions  Should be monitored throughout the survey collection  Budget too high - low productivity  Budget too low - more difficult to reach targets  Budget often based on previous survey cycles or comparable surveys  But it is exactly what we try to improve (e.g. low TPU)

17/11/2015 Statistics Canada Statistique Canada 6 2 & 3. Resources allocation within and between surveys  Within survey  Interviewer staffing levels should consider sample workload and collection progress in taking advantage of the entire collection period ●Survey productivity decreases for all CATI surveys during collection  Between surveys  Many concurrent surveys compete for data collection resources

17/11/2015 Statistics Canada Statistique Canada 7 Resources allocation within and between surveys  What is the best strategy within survey?

17/11/2015 Statistics Canada Statistique Canada 8 Resources allocation within and between surveys  The main idea is to try to maintain the survey productivity for all surveys for a given RO (red line) between reasonable boundaries around the average productivity  How?

17/11/2015 Statistics Canada Statistique Canada 9 Resources allocation within and between surveys  The strategy is to distribute system time for a given period (day or week) in function of the inverse of the proportion of the budget spent (inversely correlated to productivity) to maintain productivity at RO level

17/11/2015 Statistics Canada Statistique Canada 10 Resources allocation within and between surveys  In practice  Need to get consistent budget figures for all surveys  Need to get daily or weekly capacity system time  Need to consider the active collection period of each survey  Need to take into account operational constraints  Is it feasible? Can it be operationalized and automated?  The objective is to produce guidelines for RO to help them allocate collection resources only at the aggregate level.

17/11/2015 Statistics Canada Statistique Canada Collection process and practices  Focus of the past research  Some examples  Best time to call  Call scheduler features ●Cap on calls, customized time slices, Z-groups etc…  Increased the proportion of evening shifts versus day shifts to improve contact rate and productivity  Active management  Responsive collection design  No significant efficiency if budget and resources allocation remain the same.

17/11/2015 Statistics Canada Statistique Canada Operational constraints  RO capacity  Varies by day (interviewer availability)  LFS week  Holidays and special event  Average productivity  Varies by RO and survey  Interviewer schedule in advance  Interviewer training across surveys  Etc…

17/11/2015 Statistics Canada Statistique Canada 13 Comments  3 Separate projects  Costing metrics  Resources allocation within and between surveys  Collection process and practices  All need to take into account operational constraints  All dimensions have to be considered simultaneously to succeed in improving the cost- efficiency of data collection.

17/11/2015 Statistics Canada Statistique Canada 14 For more information, please contact Pour plus d’information, veuillez contacter François Laflamme