Session 5 – Questionnaire Checklists Survey Training Pack Session 5 – Questionnaire Checklists

What is it? A comprehensive list of checks to help ensure: Questionnaire is complete and is readable Skips have been correctly followed Codes have been used correctly Identifiers are clearly given Data are consistent between variables So what exactly is a Questionnaire Checklist? Very briefly it is a complete list of checks that help ensure data quality. It includes ensuring the questionnaire has been completed and responses are clear; ensuring that any skips in the questionnaire have been followed correctly; ensuring that codes have been used correctly; ensuring you have the correct identifiers and that data are consistent between variables. In a short while we will show you a couple of examples of data errors that could have been avoided if a checklist had been used.

Why do we need one? Checklists help ensure that nothing is overlooked Adds credibility to your data Helps eliminate errors early in the survey process A checklist helps ensure that nothing is overlooked. It’s a bit like a task list where you tick the checks as they are carried out. It can add credibility to your data as you will be able to show that you have considered data errors and have systematically checked for potential problems. The checklist can and should be archived along with your survey materials. It also helps you trap errors as early as possible in the survey process.

When should we use it? In fieldworker training sessions During fieldwork While developing data entry system – some checks can be programmed into the system During data checking – before analysis The checklist has several uses: during fieldworker training it will help in making enumerators and supervisors aware of any potential issues so that they know what to look for during fieldwork and data recording. If you are developing a data entry system, whether for direct data entry onto mobile devices or for data entry from paper questionnaires, the checklist can be very useful during the programming as some checks can be programmed into the system. When you are checking your data prior to analysis a checklist can be useful in making sure you don’t miss anything.

How should it be used? Each item on the list should include a checkbox One copy per questionnaire – attach to paper questionnaire Tick each item as the check is done When complete sign and date the list – include space for this If you are using paper questionnaires then you should have one copy of the checklist per questionnaire. Make sure it includes the unique identifiers for that particular questionnaire. Include checkboxes so that you can tick each item as the check is done. Signing and dating the checklist will help with tracking progress.

Dealing with errors Can you check the values? Possible if you are still in the field Otherwise make a decision: Is the value feasible? If no, set to missing e.g. birth weight > 10kg Otherwise do you include it? Depends on circumstances Document your decision and reasons for that decision So what do you do if and when you find errors? The ideal solution would be to check the value – this might be possible if you are still in the field so it is useful to check values before you leave the area. If that isn’t possible you need to decide what to do and this is where you need to make a decision based on experience. If the value is not feasible then it should be set to missing – values that are clearly wrong would lead others to question the validity of the rest of your data. Whatever you decide you should document your decisions and your reasons.

Exercise This example is taken from a survey looking at childhood poverty – this is part of the household roster. The survey was concentrating on a particular child in the household referred to in this extract as “NAME OF CHILD” and in the roster we collected the names, ages, etc. of other household members included their relationship to the child. There are some obvious errors here.

Exercise This example is adapted from a clinical trial; data was initially recorded at a screening visit, then during the treatment visit and again two weeks later. Here we can see the height and weight at the screening visit, and the weight at treatment and follow-up. Height is in cms and weight in Kgs. Can you see any problems here?

Exercise Review the checklist for the Savings’ Group Survey Make sure you understand the purpose of each check Produce a checklist for the 2014 rice survey

Summary A comprehensive list of checks helps ensure data integrity Include every check you can think of – even if it seems obvious to you Do not make assumptions Remember: If your data includes obvious errors (e.g. 12 year old grandparents) others will not trust your results