Presentation on theme: "Facilitated by Elizabeth Wiredu"— Presentation transcript:
1 Facilitated by Elizabeth Wiredu Questionnaire DesignFacilitated by Elizabeth Wiredu
2 Course AimThe aim of this course is to facilitate the design of valid questionnaire to support good data collection and the subsequent analysis.
3 Learning Outcomes By the end of the course, you will be able to: Identify the principles that will guide the design of a good questionnaire.Learn how to put a good design together.Examine how response types determine the choice of summaries and statistics you could generate on your data.Critique a poorly designed questionnaire.Create a sample good questionnaire from scratch.
4 Questionnaire What is questionnaire? When to use questionnaire? Is validated questionnaire the solution?Checked the literature for what others have measured before in similar studies?Could you use a pilot interview to inform what to measure?Using existing validated questionnaireUsing a previously validated and published questionnaire will save you time and resources;You could use professional interviewers if necessary to collect more complex data – probe, prompt, use follow up questions, assist with writing for poor spellers.Using standardised questionnaires, you will be able to compare your own findings with those from other studies,You may find it easier to get published.The Cost besides Time and Resources?Poorly design questionnaire lead to poor quality data, sometimes ‘impossible to analyse’ data situation. No room for correction since you’ve only one opportunity with the respondent.Lead to misleading conclusions, unreliable recommendations.If there is no ‘off the shelf’ questionnaire available, then this lets look at how to write a bespoke questionnaire.
5 The Framework Pose the research question. Develop and write the questionnaire, and do validity checks.Summarise and analyse data.Interpret and report findings.
6 Stage 1- Pose the Research Question Funnel the research question down into specific key questions or objectives which will then be grouped as themes.Write down the hypotheses and the conclusions you want to arrive at.Establish the relationships, comparisons, trends, patterns or summaries you wish to make.What data?Work with your co-researcher to clarify this.Do a thorough literature search for validated measures such as SF-36 questionnaire. This will increase the chance of your work to be acceptable for publication and be accepted by the ethic committee.If you are not familiar enough with the research area and literature cannot help, use focus groups interview to find out what data to collect.ValidationValid questionnaire measures what it claims to measure. Refer to examples from Wording to Vague questions. Another identifier is in the areas of context. For example, An instrument developed in a different time, country, or cultural context may not be a valid measure in the group you are studying.ReliabilityReliable questionnaires yield consistent results from repeated samples and different researchers over time. Differences in results come from differences between participants, not from inconsistencies in how the items are understood or how different observers interpret the responses. A standardised questionnaire is one that is written and administered so all participants are asked the precisely the same questions in an identical format and responses recorded in a uniform manner. Standardising a measure increases its reliability.
7 Stage 2- Develop and Write the Questions Formulate each key question into a theme and choose which variables you must collect data on.Decide how each variable is to be measured i.e. how to frame the question (open/closed) and its response set.Get help from your literature reviews.
9 Examples of Closed Questions Design DichotomousQuestions???Response: Yes No Male FemaleSimple Choice Exceeded expectations Met expectations Fell short of expectationsLikert Scale Very satisfied Somewhat satisfied Undecided Somewhat dissatisfied Very dissatisfiedAnchor Scale1-directional design:On a scale of 1 to 10, where 1 is virtually no pain and 10 is excruciating pain.2-directional design:On a scale of 1 to 5, where 1 is highly unsatisfied and 5, highly satisfied.
10 Examples of Closed Questions Design Multiple ChoiceWhich of the following products do you own? (select all that applies) Pen drive Computer Printer Fax machine Scanner Digital cameraMatricesPlease rate your purchase decision based on the following characteristicsV. Important Will Consider Not ImportantReliability Price Features The look Functionality
11 Opened Question Design Example Open ended question…???Do you think smoking and wound-healing are linked, and if so, how? Please write your response in the box below.
12 Stage 2- Questions Designs WordingPhrase your questions in positive toneLogical Arrangement, Appearance, and LayoutAmbiguity and ImprecisionAcronyms, Jargons, Technical LanguageAssumptionMemory and KnowledgeDouble Questions or complex and heavy loaded questionsLeading QuestionsWording Answers before the QuestionsSubjective QuestionsVague Questions
13 Reliability and Validity Reliability. A questionnaire is considered to be reliable if the results of a study can be reproduced at different times using the same questionnaire under a similar methodology.Use test-retest method to check reliabilityValidity refers to whether the questions (questionnaire) are measuring what they are supposed to measure.Validity checking: use pre-testSplit-half and internal consistency methods. Cronback Alpha correction usually gives a greater estimate of reliability than the split-half correlation. Nunnally’s(1978) indicated that internal consistency methods such as Cronbach’s alpha set an upper limit to a test’s reliability. Nunnally, J. C. (1978) Psychometric theory. New York; McGraw Hill. Quoted in
14 Stage 2- Sampling How much sample do you need?. Decide on the appropriate Sampling Method(s)Simple RandomStratified RandomSystematic SamplingCluster SamplingQuota Sampling.Some data collection methods:Consider the appropriate data collection method – self completion, interviewer assisted completion, postal, telephone completion, , internet hosting page.
15 Pay attention to the following during the planning stage Have you budgeted for non-response by sending (10-40%) more to protect your specified sample size for your power calculation?Have you carefully considered the ethics of the research, e.g. questions you should not ask? The protection of the respondents’ data?Have you cleared access to your respondents?Are you going to use software to summarise and analyse the responses? If so, do you know how to use the software or have you got access to support?
16 Pay attention to the following design issues: Themes and their logical arrangementPlace a text box under each theme to ask for an overall opinion about the themeAvoid ‘Any comment/suggestion’ question at the very end. It does not work!Clear instructions to reduce response errorNavigation – adopt question numbering and goto jumps, etc.Appearance, layout and sequenceLogical, inclusive/complete response sets
17 Piloting the Questionnaire Proofreading and EditingPre-test pilotingFeedback promptsDo you believe other people may have difficulty in answering this questionnaire? If so, which question in particular may pose a problem?Given the purpose of my study outlined in the introduction section, would you suggest a question, may perhaps be added or removed to enhance the study?How long did it take you to complete the questionnaire?Did you read the instructions?Were the instructions clear and neatly emphasised?Did you object to answering any of the questions?Was the layout of the questionnaire clear, logical and attractive?Is there any question that the wording could be enhanced or clarify?
18 Stage 3- Summarising and Analysing the data Summarise and organise the dataLook for common and unusual patterns, trends, differences, features in the dataRelate the data to the research questionUndertake appropriate statistical analyses of the dataClearly communicate the relevant findingsAccess the limitations of the study
19 Stage 4- Interpretation and Reporting of Findings Relate the findings to the research questionDraw appropriate conclusionsCompare and contrast findings with those of other relevant researchSuggest future research question leading on from here.
20 Introduction to the Questionnaire A short introduction explaining:The purpose of your research.How this study will benefit the respondent directly or indirectly.State official backing or sponsorship if any.A note of appreciation to the respondent for their contribution to your study.Confidentiality if applicable and Data protection statement.Contacting information if respondent wish to make a contact.The introduction letter MUST be part of the questionnaire and also the separate covering letter.
21 Concluding Note A short concluding note containing: A note of appreciation to the respondent for their contribution to your studyReturns addressReturns deadline
22 Frequently Asked Questions Question: Can I use tick boxes for all the questions?Answer: It depends on your intended statistical analysis. E.g. if you intend to do a comparative testing (Ttest/ANOVA) you will need the dependent variable to be a continuous data (scale or ordinal measure) and the factor variable, categorical data (choice response).Question: After the data is collected, can it be manipulated to allow certain statistics?Answer: It is possible to regroup continuous variables into categorical, or even reduce categories of a categorical variable to take advantage of certain statistics. However, it is not possible to convert categorical variable to continuous variable.
23 What Next? - Practical Session Evaluate a sample questionnaireGroup Feedback and Discussion of the sample questionnaireCreate a questionnaire from scratchExamine the design of your questions and the summaries or statistical analysis opportunities available.
24 How can I help you? We provide: Questionnaire design feedback. A guide to summary statistics and inferential statistical analysis.Assistance with reporting the statistics for publication.