Designing and Analyzing Questionnaires Dr. Gene Education Institute

Purpose of Questionnaires Questionnaires are the most common data collection methods used in research. They can be used to assess: Attitudes Opinions Interests Values

Advantages and Disadvantages Can assess a large group quickly Easy to analyze if constructed correctly Disadvantages Requires “good” language skills Some people give answers they think you want Not very good for getting in-depth information

Types of Questionnaires Structured Respondents select a response from those given Unstructured Respondents create a response Combination Contains structured and unstructured items

Sample Structured Questionnaire Why did you choose teaching as a career? Tick each reason that applies to you. ___ I like working with children. ___ The pay is better than most jobs. ___ I can help my country and society. ___ It is a safe profession for women.

Sample Unstructured Questionnaire Why did you choose teaching as a career? In the space below give the reasons why you chose to become a teacher.

Types of Questions Closed questions Open questions Choices are provided to the respondents Open questions Respondents must create responses Demographic questions Questions about the characteristics of the respondents

Types of Closed Questions Dichotomous Questions (2 choices) Yes/No Agree/Disagree Do you think a paper recycling program should be started in your school? Yes No

Types of Closed Questions Multiple-choice Questions Give respondents options May ask for single or multiple answers Example: How did you hear about our Website? ___ Newspaper ___ Magazine ___ Radio ___ Internet ___ Other: Please specify __________

Types of Closed Questions Rank Order Questions Respondents place things in order Example: Which activities do you like to do in your spare time? Place a “1” next to the activity that you like to do most, a “2” by the next favorite, and so on to the least favorite. ___ Watch TV ___ Read ___ Visit friends ___ Surf the Internet ___ Shop

Types of Closed Questions Rating Scale (Semantic Differential) Also called a “Likert Scale” Give a statement; choose your response along a scale Example: My students are motivated to learn. Strongly Disagree Disagree Not Sure Agree Strongly

Open Questions Used to explore topics in-depth Gives people a chance to respond in detail They are time-consuming to summarize and analyze

Types of Open Questions Open-ended Questions Example: What changes would you like to see in our science class? Stem Plus Questions Some things I would like to see change in our science class are _________________

Demographic Questions Demographic questions may ask about personal characteristics such as -- age grade ethnicity gender home characteristics

Types of Scales 2-level (yes/no; agree/disagree) 3-level (yes/no/not sure) 5-level (Likert scale – Strongly Agree, Agree, Undecided, Disagree, Strongly Disagree) Larger scales are difficult to analyze

Neither Agree or Disagree Effects of Scales The police need to do a better job of enforcing traffic laws. Disagree Agree 10% 90% Disagree Agree 10% 90% 2-level 2-level Disagree Neither Agree or Disagree Agree 2% 28% 70% 3-level

Effects of Scales The police need to do a better job of enforcing traffic laws. Strongly Disagree Disagree Undecided Agree Strongly Agree 1% 28% 63% 7% 5-level

Problems to Avoid in Writing Questions Leading Questions (Loaded Questions or Biased Questions) Examples: Don’t you agree that discipline is a major problem in this school? Do you believe it is all right to spank young children despite the SEC recommendations not to do so?

Problems to Avoid in Writing Questions Double-Barreled Questions (asking two things in one question) Examples: Did this class increase your interest in science and motivate you to study hard? Do you agree that pay is low for teachers and that the government should be working hard to increase it?

Problems to Avoid in Writing Questions Unclear or ambiguous questions Examples: What do you think about school? What role should the principal play in educating students?

Problems to Avoid in Writing Questions Halo Effect (questions that link a position to a particular person or group) Examples: Do you agree with the Emir that research should be one of the top priorities of Qatar? Qatar Petroleum wants the Independent Schools to do a better job of teaching science so that students are prepared for the workplace. Would you agree?

Problems to Avoid in Writing Questions Invasive questions about personal information Examples: Do you ever argue with your parents? What are the major problems in your home?

Tips for Creating Good Questionnaires Know your audience Make sure that the length, content, and wording matches the intended audience Keep questions clear and concise Avoid technical wording

Tips for Creating Good Questionnaires Be certain of your goals Before you start writing questions, be clear in your own mind what you want to find out

Tips for Creating Good Questionnaires Make the directions and format clear and easy to follow Put your directions and demographic questions at the beginning of the questionnaire. If possible, read and explain the directions orally

Tips for Creating Good Questionnaires Try the questionnaire out before you use it Find a similar group that is not in your study. Ask them to read the questionnaire and tell you if something is not clear. Revise your questions based on the information you get from your tryouts.

Tips for Creating Good Questionnaires Don’t ask too many questions Keep your questionnaire short and to the point People get tired and lose interest on long questionnaires so the responses you get toward the end may not be valid.

Ways to Deliver Questionnaires Paper-pencil/face-to-face Mail or take home Telephone Online delivery and analysis

Ways to Analyze The Results of a Structure Questionnaire Summarize the percentages for each question Students should be required to wear uniforms to school. Strongly Disagree Disagree Undecided Agree Strongly Agree 10% 20% 40%

Ways to Analyze The Results of a Structure Questionnaire Calculate the mean (average) for each item. Students should be required to wear uniforms to school. Strongly Disagree (1) Disagree (2) Undecided (3) Agree (4) Strongly Agree (5) 10 20 40

Ways to Analyze The Results of a Structured Questionnaire Calculate the mean (average) for each item. 20 x 5 = 100 340 / 100 = 3.4 40 x 4 = 160 10 x 3 = 30 20 x 2 = 20 10 x 1 = 10 340 Strongly Strongly Diagree Disagree 5 4 3 2 1 3.4

Ways to Analyze The Results of an Unstructured Questionnaire Summarize the comments Look for patterns or themes that run through the comments Give percentages for major themes