1. Diploma Plus Effective Supports Network Meeting Winter 2009
Questionnaire Design
Diploma Plus Effective Supports Network Meeting
Winter 2009
Diploma Plus

2. Questionnaire Design
Questionnaire: a set of questions designed to generate the data necessary to accomplish a research project’s objectives
Diploma Plus

3. Questionnaire Design No rules can guarantee a flawless questionnaire
Flaws may only be discovered after collection of data
Pre-test if possible – small number of key stakeholders (students, parents, staff)
Diploma Plus

4. Questionnaire Design Process
Have a clear definition of the research problem or objectives
Diploma Plus

5. Questionnaire Design Process
Does each question have the most appropriate form (e.g., structured vs. nonstructured)?
Is each question relevant and properly worded to obtain meaningful, valid responses?
Diploma Plus

6. Questionnaire Design Process
Is the sequencing of the questions likely to introduce any bias?
Are the layout and appearance conducive to accurate and easy data collection (this includes length)?
Diploma Plus

7. Question Relevance and Wording
Can the respondent answer the question?
How many formative assessments have you administered to your students in the past 6 months? ______
In your opinion, how many of your students are interested in contemporary philosophy?
Ten percent ____
Twenty percent ____
Thirty percent ____
Etc…
Diploma Plus

8. Question Relevance and Wording
Will the respondent answer the question?
If sensitive or embarrassing
May refuse
May be less truthful
May irritate
Diploma Plus

9. Question Relevance and Wording
Avoid Double-Barreled Questions
Do you feel your teacher’s lessons are relevant and engaging?
Yes ____
No ____
No opinion ____
Diploma Plus

10. Question Relevance and Wording
Avoid Leading/Loaded Questions
Steer respondents toward a certain answer, regardless of what their true responses are
High risk when the focus is on people’s attitudes, beliefs, or opinions
Diploma Plus

11. Question Relevance and Wording
Avoid Leading/Loaded Questions
“Do you think your teachers this year are worse than your teachers last year?”
Diploma Plus

12. Question Relevance and Wording
Avoid One-Sided Questions
Presents only one aspect of the issue
Do you feel your school is concerned about their teachers?
____ Yes
____ No
____ No opinion
Do you feel your school is concerned or unconcerned about their teachers?
____ Concerned
____ Unconcerned
Diploma Plus

13. Question Relevance and Wording
Avoid Questions with Implicit Assumptions
Why don’t you come to school some days?
___ Can’t wake up ___ Don’t feel like it ___ To avoid a test
Why are there some classes you don’t pass?
___ They’re too hard ___ I don’t understand the material
___ Passing is not important to me
Diploma Plus

14. Question Form Nonstructured Questions (Qualitative Data)
Permit free responses
When responses are expected to be brief
Beginning an interview with an open-ended question is worthwhile
Help to ensure smooth transition from one major topic to another
Diploma Plus

15. Question Form Structured Questions (Quantitative Data)
Two basic forms:
Dichotomous questions; two answer choices, typically yes/no format
Multiple-category questions; >2 choices
Diploma Plus

16. The Likert Scale
For each of the listed statements, please check the one response that best expresses the extent to which you agree or disagree with that statement.
Statements
Definitely Agree
Generally Agree
Slightly Agree
Slightly Disagree
Generally Disagree
Definitely Disagree
I enjoy my math classes.
___
I feel safe at school.
My teachers respect me.
This slide, along with the next three draw heavily upon the excellent examples provided by the authors in Exhibits 13.6 through of our text. By taking your class participants through each of these question formats with an eye to the exhibits they should begin to become familiar with the wide variety of options available to them in this final stage before they set about fine tuning the survey instruments they’ve developed to date. It’s recommended that you draw upon some of the excellent “hands-on” activities regarding these question formats which surface in the review and discussion questions and “Student Projects”. You can either cover all the formats and then compliment them with exhibits and examples or interject the exhibits and examples as you proceed with each format.
Diploma Plus

17. The Behavior Intention Scale
Type
of School Event
Definitely Would Consider Attending
Probably Would Consider Attending
Probably Would Not Consider Attending
Definitely Would Not Consider Attending
Talent show □
Football game
Jazz Concert
Step show
Quiz Bowl
Craft fair
Student-Teacher basketball game
Diploma Plus

18. A. Graphic Rating Scales Usage (Quantity) Descriptors
Use DP.net All the Time
Never Use DP.net 10 20 30 40 50 60 70 80 90
100
Smiling Face Descriptors: 1 2 3 4 5 6 7
Before moving into this “value added” territory it’s important to emphasize that balance is the goal, especially when it comes to considering what “belongs” and “what doesn’t” in the questionnaires designed by your students. A crucial distinction for them to take away is between comparative and noncomparative scale formats. Be sure to spend some time on it before going on to cover the five options which conclude this coverage of question formats in Chapter Thirteen.
B. Performance Rating Scales Performance Level Descriptors:
Truly Terrible
Truly Exceptional
Poor
Fair
Average
Good
Excellent 1 2 3 4 5 6 7
Diploma Plus

19. Questionnaire Design Sequencing of Questions
Personal or demographic at the end
May irritate; affect willingness to complete
Put at beginning only if they serve as a filter
Sensitive questions at the end
Simple questions at the beginning
Diploma Plus

20. The Role of a Cover Letter
A solid cover letter is set up with the intention of:
Increasing a respondent’s willingness to complete the survey and return it in a timely fashion
Improving the response rate associated with the overall goals of the research endeavor
Introducing the research project and the research team to the respondent
Instilling a feeling of trust and respect with the respondent, for the study and the client/research team soliciting the data
As noted in the Instructor’s Manual, you need to decide how much emphasis you want to place on the next two slides. Recently, class participants have provided evidence that they’ve been provided with ample information about the role of a cover letter, the guidelines which need to be brought alongside its development and examples. Still, the rewarding thing about this discussion is how it conveys the information (which students may already have via a career development class or student success programs) with an eye to the principles and practice of doing marketing research.
Diploma Plus

21. Guidelines for Developing a Cover Letter
Factors
Description
Personalization
Cover letter should be addressed to the specific prospective respondent; use research firm’s professional letterhead stationery.
Identification of the organization doing the study
Clear identification of the name of the research firm conducting the survey or interview; decide on disguised or undisguised approach of revealing the actual client (or sponsor) of the study.
Clear statement of the study’s purpose and importance
Describe the general topic of the research and emphasize its importance to the prospective respondent.
Anonymity and confidentiality
Give assurances that the prospective respondent's name will not be revealed. Explain how the respondent was chosen, and stress that his or her meaningful input is important to the study’s success.
Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely.
Diploma Plus

22. Guidelines for Developing a Cover Letter
Factors
Description
General time frame of doing the study
Communicate the overall time frame of the survey or interview.
Reinforcement of the importance of the respondent’s participation
Where appropriate, communicate the importance of prospective respondents’ participation.
Acknowledgment of reasons for nonparticipation in survey or interview
Point out “lack of leisure time,” “surveys classified as junk mail,” and “forgetting about survey” reasons for not participating, and defuse them.
Time requirements and compensations
Clearly communicate the approximate time required to complete the survey; discuss incentive program, if any.
Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely.
Diploma Plus

23. Guidelines for Developing a Cover Letter
Factors
Description
Completion date and where and how to return the survey
Communicate to the prospective respondent all instructions for returning the completed questionnaire.
Advance thank-you statement for willingness to participate
Thank the prospective respondent for his or her cooperation.
Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely.
Diploma Plus

24. CONSUMER BANKING OPINION STUDY BATON ROUGE, LOUISIANA
Marketing Resources Group 3205 Windsor Oaks Drive, Suite 1105 Baton Rouge, Louisiana 70814
CONSUMER BANKING OPINION STUDY BATON ROUGE, LOUISIANA
If you have a bank account—
We need your opinion,
With more bank locations, new banking services, and the growth of credit unions and savings and loans, financial institutions are indeed changing. These changes will have an effect on you and your family, and that’s why your opinion is important.
Your name has been selected in a sample of Baton Rouge residents to determine what people in our community think about the present products and services offered by banks. Your individual opinions in this survey can never be traced back to you, and all results will be held strictly confidential. The results of the study will provide the banking industry with insight into how to better serve the needs of its customers.
The brand-new quarter enclosed with this letter is not enough to compensate you for your time, but it may brighten the day of a youngster you know.
Thank you for your assistance.
Sincerely,
Thomas L. Kirk
MRG Project Director
P.S. Please return no later than June 21, A postage-paid envelope enclosed.
In an effort to “add spice” to this discussion, it would be useful if you can access any cover letters which might have accompanied research efforts performed by a Department for Institutional Research or a Research Center on campus.
Diploma Plus