1. First Steps in Designing Quantitative Research: Asking the Right Questions
CETL Workshop, July 10, 2012
Susan R. Hutchinson, Ph.D.
Department of Applied Statistics & Research Methods

2. Workshop Outline
Overview of purposes for conducting quantitative research
Role(s) served by research questions
Attributes of useful RQs
Group activity on evaluating RQs
Exercise on constructing RQs
CETL Workshop, July 10, 2012

3. Pretest
CETL Workshop, July 10, 2012

4. All quantitative studies include one or more research questions.
False
CETL Workshop, July 10, 2012

5. Research questions and research hypotheses are the same thing.
False
CETL Workshop, July 10, 2012

6. Research questions should “foreshadow” the data analyses.
True
CETL Workshop, July 10, 2012

7. Research questions should be posed after data are collected.
CETL Workshop, July 10, 2012
False

8. Research questions should be written using non-specific language to allow flexibility in designing the study.
CETL Workshop, July 10, 2012
False

9. All of the following are types of variables that might be included in research questions: dependent, moderator, mediator, extraneous, independent
CETL Workshop, July 10, 2012
True

10. Writing quantitative research questions requires knowledge of statistics.
True
CETL Workshop, July 10, 2012

11. What are the major purposes for conducting quantitative research?
CETL Workshop, July 10, 2012

12. Quantitative research is intended to examine:
Differences between groups
Statistical significance, magnitude, and direction of differences
Change over time
Direction, magnitude, and between-group differences
Relationships between 2 or more variables
Direction, magnitude, type/form, between-group differences
Descriptions of populations and phenomena
CETL Workshop, July 10, 2012

13. As you’re constructing your research questions, keep in mind the overriding purpose of quantitative research is to understand differences
CETL Workshop, July 10, 2012

14. What role do research questions play in designing quantitative research?
CETL Workshop, July 10, 2012

15. What information should research questions include?
Indicate the target population
Identify the variables/constructs and what we want to know about them
Foreshadow the type of data analysis (and sometimes the research design)
CETL Workshop, July 10, 2012

16. What characterizes a “good” research question?
Clearly conveys the variables/constructs and the nature of their relationships
Includes independent, dependent, moderator, and mediator variables (where applicable)
Must be answerable based on data from the study
RQs should not be rhetorical or require a value judgment to answer
RQs should match the research design
Should be worth investigating
A good RQ does not address a trivial issue; “so what?”
Answers to RQs should further the researcher’s understanding of the phenomenon
CETL Workshop, July 10, 2012

17. “It is better to have an approximate answer to the right question than an exact answer to the wrong one.” John Tukey
CETL Workshop, July 10, 2012

18. Types of variables that might appear in research questions
Dependent
Independent
Extraneous
Moderator
Mediator
CETL Workshop, July 10, 2012

19. Dependent variable(s)
The dependent variable is a phenomenon one is attempting to explain or predict
Also referred to as the outcome or criterion variable
In a survey study, often this is the variable or construct of primary interest to the researcher
It is something which the researcher wishes to understand more fully
For many researchers conducting nonexperimental research this is the starting point in planning a study
SRM 627 Survey Research Methods, Hutchinson

20. Independent variable(s)
In an experimental or quasi-experimental design, an independent variable would be the manipulated or treatment variable in your study
In a survey or other type of nonexperimental study, where no variables are manipulated, independent variables are the factors used to explain or predict the DV
SRM 627 Survey Research Methods, Hutchinson

21. Extraneous variables
Extraneous variables are also called nuisance variables
These are variables which are not usually of primary interest but are believed to be related to the independent and/or dependent variables
Their effects need to be controlled in order to obtain meaningful results
Extraneous variables are of two broad types:
dispositional and situational
SRM 627 Survey Research Methods, Hutchinson

22. Illustration of controlling for extraneous variables
salary
job
satisfaction
SRM 627 Survey Research Methods, Hutchinson
A researcher is interested in examining the relationship
between salary and job satisfaction. However, the
researcher suspects that factors other than salary (such
as job autonomy) are also responsible for an
employee’s level of job satisfaction.

23. Relationship with “job autonomy" taken into account or controlled
satisfaction
salary
job
autonomy
SRM 627 Survey Research Methods, Hutchinson
Relationship with “job autonomy" taken
into account or controlled

24. Moderator variable(s)
A moderator variable influences the strength of the relationship between the primary independent variable and the dependent variable
Usually a moderator variable is of secondary interest
A moderator variable would be the variable that interacts with the independent variable
SRM 627 Survey Research Methods, Hutchinson

25. Illustration of moderator effect
Satisfaction with
online course
low aptitude
high aptitude
SRM 627 Survey Research Methods, Hutchinson
Computer skills/comfort

26. Mediator variable Also referred to as an intervening variable
A mediator (or mediating) variable is some construct through which the independent variable indirectly affects the dependent variable
SRM 627 Survey Research Methods, Hutchinson

27. Illustration of mediator variable
Parents’
educational
level
Achieve-
ment
motivation
Performance
on standard-
ized tests
SRM 627 Survey Research Methods, Hutchinson

28. Examples of less than ideal RQs
What are psychologists’ perceptions of professional development?
Are university faculty depressed?
What is the relationship between self-esteem and happiness?
How many undergraduate students dropped out of college in 2011?
What factors predict job satisfaction?
How are exercise and blood pressure related?
Is there a difference between gender and depression?
What is the impact of test anxiety on test performance?
Is religion good for society?
CETL Workshop, July 10, 2012

29. What are psychologists’ perceptions of professional development?
What is the researcher hoping to learn?
What are the variables/constructs in the RQ?
What is the target population?
How might this RQ be improved?
Which aspects of professional development do psychologists rate most versus least positively?
CETL Workshop, July 10, 2012

30. Or, instead of posing a descriptive question about a particular construct, consider rewording so that you can explain differences on the construct.
CETL Workshop, July 10, 2012

31. To what extent are private practice psychologists’ perceptions of the need for professional development explained by their years of experience, area of specialization, and professional degree?
CETL Workshop, July 10, 2012

32. Are university faculty depressed?
What are the independent and dependent variables?
What assumption about university faculty is implied by the research question?
What are possible revisions?
Does level of depression among university faculty differ based on their gender, departmental affiliation, and tenure status?
CETL Workshop, July 10, 2012

33. Are university faculty depressed?
What are the independent and dependent variables?
What assumption about university faculty is implied by the research question?
What are possible revisions?
Does level of depression among university faculty differ based on their gender, departmental affiliation, and tenure status, after controlling for marital status and life satisfaction?
CETL Workshop, July 10, 2012

34. Hypothetical depression scores
CETL Workshop, July 10, 2012

35. For complex constructs that are likely to be multiply determined, you should attempt to control for potentially extraneous variables.
CETL Workshop, July 10, 2012

36. What is the relationship between self-esteem and happiness?
What do you think the researcher wants to know?
How might the question be revised to more clearly convey the type of information sought?
What is the strength of relationship between self-esteem and happiness?
Is self-esteem linearly related to happiness among college-educated adults?
CETL Workshop, July 10, 2012

37. How many undergraduate students dropped out of college in 2011?
What is wrong with this question?
How might it be improved in order to provide more useful information?
Which of the following variables predicted undergraduate college student dropout in 2011? (grade point average, perceived social support, ….)
CETL Workshop, July 10, 2012

38. What factors predict job satisfaction?
What are the independent and dependent variables?
What is the target population?
How might you reword this RQ so that it provides a clearer focus the researcher?
To what extent do salary, autonomy, ….
predict job satisfaction among full-time employees?
CETL Workshop, July 10, 2012

39. How are exercise and blood pressure related?
What is the independent and dependent variable?
Target population?
How would you reword this RQ if you had reason to believe gender could be a moderator variable?
Does the strength of relation between amount of daily aerobic exercise and blood pressure differ for male and female adults?
CETL Workshop, July 10, 2012

40. Is there a difference between gender and depression?
What flaw do you notice in this RQ?
What do you think the researcher actually wants to know?
What is a possible revision?
CETL Workshop, July 10, 2012

41. What is the impact of test anxiety on test performance?
Assuming the researcher plans to collect data via survey, what part of this RQ needs to be reworded and why?
How might you revise this item if you wanted to determine if study strategies was a mediator variable?
Do study strategies mediate the relationship between test anxiety and test performance among doctoral level social science majors?
CETL Workshop, July 10, 2012

42. Is religion good for society?
Why is this not a suitable research question?
How could you revise the question so that it is suitable?
Are rates of violent crimes lower in communities with higher attendance at formal religious services?
CETL Workshop, July 10, 2012

43. Other considerations when writing RQs
Be careful about your choice of words
Verbs such as improve, impact, affect, and predict imply certain types of research designs
Think about how you plan to answer your research questions as you write them
Consider the research design, measurement, and data analysis
CETL Workshop, July 10, 2012

44. Research question templates
Do [groups of interest] differ on [name of construct]?
Do male and female undergraduate students differ in their level of statistical test anxiety?
CETL Workshop, July 10, 2012

45. Templates – cont’d Does [name of construct] differ by [variable]?
Does level of participation in Greeley Parks & Recreation Center classes differ by age, educational level, gender, or ethnic group?
CETL Workshop, July 10, 2012

46. Templates – cont’d
To what extent does a set of variables explain (or predict) a given outcome variable?
To what extent do salary, working conditions, and types of job tasks predict job satisfaction?
CETL Workshop, July 10, 2012

47. Templates – cont’d
Is there a relationship between [variable] and [variable]?
Is there a relationship between teachers’ job satisfaction and perceived level of administrative support?
CETL Workshop, July 10, 2012

48. Templates – cont’d
Is the relationship between [variable] and [variable] the same for [names of groups]?
Is the relationship between teachers’ job satisfaction and perceived level of administrative support the same for novice versus experienced teachers?
CETL Workshop, July 10, 2012

49. Templates – cont’d
Does a given explanatory variable (or set of variables) explain a criterion, after taking into account one or more extraneous variables?
Does child’s achievement motivation explain performance on statewide standardized tests, after controlling for parents’ SES and academic self-concept?
CETL Workshop, July 10, 2012

50. Templates – cont’d
Does [dependent variable] change [or improve or decline] across time among [population of interest]?
Does reading level improve during a two-month summer intensive reading program for elementary school English language learners?
CETL Workshop, July 10, 2012

51. Templates – cont’d
Is the pattern of change [or improvement or decline] on [name of dependent variable] the same for [names of groups]?
Is the rate of improvement in reading level the same for males and females?
CETL Workshop, July 10, 2012

52. Question Writing Exercise
Consider a quantitative study you might conduct
Think about the overall purpose and how the research questions will help support the purpose
You are to compose at least two research questions, each of which should include a dependent variable and one or more independent variables.
One of your RQs will include one or more extraneous variables and the other will include a moderator variable.
CETL Workshop, July 10, 2012