1. Research Methods in Education, ECC511
Dr. Jill Taggart
City University, of Seattle, 2014
Contact:

2. Principles of Research Design
Outline of today’s lecture
Key principles
Purpose of research in education
Aims & Predictions
Validity
Design

3. Why do you need this course?
People are basically nosy – we love to pry into the lives of others. That’s part of the attraction of reality TV. Psychological research allows us to pry into the lives of others to see how they tick.

4. Principles of Research Design
Psychological research:
To explain why and how “stuff” occurs
In psychology “stuff “is the mind or behaviour
Two general questions:
What does the research consist of?
What is the explanation good for?

5. Principles of Research Design
Explanations:
Predictions:
Conceptual models that help us infer the future from the past
Theory
Conceptual models that increase our understanding of the issue
Influence
Conceptual models that help us design intervention programs
So we’re interested in explanations – what they involve and what they can be applied towards.

6. The hypothetico-deductive method
The hypothetico deductive method aims to work from an observation to a testable explanation that can be modified in the light of findings.
In amongst this method we also have categories of research:
Quantitative: numerical data
Which also includes experimental, observational, surveys, correlations
Qualitative: which focuses on explaining the meaning of events
Contexts/experience
Case studies
Interviews
Focus groups
Questionnaires can provide both qualitative and quantitative

7. Making Predictions Research questions Aims Hypotheses
Null and alternative hypotheses

8. Research Questions From where to they emanate?
What questions do you as educators want to answer?
Examples from research?

9. Aim Aim: An achievable goal
How does gender affect the nature of play between parents and their children? (Leaper, 2000)
Is there a relationship between religious belief and safe sex in young people? (Zakeski & Schiaffino, 2000)
Can dogs calm other dogs using visual communication ? (Gazzano et al. 2010)
After deciding on the research question, it now needs to be translated into an aim.
What would he aims be for these research questions.
To “investigate”…..

10. Example
The social psychology of sitting Most of our important social behaviours occur at work. This includes interactions with management, colleagues, external individuals, etc. Students attend lectures, and interact with other students and instructors. An interesting question is whether group meetings are more effective if participants are either standing or sitting. For example, many American companies insist on stand-up meetings because it saves time. The rationale behind this is that the comfort of sitting encourages longer discussion and more diversions.

11. Example, con’t
To test this hypothesis, Bluedorn et al. (1999) compared the effects of a sitting versus a standing meeting.
Researchers found that although seated meetings were 34% longer, there was no difference between the two formats in the quality of decision making. However, participants expressed greater contentment sitting rather than standing.
The authors suggest that a standing meeting format may be more effective for some companies (Bluedorn et al. 1999).

12. Example, con’t
What is the research question being investigated by Bluedorm et al (1999)?
Why was it a useful question to ask?
How would you express the aim of their experiment?

13. Research Hypotheses Statements about causal relationships
A theoretical guess about a relationship
They measure the outcome of a piece of research
It must be testable
Alternative hypothesis:
HA or H1
Is a testable statement the proposed the expected outcome of the study. Is a prediction based on the researcher’s knowledge from observations, a literature review.
In this case, he might have predicted that there would be a reduction in heart rate when introducing a dog to an older person.

14. Principles of Research Design
And since we do not know beforehand whether our research is going to support or reject the hypothesis we need 2:
The alternative hypothesis: Other possibilities that might be true
The null hypothesis: there is no difference in the relationship
Could be due to chance

15. Principles of Research Design
Experimental hypotheses
H1 proposed that there will be a difference between conditions/groups/individuals
H0 proposes that there will be no difference found between condition/groups/individuals
Example?
Correlational hypotheses
H1 proposed that there will be a difference between groups/individuals
H0 proposes that there will be no difference found between groups/individuals

16. Principles of Research Design
It is the null hypothesis we are testing:
Why?
It is much easier to decide the truth of a negative statement.
All flowers are yellow: only one purple flower will refute this statement …
Vs. the negative: Not all flowers are yellow
The null hypothesis gives us a starting point.
It must be stated from the outset.
The alternative hypothesis is essential to the design of the study, but it is the null hypothesis that we use when analysing the results so it is more appropriate to state the null hypothesis from the outset.

17. Are dogs good for old people’s hearts?

18. Are dogs good for old people’s hearts?
What is the research question?
What is the research aim?
What is being predicted?
What is the null hypothesis?
What is the alternative hypothesis?

19. Research hypotheses Derive from many sources
A broader theory
Experience and intuition
Prior research findings
The research is dialectical (theory) and is ongoing
Theories suggest tests
Test results then modify theory
The process continues….
Dialectical: discussion of theoretical concepts

20. Research Hypotheses Theories and Testing Testing tools Testing goals:
Measures: devise ways to assess the phenomenon
Manipulations: defines ways to change the phenomenon
Testing goals:
Verification: finding evidence to support the research hypothesis
Falsification: finding evidence to reject the research hypothesis
Testing is not definitive. There are often mixed and complex findings.
Often you need to be able to criticise the research:

21. Research Hypotheses Testing Theories: The best evidence
Criticizes previous measures, manipulations and the interpretation
The best evidence
Disconfirms an existing theory
Or shows a new application to an existing theory
Or distinguishes between two or more different hypotheses.

22. Research Hypotheses The best theories: In psychology, theories are:
survive repeated attempts at disconfirmation
Are general, efficient, powerful and useful
In psychology, theories are:
Generally smaller scale and specific
Tend to be heavily moderated
What is the difference between a theory and an hypothesis?
In chapter 2 Haslam and McGarty point out some important principles of psychological research. Do you remember what they were?:
Reliability = confidence that it can be replicated
Validity = a finding shows what it purports to show : have the findings been correctly interpreted; correct statistical tests. For example, are IQ tests reliable, or subject to cultural bias.
Parsimony: the best theory is often the simplest: Clever Hans

23. The Research Cycle (Cooligan, 1999, p.11)

24. Planning Research Variables: What shall we study?
Samples: Who shall we study?
Design: How should we study these?
Analysis: What sort of evidence will support in and in what form?

25. Planning Research Example:
Possible hypothesis: “Students are more irritable in hot weather.”
What is/are the variables above?
Is it heat? How can we make sure its not humidity rather than temperature that’s making people more irritable?
How do you measure irritability? - survey, observation, experimental study in the lab?

26. Planning Research Example:
Possible hypothesis: “Students are more irritable in hot weather.”
What is the sample?
Is it representative of the target population or an “opportunity sample?”
Is it random: each member of the population has an equal change of being selected?
Systematic: select every nth person
Stratified sample: individuals representing each layer or strata within the population
Quota: allocating participants to stratum after questioning all
Sample:
What is the target population?
How narrow do you want to make it?
Should be dictated by the aims of the research:
What about demographic factors such as age or gender
Could be defined by a particular experience (all students, all suffering from claustrophobia, all left handed)
Once you pick the target population you have to go about determining how you get a sample of the population to participate
And it is going to be representative of the sample or an opportunity sample – it is important because the sample may fail to accurately represent the population so it would be inappropriate to generalise the results.
Systematic
Stratafied: looking at bullying you would need a certain number from each year.
Quota – selecting participants from each chosen stratum of the population after questionning until suffficient numbers of each category have been obtained.

27. Sample Size is dependent on: Size of target population
Nature of the research
Statistical analysis
Practicalities
Ethics
Size of target population: could sample everyone if you are looking at something very specific such as a particular neural disorder or Siamese twins
Nature of the research: what will it be used for: if it is going to have an impact on policy making it has to be larger than a student project.
Stats depend on statistical significance where you generally have to have a minimum sample size. i.e. statistical power and effect sizes
Practicalities: access to participants will be limited by your time and resources

28. Planning Research Example:
Possible hypothesis: “Students are more irritable in hot weather.”
Design: How are you going to study this?
Aspects to consider in design:
1. Resources: realistic about funding, staff, technical resources, time constraints
2. Nature of the research: children (can’t do experiments-observation is better) or use animals instead
3. Previous research: if we are replicating a previous study, the design has to be identical. This might happen if you are extending the findings of a previous study.
It is realistic: can you control for confounding variables?

29. Planning Research Example:
Possible hypothesis: “Students are more irritable in hot weather.”
Analysis: What evidence will we get?
Analysis: you need to consider your analysis before setting out the design. You don’t want to move ahead with an observational study if you don’t have the time to analyse the results correctly. “garbage in/garbage out”

30. Scientific Research….
…needs to be seen for what it truly is; a way of preventing me from deceiving myself in regard to my creatively formed subjective hunches which have developed out of the relationship between me and my material.
(Reason & Rowan, 1981, p. 240)

31. Critical Reading Empirical or theoretical weaknesses:
Does the research back up claims?
Does literature support their assumptions?
Inductive reasoning:
Have they developed their research combining theory with novel means of research methods?
Re-description:
Are they describing something in a different way rather than explaining it? (word for word)
Reification
Considering an abstract concept as factual
“students get good math results because they are good with numbers”

32. Critical Reading Limited reliability Limited validity
Can findings be replicated?
Limited validity
Does it show what the researchers believes it shows?
Non-cumulativeness
Has it missed an important past source of knowledge in the literature?
Lack of parsimony:
Is it too complicated?
Lack of Scrutiny or Accountability
Has it been peered reviewed? Have the findings been communicated within the scientific community?

33. Abstracts for review
In groups of 3 Review one of the following abstracts and report back looking for:
Theoretical strengths/weaknesses
Reliability
Validity
Parsimony
What were the population samples?
Were they representative?
Were the study sizes ample?
Methodological/design issues?
Analysis issues?