1. CMNS 260: Empirical Communication Research Methods 13-Review and Overview of the Course
Professor: Jan Marontate
Teaching Assistants: Nawal Musleh-Motut, Megan Robertson
Lab Instructor: Chris Jeschelnik
School of Communication.
Simon Fraser University
Fall 2011

2. Outline of Class Activities Today
Syllabus & Outline of Class Sessions
Objectives
Selected excerpts of lecture material to review for final examination
Study tips for final examination
Discussion of last assignment

3. Course content Introduce different forms of research
Analyze relationships between goals, assumptions, theories and methods
Study basic data collection and analysis techniques
Research process—focusing on empirical methods

4. Why study methods? Practical aspects
learn to read other people’s research & critically evaluate it
learn ways to find your own “data” to answer your own research questions
acquire skills potential employers seek
self-defense (against misinformation) & responsible citizenship

5. The Research Process
Babbie (1995: 101)

6. Why study methods?
“Knowledge is power” (to acquire skills for social action or change)
“Savoir pour pouvoir, Pouvoir pour prévoir” (Auguste Comte)
«To know to do (have power), to do (have power) in order to predict the future and plan for it  »
« Knowledge is understanding »
“décrire, comprendre, expliquer ” (Gilles Gaston Granger)
“to describe, to understand and to explain”

7. Research has the potential to inform and misinform
even well-done research is not always used accurately
some research is technically flawed
knowledge of methods an important tool for understanding logic and limits of claims about research

8. Research Methodology (Scholarly Perspectives)
Process
methods
logic of inquiry (assumptions & hypotheses)
Produces
laws, principles and theories that can be tested
(Karl Popper & notion of falsifiability for politically engaged scholars interested in the fight against genocide in the early 20th century)

9. Research has the potential to inform and misinform
even well-done research is not always used accurately
some research is technically flawed
knowledge of methods an important tool for understanding logic and limits of claims about research

10. Other Ways of Knowing
authority (parents, teachers, religious leaders, media gurus)
tradition (past practices)
common sense
media (TV. etc.)
personal experience
Talk show host Oprah Winfrey
Cory Doctorow
Electronic Frontier Assoc. &
Boingboing.net

11. Ordinary Inquiry vs. Scholarly Inquiry
Risks of “Errors” associated with non-scholarly knowledge
selective observation--only notice some phenomena-- miss others
overgeneralization-evidence applied to too wide a range of conditions
premature closure--jumping to conclusions
halo effect--idea of being influenced by prestige

12. Communication as a Science?
Field more recent
affiliations with the sciences, social sciences & the humanities
Scholarly work (like old ideas of science) distinguished from mythology by methods AND goals
many different approaches

13. Relations between theory and empirical observation
Theory and empirical research
Testing theories through empirical observation (deductive)
Using empirical observation to develop theories (Inductive)

14. Source: Singleton & Straits (1999: 27); Babbie (1995: 55)
Empirical and Logical Foundations of Research (does not have to start with theory)
Theories
The
Scientific
Process
DEDUCTION
Empirical
Generalizations
Predictions
(Hypotheses)
INDUCTION
Observations
Source: Singleton & Straits (1999: 27); Babbie (1995: 55)

15. Scholarly Communities--Norms
universalism -- research judged on “scientific” merit
organized scepticism -- challenge and question research
disinterestedness-- openness to new ideas, non-partisan
communalism--sharing with others
honesty

16. Research Questions
Questions researchers ask themselves, not the questions they ask their informants
Must be empirically testable
Not
too vague
too general
untestable (with implicit, untested assumed outcomes)

17. Developing research topics

18. “Dimensions” of Research
Purpose of
Study
Intended Use of Study
Treatment of Time in Study
Space Unit of Analysis
(examples)
Exploratory
Descriptive
Explanatory
Basic
Applied
-Action
-Impact
-Evaluation
Cross-sectional
Longitudinal
-Panel
-Time series
-Cohort analysis
-Case Study
-Trend study
-dependent individual
-independent -family
-household
-artifact
(media,
technology)
Neuman (2000: 37)

19. Exploratory Research When not much is known about topic
Surprises (e.g. Serendipity effect)
Acquire familiarity with basic concerns and develop a picture
Explore feasibility of additional research
Develop questions

20. Descriptive Research Focuses on “who”, “what” and “how”
Background information, to stimulate new ways of thinking, to classify types, etc.

21. Explanatory Research To test theories, predictions, etc…
Idea of “advancing” knowledge

22. Intended Use of Study Basic Applied
action research (We can make a difference)
social impact assessment (What will be the effects?)
evaluation research (Did it work?)
needs assessment (Who needs what?)
cost-benefit analysis (What is it worth?)

23. Basic or Fundamental Research
Concerns of scholarly community
Inner logic and relation to theoretical issues in field

24. Applied Research
commissioned/judged/used by people outside the field of communication
goal of practical applications
usefulness of results

25. Types of Applied Research
Action Research
Social Impact Assessment
Needs Assessment
Evaluation Research
formative (built in)
summative (final outcomes)
Cost-benefit analysis

26. Treatment of Time Cross-sectional (one point in time) Longitudinal
(more than one point in time)

27. Main Types of Longitudinal Studies
Panel study
Exactly the same people, at least twice
Cohort Analysis
same category of people or things (but not exactly same individuals) who/which shared an experience at at least two times
Examples: Birth cohorts. Graduating Classes, Video games invented in the same year
Time-series
same type of info., not exactly same people, multiple time periods, e.g. Same place
Burnaby residents Burnaby residents
Case Studies may be longitudinal or cross-sectional

28. Lexis Diagram (To study Cohort Survival)

29. Importance of Choosing Appropriate Unit of Analysis
example: Ecological Fallacy (cheating)

30. Ecological Fallacy

31. Ecological Fallacy

32. Ecological Fallacy & Reductionism
ecological fallacy--wrong unit of analysis
(too high)
reductionism--wrong unit of analysis (too low)
reductionism--wrong unit of analysis (too low)

33. Relationship of Theory & Empirical Observation (Wheel of Science)

34. Deductive & Inductive Methods (p. 71)

35. Conceptualization & Operationalization of Research questions
Development of abstract concepts
Operationalization:
Finding concrete ways to do research

36. Reliability & Validity
dependability
is the indicator consistent?
same result every time?
Validity
measurement validity - how well the conceptual and operational definitions mesh with each other
does measurement tool measure what we think ?

37. Hypothesis Testing

38. Possible outcomes in Testing Hypotheses (using empirical research)
support (confirm) hypothesis
reject (not support) hypothesis
partially confirm or fail to support
avoid use of PROVE

39. X Y Causal diagrams Direct relationship (positive correlation)
Indirect relationship (negative correlation)

40. Causal Diagrams Y X + X Y Z + _ X1 X2 Y + _ X1 X2 Z Y _ + X Z Y +
Neuman (2000: 56)

41. Types of Errors in Causal Explanation
ecological fallacy
reductionism
tautology
teleology
Spuriousness

42. Double-Barrelled Hypothesis & Interaction Effect
Means one of THREE things 1 2 OR

43. Interaction effect

44. Recall: Importance of Choosing Appropriate Unit of Analysis
Recall example: Ecological Fallacy (cheating)

45. Ecological Fallacy (cheating)

46. Ecological Fallacy (cheating Box)

47. Ecological Fallacy & Reductionism
ecological fallacy--wrong unit of analysis
(too high)
reductionism--wrong unit of analysis (too low)
reductionism--wrong unit of analysis (too low)

48. Teleology & Tautology
tautology--circular reasoning (true by definition)
teleology--too vague for testing
Neuman (2000: 140)

49. Spurious Relationship
spuriousness--false relationship
(unseen third variable or simply not connected)
Neuman (2000: 140)

50. Example: Storks & Babies
Observations:
Lots of storks seen around apartment buildings in a new neighbourhood with low cost housing
An increase in number of pregnancies
Did the storks bring the babies??? ?

51. But... The relationship is spurious.
The storks liked the heat coming from the smokestacks on the roof of the building, and so were more likely to be attracted to that building.
The tenants of the building were mostly young newlyweds starting families.
So…the storks didn’t bring the babies after all.

52. Causal Diagram for Storks
Baby = B
Newlywed = N
Chimneys on Building = C N B + S B + C S +

53. Another example of spurious relationships: number of firefighters & damage
The larger the number of firefighters, the greater the damage

54. But...
A larger number of firefighters is necessary to fight a larger fire. A larger fire will cause more damage than a small one.
Debate about Hockey Riots in Vancouver.
Did the size of the crowd & amount of drinking cause the riots?
Did bad planning and inadequate policing cause the fire?

55. Causal Diagram F S + D + Firefighter = F Damage = D Size of Fire = S F

56. Ethics & Legality Typology of Legal and Moral Actions in Research
Ethical
Both
Moral and
Legal
Illegal
Only
Immoral
Only
Illegal
Legal
Both
Immoral
and Illegal
Unethical
Source: figure adapted from
Neuman (2000:91)

57. Privacy, Anonymity, Confidentiality
privacy: a legal right (note : public vs. private domain)--even if subject is dead
anonymity: subjects remain nameless & responses cannot be connected to them (problem in small samples)
confidentiality: subjects’ identity may be known but not disclosed by researcher, identity can’t be linked to responses

58. 4-Measurement—Scales & Indices (Part 2 of 2 slideshows)
Neuman & Robson
Chapter 6
systematic observation
can be replicated

59. Creating Measures Measures must have response categories that are:
mutually exclusive
possible observations must only fit in one category
exhaustive
categories must cover all possibilities

60. Composite Measures
Composite measures are instruments that use several questions to measure a given variable (construct).
A composite measure unidimensional (all items measure the same construct)
Indices (plural form of index) and scales

61. Logic of Index Construction actions combined in single measure, often an ordinal level of measurement
Course Syllabus
Objectives
Course Administration
Tentative Schedule of Class Sessions

62. Logic of Scales actions ranked
Grading
Quizzes, Mid-Year and Final Exam
50%; best 3 of 4 for final grade, all must be written
Term Assignments (includes round-tables)
50%; 25% each term

63. Logic Index--example

64. Logic Scale-example

65. Treatment of Missing Data
eliminate cases with missing data?
substitute average score ?
Guess ?
insert random value ?

66. Rates & Standardization:
deciding what measure to use for reference populations example: employment rates

67. Sampling: key ideas & terms

68. Bad sampling frame
= parameters do not accurately represent target population
e.g., a list of people in the phone directory does not reflect all the people in a town because not everyone has a phone or is listed in the directory.

69. Types of Nonprobability Samples4

70. Types of Probability Samples link to useful webpage: http://www16

71. Stratified

72. Evaluating Sampling
Is the sample representative of the population under study?
Assessing Equal chance of being chosen
Examine Sampling distribution of parameters of population
Use Central Limit Theorem to calculate Confidence Intervals and estimate Margin of Error

73. Asking Questions that can be answered

74. Types of Surveys & Survey Instruments
Self-administered Surveys
Mail
Web
Surveys based on Interactive Interviews
Telephone
Online (interactive)
Face-to-face
Individuals
Focus groups
Survey Instruments:
Questionnaires
self-administered
Respondent reads questions & records answers
Interview Schedules
interviewer reads questions & records responses

75. Main Types of Unobtrusive Measures
Physical traces
Erosion (ex. wear on floor in museum displays as measure of popularity of display)
Accretion (ex. garbage)
Simple observation
Media analysis such as content analysis, critical discourse analysis (ex. advertisements, news reports, films, music lyrics etc…)
Analysis of archives, existing statistics & running records (ex. shoppers’ records, library borrowers’ histories)

76. Types of Equivalence for comparative research using existing statistics
lexicon equivalence (technique of back translation)
contextual equivalence (ex. role of religious leaders in different societies)
conceptual equivalence (ex. income)
measurement equivalence (ex. different measure for same concept)

77. Discrete & Continuous Variables
Variable can take infinite (or large) number of values within range
Ex. Age measured by exact date of birth
Discrete
Attributes of variable that are distinct but not necessarily continuous
Ex. Age measured by age groups (Note: techniques exist for making assumptions about discrete variables in order to use techniques developed for continuous variables)

78. Cleaning Data checking accuracy & removing errors
Possible Code Cleaning
check for impossible codes (errors)
Some software checks at data entry
Examine distributions to look for impossible codes
Contingency cleaning
inconsistencies between answers (impossible logical combinations, illogical responses to skip or contingency questions)

79. Treatment of Missing Data (%)
Comparison with medium & low collapsed
Table 5-1 Alienation of Workers
Level of Alienation F %
High Medium & Low
No Response
(Total)
Table 5-1 Alienation of Workers
Level of Alienation F %
High Medium & Low
(Total)
Non-respondents eliminated
Non-respondents included

80. Grouping Response Categories(%)
Comparison of with high & medium response categories collapsed
Table 5-1 Alienation of Workers
Level of Alienation Freq %
High & Medium
Low
No Response
(Total)
Table 5-1 Alienation of Workers
Level of Alienation Freq %
High& medium
Low
(Total)

81. Core Notions in Basic Univariate Statistics
Ways of describing data about one variable (“uni”=one)
Measures of central tendency
Summarize information about one variable
three types of “averages”: arithmetic mean, median, mode
Measures of dispersion
Analyze Variations or “spread”
Range, standard deviation, percentiles, z-scores

82. Normal & Skewed Distributions

83. Details on the Calculation of Standard Deviation
Neuman (2000: 321)

84. The Bell Curve & standard deviation

85. If Time: Begin Bivariate Statistics (Results with two variables)
Types of relationships between two variables:
Correlation (or covariation)
when two variables ‘vary together’
a type of association
Not necessarily causal
Can be same direction (positive correlation or direct relationship)
Can be in different directions (negative correlation or indirect relationship)
Independence
No correlation, no relationship
Cases with values in one variable do not have any particular value on the other variable

86. Recall (Lecture 2) *Types of variables*
independent variable (cause)
dependent variable (effect)
intervening variable
(occurs between the independent and the dependent variable temporally)
control variable
(temporal occurance varies, illustrations later today)

87. Causal Relationships proposed for testing (NOT like assumptions)
5 characteristics of causal hypothesis (p.128)
at least 2 variables
cause-effect relationship (cause must come before effect)
can be expressed as prediction
logically linked to research question+ a theory
falsifiable

88. Types of Correlations & Causal Relationships between Two Variables
X=independent variable Y=dependent variable
Positive Correlation (Direct relationship)
when X increases Y increases or vice versa
Negative Correlation (Indirect or inverse relationship)
when X increases Y decreases or vice versa
Independence
no relationship (null hypothesis)
Co-variation
vary together ( a type of association but not necessarily causal) Y X + Y X -

89. Five Common Measures of Association between Two Variables

90. General Idea of Statistical Significance
In general English ‘significance’ means important or meaningful but this is NOT how the term is used in statistics
Tests of statistical significance show you how likely a result is due to chance.

91. Multi-variate Statistics: Elaboration Paradigm (Types of Patterns)
Replication: same relationship in both partials as in bivariate table
Specification: bivariate relationship only seen in one of the partial tables
Interpretation: bivariate relationship weakens greatly or disappears in partial tables (control variable is intervening—happens in between independent & dependent)
Explanation: Bivariate relationship weakens or diappears in partial table (control variable is before independent variable)
Suppressor: No bivariate relationship; relationshp only appears in partial tables.

92. Elaboration Paradigm Summary

93. Study Tips for Final Exam
Practice questions
Other ideas for preparation