1. DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO A. SEALEY Week One: An Introduction to the Course Political Science 242: ______________________ An Introduction to Research Methods

2. Week One: ______________________ An Introduction to The Course A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

3. Lecture Two ______________________ An Introduction to Some Key Concepts and Tools A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

4. Part One ______________________ An Introduction to Some Key Concepts A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

5. Key Concepts to be Discussed __________________________ 1) Normative and Empirical Analysis 2) Deductive and Inductive Logic 3) Theories, Concepts and Variables 4) Measures and Indicators A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

6. Key Concepts to be Discussed __________________________ 5) Independent, Dependent, and Intervening Variables 6) Null and Alternative Hypotheses 7) Falsifiability and Replicability 8) Correlation, Causation and Spuriousness A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

7. Theories, Concepts and Variables __________________________ In the social sciences, we use the term ‘theory’ to refer to explanations of observable regularities or patterns. Theory tell us why what we know to be true is true. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

8. e.g. Some people believe that Canadian political culture is more ‘deferential’ thanAmerican political culture and that Canadians are more likely to favour a greater level of government intervention that Americans. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

9. Seymour Martin Lipset’s ‘Formative Events’ theory suggests that this difference can be explained by the differences in the events which led to the creation of these two countries. While the United States was an outcome of a violent revolution against the British government, the Canadian state was created peacefully. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

10. ‘Concepts’ can be thought of as theoretical building blocks which we use to build theories. Concepts are often quite abstract, but can also be quite concrete. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

11. ‘Concepts’ can be thought of as theoretical building blocks which we use to build theories. Concepts are often quite abstract, but can also be quite concrete. e.g. ???? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

12. ‘Concepts’ can be thought of as theoretical building blocks which we use to build theories. Concepts are often quite abstract, but can also be quite concrete. e.g. Power, democracy, left- vs. right-wing, feminism, moral traditionalism. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

13. Broadly speaking, a ‘variable’ is anything that can take on a variety of possible values. The creation of variables is an important step toward the operationalization of a concept. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

14. When we operationalize a concept, we provide a specific definition of it that allows us to measure it. The creation of specific variables helps us to do this. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

15. e.g. For the purposes of this analysis, a ‘war’ is any armed conflict in which at least 1000 people killed in armed conflict in a given year. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

16. Measures and Indicators __________________________ A measure is a variable that has been constructed in order to operationalize a specific concept. An indicator is a variable that is combined with other indicators in order to create a measure. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

17. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course e.g. Creating a Measure for ‘Social Progressivism’ Social Progressivism Outlooks on Gay Rights Outlooks on Prostitution Outlooks on Abortion

18. Notice that in some case, a given variable can be conceived of as either a measure or an indicator. e.g. ‘Outlooks on gay rights’ can be used either as a measure of ‘homophobia’ or an indicator of a measure of ‘social progressivism’. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

19. Questions ??? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week One: An Introduction to the Course

20. Week One: An Introduction to Research Methods A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO

21. Week Two: Approaches to Comparative Research A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Political Science 242: ______________________ An Introduction to Research Methods

22. Week Two: Approaches to Comparative Research Week Two ______________________ Approaches to Comparative Research A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO

23. Week Two: Approaches to Comparative Research A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Lecture One ______________________ Key Concepts in Comparative Research

24. Week Two: Approaches to Comparative Research DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO A. SEALEY 1) Qualitative and Quantitative Approaches 2) Descriptive, Exploratory and Explanatory Research 3) Ideographic and Nomothetic Explanations 4) Building and Testing Theories 5) Populations and Samples 6) Measurement, Sampling and Error Key Concepts to be Discussed __________________________

25. Week Two: Approaches to Comparative Research DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO A. SEALEY 7) Credibility, Transferability and Validity 8) Dependability and Reliability 9) Confirmability and Replicability 10) Validity, Reliability and Bias 11) Cases, Units of Analysis and Bases of Comparison Key Concepts to be Discussed __________________________

26. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Qualitative and Quantitative Approaches to Comparative Research __________________________ Broadly speaking, most of the principal research techniques used by political scientists can be located within one of two principal categories: qualitative and quantitative. Week Two: Approaches to Comparative Research

27. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Qualitative approaches to research are often centered on linguistic or discursive analysis, whereas quantitative approaches tend to focus on statistical analyses of quantifiable data. Drawing comparisons between similar qualitative and quantitative approaches are helpful for illustrating this distinction. Week Two: Approaches to Comparative Research

28. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #1 Interviews vs. Survey Research One qualitative approach to political science research is to interview subjects about their knowledge of particular political events or processes. Such an approach is highly analogous to survey research, which is a quantitative approach that also asks research subjects to respond to a series of questions. Week Two: Approaches to Comparative Research

29. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #1 Interviews vs. Survey Research Perhaps the biggest difference between these two approaches is that the set of responses that research subjects can provide are highly structured in survey research, whereas interviews allow research participants greater latitude to respond to questions in whichever way they prefer. Week Two: Approaches to Comparative Research

30. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #1 Interviews vs. Survey Research For the survey researcher, having a fixed set of possible responses is crucial, as it allows for the application of statistical techniques to the data once it is aggregated; but from the interviewer’s perspective, forcing subjects to respond in a limited number of ways eliminates the potential for original, creative or unexpected responses. Week Two: Approaches to Comparative Research

31. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #2 Qualitative vs. Quantitative Content Analysis Political scientists use both qualitative and quantitative techniques to analyze the content of political document. Week Two: Approaches to Comparative Research

32. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #2 Qualitative vs. Quantitative Content Analysis Qualitative content analysis generally involves carefully reading and coding passages of the text of each document. Conversely, quantitative content analysis usually involves more mechanized analytic processes, such as generating counts of specific words in given sets of documents. Week Two: Approaches to Comparative Research

33. Here, the biggest difference between these two approaches is the degree of involvement of the researcher in the analytic process. But again, the outcome is a relative de-emphasis of linguistic or discursive detail in quantitative research. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #2 Qualitative vs. Quantitative Content Analysis Week Two: Approaches to Comparative Research

34. Sometimes the goal of political science research is to determine the effects of specific policies on particular social outcomes. e.g. What is the effect of varying approaches to health policy on the overall level of citizens’ health? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #3 Qualitative vs. Quantitative Public Policy Analysis Week Two: Approaches to Comparative Research

35. In some instances, researchers use qualitative approaches to assess these relationships by providing detailed case studies of particular countries’ social policies and citizens’ health. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #3 Qualitative vs. Quantitative Public Policy Analysis Week Two: Approaches to Comparative Research

36. In other instances, researchers use quantitative approaches to assess these relationships by attempting to establish correlations between types of social policies and levels of citizens’ health. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. #3 Qualitative vs. Quantitative Public Policy Analysis Week Two: Approaches to Comparative Research

37. So generally speaking, it seems reasonable to conclude that qualitative approaches often allows for the inclusion of more fine-grained, detailed analysis by the researcher. On the other hand, because it requires less attention to detail, quantitative research often allows for an analysis of a greater breadth and quantity of data. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research

38. What do we mean by a ‘greater quantity of data’? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research e.g. #1 Interviews vs. Survey Research > more subjects. e.g. #2 Content Analysis > more texts. e.g. #3 Public Policy Analysis > more countries (or provinces, etc.).

39. Questions ??? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research

40. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Populations and Samples __________________________ Often researchers are interested in making general claims about relationships between particular political concepts. The complete set of all things to which the specified relationship is thought to apply is referred to as the ‘population’ of the analysis. Week Two: Approaches to Comparative Research

41. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research e.g. What was the population being analyzed when we investigated the relationship between gender and attitudes towards same-sex marriage?

42. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO While the population is what is being analyzed, it is often impractical to gather information on the complete set of things included in the population. For this reason, researchers often gather information about a subset of the population – referred to as a sample – and try to draw inferences about the population based on the information gathered from the sample. Week Two: Approaches to Comparative Research

43. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO In many respects, the best possible type of sample is a ‘random’ sample, because randomization generally ensures that samples are representative and allows us to determine the likelihood that a given sample is unrepresentative. In many instances, however, non-random sampling techniques are more convenient and sometimes even preferable. Week Two: Approaches to Comparative Research

44. DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO A. SEALEY 1) Systematic Sampling 2) Stratified Sampling 3) Cluster Sampling 4) Purposive Sampling 5) Deviant Case Sampling 6) Snowball Sampling Non-Random Sampling Techniques __________________________

45. Questions ??? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research

46. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Measurement, Sampling and Error __________________________ Notice that we now have two possible sources of error from the process of operationalizing our concepts. The first source of error comes from the measurement process (measurement error). The second source of error comes from the sampling process (sampling error). Week Two: Approaches to Comparative Research

47. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO However, it is possible (although potentially dangerous) to think of sampling error as a type of measurement error. Week Two: Approaches to Comparative Research

48. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research It is also worth drawing attention to the fact that in quantitative analysis, the availability of measures often drives the selection of measures.

49. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research e.g. Measuring attitudes towards feminism in the World Values Survey: Compare outlooks on the statement: D059 – “On the whole, men make better political leaders than women do.” with outlooks on this statement: D062 – “A job is alright but what most women really want is a home and children.”

50. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research Now let’ compare data availability:

51. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research Now let’ compare data availability: little data is missing for ‘femism1’

52. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research Now let’ compare data availability: all of the data is missing for ‘femism2’

53. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research So what do we do? We use ‘femism1’ (D059) not because it’s a more valid measure than ‘femism2’ (D062), but because ‘femism2’ isn’t available.

54. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research Finally, it is important to note that in many instances, the operationalization of measures is often highly controversial and affected by the values and beliefs that scholars bring to their research. e.g. ‘Relative’ vs. ‘absolute’ measures of poverty.

55. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research ‘Measurement’ clip from: The Gapminder Foundation http://www.gapminder.org/videos/http://www.gapminder.org/videos/human-rights- democracy-statistics//

56. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Credibility, Transferability and Validity __________________________ Validity is a concept most easily identifiable with quantitative research. The term has a wide range of possible meanings in the field of research methods, but the central idea revolves around notions of accuracy and truthfulness. Week Two: Approaches to Comparative Research

57. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO First, we can think of ‘measurement validity’. For a measure to be valid, it must accurately represent the concept that it is intended to operationalize. Week Two: Approaches to Comparative Research

58. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO One aspect of measurement validity is ‘face validity’. A measure has face validity if it is an appropriate operationalization of the concept. Week Two: Approaches to Comparative Research e.g. Which has greater face validity as a measure of ‘animal rights activism’: whether someone owns a pet or whether an individual donates to animal shelters?

59. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO The text also discusses the ideas of ‘convergent’ and ‘divergent validity’. These notions of validity can be applied to indicators. Indicators are said to have convergent validity if the variables are thought to be indicators of the same measure and they yield similar results for most cases. Week Two: Approaches to Comparative Research

60. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research e.g. The indicators ‘opposition to same-sex marriage’ and ‘opposition to abortion rights’ are said to have ‘convergent validity’ if they are thought to be indicators of a measure of ‘moral traditionalism’ and they yield similar results for most cases.

61. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Indicators are said to be ‘divergently valid’ if the variables are thought to be indicators of the same measure – but have reverse directionalities – and they yield opposing results for most cases. Week Two: Approaches to Comparative Research

62. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research e.g. The indicators ‘support for same-sex marriage’ and ‘opposition to abortion rights’ are said to have ‘divergent validity’ if they are thought to be indicators of a measure of ‘moral traditionalism’ but have reverse directionalities and they yield opposing results for most cases.

63. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO We can also apply the notion of validity to studies themselves. One such application is the idea of ‘external validity’. An analysis is said to have external validity if its findings can be generalized from the sample included in the analysis to cases outside the sample. Week Two: Approaches to Comparative Research

64. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Credibility and transferability are concepts that have been developed by qualitative researchers in as parallels to the notions of measurement and external validity in quantitative research. Week Two: Approaches to Comparative Research

65. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Qualitative research is said to be ‘credible’ if the data used in the qualitative account fits the world being described; the qualitative account must be believable. Qualitative research is said to be ‘transferable’ if the findings can be applied to other contexts. Week Two: Approaches to Comparative Research

66. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Dependability and Reliability __________________________ Another important characteristic of quantitative measures is that they should be reliable. A measure is said to be reliable if it consistently obtains comparable results in a variety of instances of measurement. Week Two: Approaches to Comparative Research

67. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO In qualitative research, the analogous attribute is often described as ‘dependability’, but again refers to the idea of a consistency between the collected data and the conclusions drawn (the results). Another way of thinking about this is to ask: would the results be consistent if the analysis of the collected data is repeated by other researchers? Week Two: Approaches to Comparative Research

68. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Confirmability and Replicability __________________________ Some qualitative researchers also draw a distinction between the ideas of confirmability and replicability. Such a distinction is quite subtle, however, and probably exaggerates the extent to which quantitative analyses are actually replicable. Week Two: Approaches to Comparative Research

69. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO The key idea for both is to ask: if we were to redo the study again, would the conclusions drawn be the same again? Week Two: Approaches to Comparative Research

70. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Terminological Summary __________________________ Week Two: Approaches to Comparative Research Quantitative Research Qualitative Research Measurement Validity Credibility External ValidityTransferability ReliabilityDependability ReplicabilityConfirmability

71. Questions ??? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research

72. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Validity, Reliability and Bias __________________________ As we have seen, the concept of ‘validity’ has a broad range of possible applications. However, two important criteria by which to conceptualize validity involve reliability and biasedness. Valid measures should be both reliable and unbiased. Week Two: Approaches to Comparative Research

73. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO A reliable or consistent estimator is one that tends to produce estimates that do not differ significantly from each other (i.e. the variance of the estimates is low). An unbiased estimator is one for which the average of all possible sample statistics is equal to the population parameter that it is estimating. Week Two: Approaches to Comparative Research

74. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research     e.g. #1: Reliable but Biased

75. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research     e.g. #2: Unbiased but Unreliable

76. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research     e.g. #3: Biased and Unreliable

77. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research     e.g. #4: Reliable and Unbiased

78. A Schematic Representation of Some Aspects of the Concept of Validity A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research

79. Validity Internal Validity Measurement Validity Reliability Face Validity Unbiased- ness Convergent Validity External Validity

80. Questions ??? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Two: Approaches to Comparative Research

81. Week One: An Introduction to Research Methods A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO

82. Week Six: Quantitative Measurement A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Political Science 242: ______________________ An Introduction to Research Methods

83. Week Six: Quantitative Measurement Week Six ______________________ Quantitative Measurement: How Indicators Fit Together A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO

84. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Lecture One ______________________ Indexes Week Six: Quantitative Measurement

85. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Recall that, in the very first week of the course, we discussed the measurement of a concept. One of the key points we discussed was the idea that in some instances measures could be constructed using more than one indicator of a given concept. Week Six: Quantitative Measurement

86. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. Creating a Measure of ‘Social Progressivism’ Social Progressivism Outlooks on Gay Rights Outlooks on Prostitution Week Six: Quantitative Measurement Outlooks on Abortion

87. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO When we use multiple indicators in order to measure a concept, we call the resulting concept an ‘index’. We can refer to the underlying dimension that we are attempting to measure as a ‘latent variable’. Week Six: Quantitative Measurement

88. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO So in the previous example, we might combine the indicators of outlooks on gay rights, outlooks on abortion, and personal feelings of religiosity into an index that we use to measure the latent variable ‘social progressivism’. Week Six: Quantitative Measurement

89. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO The major steps involved in the construction of indexes are: 1) Find a set of indicators that you think will be closely related to the underlying latent variable that you are interested in measuring. 2) Recode the indicators so that they have the same direction and range. Week Six: Quantitative Measurement

90. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO 3) Test the indicators to determine how well they fit together. 4) Once you’ve identified a set of indicators that are a good fit for the underlying latent concept, combine them into a single index. Week Six: Quantitative Measurement

91. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Step One: Finding Indicators __________________________ While the construction of indexes is really quite simple, it often involves quite a bit of work. Much of this work often involves combing through the data set that you are interested in using in order to find suitable indicators of the key concepts you are looking for.

92. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Step Two: Recoding Indicators __________________________ Once you have identified indicators of the latent concept that you think will fit well together, the next step is to recode them in order to allow for greater comparability. To do this you need to consider both direction and range.

93. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO When recoding in order to ensure that all of your indicators have the same direction, conceive of your latent variable in terms of one end of the spectrum of views that you intend to consider. Week Six: Quantitative Measurement e.g. ‘Social Progressivism’

94. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Then recode each of your variables so that the higher values of your indicators represent this end of the spectrum of views that you are intending to measure. Week Six: Quantitative Measurement

95. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO When recoding in order to ensure that all of your indicators have the same range, make sure that each of your indicators have the same minimum (0) and maximum (1), and that intermediary values are equally spaced in between. Week Six: Quantitative Measurement

96. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Then recode each of your variables so that the higher values of your indicators represent this end of the spectrum of views that you are intending to measure. Week Six: Quantitative Measurement

97. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Step Three: Fitting Indicators __________________________ Once you have recoded your indicators so that they each have the same direction and range, the next step is to test the indicators to determine how well they fit together. In order to do this we perform a ‘reliability’ analysis.

98. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO The key component of a reliability analysis is the Cronbach’s alpha score. There are two such scores for any given indicator, an unstandardized and a standardized score. We want to focus on the standardized score. Week Six: Quantitative Measurement

99. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Different researchers have different perspectives about how high these scores should be in order to conclude that a given index is sufficiently reliable. For the purposes of this course we will use a standardized alpha score of 0.50 as our cut-off. Week Six: Quantitative Measurement

100. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Step Four: Combining Indicators __________________________ Once we have tested the indicators to ensure that their fit is good, the next step is to combine them into a single indicator. In order to do this we simply add them up and divide by the number of indicators.

101. Questions? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

102. A worked example … A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

103. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Step One ______________________ Finding Indicators Week Six: Quantitative Measurement

104. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Let’s try using World Values Survey data that’s been collected on citizens’ views towards homosexuality, prostitution and abortion as indicators to build a measure of outlooks on ‘social progressivism’. Week Six: Quantitative Measurement

105. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Step Two ______________________ Recoding Indicators Week Six: Quantitative Measurement

106. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO First let’s take a look at the first potential indicator, views towards homosexuality. Recall that we need to consider whether or not we need to recode for either direction or range. Here’s the distribution of the variable: Week Six: Quantitative Measurement

107. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

108. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement minimum value

109. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement maximum value

110. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO We can see that his variable ranges from 1 to 10. In this case, a response of 1 indicates that the respondent thinks homosexuality is ‘never justifiable’ and a response of 10 indicates that he or she thinks that homosexuality is ‘always justifiable’. Week Six: Quantitative Measurement

111. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Do we need to recode for direction? Week Six: Quantitative Measurement

112. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Do we need to recode for direction? Week Six: Quantitative Measurement In this case we do not, because those who believe that homosexuality is always justifiable are more ‘socially progressive’, and so higher scores on this indicator suggest higher levels of what we are trying to measure.

113. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Do we need to recode for range? Week Six: Quantitative Measurement

114. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Do we need to recode for range? Week Six: Quantitative Measurement In this case we do, because the variable ranges from 1 to 10, while we want our indicators to range from 0 to 1.

115. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement How would we do this? Well, we want to turn the 1s into 0s and 10s into 1s. If we first subtract 1 from each value, this turns our 1s into 0s. But now our 10s will be 9s. To turn these into 1s, all we need to do is to then divide by 9!

116. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement In other words, we could use this simple SPSS code to create our first indicator of social progressivism: compute socprogin1 = (homosexuality-1)/9.

117. Questions? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

118. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO But what if we were creating a measure of ‘moral traditionalism’ instead of a measure of ‘social progressivism’? Week Six: Quantitative Measurement

119. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO If this were the case we would need to recode for direction, as those who believe that homosexuality is always justifiable are less ‘morally traditional’, and so higher scores on this indicator suggest lower levels of what we’re trying to measure. Week Six: Quantitative Measurement

120. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement How would we do this? Well, we want to turn the low scores into high scores and the high scores into low scores. In other words, we want to turn 1s into 10s and 10s into 1s.

121. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement How do we turn a 10 into a 1? Start with 11, and subtract 10 from it. How do we turn a 1 into a 10? Start with 11, and subtract 1 from it.

122. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement In other words, we could use this simple SPSS code to recode our first indicator of moral traditionalism for direction: compute mortradin1 = 11 – homosexuality.

123. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Once we’ve done this, our newly-created indicator will also range from 1 to 10, so we’ll next have to recode it for range …

124. Questions? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

125. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Step Three ______________________ Fitting Indicators Week Six: Quantitative Measurement

126. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement In order to determine how well our indicators fit together, we perform a reliability analysis.

127. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement To do this for our three indicators of social progressiveness ‘socprogin1’, ‘socprogin2’, and ‘socprogin3’, use the following SPSS code: RELIABILITY /var=socprogin1, socprogin2, socprogin3 /SCALE(’socprogrel') All /summary=ALL.

128. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Once we do this, we want to look at two key components of the output: (1) the standardized Cronbach’s alpha coefficient, and (2) the ‘alpha if item deleted’ score for each of the indicators.

129. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement These three indicators combine for a standardized alpha score of 0.728, which exceeds our 0.5 threshold. These indicators fit quite well together.

130. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement These three indicators combine for a standardized alpha score of 0.728, which exceeds our 0.5 threshold. These indicators fit quite well together. standardized alpha

131. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement If one of our ‘alpha if item deleted’ scores is higher than this standardized alpha value, this suggests that our measure would be a better fit if we removed this indicator.

132. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Since each of these values is below 0.728, this means that each of our indicators positively contributes to the fit of the measure.

133. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Since each of these values is below 0.728, this means that each of our indicators positively contributes to the fit of the measure. alpha if deleted for first indicator

134. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Since each of these values is below 0.728, this means that each of our indicators positively contributes to the fit of the measure. alpha if deleted for second indicator

135. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Since each of these values is below 0.728, this means that each of our indicators positively contributes to the fit of the measure. alpha if deleted for third indicator

136. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Step Four ______________________ Combining Indicators Week Six: Quantitative Measurement

137. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement The final step is to combine our indicators to create a new measure. In order to do this, we want to simply sum up our indicators and then divide by the number of indicators:

138. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement To do this for our three indicators of social progressiveness ‘socprogin1’, ‘socprogin2’, and ‘socprogin3’, use the following SPSS code: compute socprog = (socprogin1+socprogin2+socprogin3)/3.

139. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Once we’ve created our new measure of social progressivism, take a look at it by running a frequency: freq var = socproginx.

140. Questions? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

141. Looking forward to next class… A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

142. Looking forward to next class… A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Lecture Two ______________________ Factor Analysis Week Six: Quantitative Measurement

143. Week One: An Introduction to Research Methods A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO

144. Week Six: Quantitative Measurement A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Political Science 242: ______________________ An Introduction to Research Methods

145. Week Six: Quantitative Measurement Week Six ______________________ Quantitative Measurement: How Indicators Fit Together A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO

146. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Lecture Two ______________________ Factor Analysis Week Six: Quantitative Measurement

147. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Recall that in the last lecture, we learned that the key component of the construction of indexes was a reliability analysis based on standardized Cronbach’s alpha scores, which helps us to determine the extent to which potential indicators of a latent variable are a good fit with each other. Week Six: Quantitative Measurement

148. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Factor Analysis __________________________ Factor Analysis falls within a broad category of methodological approaches that are useful for identifying patterns and commonalities in sets of indicators that might be conceptualized using a variety of alternative sets of concepts.

149. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Other similar approaches include: 1) principal components analysis 2) cluster analysis 3) multidimensional scaling Week Six: Quantitative Measurement

150. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Not only is factor analysis only one of a variety of comparable approaches, but there is a variety of approaches to factor analysis. We will focus on one in particular that can be a helpful complement to the identification and construction of indexes. Week Six: Quantitative Measurement

151. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Instead of relying on Cronbach’s alpha to assess the extent of the fit of the indicators of each measure in isolation from other indicators of other measures, we can use factor analysis to simultaneously assess the extent to which different sets of indicators correspond to unique concepts that are identifiable in the data. Week Six: Quantitative Measurement

152. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Put more simply: factor analysis allows us to simultaneously assess the degree of fit of the indicators of multiple measures. Week Six: Quantitative Measurement

153. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO e.g. Measures of ‘Authoritarianism’ vs. ‘Feminism’ vs. ‘Moral Traditionalism’ vs. ‘Democratic Values’. Week Six: Quantitative Measurement

154. Questions? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

155. A worked example … A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

156. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Let’s try using World Values Survey data to try to determine whether ‘abortion’ is really a ‘social progressivism’ issue at all. Is it instead really a feminism issue? Perhaps we shouldn’t use it as a measure of social progressivism at all? Week Six: Quantitative Measurement

157. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Recall that because of data availability, we’re confined to one measure of feminism. Let’s use a varimax rotated factor analysis to determine whether abortion fits better with the feminist dimension or the other social progressivism indicators. Week Six: Quantitative Measurement

158. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement To do so we can use this SPSS code: factor /variables socprogin1 socprogin2 socprogin3 femism1 /print initial det kmo repr extraction rotation fscore univaratiate /format blank(0.20) /criteria factors(2) /extraction paf /rotation varimax.

159. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement To do so we can use this SPSS code: factor /variables socprogin1 socprogin2 socprogin3 femism1 /print initial det kmo repr extraction rotation fscore univaratiate /format blank(0.20) /criteria factors(2) /extraction paf /rotation varimax. here we ask the program to include these variables

160. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement To do so we can use this SPSS code: factor /variables socprogin1 socprogin2 socprogin3 femism1 /print initial det kmo repr extraction rotation fscore univaratiate /format blank(0.20) /criteria factors(2) /extraction paf /rotation varimax. here we ask the program to ignore any factor loadings less than 0.20

161. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement To do so we can use this SPSS code: factor /variables socprogin1 socprogin2 socprogin3 femism1 /print initial det kmo repr extraction rotation fscore univaratiate /format blank(0.20) /criteria factors(2) /extraction paf /rotation varimax. here we ask the program to locate exactly two factors for us

162. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement In this case the key output to look for is the ‘rotated factor matrix’: here we see that our three indicators load best together on the first factor

163. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement In this case the key output to look for is the ‘rotated factor matrix’: while this blank indicates that femism1 does not fit well with these indicators

164. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement In this case the key output to look for is the ‘rotated factor matrix’: but notice that ‘socprog1’ also loads nearly as highly on the second factor with ‘femism1’

165. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement Interestingly, ‘socprog1’ is not the indicator for outlooks on abortion. abortion indicator is ‘socprog3’

166. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO What have we learned? 1)That the indicator of outlooks on abortion clearly fits better with our other indicators of social progressivism than with the feminism indicator. Week Six: Quantitative Measurement

167. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO What have we learned? 2)That if anything, the social progressivism indicator that fits the best with our feminist indicator is the indicator of attitudes towards homosexuality (socprog1). Week Six: Quantitative Measurement

168. Questions? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

169. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Assignment Three ______________________ Building Measures Week Six: Quantitative Measurement

170. Questions? A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

171. Week One: An Introduction to Research Methods A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO

172. A. SEALEY Finally! ______________________ A focus on course objective #3: Developing a marked level of expertise with one key data set. DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

173. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Choosing a Data Set: Webstats vs. SPSS vs. R Week Six: Quantitative Measurement

174. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Key Fields in Political Science (and Public Policy) __________________________ 1) Canadian Politics 2) Comparative Politics 3) Development Politics 4) International Relations 5) Political Theory Week Six: Quantitative Measurement

175. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Exercise Objectives __________________________ 1) To identify and meet colleagues who share common academic interests. 2) To begin to discuss and decide upon which key data sets that members of your field group will develop a marked level of expertise with. Week Six: Quantitative Measurement

176. Field Group Placement __________________________ A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Canada Group _____________________________ IR Group _____________________________ Developing Group _____________________________ Comparative Group _____________________________ Theory Group _____________________________ Week Six: Quantitative Measurement

177. A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Exercise Objectives __________________________ 1) To identify and meet colleagues who share common academic interests. 2) To begin to discuss and decide upon which key data sets that members of your field group will develop a marked level of expertise with. Week Six: Quantitative Measurement

178. Looking forward to next week… A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO Week Six: Quantitative Measurement

179. Week Seven: Midterm Break Week Seven ______________________ Midterm Break A. SEALEY DEPARTMENT OF POLITICAL SCIENCE UNIVERSITY OF TORONTO