1. Measurement and Concept Formation Meredith Rolfe Centre for Corporate Reputation, Saïd Business School Nuffield College, University of Oxford

2. In physical science the first essential step in the direction of learning any subject is to find principles of numerical reckoning and practicable methods for measuring some quality connected with it. I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be. Baron William Thomson Kelvin, Popular Lectures and Addresses 1:73 (originally: Lecture to the Institution of Civil Engineers, 3 May 1883 )

3. Theoretical System

4. Steps in a Measurement Process 1. Identify theoretical concepts 2. Choose one or more indicators/variables 1. Brainstorm possible variables 2. Choose operational rule or definition 3. Level of measurement 4. Measurement technique 3. Assess Measurement Reliability & Validity

5. Theoretical concepts related to states: democracy regime type economic growth state capacity Individuals income authoritarianism political interest social class Step 1) Identify theoretical constructs or concepts

6. There have been many authorities who have asserted that the basis of science lies in counting or measuring, i.e. in the use of mathematics. Neither counting nor measuring can however be the most fundamental processes in our study of the material universe—before you can do either to any purpose you must first select what you propose to count or measure, which presupposes a classification. Roy Albert Crowson, Classification and Biology (1970)

7. Step 2) Variables: operational rules to turn observations into data Democracy fair elections GNP/capita State capacity size of military expenditures Violence, war or military conflict Deaths (civilian or otherwise?) expenditures

8. Possible indicators of democracy Procedural (free and contested elections) Multi-party Campaign financing Balance of powers Universal suffrage One person, one vote Collective deliberation Accountability of elected officials Property rights Citizens have a voice in government economic outcomes

9. Variables: Levels of measurement Nominal Algeria, Russia (country) gender or ethnicity of respondent Revolution, civil war (0/1) Ordinal democracy in 7 categories war Interval agree/disagree 5 point scale Ratio (Continuous) income, length of war

10. Discrete “Variables” WARPEACE CONFLICT

11. Variables: Measurement techniques Counting (words, elections, and so on) Sources: government statistics, newspapers, archives Includes: basic content analysis Includes: Is there a senate? Are there two large parties? Asking people Survey questions Fixed format Open-response Interviews (elite or random sample) Implicit attitudes (psychology, often experimental)

12. Variables: Measurement Techniques (cont.) Observing people Discourse analysis Content/semantic analysis Participant observation Coding of gestures/speech (psychology) Interpretive Compare measurement techniques/sources – why are they different? Semiotic practices (observing people)

13. Step 3) Assess Measurement Reliability & Validity Reliable measures are… consistent (repeated measurement gives similar results for the same case) inter-coder reliability (qualitative data) unbiased (right answer, on average) precise/low error (close to the right answer)

14. Reliable Measures Consistent

15. Step 3) Assess Measurement Reliability & Validity Reliable measures are… consistent (repeated measurement gives similar results for the same case) inter-coder reliability (qualitative data) unbiased (right answer, on average) precise/low error (close to the right answer)

16. Reliable Measures Consistent Unbiased

17. Step 3) Assess Measurement Reliability & Validity Reliable measures are… consistent (repeated measurement gives similar results for the same case) inter-coder reliability (qualitative data) unbiased (right answer, on average) precise/low error (close to the right answer)

18. Reliable Measures Consistent Unbiased Consistent Unbiased Low Variance

19. Valid measures are… Content or substance of measure captures construct face validity – ask an expert “what are you thinking about” probes manipulation checks (experiments) Criterion-related validity Predictive: Correct relationship with some predicted outcome variable Decision validity: Useful or pragmatic applications

20. Valid Measures are… (cont.) Construct validity Convergent validity: Related to other theoretically related variables/indicators (correlations, alpha scores, factor analysis) Example: GNP/capita, tax revenue/capita Discriminate validity: empirically discriminates between related constructs (correlations, factor analysis) Example: poverty, education

21. Though he may not always recognise his bondage, modern man lives under a tyranny of numbers. Nicholas Eberstadt The Tyranny of Numbers; Mismeasurement and Misrule (1995)

22. Debates Can we measure everything? Should we measure observable behaviors or social meaning? Do concepts need to be better defined or better measured?

23. Debates: Can you measure everything? Measurement people – yes, in theory (Kerlinger – but it is just a game, play it well) Sartori – no. Example: party system.

24. Debates: Observation versus meaning Example: voting in authoritarian regimes versus democratic Example: survey responses in Africa versus United States Example: organization membership across social class Solution: contingent/contextual measures, more complete specification

25. Debates: concept definition versus concept measurement Defining concepts versus Measuring concepts

26. Go, wondrous creature, mount where science guides. Go, measure earth, weigh air, and state the tides; Instruct the planets in what orbs to run, Correct old Time, and regulate the sun; Go, teach Eternal Wisdom how to rule, Then drop into thyself and be a fool. Alexander Pope Quoted in James Wood Dictionary of Quotations from Ancient and Modern, English and Foreign Sources (1893), 125.

27. Graph based measures: Social Networks and Distances Measurement goes beyond one dimension!

28. Who buys political books? Source: Krebs, orgnet.com

29. Source: Ronald Inglehart and Christian Welzel, Modernization, Cultural Change and Democracy New York: Cambridg University Press, 2005: page 63.