1. Measurement Chapter 6

2. Measuring Variables Measurement Classifying units of analysis by categories to represent variable concepts

3. Measuring Variables Quantitative research Research focused on variables, including their description and relationships

4. Measuring Variables Qualitative research Research focused on the interpretation of the action of, or representation of meaning created by, individual cases

5. Measuring Variables Measure A specific way of sorting units of analysis into categories

6. Measuring Variables The concepts in social science research can often be difficult to measure Measurement is a way to define a concept

7. Conceptualization The process of clarifying what we mean by a concept

8. Conceptualization Conceptual Definition A definition of a concept through other concepts

9. Conceptualization Dimensions If a concept is so large that it may need to be divided into different aspects or dimensions to further clarify it Dimensions are aspects or parts of a larger concept

10. Conceptualization Multidimensionality A concept that refers to a phenomenon that is expected to have different ways of showing up or manifesting itself is said to be multidimensional Multidimensionality is the degree to which a concept has more than one discernible aspect

11. Operationalization The process of specifying what particular indicator(s) one will use for a variable

12. Operationalization Indicators Observations that we think reflect the presence or absence of the phenomenon to which a concept refers

13. Operationalization Operationalization involves providing operational definitions Declarations of how a specific phenomena described by concepts are determined for a specific instance.

14. Operationalization Index A composite measure that is constructed by adding scores from several indicators

15. Operationalization

16. Composite Measures A measure with more than one indicator Designed to solve problems of ambiguity that are associated with single indicators by including indicators of a variable in one measure

17. Operationalization Scale Indexes in which some items are given more weight than others in the determination of the final measure of a concept

18. Operationalization Visual analysis A set of techniques used to analyze images] refers to a set of techniques used to analyze images

19. Operationalization Coding Assigning observations to categories

20. Operationalization Exhaustive The capacity of a variable’s categories to permit the classification of every unit of analysis

21. Operationalization Mutually exclusive The capacity of a variable’s categories to permit the classification of each unit of analysis into one and only one category

22. Operationalization Measurement error The kind of error that occurs when the measurement we obtain is not an accurate portrayal of what we tried to measure

23. Quality of Measurement Reliability & Validity Reliability - the degree to which a measure yields consistent results A measure is reliable if it yields consistent results time after time Validity – the degree to which a measure taps what we think it’s measuring A measure is valid if you measure what you think your are measuring

24. Quality of Measurement Checking Reliability A yard stick will consistently measure 30 inches time after time, so it is reliable—but it is possible to have the first inch sawed off, so we need to check reliability.

25. Quality of Measurement Checking Reliability Test-retest method Split-half method Interobserver Reliability Interrater Reliability method

26. Quality of Measurement Checking Reliability Test-retest A method of checking reliability of a test that involves comparing its results at one time with results, using the same subject at another time

27. Quality of Measurement Checking Reliability Test-retest Measuring the width of a desk two or more times in close succession with the same yard stick Problems:  Phenomenon under investigation might actually change between the test and the retest – change in life satisfaction  Earlier test results may influence you the second time you test

28. Quality of Measurement Checking Reliability Split-half method A method of checking the reliability of several measures by dividing the measure into two sets of measures and determining whether the two sets are associated with each other.  Deals with the problems of the test-retest method  Making more than one measurement of a phenomenon at the same time

29. Quality of Measurement Checking Reliability Interobserver reliability method – compares the results obtained by one observer with results obtained by another using exactly the same method. Assessing the traits of individual characters in children's’ books using a pre-defined measurement scheme, they agreed 88 percent of the time on average

30. Quality of Measurement Checking Reliability Alternate-forms method – a way of checking the reliability of a measurement strategy by comparing its results with those obtained using a measurement strategy meant to measure the same variable.

31. Quality of Measurement Checking Validity Does this measurement strategy feel as if it’s getting what it is supposed to? Face validity Content validity Predictive validity Construct validity

32. Quality of Measurement Checking Validity Face Validity The degree to which a measure seems to be measuring what it’s supposed to be measuring

33. Quality of Measurement Checking Validity Content validity How well a measure covers the range of meanings associate with a concept If most people know something about a concept, feel that a given set of questions “gets at” that concept and the set of questions asked about a question cover the usual range of “content” implied by the concept

34. Quality of Measurement Checking Validity Predictive validity How well a measure is associated with future behaviors you would expect it to be associated with Involves comparing the results of your measurement scheme with a criteria variable that it should predict

35. Quality of Measurement Checking Validity Difficult to validate social science measures using behavioral criteria, b/c limited in finding behaviors that they obviously predict to Many of the social science indicators are meant to measure abstract concepts Social scientists tend to use construct validity

36. Quality of Measurement Concurrent Criterion Validity How well a measure is associated with behaviors it should be associated at the present time

37. Quality of Measurement Construct Validity How well a measure of a concept is associated with a measure of another concept that some theory says the first concept should be associated with

38. Quality of Measurement Sensitivity The ability of a system of measurement to identify correctly those who have a condition of concern

39. Quality of Measurement Specificity The ability of a system of measurement to identify correctly those who do not have a condition of concern

40. Level of Measurement Nominal level variables Describes a variable whose categories have names Nominal level variables permit us to sort our data into categories that are mutually exclusive Lowest level of measurement, it is virtually impossible to create a variable that is not nominally scaled

41. Level of Measurement Ordinal Level Variables Describes a variable whose categories have names and can be rank-ordered in some way Ordinal level variables guarantee that categories are mutually exclusive and that the categories can be ranked All ordinal variables can be treated as nominal, but not all nominal variables may be treated as ordinal

42. Level of Measurement Interval Level Variables Describes a variable whose categories have names, can be ranked-ordered, and whose adjacent categories are a standard distance from another All interval variables can be treated as nominal or ordinal variables, but ordinal and nominal variables cannot be treated as interval

43. Level of Measurement Interval Level Variables Categories of interval variables can be meaningfully added and subtracted

44. Level of Measurement Ratio level variables Describes a variable whose categories have names, can be rank-ordered, has a standard distance from another and has an absolute zero Absolute zero is the point at which there is a complete absence of the phenomenon in question Ratio level variables can also be interval, ordinal, and nominal level variables

45. Level of Measurement The practical significance of level of measurement Researchers find themselves wanting to summarize the information that has been collected Certain statistics require different levels of measurement Three ways to describe central tendency Mean – interval, ratio Median – ordinal Mode - nominal

46. Level of Measurement Given a choice, measure a variable in a higher level of measurement than a lower one The higher the level of measurement, the more statistics that can be performed on the data

47. Summary The importance of measurement The best measurement strategies are reliable and valid

48. Quiz – Question 1 Professor Johnson is interested in examining the connection between age at first sexual experience and high school completion. Which of these concepts must be operationalized? a. First sexual experience b. High school completion c. A and B d. None of the above

49. Quiz – Question 2 What is the problem with the following survey response set for the question “how old are you”? 25 or less 25 to 65 65 and older a. The concept is not operationalized b. The response set is not mutually exclusive c. The constructs are multi-dimensional d. The conceptual definition

50. Quiz – Question 3 If we administer a survey to measure school- related stress and re-administer it one month from today to the same group and obtain different results, what does this indicate to us about our measurement? a. It may not be a reliable measure b. Stress may have changed in the group c. We may need to re-administer the survey again to determine more conclusively if the measure is reliable d. All of the above e. None of the above

51. Quiz – Question 4 Letter grades given at the end of the term are an example of _____measurements; calculated GPAs are a type of _____measurements. a. ordinal, interval b. ordinal, ratio c. interval, ratio d. Nominal, ratio

52. Quiz – Question 5 A researcher designs a survey question asking respondents to indicate their major: sociology, psychology, biology, chemistry, etc. This is a _____ measure. a. nominal b. ordinal c. interval d. ratio