Social Research Methods: Qualitative and Quantitative Approaches, 5e This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public performance or display, including transmission of any image over a network; preparation of any derivative work, including the extraction, in whole or in part, of any images; any rental, lease, or lending of the program

Chapter 7: Qualitative and Quantitative Measurement Why measure? Quantitative and Qualitative Measurement Parts of measurement process. Reliability and Validity. A Guide to Quantitative Measurement Index construction. Scales.

This includes one or more of its Introduction To study a concept, by itself or as part of a larger framework (theory, H, etc.), you must be able to measure it. This includes one or more of its Presence or absence Amount Type

Introduction, continued There are two key aspects of measurement Conceptualization: define the concept in words Operationalization: identify specific ways to measure it For example, intelligence Are you satisfied with IQ as a measure of intelligence? Does it conceptualize intelligence correctly and fully? Or would you include a wider range of types of intelligence – see text p. 170 Does it operationalize abstract reasoning ability correctly and fully? Or is it measuring, in part, cultural knowledge and/or other concepts Or are there other problems with this form of operationalization?

Why Measure? Scientific measurement Provides clarity (conceptualization) and precision (operationalization) Extends our senses Enables us to see the invisible Physical science Social science

Chapter 7: Qualitative and Quantitative Measurement Why measure? Qualitative and Quantitative Measurement Parts of measurement process. Reliability and Validity. A Guide to Quantitative Measurement Index construction. Scales.

Qualitative and Quantitative Measurement Both gather data systematically and carefully Quantitative: conceptualization and operationalization precede data collection Qualitative: conceptualization and operationalization accompany data collection

Chapter 7: Qualitative and Quantitative Measurement Why measure? Quantitative and Qualitative Measurement Parts of the measurement process. Reliability and Validity. A Guide to Quantitative Measurement Index construction. Scales.

Conceptualization Process of clearly defining a concept Think, observe, consult, read, try out Needs to be clear, unambiguous Typically are linked to theories, often to values Social class – relational or gradational Democracy – political only, or economic also Intelligence – single, multiple, individual vs. group Concepts vary from concrete to abstract

Quantitative Conceptualization and Operationalization A deductive process See figure 7.1, p. 175

H: Class size affects teacher morale | conceptualization | Measurement deals with one concept at a time Hypotheses deal with more than one – measure them separately (you measure concepts, not Hs) Example: H: Class size affects teacher morale | conceptualization | [Conceptual def.] [Conceptual def.] | operationalization | [Operational def.] [Operational def.]

Conceptualization Example of teacher morale, text p 173: Get different definitions from different sources Sort them out and narrow down. Decide if this is multidimensional Specify unit of analysis Distinguish from related concepts

Operationalization Turns conceptual definition into operational definition – a specific procedure or way to measure a concept E.g. intelligence: IQ test, or multiple tests

Qualitative Conceptualization and Operationalization Grounded theory approach: inductive Reconstruction of theory: both deductive and inductive Conceptualization: develop conceptual definition while collecting and analyzing data; gradually increasing clarity and specificity Operationalization: Description of how specific observations [and existing concepts] contributed to development of new or revised concepts

Qualitative Conceptualization and Operationalization, continued Example – see figure 7.3, p 178

Chapter 7: Qualitative and Quantitative Measurement Why measure? Quantitative and Qualitative Measurement Parts of measurement process. Reliability and Validity. A Guide to Quantitative Measurement Index construction. Scales.

Reliability and Validity Are properties of a measure Both address the question: “How good is this measure?” First approximation: Reliability = dependability Validity = truthfulness or accuracy Can you have one without the other?

Reliability and Validity in Quantitative Research Reliability = dependability: Measurements do not vary because of the process or the measure Why then would they vary? That which they measure actually changes Examples Scales Speedometers Social science:

Three Types of Reliability Stability: reliability across time Measure (estimate) by test-retest method Representative: reliability across subgroups Measure (estimate) by subpopulation analysis Equivalence: reliability across different indicators For long questionnaire, measure (estimate) by split-half method Intercoder reliability

How to Improve Reliability Clearly conceptualize constructs Increase the levels of measurement Use multiple indicators of a variable Use pretests, pilot tests, replication

Validity (Measurement) validity is the accuracy of a measure How well do the conceptual and operational definitions mesh? Validity cannot be observed directly Establishing validity rests upon the accumulation of evidence over time

Four Types of Measurement Validity Face validity - expert judgment, consensus Content validity - measure covers full content of the concept Criterion validity – comparison with known valid measure Concurrent – with existing tests Predictive – of future events or behavior

Four Types of Measurement Validity, continued Construct –consistency across different indicators (for measures with such) Convergent Divergent C

Reliability and Validity in Qualitative Research Reliability – goal of consistency in observation, similar to stability reliability in quantitative research Balanced by recognition that observation is an evolving process of uniqueness of individual perspectives and observations

Reliability and Validity in Qualitative Research, continued Concerned with truth, although less concerned with precise match between abstract concept and empirical data Established by Plausibility Diverse empirical data Connections among different pieces of data

Relationship between Reliability and Validity Reliability is necessary but not sufficient for validity A measure can be (highly) reliable, but not (highly) valid a scale that always reads 10 pounds too high a watch that is always 20 minutes fast If a measure is valid, it must also be reliable

Chapter 7: Qualitative and Quantitative Measurement Why measure? Quantitative and Qualitative Measurement Parts of measurement process. Reliability and Validity. A Guide to Quantitative Measurement Index construction. Scales.

Four Levels of Measurement Nominal measurements are mutually exclusive and exhaustive categories. Ordinal measurements are mutually exclusive and exhaustive categories that are ranked. Interval measurements are mutually exclusive and exhaustive categories that are ranked with equal distance between the categories being measured. Ratio measurements are mutually exclusive and exhaustive categories that are ranked with equal distance between categories being measured, and there is a true zero.

Indices and Scales in Social Research An index is a measure in which a researcher adds or combines several distinct indicators or constructs into a single score. A scale is a measure in which a researcher captures the intensity, direction, level, or potency of a variable construct, arranging responses on a continuum.

Chapter 7: Qualitative and Quantitative Measurement Why measure? Quantitative and Qualitative Measurement Parts of measurement process. Reliability and Validity. A Guide to Quantitative Measurement Index construction. Scales.

Indices Develop a set of questions on the same topic. Decide whether to weight each item. Decide how to handle missing data. Decide whether to use rates and standardization.

Chapter 7: Qualitative and Quantitative Measurement Why measure? Quantitative and Qualitative Measurement Parts of measurement process. Reliability and Validity. A Guide to Quantitative Measurement Index construction. Scales.

Scales The Likert Scale is called a summated rating or additive scale because a total score is computed along a continuum such as: “I really like research methods,” with the choices being: SA A Unc D SD Thurstone Scale consists of many statements that are rated by judges to reduce a number of items to a range that spans all opinions on the topic. Bogardus Social Distance involves responses that are gathered into a series of ordered statements from very threatening to least threatening.

Scales continued…. Semantic Differential involves responses that are gathered to a set of polar opposite adjectives that create a rating scale. Bad __ __ __ __ __ Good Fast __ __ __ __ __ Slow The Guttman Scaling is called cumulative scaling. This is a technique researchers use after data has been collected to determine whether a set of items can be combined to form a scale.