Research: Conceptualization and Measurement Conceptualization Steps in measuring a variable Operational definitions Confounding Criteria for measurement quality Techniques of dealing with problems in measurement Ensuring reliability Validity Face validity Content validity Criterion related validity Construct validity 1

Conceptualization We want to speak of abstract things: “Intelligence” “Ability to cope with stress” “Life satisfaction” “Happiness” We cannot research these things until we know exactly what they are. Everyday language often vague and unspecified meanings. 2

Conceptualization Specify exactly what we mean and don’t mean by the terms we use in our research. No “true” (final) definitions of “the stuff of life” Conceptualization: The process of identifying and clarifying concepts We specify what we mean by using certain terms Indicators=the presence or absence of the concept we are studying. These are often multi-dimensional More than one specifiable aspect or facet What do we mean by happiness? 3

Steps in reaching a measurement of a variable We all have conceptions of what we understand by compassion, prejudice, poverty, etc. People do not always agree about the meanings Begin by asking people to describe what they mean when they use certain terms such as “intelligence” Consult the EXPERTS Literature review Even the experts do not agree Coming to an agreement on what we understand is called conceptualization. Result of this process is a concept e.g., “prejudice” 4

Operational definitions Specifying exactly what we are going to observe, and how we will do it. Make the variable directly measurable Describe of the “operations” used to measure a concept 5

Examples “Socio-economic Status” (SES): What was your total family income during the past 12 months? What is the highest level of school you completed? How would you operationalize “success at university?” If you operationalize badly, you end up not studying what you want (invalid operational definitions) E.g., operationalizing “success in career” by looking only at pay check 6

Confounding Intelligence tests require knowledge of the language in which they are given Also measuring acquired language skills Juvenile delinquency can be defined in terms of convictions in court But convictions are more frequent when they are not legally represented - thus also measuring economic status Confounding = When operational definitions measure more than one thing 7

Criteria for measurement quality Reliability Does it yield the same result every time? Stability over time. If I measure you now and again in half-an- hour, do I get the same reading? Maximum reliability depends on the construct – some constructs are unstable, e.g., heart rate. Single observers or raters 8

Techniques of dealing with problems in measurement reliability Test-retest method – Make the same measurement more than once (external) Split-half method – Divide the instrument in two halves - Cronbach's alpha (internal) 9

Ensuring reliability Reliability suffers when respondents or researchers have to interpret Objective scales are always more reliable Allow for little interpretation Using a fixed-response format helps e.g. Multiple choice, Likert type response formats Researcher does not have to interpret what the respondent meant 10

***Validity The extent to which an empirical measure adequately reflects the meaning of the concept under investigation For a scale: The degree to which it measures what it is supposed to measure Validity is divided into many types: Content validity Criterion-related validity Construct validity Face validity 11

Face validity How a measure conforms to our common agreements Examine the wording of the items Submit items to expert judges 12

Content validity How much a measure covers every element of a concept. Example: Measuring only the affective aspect of love, but not the behavioral. Experts in a given domain generally judge content validity. For example, the content of the SAT Subject Tests™ is evaluated by committees made up of experts who ensure that each test covers content that matches all relevant subject matter in its academic discipline. 13

Criterion-related validity Sometimes called predictive validity How a measure predicts performance on an external criterion e.g., How ACT or SAT results predict academic success at university, as a way of saying they have validity 14

Construct validity Closely tied to into the theoretical underpinnings of the concept. Variables ought to be related, theoretically, to other variables. This kind of validity is based on logical relationships between variables So: Does the instrument actually measure the concept (or construct)? e.g., Measure cranial circumference or brain weight to measure intelligence Most difficult to achieve, most important – measures lacking in construct validity are almost useless 15

How to check for construct validity How can you show that a measurement truly measures what it claims to? How would you show that your depression scale has construct validity? 16

How to check 1. See how it relates to similar and dissimilar concepts: Show that your depression scale relates positively to similar concepts e.g., People who score high on your depression scale will have many sad thoughts 17

How to check 2. Show that your depression scale relates negatively to opposite concepts. Examples: People who score high on it will have very low energy levels Husbands who score high on a measure of marital satisfaction have fewer extra-marital affairs 18

Conclusion The more a scale or instrument has the qualities of reliability and validity, the better it is. Reliability and validity need to be sorted out before you run the study 19

Conclusion Tension between validity and reliability Richness of meaning of concepts Operational definitions and measurements seem to rob concepts of their “richness of meaning” But the more conceptual variation and richness we allow in our study, the more opportunity for disagreement on how it applies in this situation Related to the tension between quantitative, structured techniques such as surveys, and qualitative, semi-structured methods such as interviews 20