1. MEASUREMENT & SAMPLING
BUSN 364 – Week 9
Özge Can

2. Measurement
It connects invisible ideas or concepts in our mind with specific things we do or observe in the empirical world to make those ideas visible
It lets us observe/ helps to see things that were once unseen and unknown but predicted by theory
We need measures:
To test a hypothesis, evaluate an explanation, provide empirical support for a theory or study an applied issue
Makes this connection by using some specific techniques, processes or procedures
It lets us observe/ helps to see things that were once unseen and unknown but predicted by theory

3. Measurement Physical world or features are easier to measure
E.g. age, gender, skin tone, eye shape, weight
Measures of the nonphysical world are less exact
E.g. attittudes, preferences, ideology, social roles
“This restaurant has excellent food”, “Deniz is really smart”, “Ali has a negative attitude towards life”, “Mert is very prejudiced”, “Last nights’s movie contains lots of violence”

4. Quantitative and Qualitative Measurement
Quantitative mesurement:
It is a distinct step in the research process that occurs before data collection
Data are in a standardized, uniform format: Numbers
Qualitative measurement:
We measure and create new concepts simultaneously with the process of gathering data
Data are in nonstandard, diverse and diffuse forms

5. Measurement Process Two major steps:
1. Conceptualization => the process of developing clear, rigorous, systematic conceptual definitions for abstract ideas/concepts
Conceptual definition: A careful, systematic definition of a construct that is explicitly written down
Some constructs are highly abstract and complex; some are concrete and simple
A good definition has one clear, explicit, and specific meaning. There is no ambiguity or vagueness.

6. Measurement Process
2. Operationalization => Process of moving from a construct’s conceptual definition to specific activities or measures that allow the researcher to observe it empirically
Operational definition: A variable in terms of the specific actions to measure or indicate in the empirical world
Example: Professional work environment => level of teacher morale
Theorize the causal relationship
Conceptualize the variables
Operationalize the variables
Test the empirical hypothesis

7. Measurement Process
Illustrates the measurement process linking two variables in a theory and a hypothesis. We link the three levels together and move deductively from the abstract to the concrete.
First we conceptualize a variable giving it a clear conceptual definition; next, we operationalize it by developing an operational definition or set of indicators for it. Lastly, we apply indicators to collect data and test empirical hypotheses.

8. Reliability Dependebility or consistency of the measure of a variable
The numerical results that an indicator produces do not vary because of the characteristics of the measurement process or instrument
E.g. A reliable scale shows the same weight each time

9. Reliability How to Improve Reliability?
Conceptualization: clearly conceptualize all constructs
Increase the level of measurement: detailed info the measurement shows
Use multiple indicators of a variable: triangulation
Use pilot studies and replication

10. Validity
How well an empirical indicator and the conceptual definiton “fit” together
The better the fit, the higher the validity
Four types of measurement validity:
Face validity
Content validity
Criterion validity
Construct validity
More difficult to achieve than reliability
We cannot have absolute confidence about validity but some measures are more valid than others

11. Validity
Face Validity: It is a judgement by the scientific community that the indicator really measures the construct.
The construct “makes sense” as a measurement

12. Validity
Content Validity: Requires that a measure represent all aspects of the conceptual definition of a construct
Is the full content of a definition represented in a measure?
Conceptual definiton: it is a “space” containing ideas and concepts

13. Validity
Criterion Validity: Uses some standard or criterion to indicate a construct accurately. Validity of an indicator is verified by comparing it with another measure
Concurrent and predictive validity
Concurrent: we need to associate an indicator with a preexisting indicator that we already judge to be valid
Predictive: an indicator predicts future events that are logically related to a construct

14. Validity
Construct Validity: Is for measures with multiple indicators. Do the various indicators operate in a consistent manner?
Convergent and divergent validity
Convergent: validity based on the idea that indicators of a single construct will act alike or converge
Divergent: validt based on the idea that indicators of different constructs diverge

15. Relationship between Reliability and Validity
Reliability is necessary for validity and easier to achieve BUT
It does not guarantee that the measure will be valid!
Sometimes there is a trade-off between them:
As validity increases, reliability becomes more difficult to attain or vice versa
This occurs when the construct is highly abstract and not easily observable but captures the “true essence” of an idea

16. Relationship between Reliability and Validity

17. Levels of Measurement
A system for organizing information in the measurement of variables. It defines how refined, exact and precise our measurement is.
Continuous variables: Variables that contain large number of values or attributes that flow along a continuum
Ex: temperature, age, income, crime rate
Discrete variables: Variables that have a relatively fixed set of separate values or attributes
Ex: gender, religion, marital status, academic degrees

18. Levels of Measurement
The four levels from lowest to highest precision:
Nominal: indicates that a difference exists among categories
Ordinal: indicates a difference and allows us to rank order the categories
Interval: does everything the first two do and allows us to specifiy the amount of distance between categories
Ratio: does everything the other levels do and it has a true zero.
Nominal => religion, ethnicity
Ordinal => letter grades
Interval => IQ scores, Celsius, Fahrenheit, Kelvin scales (water freezes at 0, 32 and 273) = arbitrary zero
Ratio => Money income, years of schooling. This feature makes it possible to state relationships in terms of proportions and ratio

19. Levels of Measurement
*Discrete variables are at nominal or interval levels
*Continuous variables are at interval or ratio levels

20. Levels of Measurement

21. Principles of Good Measurement
Mutually exclusive attributes:
An individual or case will go into one and only one variable category
Exhaustive attributes:
Every case has a place to go or fits into at least one of a variable’s categories
Unidimensionality:
A measure fits together or measures one single, coherent construct

22. Scales and Indexes
Scale => a measure in which a researcher captures the intensity, direction, level or potency of a variable and arrange responses/observations on a continuum
Likert Scale: ask people whether they agree or disagree with a statement
Index => a measure in which a researcher adds or combines several distinct indicators of a construct into a single score
Ex: crime index, consumer price index
We mostly use scales when we want to measure how an individual feels or thinks about something
The composite score is often a simple sum of the multiple indicators.

23. Likert Scale – Examples:

24. Sampling
Sample: a small set of cases a researcher selects from a large pool and generalizes to the population
Population: large collection of cases from which a sample is taken and to which results from a sample are generalized
In quantitative studies, we select cases/units and treat them as carriers of aspects/features of a population

25. Sampling In quantitative research:
Primary use of sampling is to create a representative sample. If we sample correctly, we can generalize its results to the entire population
We select cases/units and treat them as carriers of aspects/features of a population
Probability sampling techniques
In quantitative studies, we select cases/units and treat them as carriers of aspects/features of a population

26. Sampling In qualitative research:
Primary use of sampling is to open up new theoretical insights, reveal distinctive aspects of people or social settings, or deepen understanding of complex situations, events, relationships
We sample to identify relevant categories at work in a few cases
We do not aim for representativeness or generalization
Non-probability sampling techniques
In qualitative studies, the aim is to sample aspects/features of the social world in general

27. Probability Sampling
It is the “gold standard” for creating a representative sample
We start with conceptualizing a target population
We then create an operational definition for this population: sampling frame
A list of cases in a population or the best approximation of them
E.g. telephone directories, tax records, school records
We choose a sample from this frame

28. Probability Sampling
Model of the Logic of Sampling:

29. Probability Sampling Probability samples involves randomness
Random sampling => using mathematically random method so that each elements will have an equal probability of being selected
Four ways to sample randomly:
Simple random sampling
Systematic sampling
Stratified sampling
Cluster sampling

30. Probability Sampling
Simple random sampling: Using a pure random process to select cases so that each elements in the population has equal probability of being selected
Systematic sampling: Everyting is the same as in simple random sampling except, instead of using a list of random numbers, we calculate a sampling interval (i.e. 1 in k, where k is some number)
There should not be some kind of pattern in the list
Simple random => We need to define the target population and the sampling frame accurately

31. If there is a pattern in the list...

32. Illustration of stratified sampling:

33. Probability Sampling
Stratified sampling: We first divide the population into sub-populations (strata) and then use random selection to select cases from each category
Example categories => gender, age, income, social class
Cluster sampling: Uses multiple stages and is often used to cover wide geographic areas. Units are randomly drawn from these clusters.
Addresses two problems: 1) lack of a good sampling frame for a dispersed population, 2) high costs to reach an element
We used stratified sampling if we have special interest in a small percentage of the population and random processes could misss this category by chance.
We draw several samples in stages in cluster sampling. Less expensive than simple random sampling but less accurate. The higher number of stages increases sampling error. A design with more clusters and less stages is better.

34. How Large Should a Sample Be?
The best answer is: It depends!
It depends on population characteristics, the type of data analysis to be employed, and the degree of confidence in sample accuracy is needed
Large sample size alone does not guarantee a representative sample
For small populations we need a large sampling ratio, while for large populations the gain is not that big
Everything else being equal, the larger the sample size, the smaller the sampling error
A large sample without good sampling frame or without random sampling created a less representative sample than a smaller one
Sampling errors => how much a sample deviates from being representative of the population.
For equally good sampling frames and precise random selection processes, the sampling error is based on two factors: the sample size and the population diversity

35. How Large Should a Sample Be?

36. Nonprobability Sampling
Convenience sampling (Availability/accidental sampling): A nanrandom sample in which the researcher selects anyone he or she happens to come across.
Quick, cheap and easy but very unrepresentative
Quota sampling: Researcher first identifies general categories and then select cases to reach a predetermined number in each category
In quota sampling, we again use convenience sampling in order to choose the units under each category. The logic is to reflect the diversity of the population

37. Nonprobability Sampling
Purposive sampling (Judgmental sampling): getting all possible cases that fit particular criteria, using various methods It is mostly used in exploratory research or in field research.
Often used for difficult-to-reach, specialized populations
Snowball sampling: The researcher begins with one case, and then, based on information from this case, identifies other cases. Begins small but becomes larger.
A method for sampling the cases which are in an interconnected network
Purposive sampling: it selects cases with a specific purpose in mind. It is appropriate to select unique cases that are especially informative. We often use it to select members of a difficult-to-reach, specialized, hidden population; such as illegal drug users, drug dealers, people with AIDS
Snowball sampling = also called “chain referral”, “network”, “reputational” and “respondent-driven” sampling
Example: friendship networks of teenagers in a community. Network of scientists investigating the same problem, the elites of a medium-size city, members of an organized crime family, people on a university campus who have some relationships with each other
Where do we stop, where these samplings end? The principle is to gather cases until we reach a saturation point.

38. Example: Snowball Sampling

39. Nonprobability Sampling
Deviant case sampling (Extreme case sampling): The goal is to locate a collection of unusual, different or peculiar cases that are not representative of a whole
We are interested in cases that differ from the dominant pattern, mainstream
Theoretical sampling: Selecting cases that will help reveal some features that are theoretically important about a particular setting/ topic. A theoretical interest guides the sampling