Beginning the Research Design Theory, Questions, Hypotheses Designing Tests for the above: Conceptualization, Operationalization, and Measurement

Conceptualization Process of specifying what we mean when we use particular terms. Produces an agreed upon meaning for a concept for the purposes of research. Describes the indicators we'll use to measure the concept and the different aspects of the concept.

From Concept to Measurement Progression from what a term means to measurement in a scientific study: Conceptualization Nominal Definition Operational Definition Measurements in the Real World

Four Levels of Measurement Nominal - offer names for labels for characteristics (gender, birthplace). Ordinal - variables with attributes we can logically rank and order.

Four Levels of Measurement Interval - distances separating variables (temperature scale). Ratio - attributes composing a variable are based on a true zero point (age).

Measurements Things Scientists Measure Direct observables - things that can be observed simply and directly. Indirect observables - things that require more subtle observations. Constructs - based on observations that cannot be observed.

Measurement Quality Reliability Validity

Reliablity GENERAL DEFINITION: Accuracy or precision of a measuring instrument. SPECIFIC DEFINITIONS: 1. Similar results - stability, dependability predictability 2. Accuracy – consistency 3. Absence of random or chance error -- extent to which errors of measurement are present in a measuring instrument

Tests for Checking Reliability Test-retest method - take the same measurement more than once. Equivalence: use "essentially the same" measurement items on the same instrument or on different instruments and compare the answers (same time period). Split-half, Random half, alternate forms. Use established measures.

Internal Validity DEFINITION: the ability of the measuring instrument to measure one's theoretical concepts. METHODS OF ASSESSING VALIDITY: PRAGMATIC (or Criterion) VALIDITY: predict to an outside criterion and compare the outcome to the outside criterion a. Concurrent: comparison to an existing or current outside criterion b. Predictive: comparison to a future outside criterion FACE VALIDITY: obvious and self-evident content

Validity (cont.) CONTENT VALIDITY: representativeness of what is being measured to the intended concepts (capturing all the dimensions of the social concept) CONSTRUCT VALIDITY: adequacy of the measuring instrument for measuring the theoretical concepts and relationships; also adequacy of the logical structure of the conceptualization and operationalization.

Construct Validity

External Validity

Political Polls and Survey Sampling In the 2000 Presidential election, pollsters came within a couple of percentage points of estimating the votes of 100 million people. To gather this information, they interviewed fewer than 2,000 people.

Terms Population Sample Element Sampling frame Enumeration units Sampling error Standard error of the mean

Election Eve Polls - U.S. Presidential Candidates, 2000 Agency Gore Bush Nader Buchanan 11/6 IDB/CSM 47 49 4 CBS 48 1 CNN/USA Today] 46 Reuters/ MSNBC 5 Voter. com 45 51 11/7 Results 3

Observation and Sampling Polls and other forms of social research, rest on observations. The task of researchers is to select the key aspects to observe, or sample. Generalizing from a sample to a larger population is called probability sampling and involves random selection.

Types of Nonprobability Sampling Reliance on available subjects: Only justified if less risky sampling methods are not possible. Researchers must exercise caution in generalizing from their data when this method is used.

Types of Nonprobability Sampling Purposive or judgmental sampling Selecting a sample based on knowledge of a population, its elements, and the purpose of the study. Used when field researchers are interested in studying cases that don’t fit into regular patterns of attitudes and behaviors

Types of Nonprobability Sampling Snowball sampling Appropriate when members of a population are difficult to locate. Researcher collects data on members of the target population she can locate, then asks them to help locate other members of that population.

Types of Nonprobability Sampling Quota sampling Begin with a matrix of the population. Data is collected from people with the characteristics of a given cell. Each group is assigned a weight appropriate to their portion of the population. Data should provide a representation of the total population.

Probability Sampling Used when researchers want precise, statistical descriptions of large populations. A sample of individuals from a population must contain the same variations that exist in the population.

Probability Sampling Most effective method for selection of study elements. Avoids researchers biases in element selection. Permits estimates of sampling error.

Populations and Sampling Frames Findings based on a sample represent the aggregation of elements that compose the sampling frame. Sampling frames do not always include all the elements their names imply. All elements must have equal representation in the frame.

Types of Sampling Designs Simple random sampling (SRS) Systematic sampling Stratified sampling

Simple Random Sampling Feasible only with the simplest sampling frame. Basic method assumed in most statistical computations

Systematic Sampling At least or slightly more accurate than simple random sampling assuming ... Arrangement of elements in the list can result in a biased sample.

Stratified Sampling Rather than selecting sample for population at large, researcher draws from homogenous subsets of the population. Results in a greater degree of representativeness by decreasing the probable sampling error.

Multistage Cluster Sampling Used when it's not possible or practical to create a list of all the elements that compose the target population. Involves repetition of two basic steps: listing and sampling. Highly efficient but less accurate.