1. Research validity can be simply understood as the quality of a research study. There are two kinds of quality issues that are referred to as internal validity and external validity. Internal validity is the extent to which the outcomes of a study result from the variables which were manipulated, measured, or selected in the study rather than from other variables not included in the study. Internal validity is about how confident we are about the stated “causal” relationship between the independent variables and the dependent variable. That is, the dependent variable is due to independent variable but not due to something else.

2. Internal Validity Threat 1. History: Events take place during the study that might affect its outcome in the same way that the independent variable is hypothesized to affect the outcome. 2. Maturation: Especially for developmental studies where children grow with the passage of time to become more mature in certain developmentally related abilities. 3. Testing: When people are measured repeatedly, e.g., pretest-posttest, they become better not because of the independent variable but because they become test smarter. 4. Instrumentation: The effect on the dependent variable is not due to the independent variable but due to aspects of the instrument used in the study. 5. Regression towards the mean: Particularly problematic when subjects are chosen because of extreme scores where high scoring individuals are more likely to score lower and low scoring individuals are likely to score higher the next time they are tested merely due to random measurement error.

3. 6. Selection: Results due to assignment to different treatment or control groups but not due to the independent variable that makes the two groups. 7. Mortality: Especially for longitudinal studies that last for an extended period of time, attrition or dropping off from the study in a non-random manner may affect the outcome of the study. 8. Diffusion or imitation of treatment: The control group or one of the treatment groups somehow end up receiving some of the same treatment as the other groups resulting in few differences among the treatments. 9. Hawthorne effect refers to the fact that when participants received unusal treatment in a field experiment, they may temporarily change their behavior or performance not because of the manipulation of the independent variable but because of the special attention they received during the experimentation. 10. John Henry effect: Whereas the Hawthorne effect is due to some unusal performance of the experiemental group, sometimes the control group may also put up some extraordinary performance to outperform the experimental group due to a sense of demoralization for not being included in the “special treatment” experimental group.

4. External validity is the extent to which the findings of a particular study can be generalized to people or situations other than those observed in the study. Many threats to external validity can be understood in the form of an interaction: – Treatment-attribute interaction: Certain personality and other characteristics may interact with the independent variable so that the effect of the independent variable may be different on people having different personality characteristics. – Treatment-setting interaction: The independent variable may interact with other external factors or contexts to result in different effects for different settings so that the effect cannot be generalized to all settings.

5. Pretest sensitization: The effect of the independent variable may be due to the pretest which serves to sensitize the subjects whereas in the population (the real world to which the findings are to be generalized), there is no pretest and thus the treatment (independent variable) may not work as it does in the study. Posttest sensitization: The effect of the independent variable is part due to sensitization or exercising effect of the posttest which is not available in the population to which the results are to be generalized.

6. Sampling Techniques A population is all the elements in a defined set about which we wish to make an inference. Sampling units are non-overlapping collections of elements from the population. Sampling frame is a list of sampling units. A sample is a collection of sampling units drawn from a frame.

7. Simple random sample If a sample of size n is drawn from a population of size N in such a way that every possible sample of size n has the same chance of being selected. M = Σxi / n is the sample estimate of population mean, μ. Stratified random sampling The population of N units is divided into subpopulations of N1, N2... Nh units which are non-overlapping so that N1 + N2 +..+ Nh = N. The subpopulations are called strata. A sample is drawn from each stratum. The sample is denoted as n1, n2, nh. If a simple random sample is taken from each stratum, the whole procedure is called stratified random sampling. Cluster sampling A cluster sample is a simple random sample in which each sampling unit is a collection, or cluster, of elements. Systematic sampling Randomly selecting one element from the first k elements in the frame and every k th element thereafter is called a one-in-k systematic sample.

8. Sample size The method used to select the sample is of utmost importance in judging the validity of the inference made from the sample to the population. The representativeness of the sample is more important than the size of the sample. Sample size is considered in relation to power of the test to be used, effect size to be expected, and number of variables investigated.

9. Research Design The various ways to control extraneous variables and to make sure that the independent variables indeed “cause” the dependent variable make up the research design. Experimental design There is experimental manipulation of the independent variable and random assignment of subjects into different treatment conditions. Quasi-Experimental Design There is experimental manipulation of the independent variable but the assignment of subjects into different treatment conditions is not random but is based on existing non-equivalent groups. For lack of randomization, pretest is an integral part of quasi-experiment that enables comparisons among the nonequivalent groups, whereas, in most experiments, a pretest is often unnecessary or undesirable.

10. Non-Experimental or Ex Post Facto Research There is no manipulation of the independent variable and, thus, no random assignment of subjects into different treatment groups. In experimental and quasi-experimental research, inferences are made from the independent variables (the causes) to the dependent variable (the effect). In non-experimental research, also called "ex post facto research", inferences are generally made in the opposite direction. That is beginning with the observation of the dependent variable, attempts are made to uncover, detect, or find the reasons (independent variable) for the existing variations. Variations are not the result of the manipulation of the independent variables but are pre-existing and are explained post hoc of after the fact.

11. Two general strategies to protect internal validity are using: –(1) large samples to compensate for the lack of random assignment and, –(2) large numbers of "independent" variables to eliminate rival explanations. According to some authors, there are two kinds of non-experimental designs, causal comparative and correlational studies. The difference lies in the measurement of the "independent" variable which can be either categorical or continuous. The categorical or continuous "independent" variable is also called a grouping variable in causal comparative studies and an exogenous variable in correlational studies.

12. Manipulation of Independent V. Random Assignment Sample Size Variable Number ExperimentalYes Small Quasi-Exp.YesNoMedium Non-Exp.No Large Summary of Research Designs

13. Statistical Techniques to Control for Extraneous Variances Partial correlation. The calculation is complicated but the idea of partial correlation is simple. It is an estimate of the correlation between two variables in a population that is homogeneous on the variable (or variables) that is being controlled. Spurious effect. When two variables are correlated solely because they are both affected by the same cause, the correlation between these two variables is spurious. Mediating variable. The correlation between two variables can be the result of a mediating variable. Suppressor variable. A special case when a partial correlation is larger than its zero-order correlation is called a suppressor variable effect. In general, correlational studies include many relevant variables in an attempt to statistically tease out the “cause and effect.”

14. Quantitative versus Qualitative Research Quantitative focuses on the specific or most salient causal link whereas qualitative takes into consideration the chain of events as contributing to a specific social process. Alternative conditions: sufficient but not necessary Presence of the conditions associates with the presence of the outcome; but the absence of conditions does not associate with the absence of the outcome. It is sufficient by itself but not necessary. Flue virus is a sufficient but not necessary condition of headache. Contingent conditions: necessary but not sufficient Absence of the conditions indicates the absence of the outcome; but presence of the conditions does not indicate the presence of the outcome. It is necessary but not sufficient by itself. Ability to discriminate letters is necessary but not sufficient to reading.

15. Conclusion drawn from the two philosophies: Quantitative: To be able to infer causality, conditions have to be both sufficient and necessary; i.e., the presence of the conditions is accompanied by the presence of the outcome and the absence of the conditions is accompanied by the absence of the outcome. Qualitative: Constellation of conditions that are individually insufficient but necessary and jointly unnecessary but sufficient (INUS) to bring about the outcome

16. Quantitative: Philosophy: Isolated causal link Method: Experiment. Control extraneous variable to isolate out the particular linkage; e.g., attitude of subject, history, instrumentation, testing, maturation. standardize data collection. Following physical science tradition, studying only that is observable, measurable, and testable. Latent constructs have to operationalized. Lot of topics are not attempted for research. Qualitative: Philosophy: INUS, causal chain Method: Field study. Consider combinations of factors. Look at context. Use different data collection schemes to obtain all sorts of information. Social phenomenon and human behaviour are not directly observable which include intentions, feelings, aspirations influenced by norms, culture, values. Observable behaviour aren't any real than internal phenomenon.

17. Quantitative: P: Following the physical science tradition, study the social phenomenon or human behaviour as an objective and impartial observer. M: control internal validity threats, such as, observer bias and characteristics. Keep the subjects unaware of your research purpose. Hire data collector and standardize the data collecting condition by training them. Structured interview. Let the data speak. Terminology: Subject, researcher Qualitative: P: Take the perspective of the people being studied. See the world from the way they see it. M: Participation. Researcher is the only or major source of data collection. Subjects play a role in data interpretation. Having subjects read your report and modify afterwards. Terminology: Informants, collaborators, teachers, vs. participant.

18. Quantitative: P: Deductive reasoning, formulate theory from previous research and conduct specific empirical test. M: Hypothesis testing. Ask questions before data are collected. Use standardized test. Confirmatory or explanatory studies. Qualitative: P: Inductive reasoning, theory grounded in observation. From pieces of specific events and observations, develop an explanation. M: Start from scratch. Extensive and prolonged observation. Going back and forth between data and explanation until a theory is fully grounded by observation. No measurement. Measurement is not just asking questions but knowing what to ask. Exploratory or discovery oriented research.

19. Quantitative: P: Generalization M: Random sampling, hypothesis testing, inferential statistics. Qualitative: P: Context dependent, Generalization with caution M: Purposive sample to gather data from most representative situation to draw generalization. Informants are selected for their willingness to talk, their sensitivity, knowledge, and insights into a situation, and their ability and influence to gain access to new situations. No intention to use statistical inference. Lengthy text report. Text analysis.