1. Group Experimental Design

2. Validity
Validity is the degree to which the data are both meaningful and adequately represent the construct under study. 
Content validity is the extent to which the measured variables represent all the possible measures of content or skills. 
In surveys, do the questions represent all the possible questions that could be asked? 
With respect to performance, does the particular measure, e.g., attendance or computation, represent all possible components of the construct, e.g.,  parent participation or mathematics

3. Relationship of Reliability to Validity
Cannot have validity without reliability
A poorly worded question on a survey cannot be valid because it is not reliable,
It does not consistently assess participants’ attitude because it will be interpreted differently by the different participants. 
A well-worded question could be reliable but still may not be valid
It may be unrelated or tangentially related to the construct under study. 
Similarly, poorly worded behavioral definitions will not yield reliable data

4. Can have reliability without validity
Well-defined definitions will yield reliable data
But such data still may not meaningfully or completely represent the target construct, attendance does not adequately represent parent participation

5. Table 6. 4 p. 172 Form of Validity What is for? How do we use it?
What else do I need know?
Content Validity
Criteria Related Validity
Construct Validity

6. Unit of Analysis
The unit of analysis is the level of analysis (student, family, school, community, organization) by which the data need to be collected and analyzed to answer the questions asked.
We must not only consider the questions, but also moderating and mediating variables implicit in our questions.

7. Sampling Representative of a given population
Needs to be some target population

8. Probabilistic vs. Non-probabilistic Sampling
Quantitative Sampling
Probability Sampling
Simple Random
Stratified
Multi-stage
cluster
Non-probability sampling
Convenience
Snowball

9. Experimental Design
The purpose of an intervention (treatment or experimental) design is
To determine the effect of the intervention (the independent or manipulated variables) on the outcomes (the dependent variables)
To show causality: A caused X to occur

10. Experimental Design
To control the effect of all other potential influencing variables, except for the particular intervention
To show that it was the intervention alone that causes the outcome
Potential extraneous (“other”) influencing variables
The participants
The treatments
The procedures
To reduce threats to internal validity

11. What is Internal Validity?
Threat to Internal Validity
What is it?
Participants threat
History
Maturation
Regression
Selection
Mortality
Interactions with selection

12. What could you do about it? What do I need know?
Treatment threats
What is it?
What could you do about it?
What do I need know?
Diffusion of Treatments
Compensatory equalization
Compensatory rivalry
Resentful demoralization
Procedure Threats
Testing
Instrumentation

13. External Validity: Insuring it applies to other situations
Interaction selection and treatment
Interaction of setting and treatment
Interaction of history and treatment

14. What is common? What is not? What are challenges with each design?
True Experimental
Post test Experimental
Quasi-Experimental –Pre-Post
Quasi- experimental Post-test only

15. Between Groups Design

16. Factorial Design (2 x 3) Design a 3 X 3 X 4
Multiple Levels – we often call these grouping variables in data sets.
PPT
Clicker 5
Technology Instruction
type
Typical
At-risk
Special ED
Risk Level

17. Main Effects vs. Interaction Effects
Main Effect-Overall influence of the independent variable
Interaction Effect- when the influence on one independent variable covaries (depends on) the other independent variable
Look at p. 318 What does it mean to have an interaction effect?

18. REPEATED MEASURE DESIGN
Time 1
Time 2
Time 3