Download presentation

Presentation is loading. Please wait.

Published byJermaine Widmer Modified over 3 years ago

2
Degree to which inferences made using data are justified or supported by evidence Some types of validity ◦ Criterion-related ◦ Content ◦ Construct All part of unitarian view of validity Constructs - theoretical abstractions aimed at organizing and making sense of our environment; they are LATENT

3
A criterion is any variable you wish to explain and/or predict They are the key to well-developed theory, good measurement, and strong research design Ultimate criterion Multidimensional nature of criteria Intermediate criteria

4
Process of establishing a relationship between variables Predictive, concurrent, postdictive Usually based on correlation or regression equation Low reliability will attenuate or mask relationships

5
Selection Ratio – proportion of the individuals in the sample who are selected of the total number Base rate – percent of successful individuals under random selection Range Restriction Differential Prediction for different subgroups

6
FN FP VP VN XcXc YcYc FN+VP=BR VN+FP=1-BR VP+FP=SR FN+VN=1-SR Successful Unsuccessful RejectAccept False Negatives False Positives

7
FN FP VP VN XcXc YcYc FN+VP=BR VN+FP=1-BR VP+FP=SR FN+VN=1-SR Successful Unsuccessful RejectAccept False Negatives False Positives

8
FN FP VP VN XcXc YcYc FN+VP=BR VN+FP=1-BR VP+FP=SR FN+VN=1-SR Successful Unsuccessful RejectAccept False Negatives False Positives

9
FN FP VP VN XcXc YcYc FN+VP=BR VN+FP=1-BR VP+FP=SR FN+VN=1-SR

10
Even low correlations can lead to large increases in selection efficiency SR and BR have strong influences When SR is small (choose few), fewer FP and more FN When SR is large, fewer FN and more FP When BR is large (many can be successful), SR and validity have little effect on selection efficiency Most gains in success ratio when BR =.50 and SR is small (e.g.,.10) The tradeoffs depend on purpose of selection

11
X Y - Direct - Indirect - Ambiguous

12
X Y Same prediction for each group

13
X Y Different prediction for each group

14
Extent to which items or measures cover the content area the test purports to measure ◦ Expert judges determine if a measure came from a particular content domain ◦ Scoring and content is based upon theory ◦ If measures are from same content domain, should demonstrate high reliability ◦ If low internal consistency reliability, low content validity

15
Validity of inferences about latent unobserved variables on the basis of observed variables Does a measure assess what it is intended to assess? Do the variables relate in theoretically meaningful ways? Low reliability will make it difficult to assess the nature of a particular construct and attenuate relationships with other constructs

16
Construct Validity Can we generalize to the constructs from the measures? Theory What you think Cause Construct Effect Construct Measure or Manipulation Observed Outcomes Observed Relationship True Relationship What you see

17
Anxiety Test Score (Y) Measure of Anxiety (X) Ability to Learn 1 2 3 4 Salads Eaten (Z) Vegetarianism 5

18
Internal Structure Analysis Cross Structure Analysis Nomological network (Cronbach & Meehl)

19
Factor Analysis ◦ Used to identify factors or dimensions that underlie relations among observed variables Exploratory - Useful When: ◦ No info on internal structure available ◦ Factor structures may look different than original scale ◦ You have reservations about previous factor analyses Confirmatory - Useful When: ◦ You have some idea of the internal structure ◦ Confirming factor structures from previous studies Necessary but not sufficient to establish construct validity

20
Ability to Learn Z1Z1 Z2Z2 Z3Z3 Anxiety X1X1 X2X2 X3X3 X4X4 e1e1 e3e3 e4e4 e5e5 e6e6 e7e7 e2e2

21
Embedded in nomological network (nomological validity) Test of hypotheses by examining relationships between different indicators of underlying constructs ◦ e.g., leadership style based on reports from subordinates and leadership self-report inventory Relies on multiple methods of measurement

22
A representation of constructs of interest in a study, their observable manifestations (measures), and the interrelationships among and between them Cronbach & Meehl said this is necessary to establish construct validity Elements include: ◦ Specify linkage between constructs (hypotheses) ◦ Operationalize constructs (specify measurement)

23
Convergent validity - Convergence among different methods designed to measure the same construct Discriminant validity - Distinctiveness of constructs, demonstrated by divergence of methods designed to measure different constructs Multi-Trait Multi-Method

24
Heterotrait-Monomethod ◦ Different traits, same method Heterotrait-Heteromethod ◦ Different traits, different methods Monotrait-Heteromethod ◦ Same trait, different methods ◦ Validity diagonals Monotrait-Monomethod ◦ Same trait, same method ◦ Reliability diagonals

25
Method1Method2Method3 A1 B1 C1A2 B2 C2A3 B3 C3 M1 A1 (.89) B1.51 (.89) C1.38.37 (.76) M2 A2.57.22.09(.93) B2.22.57.10.68 (.94) C2.11.11.46.59.58 (.84) M3 A3.56.22.11.67.42.33(.94) B3.23.58.12.43.66.34.67 (.92) C3.11.11.45.34.32.58.58.60 (.85)

26
Specify the nomological net (expected + and - relationships) of expected relations Establish reliability Check convergence with other preexisting measures of the construct (convergent validity) Factor analysis Empirical studies of relatedness Empirical studies of discriminability

27
Take the hypotheses you developed in assignment 2 and the variables that were included in them. ◦ Draw a picture of what you believe the nomological network of these variables would look like ◦ What alternative measures of each variable might you use (different than those specified in Assignment 3) to establish convergent validity? ◦ Draw what an MTMM construct validity chart would look like that includes each variable in your study and the original and alternative measures you identified for each construct. Specify whether each correlation would be expected to be Hi, Low or Moderate.

Similar presentations

Presentation is loading. Please wait....

OK

Understanding Validity for Teachers

Understanding Validity for Teachers

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on main bodies of uno rules Free download ppt on carbon and its compounds Ppt on pi in maths games Ppt on boilers operations director Ppt on series and parallel circuits pdf Ppt on rational numbers for class 8 free download Ppt on file system in mobile computing Ppt on network switching device Ppt on abstract art pictures Download ppt on layers of atmosphere