Presentation is loading. Please wait.

Presentation is loading. Please wait.

UNIT Standardization and Technical Properties n Standardization Sample n Reliability Studies Internal Consistency Reliabilities at Decision-Making Points.

Similar presentations


Presentation on theme: "UNIT Standardization and Technical Properties n Standardization Sample n Reliability Studies Internal Consistency Reliabilities at Decision-Making Points."— Presentation transcript:

1 UNIT Standardization and Technical Properties n Standardization Sample n Reliability Studies Internal Consistency Reliabilities at Decision-Making Points Test-Retest Stability n Floors, Ceilings, and Difficulty Gradients n Validity Studies Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

2 Standards of Test Technical Adequacy for School-Aged Children Internal Consistency Median subtest internal consistency averaged across ages .80 Total test internal consistency averaged across ages .90 Screening test internal consistency averaged across ages .80 Test–Retest Stability Total test stability coefficient .90 Floors Average subtest floor 2 SDs or more below normative mean Total test floor is 2 SDs or more below normative mean Ceilings Average subtest ceiling 2 SDs or more above normative mean Total test ceiling 2 SDs or more above normative mean Difficulty Gradients For subtests and total test, no fewer than three measurement points per standard deviation, or each measurement point equivalent to no more than one- third standard deviation Validity Validity studies reported (e.g., group mean differences) Note. Adapted from “Limitations of Preschool Instruments and Standards for Minimal Levels of Technical Adequacy,” by B. A. Bracken, 1987, Journal of Psychoeducational Assessment, 5, pp. 313–326. Copyright 1987 by the Journal of Psychoeducational Assessment. Table 5.1 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

3 UNIT Standardization Sample n Normed on 2,100 children and adolescents aged 5.0–17.11 years n 108 sites across 38 states n Total of 3,865 children and adolescents tested for norms development and reliability, validity, and fairness studies n Sample representative of the general USA population based on 1995 census updates Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

4 UNIT Standardization Sites Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

5 UNIT Sampling Variables Age (5 years 0 months through 17 years 11 months) Sex ( Female, Male) Race (African American, Asian American, Native American, White, Other) Hispanic Origin (Hispanic, Non-Hispanic) Region (Midwest, Northeast, South, West) Community Setting (Urban/Suburban, Rural) Parental Educational Attainment (Less than High School Degree; High School Graduate or Equivalency; Some College or Technical School; Four or More Years of College) n Inclusion of Representative Exceptional Children and Adolescents Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

6 UNIT Standardization Sample Compared with U.S. Population: Sex, Race, Ethnicity, and SES UNIT Sample % U.S. Population % Sex Female49.948.9 Male50.151.1 Race African American16.016.0 White78.478.8 Other 5.7 5.2 Ethnicity Hispanic13.013.0 Non-Hispanic87.087.0 Parent Educational Attainment Less than High School14.013.7 High School Graduate29.529.1 Some College29.229.7  4 Years of College27.327.5 Note. N = 175 in each age group. Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

7 UNIT Standardization Sample Compared with U.S. Population: Geographic Region and Community Size UNIT Sample % U.S. Population % Geographic Region (38 states) Midwest24.424.1 Northeast18.018.2 South35.035.2 West22.722.5 Community Size Rural27.624.8 Urban/Suburban72.475.2 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

8 UNIT Standardization Sample Compared with U.S. Population: Special Education Services UNIT Sample % U.S. Population % Learning Disabilities5.65.9 Speech and Language Impairments2.32.4 Serious Emotional Disturbance0.91.0 Mental Retardation1.21.3 Hearing Impairments0.20.2 Intellectual Giftedness6.26.4 Bilingual Education1.83.1 English as a Second Language2.04.0 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

9 UNIT Internal Consistency Split-Half Correlations With Spearman-Brown Corrections Standard Battery Memory Reasoning Symbolic Nonsymbolic Full Scale Average Reliability.88.90.87.91.93 Clinical/ Exceptional Sample.95.96.95.97.98 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

10 UNIT Decision-Making Reliabilities Split-Half Correlations With Spearman-Brown Corrections (Also corrected for restriction or expansion in range) Standard Battery Memory Reasoning Symbolic Nonsymbolic Full Scale FSIQ = 70±10.97.96.98 FSIQ = 130± 10.96.97.95.98 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

11 Test-Retest Stability Corrected for variability on the first testing Standard Battery Memory Reasoning Symbolic Nonsymbolic Full Scale Corrected r.81.87.78.84.88 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

12 UNIT Floors and Ceilings Floors: A term describing the extent to which there are a sufficient number of easy items to distinguish between examinees of delayed to very delayed ability Ceilings: A term describing the extent to which there are a sufficient number of difficult items to distinguish between examinees of superior and very superior ability Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

13 UNIT Floors and Ceilings Floors: UNIT subtests have very good to excellent floors for low ability examinees across the entire age range, with the exception of the Abbreviated Battery at the youngest ages Ceilings: UNIT subtests have consistently excellent ceilings, even for the oldest and highest ability examinees Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

14 UNIT Item Difficulty Gradients Difficulty Gradients: A term describing the steepness of the gradations in item difficulty. An increase or decrease in a single raw score point should not result in a standard score change of more than one-third SD UNIT Difficulty Gradients: Average item gradients for UNIT subtests equal or exceed standards for every age level, indicating UNIT’s sensitivity to subtle differences in ability levels (with the exception of one subtest at the earliest ages) Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

15 UNIT Internal Validity Studies n Evidence of Content Validity n Structural Evidence of Validity Development and Growth Curves Subtest Unidimensionality Studies Subtest and Scale Intercorrelation Studies Comparison among the three UNIT Batteries Factor Analytic Investigations Exploratory Factor Analyses Confirmatory Factor Analyses Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

16 Age Progression of UNIT Subtests: Mean Raw Scores By Age Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

17 Age Progression on UNIT Subtests: Mean Rasch W Scores By Age Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

18 Age Progression of UNIT Standard Battery FSIQ: Mean Rasch W Score By Age Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

19 Exploratory Factor Analyses for Standard Battery Second-Order Principal Components Schmid-Leiman Solution for the Standardization and Clinical/Exceptional Samples Standardization SampleClinical/Exceptional Sample Second FirstSecond First Order OrderOrder Order g Memory Reasoning g Memory Reasoning Symbolic Memory.75.46 –.05.88.35.03 Cube Design.77.06.36.88.05.34 Spatial Memory.80.33.11.88.34.05 Analogic Reasoning.73 –.05.45.88.03.36 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

20 Confirmatory Factor Analyses for the Standardization Sample Factor Loadings (Standardized Regression Beta Weights) Memory Reasoning Symbolic Nonsymbolic Symbolic Memory 0.66 ——— Cube Design —0.70 —— Spatial Memory 0.74 ——— Analogic Reasoning —0.67—— Object Memory0.63 ——— Mazes —0.37—— Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

21 Confirmatory Factor Analyses for the Standardization Sample Factor Loadings (Standardized Regression Beta Weights) Memory Reasoning Symbolic Nonsymbolic Symbolic Memory——0.66 — Cube Design—— —0.68 Spatial Memory —— —0.75 Analogic Reasoning—— 0.65 — Object Memory——0.63 — Mazes—— — 0.36 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

22 Confirmatory Factor Analyses for the Clinical/Exceptional Sample Factor Loadings (Standardized Regression Beta Weights) Memory Reasoning Symbolic Nonsymbolic Symbolic Memory 0.71 ——— Cube Design —0.74—— Spatial Memory 0.78 ——— Analogic Reasoning —0.73—— Object Memory0.68 ——— Mazes —0.47—— Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

23 Confirmatory Factor Analyses for the Clinical/Exceptional Sample Factor Loadings (Standardized Regression Beta Weights) Memory Reasoning Symbolic Nonsymbolic Symbolic Memory——0.71 — Cube Design—— —0.71 Spatial Memory —— —0.78 Analogic Reasoning—— 0.70 — Object Memory——0.68 — Mazes—— — 0.47 Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

24 UNIT External Validity Studies n Correlational Studies Measures of Intelligence Measures of Academic Achievement n Studies with Clinical and Exceptional Samples Speech and Language Impairment (N = 57) Learning Disabilities (N = 205) Mental Retardation (N = 84) Intellectually Gifted (N = 160) Serious Emotional Disturbance (N = 23) Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

25 UNIT Standard FSIQ Correlations with Other IQs n WISC-III FSIQ across four samples (Learning Disabled, Mentally Retarded, Intellectually Gifted, and Native American).81–.84 n WJ-R Tests of Cognitive Ability BCA (Est):.83 n K-BIT IQ Composite:.82 n Matrix Analogies Test Standard Score:.83 n Raven’s Standard Progressive Matrices T-Score:.56 n TONI-2 Quotient:.63 Note: All correlations are corrected to control for restriction or expansion in range. r Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

26 Correlations Between UNIT Full Scale IQs and WJ-R Tests of Achievement UNIT Abbreviated Battery FSIQ133.78.84.78.86.82 95.3 23.0 UNIT Standard Battery FSIQ133.81.86.81.87.85 93.8 24.4 UNIT Extended Battery FSIQ133.80.87.81.87.85 92.1 24.0 Mean 92.4 94.3 86.494.9 88.5 SD 28.1 31.3 29.023.4 27.7 N WJ-R Broad Reading WJ-R Broad Mathematics WJ-R Skills WJ-R Broad Written Language WJ-R Broad Knowledge Mean SD Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

27 Comparative Prediction of Achievement by UNIT and Other Nonverbal Measures UNIT Abbreviated Battery FSIQ133.78.84.86.82 UNIT Standard Battery FSIQ133.81.86.87.85 UNIT Extended Battery FSIQ133.80.87.87.85 Leiter-R Brief IQ* 29.79.80 — — Leiter-R Full IQ* 29.82.82 — — TONI-3 (Form A)* 20.73.76.56.76 TONI-3 (Form B)* 20.71.74.55.70 *Note. These data come from each test's manual. N WJ-R Broad Reading WJ-R Broad Mathematics WJ-R Broad Knowledge WJ-R Skills

28 UNIT Validity Studies: Diagnoses and Exceptionalities n Speech and Language Impairment (N = 57) n Learning Disabilities (N = 205) n Mental Retardation (N = 84) n Intellectually Gifted (N = 160) n Serious Emotional Disturbance (N = 23) Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

29 Table 5.31 Abbreviated Battery FSIQMean94.16 98.324.160.28 SD 15.25 15.04 Standard Battery Memory QuotientMean94.81 99.514.790.31 SD 16.26 14.71 Reasoning QuotientMean91.42 98.537.110.47 SD 14.97 13.69 Symbolic QuotientMean93.67 99.896.220.41 SD 15.43 15.68 Nonsymbolic QuotientMean92.40 98.025.620.37 SD 15.95 14.03 FSIQMean91.98 98.796.810.45 SD 15.42 14.48 Extended Battery Memory QuotientMean93.91 99.936.020.40 SD 17.16 14.87 Reasoning QuotientMean90.42100.4210.000.67 SD 15.36 13.87 Symbolic QuotientMean93.00100.267.260.48 SD 16.62 15.56 Nonsymbolic QuotientMean91.26 99.898.630.58 SD 15.82 14.29 FSIQMean91.33100.058.720.58 SD 15.93 14.62 Note. Samples were matched according to age, sex, race, ethnicity, and parent education level. UNIT Performance by a Sample With Speech and Language Impairments and a Control Sample Clinical Sample (n = 57) Control Sample (n = 57) Difference Score Effect Size Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

30 Table 5.32 Abbreviated Battery FSIQMean91.87100.508.630.58 SD 13.61 13.27 Standard Battery Memory QuotientMean90.84100.769.920.66 SD 13.91 13.34 Reasoning QuotientMean91.85100.959.100.61 SD 14.17 13.46 Symbolic QuotientMean90.32100.059.730.65 SD 13.48 13.63 Nonsymbolic QuotientMean92.21101.519.300.62 SD 13.38 12.92 FSIQMean90.20100.8310.630.71 SD 13.67 13.00 Extended Battery Memory QuotientMean90.29100.7410.450.70 SD 13.42 13.34 Reasoning QuotientMean91.98101.049.060.60 SD 14.60 13.54 Symbolic QuotientMean89.96100.3110.350.69 SD 13.08 13.34 Nonsymbolic QuotientMean92.20101.439.230.62 SD 14.10 13.27 FSIQMean89.84100.8711.030.74 SD 13.69 13.19 Note. Samples were matched according to age, sex, race, ethnicity, and parent education level. UNIT Performance by a Sample With Learning Disabilities and a Control Sample Clinical Sample (n = 205) Control Sample (n = 205) Difference Score Effect Size Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

31 Table 5.33 Abbreviated Battery FSIQMean66.4296.9030.482.03 SD 13.0813.04 Standard Battery Memory QuotientMean67.5695.7728.211.88 SD 13.2912.65 Reasoning QuotientMean66.8897.5630.682.05 SD 11.8713.20 Symbolic QuotientMean66.5295.2528.731.92 SD 11.4012.48 Nonsymbolic QuotientMean67.1797.8730.702.05 SD 14.2912.96 FSIQMean62.4096.1333.732.25 SD 13.8012.36 Extended Battery Memory QuotientMean64.1794.6530.482.03 SD 13.4412.17 Reasoning QuotientMean62.9296.8733.952.26 SD 14.7014.07 Symbolic QuotientMean63.0794.2731.202.08 SD 12.2012.06 Nonsymbolic QuotientMean64.9697.0432.082.14 SD 14.5014.17 FSIQMean60.9695.0134.052.27 SD 12.7212.91 Note. Samples were matched according to age, sex, race, ethnicity, and parent education level. UNIT Performance by a Sample With Mental Retardation and a Control Sample Clinical Sample (n = 84 Control Sample (n = 84) Difference Score Effect Size Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

32 Table 5.34 Control Sample (n = 160) Exceptional Sample (n = 160) Difference Score Effect Size Abbreviated Battery FSIQMean117.03103.8613.170.88 SD 11.08 14.61 Standard Battery Memory QuotientMean112.99102.5410.450.70 SD 11.12 15.93 Reasoning QuotientMean118.36104.9913.370.89 SD 10.52 13.95 Symbolic QuotientMean115.63104.6311.000.73 SD 10.35 14.39 Nonsymbolic QuotientMean116.19102.9413.250.88 SD 10.75 14.22 FSIQMean117.64104.1413.500.90 SD 10.10 14.18 Extended Battery Memory QuotientMean112.69102.959.740.65 SD 11.04 15.95 Reasoning QuotientMean116.74104.8911.850.79 SD 10.78 14.24 Symbolic QuotientMean114.49104.4310.060.67 SD 10.46 14.60 Nonsymbolic QuotientMean115.04103.3011.740.78 SD 10.83 14.35 FSIQMean116.48104.2812.200.81 SD 9.85 14.55 Note. Samples were matched according to age, sex, race, ethnicity, and parent education level. UNIT Performance by a Sample With Intellectual Giftedness and a Control Sample Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

33 Table 5.35 UNIT Performance by a Sample With Serious Emotional Disturbance and a Control Sample Abbreviated Battery FSIQMean93.5793.35-0.220.01 SD 13.2714.18 Standard Battery Memory QuotientMean94.5795.520.950.06 SD 15.6515.14 Reasoning QuotientMean92.4392.650.220.01 SD 13.0314.24 Symbolic QuotientMean93.4894.871.390.09 SD 14.7114.54 Nonsymbolic QuotientMean93.4392.96-0.470.03 SD 12.9113.70 FSIQMean92.6593.090.440.03 SD 13.9713.67 Extended Battery Memory QuotientMean95.0995.650.560.04 SD 15.0214.31 Reasoning QuotientMean92.9192.78-0.130.01 SD 13.5115.07 Symbolic QuotientMean94.2695.301.040.07 SD 14.9014.04 Nonsymbolic QuotientMean93.5793.00-0.570.04 SD 12.6314.93 FSIQMean93.1393.430.300.02 SD 14.2414.18 Note. Samples were matched according to age, sex, race, ethnicity, and parent education level. Control Sample (n = 23) Clinical Sample (n = 23) Difference Score Effect Size Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.


Download ppt "UNIT Standardization and Technical Properties n Standardization Sample n Reliability Studies Internal Consistency Reliabilities at Decision-Making Points."

Similar presentations


Ads by Google