IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Michael C. Rodriguez University of Minnesota & Michigan State University Data Analysis & Scale Evaluation

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Factor Analysis  Factor Analysis examines the inter- correlations of items, identifies items that are correlated as sets  Factor Loadings  Variance Explained

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Factor Loadings Primary N=830 Questions Factor loading A..564 B..686 C..712 D..661 E..576 F..713 G..736 A factor is a unidimensional measure of “something” A loading is a correlation between the item and factor Does the item contribute to the total factor? Should be positive and relatively high (.50+)

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Variance Explained % Variance Explained44.5 Each item contributes variance The total variance is the sum of the item variances As a set, the factor accounts for variance from all the items If the factor is an efficient summary of all of the items, it will explain a large percent of the total variance

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Reliability Analysis  Reliability Analysis examines the consistency of the total score and contribution of each item to the total score  Coefficient Alpha  Item-Total Correlations

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Coefficient Alpha Primary Alpha=.85 Coefficient Alpha is an index of score reliability Technically speaking, it is the proportion of observed variance that is true (systematic) variance It tells us degree to which scores are reliable, consistent, replicable This should be above.80 for research purposes (when above.90, scores for individuals can be used) Alpha is not an index of unidimensionality, but may indicate the presence of a “common factor”

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Item-Total Correlation Primary Alpha=.85 Questions Item-Total Correlation A..518 B..626 C..643 D..601 E..529 F..645 G..667 Total score is based on the sum of items – but not necessarily a unidimensional measure Correlation between item and total score Does the item contribute to the total score (total measure) Should be positive and relative high (.30+)

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Sensitivity to Group Differences  A factor is not very useful for research purposes if it is not sensitive to group differences.  Country means were computed with 95% confidence intervals

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Country Means Country LKJIHGEDCBA Practicum Activities Scale Score

IEA Teacher Education Study in Mathematics 3 rd NRC Meeting, June 25-29, Taipei, Chinese Taipei. Notes of Caution  Field Trial Data is tentative, but useful as we select items  Country means are anonymous – only for illustrative purposes  Standard errors are not correct – final standard errors will be based on the sampling design