Presentation on theme: "School Performance Variation on COMLEX Level 1 Linjun Shen, Ph.D., MPH National Board of Osteopathic Medical Examiners."— Presentation transcript:
School Performance Variation on COMLEX Level 1 Linjun Shen, Ph.D., MPH National Board of Osteopathic Medical Examiners
Purposes n Provide more information to school administrators and educators n Help school administrators and educators to interpret and analyze COMLEX data n Demonstrate analytical approaches individual schools can adopt in digesting their own COMLEX data
Limitations n Analysis is limited to Level 1 June 2002 administration n Variables available for this analysis is limited, so as the results
A model of the medical school- licensing examination relationship
Scope of This Analysis n Interest of the analysis: school mean scores and school passing rates on Level 1. n Potentially influential factors: gender of students, sector of school, school mean MCAT. n Analytical approach: mainly descriptive and graphical with a brief statistical modeling.
School Mean Performance n Two ways to rank schools: by mean score and passing rate n Numerical ranking is not as important as qualitative grouping n Variation among school mean scores is substantial (53 point between the highest and lowest, 40 point between above and below average groups)
Significant Gender Differences on June 2002 Level 1
Gender Difference in Failing Percentage by School
Gender Difference in Osteopathic Medical Schools n There are substantial gender differences in terms of individual student Level 1 performance n For a few schools, the differences in both mean scores and passing rates are alarming n Gender differences can also be an opportunity for schools to improve if efforts are made to narrow the gap
Effects of Sector of Osteopathic Medical Schools n There is no evidence that sector significantly affects Level 1 performance n According to a study 10 years ago, sector was a powerful factor in determining osteopathic Board exam performance n It is a very positive improvement of osteopathic medical education n More formal study is needed
MCAT and Level 1 Performance n The implication of correlation of school means of MCAT and a single Level 1 administration is highly limited n Individual student MCAT and Level 1 score relationship can be very different from school mean MCAT and school mean Level 1
Results of Conventional Regression n The model is significant. Both gender and sector are significant predictor. n This regression model with two variables only explained 3% of the student. performance variation. Most of the variation remains unexplained. n More information is needed to explain student performance.
Results of HLM Analysis n Difference of students Level 1 performance across school is significant n Gender is a predictor for Level 1 scores but the difference of gender is not significant across schools n Neither Sector and school mean MCAT predict student Level 1 performance n More data will help: student MCAT, race, curriculum type, etc.