1. MENO An admission test experiment at the Vrije Universiteit Brussel Raoul Van Esbroeck

2. Pag. 23/11/20062 Vrije Universiteit Brussel Why an admission test in Flanders? Open admission High failure rate Costs for society Costs for the individual student Need for more efficiency Admission test a possible option

3. Pag. 23/11/20063 Vrije Universiteit Brussel Why the MENO-test? UCLES (University of Cambridge Local Examination Syndicate) Skills crucial to success in HE Limited to 50 multiple choice items Critical Thinking (CT) with 3 subscales Problem Solving (PS) with 3 subscales Good reliability and validity Limited bias (language, age, ethnic origin) Gender and SES bias?

4. Pag. 23/11/20064 Vrije Universiteit Brussel Admission test experiment OBPWO project (Promoter: Prof. M. De Metsenaere) Questionnaire Social and cultural variables MENO- test Administered in class during first week Cohort of entering Freshmen in 95-96 Respondents: 85% of the target group (N = 1000)

5. Pag. 23/11/20065 Vrije Universiteit Brussel Does MENO allow correct classification? Errors in classification False positives (FP) False negatives (FN) Total number of errors % refused%FN%FP 10687 251863 503939 Conclusion: too many errors

6. Pag. 23/11/20066 Vrije Universiteit Brussel Has the test an added value? Can the test predict academic success better than SE results? A model with 4 variables: SE option, class rank, graduating on time MENO Explains 18.7% of variance MENO represents only 1.75% in this model Conclusion: only limited added value

7. Pag. 23/11/20067 Vrije Universiteit Brussel Bias? There is a gender bias Girls would be more refused than boys FN error is higher with girls FP error is higher with boys There is a SES bias Lower SES will be more refused than higher SES groups FN error is higher with lower SES groups FP error is higher with higher SES groups

8. Pag. 23/11/20068 Vrije Universiteit Brussel Increase of success rate? There is limited increase If 25% of the lowest MENO-scores are eliminated success rate increases from 49% to 58% What about errors?

9. Pag. 23/11/20069 Vrije Universiteit Brussel Conclusion on MENO MENO is not an acceptable admission test Too many errors in classification Limited added value There is gender and SES bias The gains are limited Using this test would be unethical

10. Pag. 23/11/200610 Vrije Universiteit Brussel Are other tests better? There are indications that the MENO in the UK is also biased The Chrysostomos test (P. Janssen, KULeuven) also faces classification errors A “Pass” (50%) score on the test leads to 37.6% of classification errors A 60% score on the test leads to 30.9% of classification errors (with about 5% of FN) Similar problems exist in other countries where admission tests are used

11. Pag. 23/11/200611 Vrije Universiteit Brussel General conclusion There is no need for admission tests because There is limited added value Too many errors are made There is bias The gains in success rate are limited The costs for the tests are extremely high There is no real societal benefit if FN are to be avoided There will be too many wrong decisions if you want to have real gains

12. Pag. 23/11/200612 Vrije Universiteit Brussel Personal reflection The use of general admission tests for selection is unacceptable and unethical Such tests may be excellent for guidance purposes