1. 智慧型系統實驗室 iLab 南台資訊工程 1 Evaluation for the Test Quality of Dynamic Question Generation by Particle Swarm Optimization for Adaptive Testing Department of Computer Science and Information Engineering Southern Taiwan University of Science and Technology, Taiwan Shu-Chen CHENG, I-Chun PEN & Yu-Chih LIN

2. 智慧型系統實驗室 iLab 南台資訊工程 Outline 1.Introduction 2.Literature 3.Method 4.Experiment 5.Conclusion 2

3. 智慧型系統實驗室 iLab 南台資訊工程 Introduction(3/4) Item Response Theory(IRT) –Item characteristics (difficulty, discrimination, guessing) –Testee’s ability (latent trait) Divided three models –Single-parameter IRT (1PL) –Two- parameter IRT (2PL) –Three-parameter IRT (3PL) 3

4. 智慧型系統實驗室 iLab 南台資訊工程 Literature(1/8) Table 1. Computerized Test Comparison 4 Type Computer Based Tests (CBT) Computerized Adaptive Tests (CAT) Rationale Classical Test Theory (CTT) Item Response Theory (IRT) Advantage Computer-aided test and scoring ‚Reduce test error Depending on the ability to give tests ‚Fewer number of quiz will be able to estimate the ability and precision Disadvantage Test presentation method is fixed Can’t go back to check and modify the answer

5. 智慧型系統實驗室 iLab 南台資訊工程 Literature(3/8) Three-Parameter Logistic Model (3PL) 5 : correct possibility as testee’s answers item : testee’s ability : discrimination of item : difficulty of item : pseudo-chance of item

6. 智慧型系統實驗室 iLab 南台資訊工程 Experiment(1/4) Item Exposure Rate Control −Select 100, 500, 1000 testees and different-sized databases 6

7. 智慧型系統實驗室 iLab 南台資訊工程 Experiment(2/4) Test overlap rate under the condition of the add exposure control 7

8. 智慧型系統實驗室 iLab 南台資訊工程 Experiment(3/4) Test Quality Select 100, 500, 1000 testees 8

9. 智慧型系統實驗室 iLab 南台資訊工程 Experiment(4/4) Test Quality Comparison 9

10. 智慧型系統實驗室 iLab 南台資訊工程 Conclusion(1/1) Added exposure factor slightly improved the repetition rate of the test questions exposure control help Exam size greater than 300 questions, there will be better papers in the greater quality, exam greater the test quality 10

11. 智慧型系統實驗室 iLab 南台資訊工程 11 Thank you all for listening Thank you