1. Quality estimation of adaptive tutoring systems Volgograd State Technical University CAD department Pavel Vorobkalov Faculty advisor Shabalina O.A. Volgograd 2007

2. 2 The aim and tasks The aim: Quality management of learning process using adaptive tutoring systems Tasks: 1.To analyze problems of quality of adaptive tutoring systems 2.To develop method of quality estimation of adaptive tutoring systems 3.To design model of adaptive learning process 4.To develop criteria of quality estimation of adaptive learning 5.To implement computer-aided system of quality estimation of adaptive tutoring systems

3. 3 Quality management system of adaptive tutoring systems

4. 4 Analysis of approaches and methods of quality estimation of adaptive tutoring systems ‘as a whole’ approach  Questionnaires  Return of investments as a result of education  Common quality criteria (ex., according to ISO-9127)  Standard Conformance ‘layered’ approach  Estimation of interface layer  Estimation of adaptation models  Estimation of adaptation decisions  Estimation of adaptation techniques  Estimation of leaning content

5. 5 Model of human-computer interaction during adaptive learning process in layered approach Representation as a «white box» needed ? ?

6. 6 Method of estimation and prediction of quality of adaptive tutoring system 1. Collecting data about learning process using adaptive tutoring system 2. Building model of learning process 3. Analysis of built model 1. Computation of criteria of quality estimation of learning process 2. Expert estimation of learning process (detecting issues) 4. Learning course modification 1. Modification of concepts those are problematic 2. Testing changed concepts by student groups 3. Calculating new parameters of learning process model 4. Computation of predicted results of changed learning process (quality prediction)

7. 7 Adaptive learning process

8. 8 Модели процесса обучения ModelMajor advantages and disadvantages Psychological models, models based on cognition theory + Well known – No formal method of building model Grammar-based models (BNF, EBNF) + Advanced approach – Difficult to estimate Models based on algebraic and differential equations + Well formalized – High level of abstraction Machine models (Mur, Mille machines) + Wide range of described tasks – Hard to interpret Net models (Markov chains, Petri nets) + Easy to interpret – Big, hard to build

9. 9 Learning process model based on Petri nets

10. 10 Quantitative parameters of learning process As a quantitative parameters learning duration D and knowledge level KL have been chosen Knowledge level KL 1 Knowledge level KL 2 =KL 1 +  KL

11. 11 Building the learning process model

12. 12 Quantitative parameters of Petri net inferring learning process

13. 13 Criterion of adaptive decisions balance of tutoring system

14. 14 Expert estimation of learning process  KL (t 3 ) D (t 3 ) = 0,54 311  KL (t 1 ) D (t 1 ) = 0,25 155  KL (t 2 ) D (t 2 ) = 0,71 271  KL (t 4 ) D (t 4 ) = 0,80 275  KL (t 6 ) D (t 6 ) = 0,82 273  KL (t 5 ) D (t 5 ) = 0,85 255 Changes are required Learning results - ?  KL (t 1 ) D (t 1 ), t 1 :  (t 3 ) D (t 3 ), t 3 :

15. 15 Экран «Редактирование модели»

16. 16 Экран «Запуск моделирования»

17. 17 Экран «Результаты моделирования»

18. 18 Conclusions 1.Problems of quality of adaptive tutoring systems and approaches to their solution have been analyzed 2.Method of quality estimation of adaptive tutoring systems has been developed 3.Model of adaptive learning process has been designed 4.Criteria of quality estimation of adaptive learning have been created 5.The prototype of computer-aided system of quality estimation of adaptive learning has been implemented