1. 1 USING EXPERT SYSTEMS TECHNOLOGY FOR STUDENT EVALUATION IN A WEB BASED EDUCATIONAL SYSTEM Ioannis Hatzilygeroudis, Panagiotis Chountis, Christos Giannoulis and Constantinos Koutsojannis University of Patras, Dept of Computer Engin. & Informatics 26500 Patras, Hellas (Greece)

2. 2 Outline 1. Introduction 2. System Architecture 3. Domain Knowledge 4. Learning Process 5. Domain Knowledge Management 6. Student Evaluation 7. Helping the Tutor 8. Implementation issues 9. Conclusion

3. 3 1. Introduction Web-based intelligent educational systems (WBIESs) use Artificial Intelligence (AI) Techniques in order to adapt mainly to student needs for self-study. An important function of such systems is student evaluation, which provides results for adapting to both the students and the tutor. There are many techniques used for student evaluation. However, a few of them use an expert system approach

4. 4 Introduction (2) To help the students and the tutors in our Department, we constructed an intelligent e-learning system to assist learning and teaching in the context of the course of Artificial Intelligence (course 451). The system provides means to the tutors for constructing questions and tests in a structured way. Also, with the help of an expert system, evaluates the students and provide useful statistics to the tutor.

5. 5 2. System Architecture The system consists of five units: the Identification Unit (IU), the Student Model (SM), Stereotypes for initial student profile the Tutoring Unit (TU), the Evaluation Unit (EU) and the Virtual Agent (VA) Identification Unit (IU) Student Model (SM) Tutoring Unit (TU) Evaluation Unit (EU) Stereotypes Expert System (ES) Virtual Agent (VA)

6. 6 System Architecture(2) SM contains all the information about students, like their preferences, interests, knowledge level, etc TU is responsible for the teaching process. A student can select a learning goal from the domain knowledge tree The main goal of EU is to evaluate student's progress due to his/her interaction with the system. This evaluation is achieved through testing. VA has as main goal to motivate the students and improve system's effectiveness. In our case, VA is an avatar (called Merlin) that provides information to the students in a natural and recreating way. To achieve this, VA gets information from the SM and communicates with the student in the same way that a human tutor would do.

7. 7 User Interface Navigation area Content area Start – Previous - next Info-bar www.mmlab.ceid.upatras.gr/aigroup/undergrad/

8. 8 3. Domain Knowledge The domain knowledge tree, as far as the subsections level, is displayed in the navigation area of the user interface (at the left side of the screen). From that tree the student can choose a learning goal (subsection). Each subsection corresponds to a learning page, which is a php page. That is, only subsections correspond to displayable material. The learning page of the selected subsection is currently presented in the content area (which resides at the centre and the right part of the screen). Each learning page deals with a number of concepts. Each concept is linked to the corresponding concept page. Concept pages constitute the real teaching material. Subject Chapter 1Chapter N Section 1 … … Subsection 1 Section N Subsection N Concept 1 Concept N … …

9. 9 Domain Knowledge(2) The teaching material, apart from concept pages, however, includes all the available questions, which are stored in a database and are used for the creation of the tests. Each learning page is associated with a test. Each test consists of a number of multiple choice type questions that examine the concepts of the associated learning page.

10. 10 4. Learning Process The learning method (implicitly followed) is based on the traditional theory-examples-exercises paradigm (although the user can follow his own method). That is, for each concept, the theory is first presented. Then, some examples are given. Finally, the student is called to make some exercises. The student is also not forced to follow the system’s way of teaching, but he can make his own choices for studying. For example, one can jump to complex concepts without having had a look at simpler ones. In many concept pages there are links to other concepts that are prerequisite to the concept of the page.

11. 11 5. Domain Knowledge Management The system offers to the tutor the capability of domain knowledge management. The tutor can deal with management of the domain tree and the teaching material (learning pages, concept pages, questions and tests). The most important part of teaching material management is the management of the questions. A question is considered as a structured object that consists of the following parts: the name of the question the body of the question, the concept it concerns (as well as its corresponding path in the domain tree), its difficulty level (one of easy, medium and difficult), its possible answers (1 to 4) each followed by a mark and an explanation and a hint (to help the student to answer correctly).

12. 12 Screen for Question Creation (Chapter) (Section) (Subsection) (Concept) (Difficulty level) Answer1-4Mark1-4Explanation1-4

13. 13 6. Student Evaluation A student can be evaluated at two levels: (a) the concept-level and (b) the topic-level. The concept-level evaluation deals with the level of understanding of the concepts of a learning page test, whereas the topic-level evaluation deals with the level of understanding of the topic of a learning page, i.e. the test as a whole. Student Evaluation implemented via the expert system (ES).

14. 14 The structure of the ES The knowledge level of a student, as far as both a concept and a topic are concerned, is classified in one of the following five categories: (a) excellent (86-100), (b) very good (71-85), (c) good (51-70), (d) average (31-50) and (e) low (0-30). Within the parentheses are the corresponding ranges of the marks to be achieved The rules are implemented via the expert system (ES). ES is a rule-based expert system implemented in Jess, which is an expert system shell. Fact Base Rule Base JESS Inference Engine KB

15. 15 Rules for answer marking Difficulty LevelNo of triesUse of HintAnswer StatusMark easy10Correct85 easy11Correct70 easy10Wrong30 easy11Wrong0 easy>20 medium10Correct92 medium11Correct77 medium10Wrong40 medium11Wrong15 medium21Wrong15 medium>215 difficult10Correct100 difficult11Correct85 difficult10Wrong50 difficult11Wrong30 difficult20Correct70 difficult21Correct50 difficult20Wrong50

16. 16 Question weight distribution Easy (E) Medium (M) Difficult (D) Question (s) weight(s) (/100) 11120 (E), 30 (M), 50 (D) 11040 (E), 60 (M) 10130 (E), 70 (D) 01140 (M), 60 (D) 10070 (E) 01080 (M) 001100 (D)

17. 17 7. Helping the tutor The system can present some statistics about students, which are the following: The percentage of the correct answers to questions per difficulty level per concept per student. The percentage of the correct answers to questions per difficulty level per test per student. The percentage of the correct answers to questions per difficulty level per concept for all students. The percentage of the correct answers to questions per difficulty level per test for all students.

18. 18 8. Implementation issues The system is web-based, so it works only through the WWW and cannot be downloaded and function on a student's computer. It is implemented in.php The VA (avatar), one of the interesting parts of the system, is implemented in VBScript. To use the avatar, we first have to call its object from the Windows Controller. Then we have to connect to that object and load it. We are then able to call its functions, show it on the screen and make it interact with the user. The ES is implemented in Jess, an ES cell implemented in Java. After the student has finished a test, a fact for each individual question of the test is saved in a file. Then the expert system is called to evaluate the student. The expert system, after having taken the facts from the file, returns the results in another file.

19. 19 9. Conclusions We concentrate on the knowledge management and student evaluation aspects of the system. Knowledge management mainly refers to test questions construction and management. Student evaluation refers to the evaluation of the knowledge level of a student with regards to taught concepts. A rule-based expert system helps in student evaluation. A number of statistics provided by the system give valuable information to the tutor. The system, as it is now, adapts to the student needs, but not to the tutor ones. It is only adaptable as far as the tutor is concerned. So, a further effort is to make it adaptive to the tutor needs as well.

20. 20 Thank you for your attention !

21. 21 USING EXPERT SYSTEMS TECHNOLOGY FOR STUDENT EVALUATION IN A WEB BASED EDUCATIONAL SYSTEM Ioannis Hatzilygeroudis, Panagiotis Chountis, Christos Giannoulis and Constantinos Koutsojannis University of Patras, Dept of Computer Engin. & Informatics 26500 Patras, Hellas (Greece)