1. e-Testing based on Service Oriented Architecture Institute of Informatics - FNS University “Ss. Ciryl and Methodious” – Skopje, Macedonia 10-th International Computer Assisted Assessment Conference Goce Armenski, M.Sc (armenski@ii.edu.mk)

2. 10-th International Computer Assisted Assessment Conference 1. CONTENT Content - C ONCLUSION - C OMMON FRAMEWORK - Е Т ЕСТ – S YSTEM FOR ELECTRONIC TESTING - S ERVICE ORIENTED ARCHITECTURE - I NTRODUCTION - А RCHITECTURE, CONCEPTS AND FUNCTIONALITY OF Е Т EST - A PPLICATION OF E T EST - R ESULTS - SOA IN E A SSESSMENT

3. 10-th International Computer Assisted Assessment Conference 2.2.2.2. 1. INTRODUCTION Introduction Changes in the way people live influenced by: - Globalization - Increased meaning of knowledge - Information and communication revolution Changes in the education (lot more students) - Lecturing process - Delivery of materials - Knowledge assessment

4. 10-th International Computer Assisted Assessment Conference 3. Introduction New education paradigms Teacher  Student centered education

5. 10-th International Computer Assisted Assessment Conference 4. Introduction Institute of Informatics (FNS) - every year 150+ students - 4 questions (assignments) x 150 students = 600 questions - 5 min for marking a question - 3000 min / 50 hours How long the marking will take??? Ontime feedback??? Personalized feedback??? Objectivity??? MOTIVATION AND GOAL

6. 10-th International Computer Assisted Assessment Conference 5. Motivation and goal Development of a system for assessment of large number of students (more that 150) every month, which can be used in distance learning, as well as any other form of knowledge or skill assessment. - Motivation for reports creation Statistical analyses of gathered information - Motivation for test construction Different tests for every student with same weight Lowering the possibility for memorizing - Motivation for simple data entry Possibility for bulk entering of data

7. 10-th International Computer Assisted Assessment Conference 6. 2. еТEST – CONCEPTS AND FUNCTIONALITY еТest – concepts and functionality - Question bank, - Algorithms for test creation, - Systems for data presentation, - Result reports QuestionMark, BlackBoard, WEB CT, Top Class, EduSystem Basic concepts of systems for eTesting

8. 10-th International Computer Assisted Assessment Conference 7. еТest – concepts and functionality еТest Technology - Web based application - Active Server Pages (ASP) - JavaScript - SQL Server 2000 - NT Server and Win 2000 compatible - IIS 4.0 or newer Web based sollutions VS desktop based sollutions Web Browser (Netscape 4.x or Internet Explorer 4.x, and above) Win 2000 Server Email (SMTP) IIS 4.0 +ASP SQL Server Access JScript

9. 10-th International Computer Assisted Assessment Conference 8. еТest – concepts and functionality еТest Architecture Fig. 2 Three tier architecture of the eTesting system

10. 10-th International Computer Assisted Assessment Conference 9. еТest – concepts and functionality - Course organization - Types of questions - Test creation algorithm - System for data presentation - Marking and reporting еТест Concepts

11. 10-th International Computer Assisted Assessment Conference 10. еТest – concepts and functionality - learning objects - three structure Course organization Fig. 3 Course organization

12. 10-th International Computer Assisted Assessment Conference 11. еТest – concepts and functionality Multichoice questions (choose one of many, choose many of many, yes/no answers);Multichoice questions (choose one of many, choose many of many, yes/no answers); Short entry answer (text or numerical);Short entry answer (text or numerical); Essay answer.Essay answer. Types of questions Questions can have pictures or graphs in the text or offered answersQuestions can have pictures or graphs in the text or offered answers

13. 10-th International Computer Assisted Assessment Conference 12. еТest – concepts and functionality - dynamic linear tests (fixed number of questions) Marking and reporting Test creation algorithm - results are shown at the end of the test - negative marking Fig. 4 Algorithms for test creation

14. 10-th International Computer Assisted Assessment Conference 13. еТest – concepts and functionality Statistical data analyses Identification of content which is not well presented;Identification of content which is not well presented; Personalized feedback to students;Personalized feedback to students; Identification of week questions which need to be revised before used again;Identification of week questions which need to be revised before used again; Identifying the individual weaknesses of students.Identifying the individual weaknesses of students.

15. 10-th International Computer Assisted Assessment Conference 14. 4. APPLICATION OF еТEST Application of eTest - Integration in the process of learning - Controlled learning Fig. 5 Way of passing the learning objects Successful strategy?? - all questions - N questions in a row - N right questions - 3 right questions in a row Statistical analyses of the user activities

16. 10-th International Computer Assisted Assessment Conference 15. 5. RESULTS Results The use of technology in education is very dependent of the organization - logistics - social changes Synchronization with other systems Technical infrastructure Practical Implementation Institute of Informatics, FNS (2001)Institute of Informatics, FNS (2001) А.D. Mobimak (2002)А.D. Mobimak (2002) UNDP (2003-2004)UNDP (2003-2004)

17. 10-th International Computer Assisted Assessment Conference 16. Results Institute of Informatics, FNS (january 2001) - 26 courses - 26 courses - 12391 questions - 12391 questions - 589 scheduled assessments - 589 scheduled assessments - 9861 generated tests - 9861 generated tests Data gathering Does eTesting provides more effective and more objective assessment compared to the traditional forms, and does it help the learning process

18. 10-th International Computer Assisted Assessment Conference 17. Results Installed in 4 Faculties from 3 Universities - not interoperable - does not allow searching or using common question bank - does not allow cross-institutional cooperation - not fully standards compliant Problems Facing with

19. 10-th International Computer Assisted Assessment Conference Possible Solution 18. Results 10-th International Computer Assisted Assessment Conference

20. 19. 6. SERVICE ORIENTED ARCHITECTURE Service Oriented Architecture 10-th International Computer Assisted Assessment Conference IT Architecture Requirements Simplicity Simplicity Software that reflects the need of business Software that reflects the need of business Agility and flexibility Agility and flexibility Reusability Reusability Technology independent Technology independent Promote integration Promote integration Built to Change

21. 10-th International Computer Assisted Assessment Conference Evolution of the Web Generation 1 Static HTML HTML Generation 2 Web Applications HTML HTML, XML Generation 3 Web Services 20. Service Oriented Architecture 10-th International Computer Assisted Assessment Conference

22. 21. Service Oriented Architecture 10-th International Computer Assisted Assessment Conference An approach for building distributed computing systems based on encapsulating business functions as services that can be easily accessed in a loosely coupled fashion. SERVICE ORIENTED ARCHITECTURE

23. 10-th International Computer Assisted Assessment Conference SOA: It’s Not a Thing Product Product A specific technology A specific technology An application An application A specific standard A specific standard A specific set of rules A specific set of rules It’s not a: An Approach for Building Agile and Flexible Business Applications 22. Service Oriented Architecture 10-th International Computer Assisted Assessment Conference

24. 7. SOA in eLearning 23. SOA in eLearning 10-th International Computer Assisted Assessment Conference JISC Technical Framework to support e-Learning (ELF). JISC Technical Framework to support e-Learning (ELF). IMS Abstract Framework (IAF) IMS Abstract Framework (IAF) LeAPP Learning Architecture Project LeAPP Learning Architecture Project Carnegie Mellon’s Learning Services Architecture Carnegie Mellon’s Learning Services Architecture

25. 10-th International Computer Assisted Assessment Conference e-Learning (ELF) 24. SOA in eLearning 10-th International Computer Assisted Assessment Conference

26. 8. SOA in eAssessment 25. SOA in eAssessment 10-th International Computer Assisted Assessment Conference FREMA. FREMA. TENCompetence Assessment Model TENCompetence Assessment Model

27. 10-th International Computer Assisted Assessment Conference FREMA Core Services Assessment Services Assessment Services –Assign –Author item –Author assessment –Validate assessment –Take assessment –Mark assessment –Moderate assessment –Grade Assessment –View Feedback Support Services Support Services –Schedule –Notify –Track –Authorise and Authenticate Assessment Domain Definition Use Cases Service Profiles Gap Analysis Reference Impl’ Common Usage Patterns26. SOA in eAssessment 10-th International Computer Assisted Assessment Conference

28. TENCompetence - Phases in the assessment process Assessment Design ItemConstruction AssessmentConstruction Assessment Delivery ResponsEvaluation Decision Making 27. SOA in eAssessment 10-th International Computer Assisted Assessment Conference

29. eTest Core Services Assessment Services Assessment Services –Item Construction –Test Construction –Test Delivery –Results Collecting –Marking –Decision Making –Statistical Analysis Support Services Support Services –Schedule –Notify and Announce –Track –Authentication –User Management 28. SOA in eAssessment 10-th International Computer Assisted Assessment Conference

30. 29. SOA Projects in eAssessment SOA in eAssessment –Research Projects TOIA – Technologies for Interoperable assessment TOIA – Technologies for Interoperable assessment ASSIS – Assessment Sequencing ASSIS – Assessment Sequencing APIS – assessment Provision through Interoperability APIS – assessment Provision through Interoperability … and many others … and many others –Standards Bodies In Assessment – IMS, OSIDs … In Assessment – IMS, OSIDs … In Web Services - SOAP, WSDL, WSRF, W3C … In Web Services - SOAP, WSDL, WSRF, W3C …

31. 10-th International Computer Assisted Assessment Conference 30. 6. CONCLUSION and FUTURE FORK Conclusion Service Oriented Architecture Advantages over monolithic applications for TestingAdvantages over monolithic applications for Testing Review of existing SOA models for eAssessment Design of Common Framework by service identificationDesign of Common Framework by service identification Practical realizationPractical realization Extend the ASSIS project with different test delivery models Dependent on contextDependent on context Game based learningGame based learning

32. 10-th International Computer Assisted Assessment Conference Thank you for your time!