1. The design and the formative evaluation of an adaptive educational system based on cognitive style
Author: Evangelos Triantafillou*, Andreas Pomportsis , Stavros Demetriadis
Source: Computer & Education 41(2003) ,pp
Speaker: Hui - Chun, Chiang
Date: 2004/05/15

2. Outline Introduction Theoretical framework Design of AES-CS
Formative evaluation
Implement discussion
Conclusions
Commends
Appendix
Formative evaluation:形成性評量

3. Introduction
Designing AES-CS (Adaptive Educational System based on Cognitive Styles)
A formative evaluation of AES-CS
→to investigate the relationship of the learning behavior & cognitive styles
→more effective & efficient
→to assess the usability and appealing

4. Theoretical framework 1-1
Adaptive Hypermedia (AH)
Hypermedia:
Advantage:
Supporting the new constructivist way
Self regulated learning
Contribute to enhance learning
Promoting cognitive flexibility
Disadvantage
Regarding to cognitive overload
Disorientation
Adaptive
Be important for web-based educational hypermedia
Theoretical framework:理論架構：分別以Adaptive Hypermedia適性化超媒體 & Cognitive style and learning認知風格和學習來談論
Self regulated learning：自我管理學習
Constructivist way :構成的方法
Disorientation迷失

5. Theoretical framework 1-2
Adaptive Hypermedia System (AHS)
Building a user model of the goals , preferences and knowledge of the individual user
Accommodating various learner needs
Learning style & Cognitive style
Theoretical framework:理論架構：分別以Adaptive Hypermedia適性化超媒體 & Cognitive style and learning認知風格和學習來談論
Self regulated learning：自我管理學習
Constructivist way :構成的方法
Disorientation迷失

6. Theoretical framework 2-1
Cognitive style and learning
Cognitive style
Dealing with the form of cognitive activity, not content
i.e. thinking, perceiving, remembering
Learning style
Broader construct
Including cognitive along with affective and physiological styles
Perceiving:理解

7. Theoretical framework 2-2
Cognitive Style: well-known division →FD/FI ( Witkin 1971)
Field dependence
Field independence
More autonomous in interpersonal skills
Be global in their perceptions
Rely on external cues & be less able to differentiate an embedded figure
Be holistic and require external help
Required externally defined goals and reinforcements
Appeared to benefit from illustrative advance organizers
Prefer using program control
More autonomous in cognitive restructuring skills
Be analytical in their approach
Rely on internal cues & be more able to differentiate an embedded figure
Internal cues to help them to solve
Developed self-defined goals and reinforcements
Preferred illustrative post organizers
Prefer learner control
Perceptions:感覺
Illustrative:說明
Advance organizer :a bridging strategy that provides a connection between one unit and another, it acts as a schema for the learner to make sense out of the new concept.
Post organizer: a 概要and supports the reconstruction of knowledge, it’s available after the presentation of new information

8. Design of AES-CS 1 System architecture and implementation
It supports the course “Multimedia Technology Systems”, 4 years undergraduate students in Computer Science Dep. at the Aristotle University, Greece
Be adapted to the cognitive style and to the level of knowledge acquired by the student .
Be organized in the form of three basic modules
Domain model
Student model
Adaptation module
Aristotle University:亞里斯多德大學
These three components interact to adapt different aspects of the instructional process,
i.e. adapting the content according to user’s prior knowledge;
adapting the presentation of contents through selection combination of appropriate media;
adapting the teaching strategies;
modifying the selection of examples and links;
recommending appropriate hyperlinks.

9. Design of AES-CS 2 Adapting instructional strategies
AHS provides learners the ability to use different instructional modes
Its design both issues of cognitive style and teaching strategy
Teaching strategies refers to the instructional material and the instructional strategy
AES-CS adopts the instructional strategies according to their cognitive style
System screen for FI learners
System screen for FD learners
Take an example in a physical classroom: the ideal edu.environment , a tutor to experience and understand students’ personal goals and preferences and thus to promote their skill

10. Design of AES-CS 3 Adaptation flexibility
The flexibility provided is the optimization of the adaptation under the adaptive systems need to be controllable by the user using the toolbar
The system cannot be intelligent enough to appropriately adapt in all possible cases
The Toolbar

11. Formative evaluation An integral part of the design methodology
The judgments of the strengths and weakness of instruction
Improve the effectiveness and efficiency of the instruction
The types were used
Expert review
Content experts, designers, instructors
One-to-one evaluation
Small group evaluation
It’s an integral part(不可缺少的部分)of the design methodology

12. Implement discussion This AES-CS was revised twice
1st: after the expert review and the one-to-one evaluation
Five experts acted as evaluators in the expert review
A teaching/training expert, an instructional design expert, a subject-mater expert, an educational technologist , a subject sophisticates
Getting the prompt subject’s suggestions for improving from the reactions of experts and a debriefing session
Ten fourth year undergraduate students participated in the one-to-one evaluation after the expert review
It’s easy to modify the initial adaptation using the student model and interaction buttons
The structure of the course is clear and easy to understand
a subject sophisticates(精通): a student who has successfully completed the course

13. Implement discussion This AES-CS was revised twice
1st: after the expert review and the one-to-one evaluation
Some significant recommendations were implemented during the revision phase:
A dynamic technique resized the frame according to the mouse position
Keeping the same adaptive annotation (using the color) but all the links are available to be selected by the student
Giving an option regarding to adapt the background of the HTML page (before revision , the system was white background and black font)
a subject sophisticates(精通): a student who has successfully completed the course

14. Implement discussion This AES-CS was revised twice
2nd: after the small group evaluation
It’s consisted of the small group evaluation in a “real world”
A lecture ”Digital image” as the example learning material
The evaluation was conducted by collecting data about the instruction from a variety of sources
A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems .
”Digital image” is to introduce the role of digital image into the development of a multimedia application.
A variety of sources : different type of subjects (learner)
Target population : 目標的母群體

15. Implement discussion This AES-CS was revised twice
2nd: after the small group evaluation
Subjects: selecting from the volunteer represent the target population
According to the Gauss theory: 68.28% is contained in [ the mean - S.D. ~ the mean + S.D.] in a normal distribution
According to “the GEFT ( Witkin )”: classifying the participants (N=68) into FD and FI groups
The score the test ranged from 0 to 18, the mean was and S.D. was 4.35
Selecting eight students [mean score - S.D. ~ mean score + S.D.], the score was 8-17
Selecting one student from them with highest (18) & selecting one with lowest score(3)
A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems .
”Digital image” is to introduce the role of digital image into the development of a multimedia application.
A variety of sources : different type of subjects (learner)
Target population : 目標的母群體
Grouped Embedded Figure Test, GEFT(藏圖測驗)，有相當的信度與效度考驗(Witkin, Otman, Raskin, & Kats, 1971; Witkin &Morre, 1974)，在電腦教學上的應用探討甚多。
Example:受試者若能從藏圖測驗(Grouped Embedded Figure Test, GEFT)迅速而正確的找出隱藏圖的位置便屬於場地獨立的類型

16. Implement discussion This AES-CS was revised twice
2nd: after the small group evaluation
Procedure:
The initial adaptation accorded to the cognitive style
Belonging to FD (n=4), score range : 0~9
Belonging to FI (n=6), score range : 10~18
A pre-test determine subject’s prior knowledge on “Digital image”
Working at the subjects’ own pace without time limit
Finished, a post-test measures the learning gain from the instruction
An attitude questionnaire determines their experience using the system
A debriefing session assess the subjective satisfaction on the instructional and interface design
A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems .
”Digital image” is to introduce the role of digital image into the development of a multimedia application.
A variety of sources : different type of subjects (learner)
Target population : 目標的母群體

17. Implement discussion This AES-CS was revised twice
2nd: after the small group evaluation
Results:
The evaluation at this phase →improve the system →more effective and efficient
Pre-test & Post-tests have the same content in the form of open-ended iteams in each test
Each correct answer gets 2 points →total score 0~18
Performance means on pre- and pose- test
It’s demonstrated the students got a better performance in the post-test
The evaluation offers the usability and the appealing of the courseware
The learners felt challenged and wanted to use the courseware again
A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems .
”Digital image” is to introduce the role of digital image into the development of a multimedia application.
A variety of sources : different type of subjects (learner)
Target population : 目標的母群體
demonstrated:證實
Usability可用性: measure of the case of system’s learned and used . Improving usability directed at improved interface
The appealing of the courseware(魅力): through the attitude questionnaires and debriefing session

18. Implement discussion This AES-CS was revised twice
2nd: after the small group evaluation
The significant recommendations:
Concept map and graphic path indicator
FD: it’s useful to organize the structure of the knowledge domain, suggesting these tools should be active so to be sued as an extra navigation tool
FI: Don’t use the tools to organize the structure of the knowledge domain, agreeing the earlier suggestion and tools help them to easily navigate , avoiding disorientation
Users have the ability to display or not the content menu
All of them agreed on the usefulness to change amount of guidance and direction according to their individual needs
A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems .
”Digital image” is to introduce the role of digital image into the development of a multimedia application.
A variety of sources : different type of subjects (learner)
Target population : 目標的母群體
demonstrated:證實
Usability可用性: measure of the case of system’s learned and used . Improving usability directed at improved interface
The appealing of the courseware(魅力): through the attitude questionnaires and debriefing session
avoiding disorientation:避免迷失

19. Conclusions & recommendations
The design and development of an AES-CE
The system has two main components
The client running in the Web browser
Be implemented as Web pages residing in a number of liked framed.
The server running in a PC workstation
The development platform used is ASP using DHTML and JavaScript language

20. Conclusions & recommendations
Some recommendations from expert ,students, evaluation
Design : AES accommodates for cognitive styles were effective and efficient
Initial adaptation: It needs to be controllable by the user because it can’t be intelligent enough to appropriately adapt in all possible cases
Instructions : students have the ability to access the maximum amount of guidance and instructions
Structure: the concept map and the graphic path indicator should be active so to be used as an extra navigation tool

21. Comments
Providing another way (adaptive screen) to design an adaptive learning system
It only has ten sample participate the system implement, it’s not objective
It emphasizes cognitive style , but we can’t see the implement to differentiate FI / FD learner in the initial.

22. Appendix

23. Fig. 1. System architecture.
Be easy to construct & modify & reflect the characteristics of different students
1.”Adaptive presentation technique”: Conditional text, page variants
2.”Adaptive navigation support”: finding an suitable path in hypermedia-learning environment
3.By student’s prior knowledge
It serves as a basis for structuring the content of AES-CS

24. Table 1 The student model

25. Table 2 Instructional strategies
Program control versus Learner control
Global versus Analytical approach
The use of contextual organizer
Instructions
Structure
Feedback
Instructional strategies and
the implementation

26. This AES-CS was revised twice
Fig. 2. System screen with the initial adaptation for Field Independence (FI) learners.
Blue color →
Recommended
Gray color →
Not ready to be learned
Design of AES-CS 2
Discussion –
This AES-CS was revised twice

27. Fig. 3. System screen with the initial adaptation for Field Independence (FD) learners.
Toolbar will discuss later

28. Table 3 Instructional strategies and their implementation

29. Fig. 4. The Toolbar.

30. Fig. 5. The frame with instructional guidance before and after the revision
Before the revision
(fixed size)
After the revision
(flexible size)

31. Table 4 Performance means on pre- and post-test
Both the FD & FI , the performance higher in post test
The improvement of the system , FD > FI

32. Table 5 Performance on pre- and post-test

33. Adaptive module The adaptive system screen for FI learner
The adaptive system screen for FD learner
Blue color →Recommended
Gray color →Not ready to be learned