1. Difficulty Analysis for Learners in Problem Solving Process based on the Knowledge Map Speaker: Rita Kuo Rita Kuo, Wei-Peng Lien, Maiga Chang, Jia-Sheng Heh Multimedia Communication System Laboratory Dept. of Information and Computer Engineering Chung-Yuan Christian Univ., Chung-Li, 320, Taiwan

2. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge2 Outlines ► Basic Definition  Basic Problem Model  Problem Construction Steps  Knowledge and Problem Structure ► Problem Difficulty Analysis  Difficulty Features  Difficulty Dimensions ► Demonstration of Item Generating System ► Brief Conclusion

3. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge3 Basic Problem Model ► Inter-correlated Knowledge  Manipulation of inter-correlation among knowledge  Ex. Physics, Chemistry, and Mathematics ► Knowledge Object  Way to analyze Inter-correlated Knowledge  Basic concept of a specific domain ► Basic Problem Definition  One Core Knowledge Object  Example in Physics: Physics Phenomenon ► Basic Problem Model Corresponding Concepts Example Basic Problem Model Sub-Problem Flag Attribute Free Fall Phenomena Given Attributes Given value of Distance and Velocity Attributes Unknown Attributes Ask for Time Attribute

4. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge4 Problem Construction ► Knowledge Base  Knowledge Map 1. Concept Selection  Knowledge Map 2. Unknown Designation  Problem Matrix 3. Proposition Construction

5. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge5 Knowledge Map ► Concept Hierarchy  Hierarchical structure of concepts  Concept Map ► Concept Schema  Attributes  Schema  Definitions, Ability (Can-Fly), Property (Age) etc.

6. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge6 Example of Knowledge Map

7. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge7 Problem Matrix ► Manipulating concepts  Example in Physics: Physics Quantity  { “ Displacement ”, “ Time ”, “ Velocity ”, “ Acceleration ” } ► Manipulating relations  Example in Physics: Physics Law  { “ Displacement = 0.5 * Acceleration * Time ^ 2 ”, “ Velocity = Acceleration * Time ” } DisplacementTimeVelocityAcceleration Displacement = 0.5 * Acceleration * Time ^ 2 1101 Velocity = Acceleration * Time 0111

8. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge8 Example of Problem Construction ► Physics Phenomenon  Motion with constant velocity ► Problem Matrix  tf: final time; si: initial position; ti: initial time; vi: initial velocity; sf: final position; d: distance  tfsitivisfd sf – si = (vf – vi) * (tf – ti) 111110 d = sf – si 010011

9. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge9 Unknown Designer ► Select Physics Law ► Set Attributes tfsitivisfd sf – si = (vf – vi) * (tf – ti) 111110 d = sf – si 010011 tfsitivisfd52Unknown81.160 sf – si = (vf – vi) * (tf – ti) 111110 d = sf – si 010011

10. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge10 Problem Constructor (cont.) Model Corresponding Concepts Example # Sub-Problems # Unknowns Basic Problem Model Sub-Problem Flag Attribute The elevator of the building proceeds motion with constant velocity. Given Attributes The velocity of the elevator is 1.1 meter/second. The initial time of the elevator is 8 th second. The final time of the elevator is 52 nd second. The final position of the elevator is 60 meter. Unknown Attributes Ask for the initial position of the elevator

11. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge11 Difficulty Analysis ► Difficulty Features According to the problem construction process 1.Selected Concepts  Knowledge Map 2.Attributes Setting  Problem Matrix ► Difficulty Dimensions According to the problem solving steps 1.Problem Identification 2.Problem Elaboration 3.Problem Planning 4.Problem Execution

12. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge12 Difficulty Features Element Difficulty Feature Denotation Number of Sub-Problems  sub_prob Selected Number of Needed Attributes  need_attr Concepts Learning Sequence  learn_seq Concept Depth  cpt_depth Number of Unknowns  unknown Attributes Number of Giving Attributes  given_attr Setting Number of Elaborating Attributes  elb_attr Mathematical Complexity  math_cpx

13. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge13 Attributes Definition ► Basic Definition  root: the root of Knowledge Map   i : ith concept ► Knowledge Map  height(  i ): the height of sub- tree from ith concept  CS(  i ): Concept Schema of ith concept  size(CS(  i )): number of attributes in concept from full Concept Schema  attr LS (CS(  i )): learning sequence attribute stored in Concept Schema ► Problem Matrix  #given_attr: number of given attributes in the problem  #unknown: number of unknown attributes in the problem  attr manip_cpt (  i ): attributes extracted to manipulating concepts in Problem Matrix  size(attr manip_cpt (  i )): number of manipulating concepts in Problem Matrix  attr max_unknown (  i ): maximum number of unknown in ith concept

14. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge14 Example ► In sky, raindrop proceeds the motion of constant acceleration. The initial velocity (vi) is 0 m/s. Acceleration (a) is 5 m/s^2. The final time (tf) is 5 s. Ask for the value of final velocity (vf). ► Selected Concept  The motion of constant acceleration ► Attributes Setting didfvivftitfa 0unknown55 vf = vi + a * (tf – ti) 0011111 df = di + vi * (tf – ti) + 0.5 *a * (tf – ti)^2 1110111 vf ^2 = vi ^ 2 + 2 * a * (df – di) 1111001

15. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge15 Attributes Definition – Knowledge Map ► size (CS (“Motion with Constant Acceleration”) ) = 11 ► size (CS (“Free Fall”) ) = 13 ► height (“Physics”) = 4 ► height (“Motion with Constant Acceleration”) = 2 ► attr LS (“Motion with Constant Acceleration”) = 2

16. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge16 Attributes Definition – Problem Matrix ► #given_attr = 3 ► #unknown = 1 ► size(attr manip_cpt (  i )) = 7 ► attr max_unknown (  i ) = 2 didfvivftitfa 0unknown55 vf = vi + a * (tf – ti) 0011111 df = di + vi * (tf – ti) + 0.5 *a * (tf – ti)^2 1110111 vf ^2 = vi ^ 2 + 2 * a * (df – di) 1111001

17. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge17 Difficulty Features Calculation (1) ►  need_attr = size(CS(  i )) / arg max j  KM size (CS(  j )) = size(CS(“Motion with Constant Acceleration”)) / size(CS(“Free Fall”)) = 11 / 13 = 0.8

18. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge18 Difficulty Features Calculation (2) ►  learn_seq = attr LS (  i ) / arg max j  sibling (  i ) attr LS (  j ) = attr LS (“Motion with Constant Acceleration”) / attr LS (“Motion with Constant Acceleration”) = 2 / 2 = 1

19. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge19 Difficulty Features Calculation (3) ►  cpt_depth = (height(root) – height (  i )) / height(root) = (height(“Physics”) – height (“Motion with Constant Acceleration”)) / height(“Physics”) = ( 4 – 2 ) / 4 = 0.5

20. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge20 Difficulty Features Calculation (4) ►  given_attr = (size(attr manip_cpt (  i )) - #given_attr + 1) / size(attr manip_cpt (  i )) = (size(attr manip_cpt (“Motion with Constant Acceleration”)) - #given_attr + 1) / size(attr manip_cpt (“Motion with Constant Acceleration”)) = (7 – 3 + 1) / 7 = 0.7 didfvivftitfa 0unknown55 vf = vi + a * (tf – ti) 0011111 df = di + vi * (tf – ti) + 0.5 *a * (tf – ti)^2 1110111 vf ^2 = vi ^ 2 + 2 * a * (df – di) 1111001

21. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge21 Difficulty Features Calculation (5) ►  unknown = #unknown / attr max_unknown (  i ) = #unknown / attr max_unknown (“Motion with Constant Acceleration”) = 1 / 2 = 0.5 didfvivftitfa 0unknown55 vf = vi + a * (tf – ti) 0011111 df = di + vi * (tf – ti) + 0.5 *a * (tf – ti)^2 1110111 vf ^2 = vi ^ 2 + 2 * a * (df – di) 1111001

22. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge22 Difficulty Features Calculation (6) ►  elb_attr = (size(attr manip_cpt (  i )) - #given_attr - #unknown + 1) / size(attr manip_cpt (  i )) = (size(attr manip_cpt (“Motion with Constant Acceleration”)) - #given_attr - #unknown + 1) / size(attr manip_cpt (“Motion with Constant Acceleration”)) = ( 7 – 3 – 1 + 1) / 7 = 0.4 didfvivftitfa 0unknown55 vf = vi + a * (tf – ti) 0011111 df = di + vi * (tf – ti) + 0.5 *a * (tf – ti)^2 1110111 vf ^2 = vi ^ 2 + 2 * a * (df – di) 1111001

23. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge23 Difficulty Dimensions ► According to four steps of problem solving  Identification Difficulty (  idf )  Elaboration Difficulty (  elb )  Planning Difficulty (  pln )  Execution Difficulty (  exc )

24. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge24 Difficulty Dimensions (cont.) ► Identification Difficulty (  idf )  Position in Knowledge Map ► Elaboration Difficulty (  elb )  Size of Concept Schema in Knowledge Map ► Planning Difficulty (  pln )  Number of Manipulating Concepts ► Execution Difficulty (  exc )  Number of Unknowns

25. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge25 Difficulty Dimension Transformation ►  idf = w 21 *  learn_seq + w 31 *  cpt_depth = 0.5 *  learn_seq + 0.5 *  cpt_depth = 0.5 * 1 + 0.5 * 0.5 = 0.75 ►  elb = w 12 *  need_attr + w 42 *  given_attr + w 62 *  elb_attr = 0.3 *  need_attr + 0.3 *  given_attr + 0.3 *  elb_attr = 0.3 * 0.8 + 0.3 * 0.7 + 0.4 * 0.4 = 0.61 ►  pln = w 53 *  unknown + w 63 *  elb_attr = 0.5 *  unknown + 0.5 *  elb_attr = 0.5 * 0.5 + 0.5 * 0.4 = 0.45 ►  exc = w 54 *  unknown = 1 *  unknown = 1 * 0.5 = 0.5

26. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge26 Demonstration

27. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge27 Brief Conclusions ► Problem Construction Steps  Concept Selection  Unknown Designation  Proposition Construction ► Problem Difficulties  Difficulty Features  Difficulty Dimensions ► Item Generating System  Physics Domain

28. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge28 Future Works ► Complex problem construction ► Mathematical complexity analysis ► Answer of the learners diagnosis

29. Thank You

30. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge30

31. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge31

32. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge32 Presentations of Knowledge ► Concept  Basic element of knowledge  A atomic unit of knowledge pieces  Ex. “ Free Fall ”, “ Constant Acceleration Motion ” etc. ► Relation  The aggregation of concepts  Ex. Is-A, Has-A etc. ► Proposition  An integration of concepts and relation  One relation with at least two concepts  Ex. “ Free Fall ” Is-A “ Constant Acceleration Motion ”

33. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge33 Inter-correlated Knowledge ► Inter-correlated Knowledge  Manipulation of inter-correlation among knowledge  Ex. Physics, Chemistry, and Mathematics ► Knowledge Object  Way to Analysis Inter-correlated Knowledge  Basic concept of a specific domain ► Relations between Knowledge Objects  Ten types in The Frame Game [Clifford, 1981] Proposition Relation Type [OBJ]<Proceed>[PHE]PW [PHE]<Has>[LAW]PW [PHE]<Influence>[QTY]INF [LAW]<Influence>[QTY]INF [OBJ]<Has>[QTY]PW [QTY] 1~n [LAW] PW [LAW] 1~n [LAW] 0 ASO [PHE] 2 [PHE] 1 SQN [PHE] 1 [PHE] 2 EQL [QTY] 1 [QTY] 2 ASO

34. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge34 Problem Structure 1: Problem Graph ► A graph composed with proposition  Relation among concepts in the problem ► Example Concept Set {"Object", "Free Fall", Displacement", "Time", "19.6", "unknown"} Propositions Object Free Fall, Object Displacement, Object Time, Displacement 19.6, Time Unknown An object proceeds Free Fall. After the phenomenon, the displacement of the object is 19.6. Ask for the procedure time of the object. An object proceeds Free Fall. After the phenomenon, the displacement of the object is 19.6. Ask for the procedure time of the object.Object Free Fall Displace- ment Time19.6 Un- known Proceed110000 Has101000 Has100100 Is001010 Is000101

35. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge35 Problem Construction ► Basic Problem Model ► Problem Construction Steps ► Essential Data Structures

36. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge36 Problem Matrix for Item Generation ► Extract manipulating concepts and attributes ► Set Given and Unknown Attributes DisplacementTimeVelocityAcceleration unknown5102 Displacement = 0.5 * Acceleration * Time ^ 2 1101 Velocity = Acceleration * Time 0111 Corresponding Concepts Example Basic Problem Model Sub-Problem Flag Attribute Free Fall Phenomena Given Attributes Given value of Time, Velocity, and Acceleration Attributes Unknown Attributes Ask for Displacement Attribute

37. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge37 Problem Template ► Problem Format  Description Sequence  Possible Sentences ModelParameter Available Proposition Physics Example No. of Sub-Problems 1 No. of Unknowns 1 Basic Problem Model Sub-Problem Flag Attribute PW-1 Object proceeds Free Fall Given Attributes EQL-1EQL-2 The Velocity of Object is 3 Unknown Attributes PWQ-1 Ask for the Acceleration of the Object

38. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge38 Proposition Table ► Possible Sentences in problems  Corresponding syntax Proposition ID Syntax Pattern Example Proposition Transform Unit Relation Type with identify no Syntactic rule Example in Physics PW-1 [Object] [Phenomena] Object proceeds Free Fall EQL-1 [Quantity] [Object] [Value] The Velocity of Object is 3

39. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge39 Item Generating System

40. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge40 Problem Constructor ► Finding suitable Problem Template ► Finding related proposition ModelParameter Available Proposition Syntax Pattern # Sub-Problems 1 # Unknowns 1 Basic Problem Model Sub-Problem Flag Attribute PW1 [Object] [Phenomena] Given Attributes EQL1 [Physics Quantity] [Value] Unknown Attributes PWQ1 [Physics Quantity] [Object] [Physics Quantity] [Object]

41. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge41 Item Generating System

42. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge42 Conception Selection

43. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge43 Problem Matrix Construction

44. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge44 Physics Law Selection

45. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge45 Physics Quantity Setting / Item Construction

46. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge46 Knowledge Categories ► Grammatical Knowledge  usually has syntax, which has rules and formats in it.  Ex. Courses of Chinese, English, Music, and Art ► Positioning Knowledge  lays stress on the location and direction among objects.  Ex. History and geography ► Inter-Correlated Knowledge  attaches more importance to the manipulation of inter- correlation among knowledge.  Ex. Physics, Chemistry, and Mathematics

47. ICALT 2003 2002/06/27To Develop an Item Generating System with Analyzing the Problem Model in the Inter-Correlated Knowledge47 Complex Problem Model Corresponding Concepts Example Basic Problem Model 1 Sub-Problem Flag Attribute Free Fall Phenomena Given Attributes Given value of Distance, Time and Velocity Attributes Unknown Attributes Sub- Problems Connection Related Basic Problem 1 Attributes Final Position, Final Velocity, Final Acceleration, Final Time Related Basic Problem 2 Attributes Initial Position, Initial Velocity, Initial Acceleration, Initial Time Relation between Attributes Equal Basic Problem Model 2 Sub-Problem Flag Attribute Motion of Constant Acceleration Phenomena Given Attributes Given value of Distance, Acceleration and Velocity Attributes Unknown Attributes Ask for Time Attribute