2. 1 Using concept map approaches to communicate and present knowledge University of Oulu, Finland EDTECH A41857 (1 credit) – Challenges, Problems, & Future of EdTech Wednesday March 30, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.eduRClariana@psu.edu home: www.personal.psu.edu/rbc4www.personal.psu.edu/rbc4 "First we build the tools, then they build us!" -- Marshall McLuhan

3. 2 goals Your take aways: Some experiences with collaborative concept mapping, mindmapping Some understanding of how/why it works Some examples that you could implement on Monday morning in your classroom or in you research Your Digital Portfolio for future reference and for sharing

4. 3 1 credit option Digital portfolio – Formulate as a group a digital portfolio of mindmapping, which you may utilize in the future in your studies or work. For teachers, provide specific examples for using mind mapping in your classroom For researchers, provide specific examples for using mind mapping in your research

5. 4 2 credit option? Digital Portfolio plus a White paper – a 5-10 page (double-spaced) persuasive review of some aspect of mind mapping, i.e., scripting MM in CSCL, MM as an artifact, etc. [Based on your intuition, describe how a MM can work, this is your first iteration of a “solution”. The White papers is a “soft sell” for your “solution” that describes the problem (90% of the document) and then states clearly how your solution solves the problem (10%). Avoid straw man arguments.]

6. 5 If you are interested… Manuscript for presentation – I hope that we can publish this experience, i.e., based on several projects we will do, together we formulate questions, collect and analyze data, write… (this will likely go beyond the workshop time frame and is also more open-ended) For example: How does interaction develop/evolve in online collaborative mind mapping? What scripts can support online collaborative mind mapping?

7. 6 Agenda for today Welcome and introductions all around Q&A Brief overview of concept maps Intro to Cmap tools software Brainstorm activity (group roles) Set up Project 1 (see handout) Set up Project 2 (see handout) Does anyone have any student essays that we can use in Project 3 on Monday? Click here for projects handout

8. 7 some terminology Concept map – diagrams indicating interrelationships among concepts and representing conceptual frameworks within a specific domain of knowledge (vanBoxtel) Concept map – a visual set of nodes and arcs (a network representation) that embodies the relationships among the set of concepts. Also called knowledge maps, mindmaps, semantic maps (Turns, et al.). Nodes – terms/complexes/concepts (usually nouns, things, examples, ideas, categories, people, locations…) Links (arcs) – lines connecting nodes, usually labeled with a relationship term (usually verbs) Propositions – node-link-node combinations, also called “soup” (ketti) by IHMC Turns, Atman, & Adams, 2000 Vygotsky contrast Some foundation stuff

9. 8 Mindmaps vs. concept maps Bahr (2004) using concept maps to teach English to German students

10. 9 Mindmap of “group” knowledge (Anni, Anna, Paula, Esa, ja Herkko), source is the second floor hallway muista huumori ! konkretisoi ! opettajan oma tarina elävöittää kytke oppilaan arkeen ! liikuta oppilas ylös penkistä kikkoja istumajärjestys ! huiputa ! perustele ! haasta, kysele ! yllätä ! oppettajan vaikutus- mahdollisuudet pelkkä kalvoshow sama työtapa liian pitkään vältä ! työtavat vaihto kyllin usein demot, konkreettiset esimerkit tekeminen vs. pelkkä kuunteleminen nopeat oppilaiden erot lisätehtäviä hitaat tukiopetus. apu huomaa erot luokkakohtaiset erot ikäluokka vaikuttaa näennäinen keskittyminen ? hiljainen luokka vilkas luokka erityisen paljon kikkoja ei palautetta opettajalle

11. 10 Mindmaps vs. concept maps My question is, do concept maps or do mindmaps fit better with the Finnish language?

12. 11 Tools to support mapping Yellow stickies!! Pencil and paper may be best for your classroom Software – PowerPoint is pretty good Inspiration is good but expensive CMAP tool is free, but your tech person will have to agree to support it At least 22 other tools are available, some free some not

13. 12 Other concept map automatic scoring approaches CMap tools (IHMC) that we will use today C-TOOLS – Luckie (PI), University of Michigan NSF grant available: http://ctools.msu.edu/ctools/index.htmlhttp://ctools.msu.edu/ctools/index.html TPL-KATS – University of Central Florida (e.g., Hoeft, Jentsch, Harper, Evans, Bowers, & Salas, 1990). TPL- KATS: concept map: a computerized knowledge assessment tool. Computers in Human Behavior, 19 (6), 653-657. SEMNET – http://www.semanticresearch.com/about/http://www.semanticresearch.com/about/ CMAT – Arneson & Lagowski, University of Texas, http://chemed.cm.utexas.edu http://chemed.cm.utexas.edu Plus 22 other non-scoring map tools, Inspiration, Kidspiration

14. 13 Some previous uses of mapping Usually involve individuals working alone, and involve text in some way Some collaborative strategies have been used Lets look at a few…

15. 14 Using a student mindmap to “capture” a text (note taking) Textbook Text text text text text text text text text text text text Mindmap notes student text memo Examples?

16. 15 Using a student mindmap to “capture” research on a topic text Text text text text tex Text text text text textt text Mindmap notes student text memo text Text text text text tex Text text text text textt www video Examples? video

17. 16 Then using the mindmap to write an essay essay Text text text text text text text text text text text text Mindmap notes student text memo Examples?

18. 17 Using a researcher drawn mindmap to “capture” an interview transcript Interview 1 Text text text text text text text text text text text text The capability and experience of the person coding the text is critical… Interview 1 coder text memo attribute theory note issue

19. 18 Using a group drawn mindmap to “capture” an interview text The capability and experience of the person coding the text is critical… Interview 1 interviewer Qs

20. 19 Example of dyad collaboration (not online) Yergin Mindmap artefact Verbal discussion (taped) Analyze the discussion Blah blah blah blah Blah blah Hannah Blah blah blah blah Blah blah Observations: On task Abstract talk 3-propositions/min Question Answer Criticize Conflict Elaboration Co-construction van Boxtel, van der Linden, Roelofs, & Erkens (2002) Problem: Sometimes unscientific notions are ingrained Inferred: Active use of prior knowledge Acknowledged problems Look for meaningful relations Negotiation Shared objects play an important role in negotiation and co-construction The incredible value of talk! Note the attentional effects of the artifact

21. 20 Chiu et al. example of an online collaboration p.22, Chiu, Huang, & Chang (2000) Hannah (lead) Jari Yergin H: WE should … J: Did you see… Y: Yeah, but … Etc. Mindmap artefact Online chat Analyzed the chat text And the mindmap creates Mindmap session lasted 80 minutes. 3 x 12 online groups, communicate by chat, 745 messages were exchanged (avg. of 62 per group). Only the lead could alter the mindmap The ‘other 2 members used chat to “advise” Researchers

22. 21 Project 1 and 2 We will experiment with two online collaboration approaches Project 1 is a synchronous concept map collaboration using Cmap tools software Project 2 is an asynchronous concept map collaboration using PowerPoint software and email But next, we will try brainstorming with Cmap tools to become familiar with the tools and process before setting up Project 1 Click here for projects handout

23. 22 First Mind map CSCL roles… Starter: You work as a discussion moderator. Your assignment is to engage your group members to the discussion by asking questions and commenting. And if the wrapper makes small summaries during discussion you can utilize his or her work to raise new questions. Active participation in the discussions is essential. Wrapper: Your assignment is to sum up the discussion. If you think it is easier you can summarize frequently and weave ideas together. For example, if five participants of your group are having a discussion about collaborative and co-operative learning you can summarize their main points during the discussion. An alternative way is to sum up the discussions in the end of article-videoclip task (and the last course assignment). Please overview your group's discussions and make a brief summary of the main topics. Active participation in the discussions is essential. Group member: Your assignment is to participate actively into discussions by asking questions making comments and stating arguments. You are expected to be a critical inquirer. Evaluator (an optional role): You are required to evaluate your group's work during the course. Please focus on the group interaction and group dynamics, for example how the starters, wrappers and group members performed during the discussions and last course assignment. The tutors inform you when to perform evaluations. Notice that you are also a deputy starter and a deputy wrapper if the originally named persons are not available. If you are called to work as a starter or wrapper please see the instructions given above. The role of evaluators are used only if you have not had a role of starter or wrapper during this course. Mindmap activity…

24. 23 Cluster analysis Brainstorming (corpus list) Sorting (move like terms closer) Merging & Pruning (combine like terms, delete or move unlike terms, synthesize terms) enter Naming Clusters (name the categories/themes) Sorting Clusters (move like clusters closer) Naming broad themes (name the cluster of clusters) and if necessary E-document (to save/print) Build consensus!

25. 24 Brainstorm, then make the map Open IHMC Cmap tools Fill in personal information on first use (I’ll tell you what to type in here) Click Other Places Open brainstorm file Click collaborate icon if necessary Type in your first name Collaborate

26. 25 Now go back and add Small Group Roles Group Task Roles Initiator-contributor. Proposes new ideas or approaches to group problem solving; may suggest a different approach to procedure or organizing the problem-solving task Information seeker. Asks for clarification of suggestions; also asks for facts or other information that may help the group deal with the issues at hand Opinion seeker. Asks for clarification of the values and opinions expressed by other group members Information giver. Provides facts, examples, statistics, and other evidence that pertains to the problem the group is attempting to solve Opinion giver. Offers beliefs or opinions about the ideas under discussion Elaborator. Provides examples based on his or her experience or the experience of others that help to show how an idea or suggestion would work if the group accepted a particular course of action Coordinator. Tries to clarify and note relationships among the ideas and suggestions that have been provided by others Etc.. Mindmap activity…

27. 26 Project 1 – Cmap tools synchronous collaboration (see the Project handout) Set day and time to join online …….

28. 27 Project 1 IHMC Public Cmaps conv v2 on Jan 22 2004

29. 28 Oulu EDTECH Public Project 1

30. 29 Project 2 – Overview of “Pass the soup” Email to PowerPoint file (see the Project handout)

31. 30 Project 2 – “Pass the soup” PowerPoint file Slide 1 – mindmap is developed bit-by-bit here by the group by adding only 3 to 5 elements and then emailing it to the next person on the list Slide 2 – numbered list of names of group members with email address, other instructions Slide 3, 4, etc. – comments about changes that you want to make, suggestions, etc.

32. 31 How to use ALA-Reader Monday, April 4, 2005

33. 32 Agenda for today Debrief “pass the soup” activity, and come up with a better Finnish name for it Q&A Brief overview of my concept map assessment research ALA-Reader demo (English language essays) Set up Project 3 for Finnish (see handout) How can we find Finnish essays for use in Project 3?

34. 33 Final map for Project 2: Team 1 Click her to See progression Of this map

35. 34 Final map for Project 2: Team 2 Click her to See progression of this map

36. 35 Debriefing What happened? What worked? What did not work? What would you do differently next time? If you like, write this up as a team for your final paper.

37. 36 My research interests Mind map assessment – automatic scoring software tool called ALA-Mapper http://www.personal.psu.edu/rbc4/ala.htm http://www.personal.psu.edu/rbc4/ala.htm Essay assessment – automatic scoring software tool called ALA-Reader http://www.personal.psu.edu/rbc4/score.htm http://www.personal.psu.edu/rbc4/score.htm for Latent Semantic Analysis (LSA) see: http://www.personal.psu.edu/rbc4/frame.htm http://www.personal.psu.edu/rbc4/frame.htm prototypes

38. 37 Novak Novak says “Concept maps were first developed in our research program in 1972 as a way to represent changes in children’s understanding of science concepts over the 12-year span of schooling. We were using modified Piagetian clinical interviews to assess changes in their knowledge over time, but we found the interview transcripts were too difficult to analyze for changes in specific aspects of the children’s knowledge. Instead we prepared concept maps from the interviews.” From: http://wwwcsi.unian.it/educa/mappeconc/jdn_an2.htmlhttp://wwwcsi.unian.it/educa/mappeconc/jdn_an2.html

39. 38 First uses… to represent knowledge in a visual format The primary parts of the system are the heart, blood cells, and vessels. The human heart, a pump, is made of cardiac muscle Cardiac muscles have a unique feature of forming connections between two adjacent cardiac cells. This allows the muscle cells to contract powerfully and quickly involuntarily The brain is unable to increase or decrease the heart's beating The heart is comprised of four chambers; two upper chambers called atriums, and two lower chambers called ventricles The blood flows through the right side to the lungs where it picks up oxygen. The blood then returns to the right. Next, it flows into the left where it I xxxx tissue within the body by approximately 9 pints of blood through 100,000 miles of vessels The primary parts of the system are the heart, blood cells, and vessels. The human heart, a pump, is made of cardiac muscle Cardiac muscles have a unique feature of forming connections between two adjacent cardiac cells. This allows the muscle cells to contract powerfully and quickly involuntarily The brain is unable to increase or decrease the heart's beating The heart is comprised of four chambers; two upper chambers called atriums, and two lower chambers called ventricles The blood flows through the right side to the lungs where it picks up oxygen. The blood then returns to the right. Next, it flows into the left where it I xxxx The human circulatory system is a transportation system. Nutrients and oxygen are carried to living tissue within the body by approximately 9 pints of blood through 100,000 miles of vessels The primary parts of the system are the heart, blood cells, and vessels. The human heart, a pump, is made of cardiac muscle Cardiac muscles have a unique feature of forming connections between two adjacent cardiac cells. This allows the muscle cells to contract powerfully and quickly involuntarily The brain is unable to increase or decrease the heart's beating The heart is comprised of four chambers; two upper chambers called atriums, and two lower chambers called ventricles The blood flows through the right side to the lungs where it picks up oxygen. The blood then returns to the right. Next, it flows into the left where it I xxxx Novak interview data Was science content knowledge Mind Map

40. 39 Finnish research with concept maps… Mainly for knowledge representation for instructional use but also for representing the structure of a curriculum and for group communication Pasi Eronen, Jussi Nuutinenn and Erkki Sutinen, (http://www.cs.joensuu.fi/pages/avt/concept.htm), Joensuu (computer science)http://www.cs.joensuu.fi/pages/avt/concept.htm Mauri Ählberg, Helsinki (education) and Erkki Rautama (computer science) University of Art and Design, Helsinki (http://www2.uiah.fi/~araike/papers/articles/CinemaSense_Collaborativ e_Cinemastudies_DeafWay2002.htm) (see also: Future Learning Environment 3)http://www2.uiah.fi/~araike/papers/articles/CinemaSense_Collaborativ e_Cinemastudies_DeafWay2002.htm Text graphs (Helsinki): http://www.cs.hut.fi/Research/TextGraph/http://www.cs.hut.fi/Research/TextGraph/ Kari Lehtonen, Helsinki Polytechnic, concept maps as a portfolio component (http://cs.stadia.fi/~lehtonen/DPF/dpf-berlin-02- muotoiltu.doc)http://cs.stadia.fi/~lehtonen/DPF/dpf-berlin-02- muotoiltu.doc Also School astronomy and Vocational Training and Education 4th IEEE International Conference on Advanced Learning Technologies Joensuu, Finland, August 30 - September 1, 2004 4th IEEE International Conference on Advanced Learning Technologies

41. 40 Concept map for assessment: score validity??? Concept maps contains propositions These propositions scores are generally considered to be valid and reliable measures of science content knowledge organization (Ruiz-Primo, Schultz, Li, Shavelson, CREST in California...). essays interviews tests observations

42. 41 e.g.,… Rye and Rubba (2002) reported that traditional concept map scores were related to California Achievement total test scores (r = 0.73). (Note that Crocker and Algina say that validation coefficients rarely exceed r=0.50.) Concept maps (cognitive maps, concept maps) may be an appropriate approach for assessing structural knowledge (Jonassen, Beissner, & Yacci, 1993). For example, concept maps have been used to visualize the change from novice to expert.

43. 42 Scoring Concept Maps Traditionally, concept maps are scored by teachers or trained raters using scoring rubrics (e.g., Lomask’s rubric) Although this marking approach is time consuming and fairly subjective, map scores usually correlate well with more traditional measures of science content knowledge (multiple choice, fill-in-the blank, and essays) Complex scoring rubrics decrease the concept map score reliability (so keep scoring simple)

44. 43 Scoring Concept Maps C3 describes our automatic system for scoring concept maps: collect –>convert –> compare 1. Collect raw map data 2. Convert raw data into a mathematical network representation 3. Compare the mathematical network representation of two maps (e.g., student to teacher, student to expert, student to student)

45. 44 1. Collect raw data What raw data can a computer “extract” from a concept map? Term counts – in open-ended maps, count required terms included Propositions – a link connecting two terms and a link label Associations – geometric distance between pairs of terms. Small values indicate stronger relationship.

46. 45 Link and distance data Most approaches use only link label information, usually called “propositions”. (n 2 -n)/2 pair-wise comparisons

47. 46 Link and distance Link data (propositions) – are the common way to compare/assess concept maps Distance data – not common, based on James Deese’s (1965) ideas on the structure of association in language and thought, card- sorting task approaches (Vygotsky in Luria, 1979, Miller, 1969), Kintsch and Landauer’s ideas on representing text structure, and neural network methods (Elman, e.g., 1995)

48. 47 Using our Finnish Mind Map example Borrowed from Anni, Anna, Paula, Esa, ja Herkko Found in the hallway on the second floor See next slide

49. 48 muista huumori ! konkretisoi ! opettajan oma tarina elävöittää kytke oppilaan arkeen ! liikuta oppilas ylös penkistä kikkoja istumajärjestys ! huiputa ! perustele ! haasta, kysele ! yllätä ! oppettajan vaikutus- mahdollisuudet pelkkä kalvoshow sama työtapa liian pitkään vältä ! työtavat vaihto kyllin usein demot, konkreettiset esimerkit tekeminen vs. pelkkä kuunteleminen nopeat oppilaiden erot lisätehtäviä hitaat tukiopetus. apu huomaa erot luokkakohtaiset erot ikäluokka vaikuttaa näennäinen keskittyminen ? hiljainen luokka vilkas luokka erityisen paljon kikkoja ei palautetta opettajalle

50. 49 Collect Mind Map raw data 9 main terms selected here (ALA-Mapper max=30)

51. 50 Selecting terms Selecting important terms (and their synonyms) is a critical step (for example, singular value decomposition in LSA derives terms). We use an expert(s) to determine terms. Goldsmith, Johnson, and Acton (1991)

52. 51 predictive validity of PFNets directly relates to the number of terms used 0,00 0,10 0,20 0,30 0,40 0,50 0,60 0,70 0,80 0102030 predictive validity Number of terms Goldsmith, Johnson, and Acton (1991) So, perhaps the predictive validity of Concept Maps (and essays) directly relates to the number of terms used

53. 52 2. Convert raw data into scores Currently, we use a data reduction and comparison approach called Pathfinder network representation (PFNet, Schanveldt, 1990). Our future research will consider additional approaches, such as MDS and data-mining. http://interlinkinc.net/Pathfinder.htmlhttp://interlinkinc.net/Pathfinder.html PFNets describe the least weighted path to connect the terms Scores are established by comparing the participant’s PFNet to a referent (expert) PFNet, and calculating the number of common links (the intersection) Visual example 

54. 53 Finnish example: PFNet for distance data PFNet for distance data

55. 54 Compare student to expert referent Expert Referent PFNetStudent PFNet O O 6 of 8 common links

56. 55 Poindexter and Clariana Participants – 23 undergraduate students in intro EdPsyc course (Penn State Erie) Food rewards for participation Setup – complete a demographic survey and how to make a concept map lesson Text based lesson interventions – instructional text on the “heart” with either proposition specific or relational lesson approach Poindexter, M. T., & Clariana, R. B. (in press). The influence of relational and proposition-specific processing on structural knowledge and traditional learning outcomes. International Journal of Instructional Media, 33 (2), in press. link to doc file #1st

57. 56 Treatments Relational condition, participants were required to “unscramble” sentences (following Einstein, McDaniel, Bowers, & Stevens, 1984) in one paragraph in each of the five sections or about 20% of the total text content Proposition-specific condition (following Hamilton, 1985), participants answered three or four adjunct constructed response questions (taken nearly verbatim from the text) provided at the end of each of the five sections, for a total of 17 questions covering about 20% of the total text content (no feedback was provided).

58. 57 Posttests Concept map (use 26 terms provided) Link-based common scores Distance-based common scores Multiple-choice tests (Dwyer, 1976) Identification (20) Terminology (20) Comprehension (20)

59. 58 Means and sd Map-linkMap-dist

60. 59 Analysis MANOVA (relational, proposition-specific, and control) and five dependent variables including ID, TERM, COMP, Map-prop, and Map-assoc. COMP was significance, F = 5.25, MSe = 17.836, p = 0.015, none of the other dependent variables were significance. Follow-up Scheffé tests revealed that the proposition-specific group’s COMP mean was significantly greater than the control group’s COMP mean (see previous Table).

61. 60 Correlations All sig. at p<.05 Compare to Taricani & Clariana next  Map-link Map-distance

62. 61 Taricani and Clariana – Replication of Poindexter and Clariana Taricani, E. M. & Clariana, R. B. (in press). A technique for automatically scoring open-ended concept maps. Educational Technology Research and Development, 53 (4), in press. TermComp Link data0.78 0.54 Distance data 0.48 0.61

63. 62 Compare these two... Poindexter & ClarianaTermComp Link data 0.77 0.53 Distance data 0.69 0.71 Taricani & ClarianaTermComp Link data0.78 0.54 Distance data 0.48 0.61

64. 63 Clariana, Koul, & Salehi Participants – A group of 24 practicing teachers enrolled in CI 400 Lesson intervention – while researching online, completed concept maps in pairs (newsprint & yellow stickies) to describe the structure and function of the heart and then individually wrote essays on this topic from their maps. Clariana, R. B., Koul, R., & Salehi, R. (in press). The criterion related validity of a computer-based approach for scoring concept maps. International Journal of Instructional Media, 33 (3), in press. # 2nd

65. 64 Posttests Essays Multiple-raters using holistic rubric Computer-derived LSA Essay scores (http://www.personal.psu.edu/rbc4/frame.htm)http://www.personal.psu.edu/rbc4/frame.htm Concept Maps Multiple-raters using Lomask’s rubric ALA-Mapper PFNet link and distance agreement with an expert

66. 65 Correlation matrix MapEssayLSA Link Map1 Essay 0.491 LSA 0.310.731 Link data 0.360.760.831 Distance data 0.600.770.710.82 1 p <.05 shown in boldface type. HumanComputer Many investigators have noted the close relationship between maps and essays.

67. 66 Overview: Tools to score Essays ETS – PEG (Project Essay Grade), e-rater, Criterion and other products… http://www.ets.org/research/erater.html http://www.ets.org/research/erater.html Walter Kintsch (and Landau) at CU-Boulder – Latent semantic analysis, many uses, i.e., score online training for the Army - http://lsa.colorado.edu/ http://lsa.colorado.edu/ Vantage Learning essay scoring products - http://www.vantagelearning.com/ http://www.vantagelearning.com/ ALA-Reader: http://www.personal.psu.edu/rbc4/score.htmhttp://www.personal.psu.edu/rbc4/score.htm

68. 67 ALA-Reader … an electrical signal starts the heartbeat, by causing the atrium to contract. The blood then flows through the pulmonary valve into the pulmonary artery and then into the lungs. Once inside the lungs, the blood gives up the carbon dioxide (cleansed) and receives oxygen. This oxygenated blood … atrium contract lungs cleansedoxygenated P artery P valve TextPFNet Link array

69. 68 Clariana & Koul Participants – Again, a group of 24 practicing teachers enrolled in CI 400 Lesson – while researching the topic “the structure and function of the heart” online, students completed concept maps using Inspiration software and later wrote an essay on this topic from their maps. Clariana, R.B., & Koul, R. (2004). A computer-based approach for translating text into concept map-like representations. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep 14-17, pp.131-134. http://cmc.ihmc.us/papers/cmc2004-045.pdfhttp://cmc.ihmc.us/papers/cmc2004-045.pdf # 3rd

70. 69 Posttests Essays Multiple-raters using holistic rubric Computer-derived LSA Essay scores (http://www.personal.psu.edu/rbc4/frame.htm)http://www.personal.psu.edu/rbc4/frame.htm Concept Maps Multiple-raters using Lomask’s rubric ALA-Mapper PFNet link and distance agreement with an expert ALA-Reader PFNet link scores (from 1 to 5) (so far, only looked at essay scores) 

71. 70 ALA-Rater PFNet scores The scores for each text and rater-pair are shown ordered from best to worst. ALA-Reader scores were moderately related to the combined text score, Pearson r = 0.69, and ranked 5th overall.

72. 71 Comments and Questions ??

73. 72 Demo ALA-Reader Download ALA-Reader.exe Create terms file (can include 2 synonyms) Create 2 expert baseline reference texts called expert1.txt and expert2.txt (i.e., Instructor, best student) Use it (type in the students essay file name)students essay file Files created Summary file called report.txtreport.txt Multiple *.prx files (PRX folder)PRX folder Available at: www.personal.psu.edu/rbc4www.personal.psu.edu/rbc4

74. 73 Other methods for eliciting and representing knowledge structure Monday, April 11, 2005

75. 74 agenda Today is a hands-on demonstration day Brief overview of the ideas SPSS for representing Pathfinder KU-Mapper My intent, you will know enough to begin to use these approaches

76. 75 Eliciting structural knowledge Every method for eliciting knowledge should be viewed as “sampling” Caution, never forget the likely effects of contiguity (time, space, etc.) dominating over semantics (meaning) essays interviews tests observations

77. 76 Dave’s ideas Knowledge representation Knowledge comparison Knowledge elicitation Jonassen, Beissner, & Yacci (1993), page 22

78. 77 Dave’s ideas graph building similarity ratings semantic proximity word associations card sort ordered recall free recall additive trees hierarchical clustering ordered trees minimum spanning trees link weighted Pathfinder nets Networks Dimensional principal components MDS – multidimensional scaling cluster analysis expert/ novice qualitative graph comparisons quantitative graph comparisons relatedness coefficients scaling solutions C of PFNets Trees Knowledge representation Knowledge comparison Knowledge elicitation Jonassen, Beissner, & Yacci (1993), page 22

79. 78 Eliciting structural knowledge Vygotsky (in Luria, 1979); Miller (1969) card- sorting approaches Deese’s (1965) ideas on the structure of association in language and thought Kintsch and Landauer’s ideas on representing text structure, and latent semantic analysis Recent neural network representations (e.g., Elman, 1995)

80. 79 Analyzing Deese free association data with MDS Hands-on with MDS in SPSS A good description of MDS: http://www.statsoft.com/textbook/stmulsca.html http://www.statsoft.com/textbook/stmulsca.html (Aside: a good description of Factor analysis: http://www.statsoft.com/textbook/stfacan.html ) http://www.statsoft.com/textbook/stfacan.html Hands-on with Pathfinder KNOT

81. 80 Deese, free recall data (p.56) Deese, J. (1965). The structure of associations in language and thought. Baltimore, MD: John Hopkins Press, page 56 Full array (n * n): 19 x 19 = 361 Half array ((n 2 – n)/2): ((19 x 19) –19 )/2 = 171 100 participants are shown a list of related words, one at a time, and asked to free recall a related term

82. 81 Deese, free recall data (p.56) Deese, J. (1965). The structure of associations in language and thought. Baltimore, MD: John Hopkins Press, page 56 Full array (n * n): 19 x 19 = 361 Half array ((n 2 – n)/2): ((19 x 19) –19 )/2 = 171

83. 82 Using MDS in SPSS Start SPSS and open the deese.sav file Under Analyze, select Scale, then select Multidimensional Scaling (ALSCAL)… Move Variable from left to right Create distances from data Model Options Next page

84. 83 Select all of these

85. 84 Multi-dimensional scaling (MDS) of Deese data

86. 85 Both are “correct”. Side issue, the MDS obtains alternate (e.g., enantiomorphic) visual representations Oulu Pori Tampere Helsinki Jyväsklyä Oulu Pori Tampere Helsinki Jyväsklyä Is this map correct? geographic data, for example, may be oriented in different ways

87. 86 How good is the representation? many dimensions (as many as 19) reduced to 2 dimensions Check the “stress” value to estimate how strained the results are An algorithmic, power, approach rather than based on a model so no assumptions about data structure are required…

88. 87 Side trip Wordnet: http://wordnet.princeton.edu/http://wordnet.princeton.edu/ http://wordnet.princeton.edu/cgi-bin/webwn What is the Visual Thesaurus? – The Visual Thesaurus offers stunning visual displays of the English language. Looking up a word creates an interactive visual map with your word in the center of the display, connected to related words and meanings. What is the Visual Thesaurus? Type “bird” in at: http://www.visualthesaurus.com/trialover.jsp http://www.visualthesaurus.com/trialover.jsp

89. 88 Pathfinder Network (PFNet) analysis Pathfinder is a mathematical approach for representing and comparing networks, see: http://interlinkinc.net/index.html http://interlinkinc.net/index.html Pathfinder data reduction is based on the least weighted path between nodes (terms), so for example, Deese’s 171 data points become 18 data points. Only the salient or important data is retained. Pathfinder PFNet uses, for example: Library reference analysis Measuring Team Knowledge (Nancy J. Cooke) next slide  Use google to see many more

90. 89 Pathfinder for cognitive task analysis Shope, DeJoode, Cooke, and Pedersen (2004)

91. 90 PFNet of same data Now let’s try Pathfinder analysis of the same Deese data set… Find the pfnet folder Double-click to run PCKNOT.bat (notice the bat extension, see next slide below) We will do it together

92. 91 Select the right PCKNOT file

93. 92 PFNet of Deese data summer spring sunshine yellow color blue sky flower garden nature butterfly cocoonmoth wing bees bird fly bug insect

94. 93 MDS and PFNet of Deese data Pathfinder KNOT PFNet SPSS MDS

95. 94 MDS and PFNet data reduction MDS uses all of the data points to reduce the dimensions in the representation, and so may be improperly driven by noise in the data or by unimportant data points Pathfinder uses only the most important data

96. 95 Transition to your real life example Finally, you will collect *real* data (using my KU-Mapper software) and analyze it with Pathfinder KNOT

97. 96 KU-Mapper Your data, determine 15 important terms in your research area (Finnish and English), create a “terms.txt” file with the 15 terms Run KU-Mapper (do all 3 tasks: pair- wise, list-wise, and card sort) Use KNOT to analyze and compare all three prx files Download KU-Mapper from: http://www.personal.psu.edu/rbc4/KUmapper.htmhttp://www.personal.psu.edu/rbc4/KUmapper.htm

98. 97 Debrief your data activity What happened? What worked? What did not work? What would you do differently next time? If you like as your final paper, describe how you might use this approach.

99. 98 Final thoughts… I enjoyed working with you If you want a credit, Email to let me know this Then be sure to send me you paper via email as soon as possible

100. 99 Stop here

101. 100 Possible research question on optimal scripts: Under- vs. over-scripting CSCL Amount of scripting  Amount of collaboration linearS-curve with crash? J-curve Amount of scripting  Some possibilities

102. 101 generative learning strategies Generative learning (Jonassen, 1988) recall - repetition, rehearsal, review, mnemonics integration - learner paraphrases, generates questions, generates examples organization - learner analyzes key ideas by creating headings, underlining keywords, outlining, categorizing (i.e., invent table categories, populate a table with existing ideas) elaboration - generate mental images, create physical diagrams, sentence elaboration (i.e., invent stuff to fill cells in a table) + +

103. 102 I just "think" systemically and "n- dimensionally" on paper, with imagery… My essential skill is simply--If you can explain it to me, I can draw a picture of it. It doesn't matter if it's something totally new to me, if you can make a coherent explanation, and let me understand it. I can "visualize it" and make a picture that shows you what you said. This is why I work in aerospace. I'm able to sit down with SME's (Subject Matter Experts-in any discipline), let them do a "data-dump" and put a sketch in their hand at the end of the conversation that "say's it all". This skill is vital to helping disparate technical types talk to each other (communication across cultural barrier of the "dialect" of the various technical disciplines). It also provides a way for ideas to get from that rough-semi coherent stage and into a practical and "do-able" condition. For example, One day I found myself working a Kelly Temp job for a bunch of Boeing System Analysts doing a JAD (joint application development) project to design a computing architecture for a new tooling system for the 777. The first drawing came by accident, started a huge argument, and eventually (2 weeks later) resolved in a group wide "a-hah"... that put everyone on the same wavelength- allowing the new system to be built a lot more "right" than usual, quicker than usual. From: http://visual.wiki.taoriver.net/moin.cgi/MichaelEricksonhttp://visual.wiki.taoriver.net/moin.cgi/MichaelErickson