Workshop: Utilizing concept maps and other approaches for acquiring, eliciting, representing, and comparing structural knowledge By Roy Clariana University of Oulu, Finland EDTECH Team seminar 14 and 15 of March, 2005 (9am till 1pm) Tuesday March 15th

Mindmaps for qualitative data collection, a working seminar The goal today is to determine if mindmapping of interview data is useful (ECOL)  Identify themes by extracting and sorting concept terms  Capture meaning by extracting and representing propositions Products for today  Activity 1 – One (or possibly more) holistic map(s) that represents the themes across the 6 interviews  Activity 2 – One representational map of each interview Activity 1 Activity 2

The connection The purpose of most qualitative research is to explore the relationships between the data and ideas in the project (p.27, NVivo Getting Started Guide, 2002) Concept maps… are particularly effective for representing the organization that students see among concepts (p.64, Turns, Atman, & Adams, 2000)

Using a group drawn mindmap during an interview text The capability and experience of the person coding the text is critical… Interview 1 interviewer Qs examples See: William M. K. Trochim In future, experiment with this approach, contrast one group using yellow stickies on big pieces of paper to respond to the interview questions compared to regular (no map) interview. See if there is a difference in the data…

Using a researcher drawn mindmap after an interview Interview 1 Transcript text text text text text text text text text text text The capability and experience of the person coding the text is critical… Interview 1 coders text i.e., attribute theory note issue examples memo

The connection Concept maps can provide one strategy to deal with the methodologic challenges of qualitative research. A mindmap can be used to: frame a research project (develop the model) reduce qualitative data (20 page interview to one page map, visual identification of theme, one page maps facilitate comparison between groups and sites) analyze themes and interconnections (e.g., our cluster analysis activity), and present findings (similar to using participant comments, maps provide peremptory example). Daley, 2004 Mindmaps and qualitative research are philosophically consistent Activity 1 Activity 2

You must determine the unit of analysis for each pass through the interview Concept (one or two words, usually nouns) Phrase (several words, nouns and verbs) Proposition (noun-verb-noun combination)

The connection Concept maps can provide one strategy to deal with the methodologic challenges of qualitative research. A mindmap can be used to: frame a research project (develop the model) reduce qualitative data (4 page interview to one page map, visual identification of theme, one page maps facilitate comparison between groups and sites) analyze themes and interconnections (e.g., our cluster analysis activity), and present findings (similar to using participant comments, maps provide peremptory example). Daley, 2004 Mindmaps and qualitative research are philosophically consistent Words-- Phrase--

An example of a 20 page interview to one page map, visual identification of theme, one page maps facilitate comparison between groups and sites phrases Activity 2 – reduce qualitative data

ECOL data Activity 1 – Lets look at the semi-structured interview audio tape from 2 groups over 3 sessions The first activity is to derive themes (from terms) The second activity is to represent the interview (with phrases)

Activity 1 - Identifying broad themes text abstraction An alternative to brainstorming for generating a set of statements for concept mapping. A text abstraction method involves identifying text statements that are imbedded in some larger text and extracting them for use as separate statements in concept mapping. For instance, one could review the transcripts of a focus group discussion and identify key conclusion statements that were made. These could each be extracted and entered as a concept map statement set so that the results of the focus group discussion could be mapped and used in subsequent work (e.g., decision making or pattern matching).

Activity 1 – Cluster analysis, as 2 groups Brainstorming (corpus list) Sorting (move like terms closer) Merging & Pruning (combine like terms, delete or move unlike terms, synthesize terms) Naming Clusters (name the categories/themes) Sorting Clusters (move like clusters closer) Naming broad themes (name the cluster of clusters) and if necessary Then document (save/print) Finally, links may be added Using all 6 interviews, if possible Use yellow posit notes on large paper

thoughts Keep note of the evolution of the process for later discussion, especially what works and what did not Each team presents their map Discussion of Activity 1 process…

Activity 2a: Let’s try converting ECOL interview data directly into a map Method 1 Read a sentence, add a phrase or proposition to the map, repeat when done, play with structure and links Compare our maps Debrief Activity 2a

How to copy from interview, double click paste on CMAP InterviewCMAP Activity 2a

Activity 2b: Converting ECOL interview data into a map using propositions Method 2 Write propositions into a text file Open text file with cmap tools Play with structure and links Compare our maps Debrief Activity 2b

How to convert propositions  a map Write propositions into a text file Open text file with cmap tools Play with structure and links Compare our maps Debrief Activity 2b

How to convert propositions  a map Makes a “likes” text file Import into CMAP Tools Select Format  Auto layout  View hierarchical  View force directed Activity 2b

Summary Additional approaches – I will use your corpus terms and see what ALA- Reader can determine from the interviews Other comments?

readings Monday Turns, J., Atman, C.J., & Adams, R. (2000). Concept maps for engineering education: A cognitively motivated tool supporting varied assessment functions. IEEE Transactions on Education, 43 (2), Cicognani, A. (2000). Concept mapping as a collaborative tool for enhanced online learning. Educational Technology and Society, 3 (3), van Boxtel, C., van der Linden, J., Roelofs, E., & Erkens, G. (2002). Collaborative concept mapping: provoking and supporting meaningful discourse. Theory Into Practice, 44 (1), Chiu, C.H., Huang, C.C., & Chang, W.T. (2000). The evaluation an influence of interaction in network supported collaborative concept mapping. Computers and Education, 34, Kinchin, I.M. (2001). If concept mapping is so helpful to learning biology, why aren’t we all doing it? International Journal of Science Education, 23 (12), Tuesday Daley, B.J. (2004). Using concept maps in qualitative research. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep Daley, B.J. (2004). Using concept maps with adult students in higher education. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep Basque, J., Pudelko, B., & Leonard, M. (2004), Collaborative knowledge modeling between experts and novices: a strategy to support transfer of expertise in an organization. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep

Tuesday post workshop – Lessons learned Only able to complete Activity 1 for 1 interview (5 page transcript), allow more time or fewer activities next time Lessons learned for Activity 1 There are “unsaid” ideas that will not be captured; solution, add “memos” Underlining key terms is quick, about the same as just reading the transcript Individual’s underline, compare for inter-rater reliability We noted that the corpus terms depended on the perspective of the reader, and concluded that the corpus term list may be under represented and decided to do it again taking separate roles when underlining Have individuals read with different hats in order to capture different Themes began to emerge just simply during underlining Write “a has” in the margin when you think of it To prepare the interview, reduce names (i.e., James becomes J: ), number each exchange Note episodes while reading by drawing a line across the page On the yellow postit note, include line number and person (i.e., 12, J: ) as well as the phrase It takes a lot of time to go from the underlined text to the postit notes; solution, prepare postit notes ahead of time but don’t tell the group until they are ready to write the postit notes, then add their phrases as additional postit notes Need a really big white board to stick the postits on and to be able to move them around and write on it 5 pages of interview produced about 50 stickies We first used a matrix to sort the stickies, students down the left column and time going from left to right. We noticed that certain students tended to start discussions and take certain roles, like leading or group regulation (this specific matrix can be done ahead of time, but is fairly quick to complete); need a way to capture this matrix because the next step moves all the stickies The next sort was open-ended, moving like terms together etc. we needed to do this as a big group, though several people were at the board at the same time moving things, we all discussed the names of clusters, moved more stickies, and then reached consensus on the THEMES, the themes that emerged were goals, group dynamics, external factors, regulation, positive feelings and negative feelings Looking at these clusters, we asked “Who was the first to mention a theme” and “Who occurred the most within a theme cluster” We drew a new matrix with these themes down the first column and time going across the top, we observed a pattern of group dynamic-external factor-regulation that repeated several times. A group dynamic issue seems to demand a regulation response to resolve it. We concluded that this kind of “pre analysis work” as a team before going to NVIVO analysis may be critical in order to elicit emerging themes See Monday powerpoint at