A Framework for Analysing Critical Thinking in Computer Conferencing EURO-CSCL, Maastricht March 22, 2001 Walter Archer

2 Outline of This Presentation The research team Background of the study The conceptual model and its elements –social presence, cognitive presence, teaching presence Methodology, and issues of practicality Linguistic structure - a proxy for semantics? Towards automation and practicality

3 The Research Team Terry Anderson (principal investigator) Randy Garrison Walter Archer Liam Rourke (Ph.D. candidate in Education) Wolf Wikeley (Ph.D. candidate in Linguistics)

4 Rationale for the Study Distance delivery increasingly important in higher education Particularly Generation 3 (CMC, including computer conferencing) Promotion of critical thinking a central goal of higher education Our research question: How well does CMC support critical thinking?

5 The Conceptual Model and its Elements From a review of the literature and our own experience as educators, we conclude that critical thinking is usually embedded within a community of inquiry composed of students and teachers (or learners and facilitators) We propose a model of an educational experience within such a community

6 From Garrison, Anderson, & Archer (2000, p.88) with permission from Elsevier Science

7 Elements of the Model “Community” implies that participants interact with each other Social presence - project their personalities Teaching presence - the actions of (a) person(s) who structures the interaction Cognitive presence - the resulting collaborative construction of meaning

8 Social Presence Term originated by Short, Williams, & Christie (1976). The Social Psychology of Telecommunications. Toronto: John Wiley and Sons. A number of scholars have developed this concept and variously defined it Some observers of CMC have questioned whether this medium is capable of supporting social presence

9 Our Definition of Social Presence “The ability of participants in the Community of Inquiry to project their personal characteristics into the community, thereby presenting themselves to the other participants as ‘real people.’” Rourke, Anderson, Garrison & Archer (1999). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education/Revue de l'enseignement à distance: 14 (2), Available online at

10 Content Analysis of Computer Conference Transcripts From the literature, we identified several broad categories of content that would indicate social presence Within each category, we identified more specific indicators of each category Employed 2 individuals to find and code (independently) occurrences of these indicators within the course transcripts

11 Social Presence Categories & Indicators

12 Interactive a Social presence density = total number of social presence indicators coded in transcript/total number of words in transcript. b Transcript A: Number of words (n = ), aggregate social presence density = c Transcript B: Number of words (n = 6 260), aggregate social presence density = Social presence density a of all indicators in transcript A b and transcript B c From Rourke, Anderson, Garrison, & Archer (1999, p. 99) with permission of the editor.

13 Teaching Presence A new term, defined as “the design, facilitation and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes.” Anderson, Rourke, Garrison & Archer (unpublished), p. 7.

14 Teaching Presence (2) Distinguishes educational experiences from experiences of incidental learning Often provided by a single individual In CMC settings, particularly in higher education, parts of this function may be provided by other individuals such as instructional designers, program coordinators, and the students in the class

15 Teaching Presence Categories & Indicators

16 Health Course Education Course f%f% Instructional Design Facilitating Discourse Direct Instruction In the Graduate Health course, number of instructor messages =139. In the Graduate Education course, number of instructor messages = 32. Frequencies of teaching presence categories in instructor messages Teaching Presence in Two Graduate Courses

17 Cognitive Presence This concept developed more fully in: Garrison, D.R., Anderson, T., & Archer, W. (in press). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education. Grounded in the literature on critical thinking, operationalized within a model of practical inquiry

18 From Garrison, Anderson, & Archer (2000, p. 99) with permission from Elsevier Science

19 Cognitive Presence Defined Cognitive presence is defined as the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry. Garrison, Anderson, & Archer (2000)

20 Cognitive Presence Categories & Indicators

21 Relative frequencies of categories of cognitive presence From Garrison et al. (in press)

22 Methodological Issues in Content Analysis of Computer Conference Transcripts Explored in detail in Rourke, Anderson, Garrison, & Archer (in press (2001)) Issues include: unit of analysis objectivity, replicability, reliability

23 Content Analysis - Problems Quantitative content analysis is "a research technique for the objective, systematic, quantitative description of the manifest content of communication" (Berelson, 1952, p. 519). Despite the potential of this technique, researchers who have used it have described it as difficult, frustrating, and time-consuming. Very few have published results derived from a second content analysis. From Rourke et al. (in press (2001)) Since many of the indicators used by our group and others researching computer conferences are latent, rather than manifest, this problem becomes even more acute.

24 Linguistic Structure - A Proxy for Semantics? Most of our work (and the work of others doing similar research) relies on semantic (meaning) analysis of content This can’t be automated, even though we are in the year 2001 (Sorry, Hal) But analysis of lexical and structural features of content CAN be automated So let’s look for structural/semantic correlations

25 The Language of CMC More like writing or more like speech? This is not a single continuum –both speech and writing encompass a wide range of styles and registers Baselines for lexical/structural features of formal and informal speech and writing have been established in Chafe and Danielewicz (1987)

26 Chafe/Danielewicz chapter Chafe, W., & Danielewicz, J. (1987). Properties of spoken and written language. In R. Horowitz & S.J. Samuels (Eds.), Comprehending oral and written language (pp ). San Diego: Academic Press.

27 What C & D Measured - Samples Type/token ratios (variety of vocabulary) Contractions Words per intonation unit (defined by them - more or less same as clause) Conjoining two elements into a compound phrase using “and” 15 other features

28 C & D’s Results - Sample Table Table 3.11 (p. 101): Conjoining (Occurrences per 100 intonation units)

29 Towards Automation and Practicality We are now attempting to automate, to some extent, the analysis of computer conference transcripts –may make content analysis a more useful tool Use some of C & D’s measures - those that lend themselves to automation –also those that may correlate with degree or category of social presence, cognitive presence, and teaching presence

30 Issues We Now Face Unit of analysis (again!) –C & D’s “intonation unit” a rather slippery concept, and difficult to automate –we may have to go back to the punctuated sentence as the unit of analysis Lexical/structural analysis may give only a very rough cut at cognitive presence, etc. –will this be good enough to be of use to educators?

31 Our First Conceptual Paper Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text- based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 1-19.

32 The Social Presence Paper Rourke, L., Anderson, T., Garrison, D.R., & Archer, W. (1999). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education/Revue de l'enseignement à distance: 14, 2. Available online at _al.html _al.html

33 The Cognitive Presence Paper Garrison, D. R., Anderson, T., & Archer, W. (in press). Critical thinking and computer conferencing: A model and tool to assess cognitive presence. American Journal of Distance Education.

34 The Teaching Presence Paper Anderson, T., Rourke, L., Garrison, D.R., & Archer, W. (submitted for publication). Assessing teaching presence in a computer conferencing context.

35 The Methodology Paper Rourke, L., Anderson, T., Garrison, D.R., & Archer, W. (2001). Methodological issues in analyzing text-based computer conferencing transcripts. International Journal of Artificial Intelligence in Education (12, to appear).

36 Our website