1. Nov 9th 2006 Determinants of Engagement in an Online Community of Inquiry Jim Waters College of Information Science and Technology Drexel University Philadelphia james.waters@drexel.edu

2. Nov 9th 2006 Background Problem of maintaining student engagement. Online learning creates separation Alienation, lack of commitment and antisocial behavior ? Community of Inquiry ?

3. Nov 9th 2006 Pragmatism: Dewey and Addams Problematic situation, scientific attitude and community as participatory democracy Inquiry is controlled or directed transformation of an indeterminate situation There is a community engaged in inquiry. Inquiry is an open-ended process with positive feedback. Dewey (1916,1933)

4. Nov 9th 2006 Community of Inquiry Garrison et al 2000

5. Nov 9th 2006 Cycles of Inquiry Garrison et al 2000

6. Nov 9th 2006 Building on the Garrison et al Model Content Analysis of online Discussion Board Graduate Information Systems Students Open-ended debate Practical and Theoretical questions Derived behaviors that incorporated different elements of the Garrison model

7. Nov 9th 2006 Student Roles RoleAnalogyMain Behavior Types InitiatorSpiderSocial FacilitatorMiddlemanSocial, Teaching ContributorJourneymanSocial, Cognitive Knowledge-eliciterSeekerSocial, Cognitive Vicarious- acknowledger Me-tooSocial, Cognitive ComplicatorReframerTeaching, Cognitive CloserSynthesizerSocial, Teaching, Cognitive Passive-LearnerFreeloaderCognitive Waters and Gasson 2005

8. Nov 9th 2006 Research Questions 1.Are there noticeable patterns of interactions between participant roles? 2.Do patterns of interaction change over time? 3.Does the online learning environment support critical inquiry ? 4.What interactions generate greatest student engagement

9. Nov 9th 2006 Study  Post-Hoc analysis of online learning archive:  10 week graduate IS Management course at a US university  23 students, experienced professionals & managers.  3 - 4 open-ended questions posted to discussion board weekly:  1063 discussion-board messages  951 student responses (analyzed)  112 instructor postings (not analyzed).  Content analysis of postings and responses:  Each student contribution message assigned to single response type, reflecting dominant mode of behavior.

10. Nov 9th 2006 Raw results  25,937 individual reads of discussion board message (range 331 – 2179 reads per student)  951 student postings (range 1 – 154 per student)  Most active period weeks 1 & 2 (157 posts and 162 posts)  Then steady pattern of ~ 70-80 posts per week.

11. Nov 9th 2006 Student behavior  Contributor (61%)  Facilitator (22%)  Fluid patterns of class behavior  Students adopt different behaviors from week to week  Popularity and volume were unrelated  Possible connection between facilitation and popularity/reference to poster.

12. Nov 9th 2006 Detailed Analysis Nine typical threads analysed Three threads each for weeks 3, 6 and 9 The most productive debate produced 30 messages with a maximum thread depth of 7. The least productive produced 14 messages with a thread depth of 2. The mean number of messages on a discussion was 22 Four discussions had a thread depth of greater than 3. Pattern of responses analysed

13. Nov 9th 2006 Are there noticeable patterns of interactions between participant roles? FromToFrequencyPercentage AcknowledgerContributor31.7% EliciterContributor42.3% ComplicatorContributor42.3% FacilitatorFaculty42.3% CloserFaculty63.4% ComplicatorFaculty63.4% ComplicatorFacilitator74.0% FacilitatorContributor169.2% Facilitator 1810.3% ContributorFaculty10660.9% TOTAL174100.0%

14. Nov 9th 2006 Senders Acknowledger31.7% Eliciter42.3% Closer63.4% Complicator179.8% Facilitator3821.8% Contributor10660.9% Faculty Total174100% Receivers Acknowledger00.0% Eliciter00.0% Closer00.0% Complicator00.0% Facilitator2514.4% Contributor2715.5% Faculty12270.1% Ratio of receive to send Contributor = 27/106 = 0.25 Facilitator = 25/38 = 0.65 Complicator = 0/17 = 0.00

15. Nov 9th 2006 Do patterns of interaction change over time? Week 3 (n = 63)Week 6 (n=51) Week 9 (n = 60)

16. Nov 9th 2006 Does the online learning environment support critical inquiry ? Stahl 2006 Muukkonen et al 1999

17. Nov 9th 2006 Does the online learning environment support critical inquiry ? Few threads reached a definitive conclusion Closer synthesizes and ends debate –Closer often ignored Elements found –Information Gathering –Synthesis –Concrete experience –Reflective observation. –Critical evaluation –Deepening questions –Generating subordinate questions –Refining given knowledge –Generating hypotheses Open-ended debate ? Not problem centered ?

18. Nov 9th 2006 What interactions generate greatest student engagement Analysis of all 951 student messages Analysis of Read frequency for different message types Knowledge-elicitation messages (asking questions) generated significantly more (24) reads pre message than any other type of message. Average reads per message for all messages is 16.78 Some participants messages are read more frequently than others

19. Nov 9th 2006 Who are the most attended to posters ?

20. Nov 9th 2006 Why are some posters more engaging ? Does frequency of posting messages affect popularity? Does length of message affect frequency of reads? Does position of messages affect frequency of reads ? Does type of “participant” affect frequency of reads ?

21. Nov 9th 2006 Is frequency of posting related to popularity? Correlation between number of messages and total reads of a persons messages is 0.97, Weak -0.21 correlation between frequency of posting and reads/message. Most frequent poster posted 136 messages which attracted an average of 15.65 reads per message. The average messages per person was 37 Top three most attended to participants posted an above average number but subject 20 did not. Two of the least attended to participants posted well above average numbers of messages.

22. Nov 9th 2006 Does length of message relate to read frequency Correlation between length of post and reads for that post = 0.011 Grouping messages into very short ( 301 words) One-Way ANOVA on frequency of reads gives an f value of.373 and a significance level of.773, no apparent significant effect

23. Nov 9th 2006 Does position of message affect frequency of reads Messages posted in the first 2 days of a thread are read significantly more frequently (f=36.339, p= 0.000) than later messages. Messages posted after the third day are read by less than 50% of participants. If a message is one of the first 10 posted it is much more likely to be read than later messages (f=22.564, p = 0.000). However only two of the most attended to participants are “early” posters.

24. Nov 9th 2006 Does type of participant affect frequency of reads The most attended to participants posted more facilitation messages (39% of messages posted) The least attended to participants typically posted far fewer facilitation messages. (23% of messages posted).

25. Nov 9th 2006 Conclusions Peer Facilitation does work Students quickly identify valuable contributors Early stages crucial Changing Contributor to Facilitator Identification of thought leaders Asking questions gets responses Fluid patterns of behavior within the community Volume is not the same as quality

26. Nov 9th 2006 Future Work  Small, exploratory study  Initial framework  Open to debate Limitations  Influence of prior online learning-experience on patterns of behavior  Larger sample size  Deeper analysis of content  Explore vicarious learning contributions more fully  Explore why patterns change  Compare ill-defined vs. well-bounded questions.

27. Nov 9th 2006 Questions?