1. Bridging Research and Practice in Distance Education A communication management approach to a state-of-the-art Online Learning Environment. Erin Shaw CARTE/DEN Collaboration Viterbi School of Engineering

2. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Goals and Progress DEN OLE goals  Feature innovative teaching and learning facilities  Increase learner-centered participation and collaboration  Encourage instructor-initiated experimentation and assessment  Bridge research and practice in distance education CARTE progress toward goal  Deploy ISI discussion board  Automate registration via single sign on from Blackboard  Render development at ISI independent of production at DEN  Integrate discussion and artifacts (course doc, webcasts)  Allow students to attach and share documents  Focus discussion and assessment  Expand communication capabilities  Support IM (mobile), RSS feed, summarization and activity watches  Extend document capture capabilities  Generate text, html, and pdf file formats and metadata  Parse, segment, tile, and make searchable

3. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Integrated System Overview ISI Discussion Board Agent Response ISI Document Database DEN Blackboard System Student Collaboration Tools DEN File System Instructor Assessment Tools Agent Analysis Completed In Development Planned

4. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering DEN Blackboard Link to ISI Discussion Board Native Blackboard discussion board link replaced with link to ISI board

5. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering ISI Discussion Board Blackboard sends user profile and discussion board automatically registers user

6. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Current State Current state  In use since Spring 2005  Two DEN courses currently online (CSci577, CSci402)  Four archived courses available (AME599, 3xCSci402)  Extensive processing of 402 course materials completed Current focus  Utilize discussion board for research  Data benefits learning, mining, ped. agent and NLP research  Spawned AIED paper, HTL demo, and AERA, IUI submissions  Experimental work (UCLA intern and CSci577 class)  Proposal planning Now is a critical time!  Funding has ended (funds slated to run out ~12/2005)  Minimal funding would assure continued support for  current course use  research evaluation

7. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Project Rationale Addresses instructional challenges  Instructors want to experiment but have limited support  E.g. Can maintain own web site but not discussion board  registration, security, and maintenance are obstacles  Students expect to use current communication technologies  Evaluation of ‘Learner centered’ model difficult but necessary Addresses technical challenges  Building own OLE, duplicating DEN system impractical  Blackboard’s Building Blocks solution incomplete  API incomplete, integration too tight  ISI development within DEN production environment problematic Our solution  Encourages adoption (maximizes ease-of-use)  Enables development at ISI (minimizes disruptions)  Uses open source development model (maximizes code maintenance)  Allows for evolution from Blackboard  18 open source CMSs currently available (EduTools, August, 2005)

8. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Discussion Board Deployment Software  phpBB, open source, latest version, running at ISI http://www.phpbb.com, http://melodic.isi.edu/phpbb2http://www.phpbb.com  ISI modifications adhere to community development standard modularized and self-documenting we maintain changes, phpBB dev community maintains core Courses online (2005)  Spring: AME599, CSCI402  Summer: CSCI402  Fall: CSCI555, CSCI402  Archived 2004 discussion (CSCI402 and CSCI555) Usage statistics  ~130 past students, ~130 current students  Over 2000 messages

9. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Summary of Accomplishments Main accomplishments  Single sign on from Blackboard (fully integrated)  Document capture (Blackboard-integrated user interface)  Successfully used in five courses Discussion board details  Auto-creation of lecture discussions  Integration of webcasts and lecture notes to follow  Annotation system  Instant messaging/mobile learning integration  Answer mining of archived discussions  Archiving features Document capture details  Automate parsing of uploaded document  Creates PDF, HTML, text and metadata files for all documents  Makes documents searchable via discussion board

10. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Summary of New Research Research thrusts  SPADE rhetorical analysis (Shaw)  Classummary evaluation (Liang Zhou, GRA)  Q/A processing pipeline (Donghui Feng, GRA)  800 messages manually annotated for network analysis  Ontology creation (in planning with Kim) Data share tools  API for message export  XML schema for message formatting  API for agent message insertion Evaluation tools  Statistics viewer  Questionnaire creation tool (auto-send to users)  Instrumentation for Q/A evaluation (planned)

11. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Screen Shots

12. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Artifact-Based Discussion Documents and images displayed next to discussion Supports all file formats, including PDF Documents and images displayed next to discussion Supports all file formats, including PDF

13. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Topic-Based Annotation Simple markup of HTML version of document directs processor to create links to auto-generated topics Simple markup of HTML version of document directs processor to create links to auto-generated topics

14. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering BBS sends message Notification BBS sends message Notification User queries BBS for details User queries BBS for details Two way communication via instant messaging

15. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Mobile Communication via IM T*Mobile Sidekick Cingular Nokia 3595 Working Assets Sprint LG 1200 AOL Instant Messenger communication Worked on all phone types tested AOL Instant Messenger communication Worked on all phone types tested AT&T Samsung x427

16. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Statistics Viewer Accessible via ‘Administrator Panel’ Allows instructors to access ISI data Accessible via ‘Administrator Panel’ Allows instructors to access ISI data

17. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Question Answering Research Text is incoherent, non-factoid Questions barely identifiable, often include multiple sentences or paragraphs Answers require explanations Text is incoherent, non-factoid Questions barely identifiable, often include multiple sentences or paragraphs Answers require explanations

18. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Question Answering Placeholder for results screen shot Answer-bot refers questioner to most similar discussion and document segment.

19. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering XML Schema for Message Export

20. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Classummary Tool (Liang Zhou)

21. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Classummary Archive

22. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Classummary Notification Scheduling Interface

23. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Classummary Notification Email

24. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Research Potential

25. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Discourse Analysis (Shaw) Apply SPADE (Soricut & Marcu) rhetorical parser to discussion board data (spring 2005)  Run on over 1000 messages (Open University corpus)  Used to validate differences in tutor and student messages  AIED 2005 paper Assessing and Scaffolding Collaborative Learning in Online discussions Generalize work (to be proposed)  Adapt for real-time analysis  Run on DEN data  Evaluate different types of messages Question vs. comment vs. answer Tutor-instructor vs. student  Develop message classifier (done manually now)  Develop rules for promoting discussions

26. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Pedagogical Interventions (Beal) Analyze discussion board communication (spring 2005)  Analysis of small corpus of messages (Beal’s UMass course)  [More information here] Generalize work (proposed in AIED poster, Beal, Kim, Shaw)  Amalgamation of agent and ontology work, NLP, and discourse analysis  [More information here]

27. den.usc.edu Lord Corporation Foundation and USC Viterbi School of Engineering Learning Ontologies (Jihie Kim) Analyze discussion board communication (summer 2005)  Annotation of small corpus of messages (CSci555)  Classify on-topic messages using keywords  Compare technical vs general discussion messages Generalize work (to be proposed)  Acquire keywords automatically  Identify keywords in messages  Assess students’ contributions