1. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki CASE STUDY: Web Course Design with the Topic-Case Driven Methodology Leena Hiltunen & Tommi Kärkkäinen

2. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Contents of the presentation Introduces briefly the Topic-case driven methodology for web course design and realization Introduces the backgrounds of the case study Reports the students’ performance during the case study Analyses the feedback collected from the students Concluses the findings Draws some guidelines for future work

3. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Introduction Topic-case driven methodology –Describes a unified way to design and realize Web courses –Was introduced in ICNEE 2004 *) –Utilizes metaphors from software engineering –Blends ICT with educational issues (especially pedagogics) –Allows incremental and iterative development –Supports blended learning as well –Supports reusability of learning objects –Upraises the quality of online learning –Enables the creation of a Web Course Repository *) Hiltunen, L. & Kärkkäinen, T. (2004), Topic-case Driven Approach for Web Course Design. In Proceedings of 6th International Conference on New Educational Environments (ICNEE), 27th - 30th September 2004, Neuchâtel, Switzerland; Session “Web-based courseware” 5.2/B.

4. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Background Study What? Why? Who? When? How? Goal Concepts Content Material Content Design Creation of Topic Cases and representation of their hierarchical relations in Topic Case Diagram Pedagogical Design Pedagogical solutions Cognitive tools Communication tools Pedagogical usability Technical Design Platform Medias Usability Maintenance Scaling Compatibility Standards User interface Realization and assessment Completion of topics using chosen contentual, pedagogical and technical solutions Overall assessment Incremental and iterative development Phases of Topic Case driven Web course design and realization process

5. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Topic-case descriptions Basic element: a topic-case –A short but structured description of basic lines of the single course topic (or the course itself in the beginning). –Describes the necessary issues that should be treated during the course; forms the skeleton of contents of the course. –Extends during the pedagogical design with new attributes: actor(s) involved, detailed description of activity, pedagogical solution(s), and relations to other Topic Cases. –Formed during the early planning stage (central ones already during the background study). –Linked together as a topic-case diagram according to preliminary knowledge and pursued learning

6. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Name of the course Topic Case number 1 Date/Name of the developer Topic Case:Name of the Topic Case Summary:Brief description of the Topic Case Preliminary knowledge:Knowledge that is required before entering the Topic Case Material(s): Material(s) engaged with the topic Learning:Sort of post-conditions, learning that is pursued after completing the Topic Case Form of basic Topic Case description (cf. Use-case in software engineering); will be extended during the pedagogical design.

7. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Topic-case diagram Represents the relations between topic-cases. Defines the basic contentual hierarchy of the Web course (like content map). Introduces stereotypes: «requires» and «advances», that reveals which topic-cases are essential to the main concepts and which are prerequisites for other topics. «requires» indicates the knowledge that is required before certain Topic Case can be accomplished properly. «advances» indicates the knowledge that would be useful to be available, but is not compulsory for the following Topic Case.

8. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Role of the Web in teaching Role of the teacher on the course Computing and communication skills of the teacher Pedagogical model Cognitive and communication tools «requires» «advances» Part of the pedagogical design of a Web course - an example of topic-case diagram with five topic-cases (cf. use-case diagram in software engineering)

9. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Backgrounds of the case study Carried out with a group of computer science graduate students who have studied both computational and educational studies Carried out in a Web Course and Implementation Course during the autumn 2004 with 27 students Both campus and distance students Teaching activities included two-hour lectures twice a week and some four-hour exercises related to digital course material production; lectures were streamed online and recorded for later review

10. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Backgrounds of the case study Implemented on Finnish virtual learning environment called Discendum Optima *) Learning activities were divided into six learning assignments which followed the phases of the topic- case driven approach Students wrote learning diaries during the course More consistent and comprehensive feedback was collected with questionnaire at the end of the course *) see http://www.discendum.com/english/index.html

11. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Activities and assignments Weeks 1-2: Introduction to the backgrounds of web course design –Specification of the web course –Clarification of basic elements on the web course by exploring web courses found from the Internet –Introduction to basics of the web course design and the topic-case driven methodology

12. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Activities and assignments Weeks 2-4: During the Background study students –Defined and considered all those issues that affect the feasibility of the planned web-course –Chose the topic for their own courses –Explored different resources (Internet, databases, books, articles, etc.) –Created an idea bank from chosen topic as the first learning assignment

13. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Activities and assignments Weeks 4-5: During the Content design students –Reviewed their own idea bank by evaluating different ideas and choosing the most suitable ones into their own web course –Designed and documented the basic content of their web course with topic-cases by describing the necessary issues that should be treated during the course –Linked single topic-cases according to preliminary knowledge and pursued learning –Represented relations in the topic-case diagram that shows which topic-cases are essential to main concepts of the course and which are prerequisites for other topics –Documented the content of the web course with topic-cases and topic-case diagram as the second learning assignment

14. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Activities and assignments Weeks 5-8: During the Pedagogical design students –Sized up all the selected topic-cases and tried to find the best pedagogical solution for each case separately –Documented the content of the web-course and pedagogical activities for each topic with the extended topic-case descriptions as the third learning assignment As the fourth learning assignment, students produce new kind of learning materials with video photography or sound treatment into students’ own Web courses

15. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Activities and assignments Weeks 8-10: During the Technical design students –Made decisions concerning technical issues, like use of platform, medium in use, maintenance, scaling, compatibility, user interface –Used different platforms (e.g. Optima, Moodle, WebCT, R5 Generation), tools (e.g. HTML-Kit, FrontPage, Dreamviewer), and techniques freely (e.g., HTML, XHTML, Java, Java Script, or XML) –Kept usability and accessibility issues in mind –Documented the technical design in production manuscript as the fifth learning assignment

16. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Activities and assignments Weeks 10-14: During the Realization, testing and assessment students –Completed the individual topic-cases using the chosen pedagogical and technical solutions –Enlarged the contents to the final length –Described teaching and learning actions in detail in connection with the final contents and medium in use –Got another students web course to test and assess as peer evaluation assignment (the sixth learning assignment)

17. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Students’ performance 22 of students passed the course, 18 of them with good or excellent grades. Assessment of the students’ learning outcomes at the course was based on authentic assessment: students’ performance was observed during they work, and learning assignments, learning diaries, self-evaluation, and peer evaluation were included in the assessed portfolio. Students gave a lot of feedback from the course in their learning diaries. More formal feedback was collected with feedback questionnaire at the end of the course.

18. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Findings and feedback 95% of the students had only a little experience on online learning before taking this Web Course Design and Implementation Course. 77% of the students were satisfied with the technical solutions used at the course. Goals of the course were defined clearly at the beginning of the course – they all knew what to expect, and what had to be done and when. All the students felt that they learned a lot of new issues during the course; they had not realized how much different matters had to be considered while designing a web course.

19. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Findings and feedback Students liked the way that the course was designed as a web course; the best parts of the course were learning assignments, following the process model, and video recorded lectures. –The idea to bind learning assignments with the phases was considered excellent. –Using the process model worked very well; it supported the working process well, split the work in smaller pieces, and on the other hand, helped to see the big picture as well. –Streamed and recorded online video lectures were valuable help especially for distance students.

20. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Findings and feedback Detailed design and following the phasing of the process model helped a lot to achieve the one of the course goals, a well designed self-made web course. The phasing and learning assignments spread the workload equally from September to December. Designing was sometimes difficult, but diligence and exactness were rewarded in the implementation phase at the latest; many times already during the next step. Students were able to utilize designs from the previous assignment on the next one.

21. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Findings and feedback Technical design was unexpectedly fun and ultimately, by following the previous designs, quite easy. “Well designed was half done” Learning assignments related to design model were illustrative, good and authentic. “There were good examples which explained well what to do, and it was very motivational to get small pieces ready and see how bigger completeness was getting ready piece by piece”.

22. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Improvements Some of the distance students felt that they did not get enough guidance during the course and expected more feedback after returning their learning assignments. –At the same time some other distance students told that they got as much guidance as they needed. –Part of the students simply was more active and asked more guidance spontaneously. –Individual differences; some students need more guidance than the others. –There were a discussion forum for students to ask questions and discuss about the design also with other students, but only few students used it occasionally.

23. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Improvements Some distance students said that the timing was too fast and they would have needed much more time to achieve better results. –Students knew at the very beginning of the course how the course will be timetabled, but all the distance students were working during the course and most of them had also other studies at the same time, so the timing is mainly students’ personal timing problem.

24. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Improvements Some of the students wanted to design they course interface earlier than they were allowed to do according to the proposed model. –They had some image of their course interface in their minds, but before designing the interface one should be sure what issues are needed on the interface, e.g., links to content, tools, media elements, and learning assignments. –Of course, if one decides at the very beginning, during the Background study, that he is going to use some certain learning platform, then this has its’ influence on interface design, but still components needed on the interface are missing.

25. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Validity and reliability of the results Results of the case study can be too positive –Students enjoyed the course and they created excellent web courses by following the proposed web course design methodology. This positive atmosphere might have had slightly positive effects on students’ answers. –The size of the student group was relatively small (only 27 students). –Students worked alone or in pairs, so the results could not yet be generalized to bigger design groups. –The lecturer of the course was involved in inventing the design methodology, so she had different kind of relation to this proposed methodology that another teacher would have had.

26. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Validity and reliability of the results –Students were a group of computer science students who had better computational skills that teachers normally have, so they did not have problems on that area of competence; they were able to fully concentrate on implementing the content and pedagogical issues. –Students were able to get as much guidance during the design process as they needed, this might not (unfortunately) be realistic in real world web course design processes.

27. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Conclusions The case study was successful as a whole. Students designed and implemented excellent web courses and they outperformed right through the design process. Phasing activities with learning assignments that were based on the phases of the proposed design process was an excellent idea; it made designing easy and even fun.

28. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Conclusions The case study was executed with a biased and small group: twenty-seven students of computer science teacher education study line. The course was organized for the first time, so first time enthusiasm might have had slight positive effect on the results. Working habits of both blended and distance learning groups are much based on teacher’s success in organizing and activating students.

29. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Conclusions Relatively small group size and working alone or in pairs as well as better computational skill of students leads us to the conclusion that these results can not yet be generalized to bigger design groups. Still, very encouraging results were obtained concerning the topic-case driven methodology and its utilization in the corresponding course in a bootstrap fashion.

30. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Future work How to connect pedagogical patterns and scripts into this web course design process, especially into the topic-cases during the phase of pedagogical design? –Hepls also teachers in their design process when they try to find best possible solutions and activities to they web courses –Use of a web course repository - explore it and find suitable pedagogical solutions (reusable learning objects) into new courses or at least develop some new ideas.

31. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Future work How to evaluate learning outcomes? –How to evaluate students’ performance without need for pondering academic honesty and integrity? Quick answer: Authentic assessment or better authentication –How to get sufficient information on how well learners actually perform? Quick answer: By designing new kind of learning activities and assignments

32. Computer Science Teacher Education Leena.Hiltunen@mit.jyu.fi UNIVERSITY OF JYVÄSKYLÄ Depertment of Mathematical Information Technology Helsinki Thank you for your attention! Leena.Hiltunen@mit.jyu.fi University of Jyväskylä Finland