1. Copyright Mary Y. Lee, MD, 2006. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author. EDUCAUSE Dallas, Texas October, 2006

2. Formative and Competency-Based Assessment of Learners in a Longitudinal Curriculum EDUCAUSE Dallas, Texas October, 2006 Mary Y. Lee, MD Associate Provost Tufts University

3. By the end of this session you will: Exposed to examples of curriculum and assessment tools and their uses Think more longitudinally about curriculum development and assessment Reflect on linking program assessment more tightly with learning outcomes Consider engaging a broader team for curriculum and assessment design

4. How does medical education work? Four years of undergraduate education – Basic and clinical sciences Accreditation standards – Learning objectives – Written curricula – Formative and summative assessment – Learning outcomes/competencies

5. What does medical education look like? Curriculum is highly integrated – Horizontal – Vertical Online tools to – Create – Coordinate, integrate – Document progress and outcomes

6. What has Tufts used to develop online tools?

7. Tufts University Sciences Knowledgebase

8. TUSK as a Complete System INFRASTRUCTURE CONTENT USERS Flexible and Extensible Personalized tools Development and Management Tools

9. What is unique about TUSK? Rearrange/enhance/link “atoms” of UMLS-tagged information” to suit curricular needs New Links

12. Enhancing Text Documents with XML Highlight objectives, key words, key concepts, pearls, important notes

13. [Student’s notes would appear here] Add notes to personal folders

15. Online text with Imbedded Multimedia

16. Virtual Case Player that meets emerging virtual patient standards

17. Histology image used across three schools

19. What do you need to do to lay the groundwork? What student outcomes do you desire? What are the learning objectives? – What do students want/need to learn? – What learning behaviors do you want to promote? What are external requirements? – State examinations – Licensing examinations – Accreditation requirements

20. Do your online assessment methods match your objectives? Provide formative feedback to learners Document adequate preparation for the next level of learning Create synergies with other assessment methods to provide the most complete reflection of student learning

21. Starting an e-portfolio with a patient log system

22. How can a student track his/her progress?

23. How do students use logs to track objectives and find learning resources?

24. How do faculty use logs to provide formative feedback to a student?

25. How do faculty at one site assess program effectiveness?

26. How can faculty track aggregate student performance across sites?

27. How can faculty and deans assess program effectiveness across sites?

28. How can logs be combined with other assessment tools? Clinical performance in hospitals – Direct observation by faculty and residents – Cumulative evaluations each month Online quizzes, virtual cases Standardized patients National licensing examinations – MCQ subject tests – Clinical Skills Examination

29. What are the challenges of implementing these tools? Student training Faculty training Faculty and students at multiple sites Integration with other tools Integration with other parts of the curriculum

30. How does a team approach improve curriculum and tool design? Educational Technologists Educational specialists Faculty Students Librarians Administration

31. Who else is using TUSK’s infrastructure for their own content? New York Medical College University of Hawaii Boston University New York University Kenya, Tanzania, Uganda India

32. TUFTS CONTENT MUHIMBILI (TANZANIA) COTENT JOINTLY DEVELOPED CASE STUDIES Muhimbili University College of Health Sciences, Dar es Salaam Prof. J Griffiths, ”(NIH K07 GM072954) working with Prof. M Mathiu, Tanzania

34. By the end of this session you will: Exposed to examples of curriculum and assessment tools and their uses Think more longitudinally about curriculum development and assessment Reflect on linking program assessment more tightly with learning outcomes Consider engaging a broader team for curriculum and assessment design

37. TUSK Components AdministrationContent Authoring Tools Schedule Users Enrollment Evaluation Surveys Grades Logs Search Groups Text with semantic markup Literature Links Multimedia: video, audio, flash, zoomable images Quizzes Virtual patients Downloadable files Search Forums Authoring Tools – CMS Upload content Create, input, import and display administrative data Content Creation Tools Quiz Virtual patient TUSK “style” in word Survey tools Evaluation tools Patient logs TUSK Data

38. TUSK Components Programming Interface Data Model User Interface Author Interface Admin Interface External Interface

39. TUSK Technology Stack Operating system: Solaris (or Linux) Database: MySQL Webserver: Apache Application environment: Mod_perl Templating engine: HTML::Mason HTML::EmpPerl

40. How can technology support your objectives? Viewing objectives – On homepage – For each lecture – For each log/portfolio entry Linking learning resources to each objective

41. Enhancing Text Documents Slides from collections can been seamlessly integrated into syllabi text