Measuring our Impact: The Open.Michigan Initiative Emily Puckett Rodgers Open Education Coordinator, Open.Michigan OpenCourseWare Consortium Global Meeting 2011 Except where otherwise noted, this work is available under a Creative Commons Attribution 3.0 License. Copyright 2011 The Regents of the University of Michigan CC: BY-NC-SACC: BY-NC-SA, Choconancy1Flickr Choconancy1

Evaluation team Emily Puckett Rodgers Ted McCarthy Pieter Kleymeer Introduction Evaluation: started September

“Through a combination of existing and new qualitative (including surveys, interviews and user feedback) and quantitative (including web analytics, published resources) data, we aim to measure Open.Michigan’s impact on the University of Michigan campus and on the broader open education landscape, as well as its progress toward overall objectives and mission.” Evaluation Design Implementation Mission and Objectives Environmental Scans Document Study and Review Organizational analyses Research development and deployment Analysis Review

Environmental Scan General evaluations, activities in Open Education Specific evaluation projects and documentation: Tufts OpenCourseWare (2010) Free to Learn Guide (2010) MIT OpenCourseWare (2006) Oxford’s Technology-Assisted Lifelong Learning (2010) Document Study dScribe data Wiki Shared internal documents and spreadsheets Environments

Open.Michigan’s Strategic Vision Open.Michigan enables University of Michigan faculty, students, staff and others to share their educational resources and research with the world. Open.Michigan’s efforts contribute to two primary goals: 1.to sustain a thriving culture of sharing knowledge at U-M 2.to provide comprehensive public access to all of U-M’s scholarly output Strategic Vision on

Measurable Activities Building communities of OER producers and users Consulting and Outreach Services to facilitate OER production Development of Processes and Software to support OER production and publishing Next Steps: Goals and Objectives document, Communications Plan Strategic Plan Guiding Objectives How and Why is our OER being used? Who is using our OER? What value does OER bring to U-M? In what contexts are people using our OER? From the Vision document From the Evaluation plan

Impact: Growth Open.Michigan platform migrations eduCommons OER OERbit /education OER static Open.Michigan pages 4/1/2008 9/16/2010 8/27/2009 4/2011

Google Analytics Impact: Growth

Impact: Contributions Resources available ~70 courses published 2 campuses 12 schools ~9 OER modules published ~48 resources deposited in Deep Blue Resource facilitation Anishinaabe langauge and literature African Health OER Network “Textbook of the Future” discussions Center for Global Engagement Handbook

OERca, released 2008 (copyright and content clearance tool) OERbit, released 2011 (publishing platform) dScribe, process for using volunteers and community members to clear content and publish OER Code of Best Practices in Fair Use for OpenCourseWare (2009) African Health OER Network Contributions to Open Education Impact: Contributions

Institutional Support (U-M) Dean support: Dean Woolliscroft (UMMS) Dean MacKie-Mason (SI) Dean Potempa (Nursing, retired) Paul Courant (MLibrary) Financial support: UMMS Hewlett Foundation FAIMER Open Society Institute Local grants Institutional support and partnerships: Medical School School of Information MLibrary MERLOT African Health OER Network: Global Reach African Studies Center Ghana-Michigan Charter Ghana Emergency Medicine Collaborative Impact: Investment “Having the big block M on a lot of high quality produced teaching modules that are made available to the world… is mission consistent, so what’s the business that we’re in here? We create and distribute knowledge.” -Paul Courant, “Why Open is Important” interviewWhy Open is Important

Impact: Surveys 40.7% instructors never heard of OCW 75.6% students never heard of OCW CTools survey

Impact: Surveys Population SizeResponse Rate Completion Rate 39,958 students 27.5%71% 6,149 faculty 30.4%68% 2,268 staff 16.5%55% Open.Michigan and Open Educational Resource survey to U-M Themes: Sharing habits Awareness of Open.Michigan and OER Use of OER Impact of OER Support of OER efforts and Open.Michigan

Sharing Impact: Surveys "I share learning materials with my colleagues to..." ? (faculty responses)

Use Impact: Surveys “What type of OER have you used?” (student responses)

Next Steps: participating faculty surveys (Some) Results Themes from survey comments Establish Open.Michigan in main channels of U- M communication Clarify copyright in OER and provide resources Consistently define Open.Michigan and OER Describe why/how to use/create OER Make short-term improvements to OER Encourage cultural shifts in learning CC: BYCC: BY Emily Puckett Rodgers Impact: Surveys

Next Steps In the next three years, Open.Michigan will: ① Produce more and richer content as OER with the various campus units, improve modularity, instructional design, and accessibility of U- M OER ② Increase the visibility and discoverability of U-M resources through a combination of marketing and metadata ③ Draw participants from more parts of campus to expand its disciplinary coverage ④ Ensure OER production is an embedded part of the academic life on campus Evaluation Finish survey analysis Focus Groups Strategy Goals and Objectives Communications plan Advisory committee

Resources Contact Emily Puckett Rodgers Use Evaluation wiki page Evaluation plan Survey questions Strategic planning Aggregate data Our Google Analytics data Aggregate survey results