David A. Starrett, PhD Director, Center for Scholarship in Teaching and Learning ED-MEDIA 2002 – Denver Michael L. Rodgers, PhD Professor of Chemistry Supporting the Claims of Campus Transformation Through Assessment of Southeast Missouri State University’s "Technology Serving Learning Institutes"

2 Regional, comprehensive, Masters granting 9,000 Students 380 Full and Term Faculty Strong commitment to Teaching/learning Use of Instructional Technology in teaching Faculty development Centralized teaching/learning faculty development center (CSTL) incorporating an Office of Instructional Technology (OIT) Southeast Missouri State University

3 Halfhearted tech investment before of 380 faculty had no PC at work -only 16 dialup lines for remote access to University’s network -only accounts for a faculty/staff/student population >10,000 Low faculty e-learning development skills - no online courses - few Web-augmented courses - no coordinated skills development The Internet Revolution (mid-1990’s)

4 Technology Serving Learning Institutes IT Workshops for Faculty Summer session, Winter break Taught by faculty with Student Tech Consultants Diverse participant expertise levels Participants received PD$, software Customized, modular course management software developed by OIT Hardware and software instruction Model evolved: Short course  “ a la Carte ” $70,000 annual budget

5 Evolution of the Institute Changes in length, time, organization reflect new clientele Dynamic faculty interests: new software, teaching techniques, etc. Computing Services interested in offering workshops within Institute Respond to faculty requests: Chairs working around TSL schedules Campus-wide electronic grade reporting workshop using Institute software Assessment Item #1

6 TSL Institute attendance numbers Assessment Item #2

7 Declining costs for the TSL Institutes Assessment Item #3

8 Institute Evaluation instrument results… Assessment Item #4 9-point scale (9=“Excellent”; 5=“Average; 1=“Poor”) “What was your overall rating of this Institute?” “The Institute was practical and useful.” “The facilitators responded effectively to questions.” “The facilitators knew the subject.” 146/149 rated Institute “Effective” or “Somewhat Effective”

9 Institute Facilitator pool Facilitators New Facilitators TSL Veteran Facilitators New TSL Veteran Facilitators S S W S W S W S W Total 4225 Assessment Item #5

10 Number of on-line courses offered TermOn-line Courses OfferedNew On-line Courses Fall 9999 Spring 0055 Summer 0064 Fall Spring Summer Fall Spring Summer Fall Total25098 Assessment Item #6

11 Enrollment in on-line courses TermUndergraduateGraduatePercent seats taken Fall % Spring % Summer % Fall % Spring % Summer % Fall % Spring % Summer 02>628>55>75% Total ~80% Assessment Item #7 Current enrollment

12 Number of faculty IT server accounts 275/380 faculty have active IT accounts New accounts added ~weekly All new faculty receive and use accounts during orientation Assessment Item #8

13 Number of course sections supported by Websites Instructional servers house >500 course Websites Servers house >750 OIS-managed course sites Dozens of new course Websites, OIS-managed sites each semester Assessment Item #9

14 Support for locally-authored course management software (OIS) OIS – modular course management tool for Web-based & Web- assisted courses – 1st taught Winter 2001 Online Manager, Testing, Gradebook, Calendar, Forum Under development: chat, portal, student folders, etc. Faculty serve as advisors, β-testers Close collaboration between faculty and programmers Institute participants provide feedback Assessment Item #10

15 Quality, sophistication of course Websites Increased quality, complexity of course Websites for Web-assisted and Web-delivered courses since 1997 Due to TSL Institutes, increasing faculty experience, other faculty PD Assessment Item #11

16 Changes in strategic plans around the University The University Strategic Plan included technology as one of six priorities The current plan assumes technology as base for ALL plan priorities College and Department plans increasingly reflect technology integration Assessment Item #12

17 Increases in faculty attendance and presentations at IT-related conferences Increasing faculty attendance at tech-related conferences or tech- related sessions at professional society conferences since 1997 An increasing number of faculty presentations at these conferences Assessment Item #13

18 Increased faculty participation in IT committees and discussions on campus Increased number of campus IT committees since 1997 Institute veterans make up an increasing percentage of faculty involved in those committees TLT Roundtables and other IT- related discussions have occurred, with increased faculty involvement Assessment Item #14

19 Increased IT-related purchases & infrastructure development University Information Technology Committee makes budget decisions, allocates funds Increasing IT funding, including smart classrooms, computer labs, software, specialized hardware, personnel Assessment Item #15

20 Student Technology Learning Impact Survey II March 18 - April 8, student addresses from online courses taught F99 - S02 Survey URL sent to all on March invalid addresses 6 opted out URL again sent April 1 to remainder 408 valid responses (19%) Graphic Representation: % students choosing given responses. Assessment Item #16

21 Helpful Feedback from Instructor?

22 Intellectual Effort

23 Does the Instructor Care?

24 Logical Organization

25 Did the Students Learn?

26 Take an Online Course Again?

27 IDEA results IDEA (Individual Development Educational Assessment) student ratings of instruction instrument used to obtain nationally normalized data Results: online courses rated as well as traditional F2F courses; overall indicators show no statistically significant differences Assessment Item #17

28 Faculty visits, phone calls, s to CSTL Support typeAnnual number Phone6,000 1,375 Walk-in500 Equipment loan300 Assessment Item #18

29 Interest in the value of IT-related work in Promotion and Tenure decisions Assessment Item #19 Information Technology Faculty Roles and Rewards advisory document – role of IT in teaching, research, service Inclusion in college and departmental rank & tenure documentation Leadership by administrators

30 Promulgation of standards for instructional Web pages on campus Online course quality focus Numerous approaches to ensuring quality online instruction Technology Associates tasked with creating Instructional Web Page standards Based on AAHE “Seven Principles” implemented with technology Assessment Item #20

31 for further information … Mike Rodgers: David Starrett: Website: CSTL: Thank you for your attention!