1. Welcome to Teaching + Learning Tuesdays July 16, 2013 I 2:30PM to 4:30 PM

2. Title: Innovative Results Reporting: Student Assessment of Instruction Presenter: Julian Hutcheson, Statistical & Research Analyst III, Aiken Technical College

3. Innovative Results Reporting: Student Assessment of Instruction 3

4. 4 Have a “SAI” in your education!”

5. In-house report development for student course evaluation results Translating this to our vendor’s online system Features of our online reports Additional recent innovations 5 This is the Story of……

6. Fall Enrollment around 3,000 Approx. 60 FT faculty and 100 PT faculty Online course evaluation began in Fall 2009 Vendor was Campus Cruiser, Cruiser Eval product Fall 2011 began using IOTA Solutions 6 About Aiken Technical College

7. Part of FPMS, Evaluation of Instructors Student Satisfaction rates Trends in courses, departments, and programs Collect assessment data on our QEP critical thinking learning outcomes QEP Awareness Withdrawn students feedback Comparison of Online vs Regular classroom 7 How ATC Uses the SAI Survey

8. 16 core questions: Rating Very Satisfied to Very Dissatisfied Converted to numerical 4- point scale 2 QEP Awareness questions for all 6 QEP Critical thinking outcomes for implementer courses Faculty required to keep an average of 3.0 8 About our SAI Survey (through Fall 2012)

9. Does your online system have standard, built-in reports that meet your needs? 9 Reporting Results

10. 10 Raw

11. 11 Diligence

12. 12

13. 13

14. 14

15. 15

16. 16

17. Term Course IDSection Instructor Last Name Instructor First Name Q1: My instructor explains and clarifies course learning outcomes, grading and requirements. 2010FA1BUS-140011 LouisaDelay 4.00 2010FA1ECD-135003 MalloryBrassell 4.00 2010FA1MTT-112040 FernandoDesanto 4.00 2010FA1IMT-151041 KurtPrettyman 4.00 2010FA1CPT-101151 CodiSmith 3.92 2010FA1IMT-113041 TedWagoner 3.91 2010FA1WLD-102018 LouisaGarabedian 3.91 2010FA1ACR-102040 KarinaSobel 3.91 2010FA1MKT-240151 TamekaValderman 3.91 2010FA1WLD-132069 EricSturtz 3.90 2010FA1CPT-209029 LakishaTurnbough 3.89 2010FA1WLD-103040 JamieQuisenberry 3.89 2010FA1CRJ-202069 JohnSchieber 3.86 2010FA1CPT-101451 LanceVanders 3.86 2010FA1WLD-111040 ClaytonShira 3.84 2010FA1WLD-102040 SelenaDutil 2.83 2010FA1IMT-151040 HillaryFilip 3.84 2010FA1CPT-162057 ChristianTiano 3.83 2010FA1RPT-223058 LouisaDelay 3.82 2010FA1MGT-101060 MalloryBrassell 3.82 2010FA1CPT-101013 FernandoDesanto 3.82 2010FA1IST-201102 KurtPrettyman 2.80 2010FA1BAF-230063 CodiSmith 3.81 2010FA1CPT-257103 TedWagoner 3.80 2010FA1WLD-101040 LouisaGarabedian 3.80 2010FA1ECD-101064 KarinaSobel 3.73 2010FA1EET-251040 TamekaValderman 3.73 17

18. TermCourseIDSectionFirst_NameInstructor_Last_NameInstructor Overall Average with Course Section Averages 2010FA1CPT-162057LouisaDelay3.82 2010FA1CPT-168059LouisaDelay3.58 Delay Average3.70 2010FA1WLD-132069FernandoDesanto3.20 2010FA1WLD-170040FernandoDesanto3.20 Desanto Average3.20 2010FA1RPT-243069CodiSmith2.98 Smith Average2.98 2010FA1ECD-102002TedWagoner3.71 2010FA1ECD-237040TedWagoner2.95 2010FA1ECD-131015TedWagoner2.88 2010FA1ECD-133040TedWagoner2.68 Wagoner Average3.06 2010FA1CPT-162015LouisaGarabedian3.69 2010FA1CPT-212003LouisaGarabedian3.49 2010FA1CPT-162013LouisaGarabedian3.39 Garabedian Average3.52 2010FA1MKT-240151KarinaSobel4.00 2010FA1MKT-240014KarinaSobel3.70 2010FA1MKT-101058KarinaSobel3.62 2010FA1MGT-120013KarinaSobel3.07 Sobel Average3.60 2010FA1EGT-252069TamekaValderman3.00 Valderman Average3.00 18

19. Positives User friendly mostly Overall averages Sorted by course and by instructor PDF with Navigating tabs Repeated too much info on each page Hard to see pattern for instructor or for a course 116 Pages! 19 Fall 2010: Good but Lengthy

20. 20

21. 21

22. 22

23. Compact Seeing results for all questions on single page 12 pages versus 116 pages Can see areas of weakness in student satisfaction: Referring to the survey questions Patterns of red areas Online courses and QEP Implementer courses identified by color coding Sorted by course and by Instructor, 5 pages each 23 Spring 2011 Report

24. Encouraged customization Very different from current vendor Can you do this? Usually “Yes!” Site graphics made course evaluations look like ATC 24 Enter IOTA Solutions, Inc

25. Even standard reports were a significant step up from previous vendor Faculty views and Dept. Chair views impressive What is standard and what is custom? Implemented Fall 2011 Overall went well, but not all rosy 25 IOTA, continued

26. http://atc.iota.cc 26 Let’s go!

27. 27

28. 28

29. All the information of the original report, plus response rates & comments link Automatically generated Numerous interactive sorting, ranking, and filtering features Instantly look at the data in different ways Link to comments instantly Printable QEP Reports 29 IOTA Custom Report

30. We are in the middle of a journey Going to a much more customized reporting tool has its downsides and risks Bugs, Failures, Beta Customer service may be impacted by features not working at all times 30 Customization

31. Affordable Prompt attentive customer service to correct problems Automated, customized reports with features already described 31 Positive Side

32. Benefits clearly outweigh the costs For customized, automatic reporting features, more tolerance and patience are warranted to allow the development process to occur 32 Final Analysis:

33. Moved from 16 questions on instruction to: 5 Instructor questions 4 Course questions, and a combination of the following, based on delivery method Lecture, Laboratory, Clinical Site, Online Course, Hybrid Courses, Two-way Video Courses, Flexible / Open Lab, and Field Work 2 to 4 questions for each Only the 5 instructor questions feed into the FPMS and the Custom Report 33 New Development Spring 2013 Questions Based on Delivery Method

34. 34 Discussi on

35. Julian Hutcheson Aiken Technical College Office of Planning & Research 803-508-7418 hutchesj@atc.edu 35 Contact:

36. Join us every Third Tuesday for more Professional Development opportunities. Next Session: August 20, 2013 TIME2:30pm – 4:30pm TITLE Preparing for the Fifth-Year Report and Reaffirmation PRESENTERS Dr. Claudette Williams, SACS-COC