1. id@ S&T Student Course Evaluations: Beyond Question 7 – Part 2 Dan Cernusca, Ph.D. Instructional Design Specialist Missouri University of Science and Technology Active Learning FLC - March 24, 2010

2. id@ Why student evaluations? Classroom observations Instructor’s perspective Students’ perspectives GAP ?

3. id@ Focal Point: Analysis of S&T evaluation questions; Additional evaluation questions tested; Classroom strategies to enhance learning

4. id@ Current Evaluation [In]Famous 7: 1.Course educational value 2.Use of assignments and tests 3.Concern for students’ understanding of the materials 4.Preparedness for class 5.Ability to communicate 6.Ability to motivate 7.Overall teaching effectiveness

5. id@ Some challenging evaluations What is the value of Q7? Q1- course value2.13.2 Q2- use of tests2.11.8 Q3 – concern for understanding 2.61.4 Q4 – preparation for class2.61.4 Q5 - communication1.61.5 Q6 - motivation1.31.4 Means of Q2 – Q62.01.5 Required courses surveyspecialty

6. id@ What is the value of Q7? Q1- course value2.13.2 Q2- use of tests2.11.8 Q3 – concern for understanding 2.61.4 Q4 – preparation for class2.61.4 Q5 - communication1.61.5 Q6 - motivation1.31.4 Q7 - overall1.81.3 Means of Q2 – Q62.01.5 Required courses surveyspecialty

7. id@ Unfortunately not that much… 0 (poor) 123 4 (excellent) Qi 1 38106 Qi+1 49762 Meaning beyond the S&T means?

8. id@ How about written comments? “Dr. _____ knows the material but lacks the ability to communicate that knowledge to us.” “Poor communication. Looks down on everyone. Never Positive.” The tone of comments … The exception… “…I thought the classes were entertaining. It was easy to pay attention. … Thanks for always being prepared for class.” Q51.61.5 Q61.31.4

9. id@ Focal Points: Analysis of S&T evaluation questions; Additional evaluation questions tested Classroom strategies to enhance learning

10. id@ Focus on communication & motivation Q1- course value Q2- use of tests Q3 – concern for understanding 0.49*0.72** Q4 – preparation for class Q5 - communication 0.80**0.76** Q6 - motivation 0.82**0.68** Q7 - overall 1.82.9 ** p < 0.01; * p < 0.05 (Pearson Correlations)

11. id@ Additional Evaluation Questions (15 initial items) Instructor Mastery Goal Orientation (IMGO) perceived interest in student learning; Teacher Immediacy (TI) perceived personal closeness; Communication Competence (CC) perceived communication skills

12. id@ Testing and Refining The15 items first tested in small classes encouraging trends; some items appeared more powerful than others Refined in a large introductory course adapted items for S&T evaluation scale; exploratory factor analysis conducted; 7 items loaded in one major scale;

13. id@ Additional Evaluation Questions The “chosen” 7: 1.Instructor’s willingness to have you enjoy new things 2.Instructor’s focus on understanding not memorizing 3.Mistakes are okay if you are learning 4.Instructor recognizes you for trying hard 5.Gesture, movement, and dynamic delivery 6.Instructor’s good eye contact in classroom 7.Instructor’s ability to express ideas clearly IMGO TI CC

14. id@ Used at the midterm & finals good indicators for major problem zones (typically “clarity of ideas” and “recognizing us for trying hard” mirrored Q7); as a scale showed significant strong correlations with Q7 (> 0.70); as a scale the 7 additional questions predicted more than 60% of variance in Q7 (R 2 in regression analysis); Additional Evaluation Questions

15. id@ Can offer: insightful opinions related to the motivation and communication issues; additional information for each major instructional activity (lecture, lab, test) support for more informed instructional interventions; Essay Questions for Midterm

16. id@ Warnings! If you use midterm surveys, then analyze students’ input in a timely manner; Select the critical but constructive suggestions and ignore individual/empty/negative ones; [If available] choose suggestions that you consider valid, inform students, and act on them; Essay Questions for Midterm

17. id@ Focal Points: Analysis of S&T evaluation questions; Additional evaluation questions tested Classroom strategies to enhance learning

18. id@ Levels of classroom interventions Visible behavioral & presentation issues Classroom dynamics and engagement Partial restructuring/redesigning of instructional materials

19. id@ Visible behavioral & presentation issues verbal annoyances (“yes” / “OK”/ “that is”) talking to the board or the slide; being “glued” to the podium; deleting parts of existing information on the board & writing over; using “look down” terms (Students - Class)

20. id@ Balance slides and board work: slides for outlines and summaries ; board for detailed information/examples; Classroom dynamics & engagement Engage students in the classroom ask questions and insist on an answer; listen to & answer students’ questions; use in-class transfer problems for your worked-examples;

21. id@ Partial instructional restructuring Topic 1 Topic 2 Issue 1.1 Issue 1.2 … … …you do this…you expect that

22. id@ Partial instructional restructuring Potential issues to look into: content and assessment are not aligned; misconceptions are not identified and addressed before the major exams; not enough instructional support for term/team projects; good instructional tools used in a wrong way;

23. id@ Contact Information Dan Cernusca, Ph.D. 207 Norwood Hall (CERTI office) (573) 341 4632 dcernusca@mst.edu