1. PRINCIPLES OF GOOD PRACTICE IN INTRODUCTORY STEM COURSES: LISTENING TO THE VOICES OF FACULTY AND STUDENTS Gina A. Garcia Josephine Gasiewski Sylvia Hurtado Association for the Study of Higher Education Charlotte, North Carolina November 17, 2011

2. Background  Focus on introductory STEM courses since the mid-1990’s (National Research Council 1996, 1999; National Science Foundation, 1996; Seymour & Hewitt, 1997; Tobias, 1990)  Large class sizes  Lecture-based courses that encourage passive learning  Lack of engaging pedagogy  Numerous teaching and learning techniques employed, often by research scientists teaching introductory courses  Student Response Systems (“clickers”)  Just in Time Teaching (JiTT)  Web-based pedagogy  Peer-based learning (PBL)

3. Background  Criticism of curriculum continues in 2000’s (Alberts, 2005; DeHaan, 2005; Handelsman et al., 2004; McWilliam et al., 2008)  Lacks exploration of the real world of science  Lacks ways for students to learn about the discovery of science  Lacks creativity and innovation  Promising Practices in Undergraduate STEM Education (NRC, 2011)  Goals and purposes of STEM teaching and learning innovations have varied tremendously  Large scale assessment of in class techniques is lacking  Froyd (2008) promising practices in STEM undergraduate education

4. Purpose of Study  Highlight the ways in which undergraduate students and university professors at eight institutions make meaning of good teaching practices in introductory STEM courses  Provide practical suggestions for improving teaching and learning in introductory STEM courses using empirical data

5. Methods  Sample  8 Institutions (diverse by type, size, and control)  41 Student Focus Groups (n = 239)  25 STEM Faculty Interviews  Data Collection  60-90 min in-depth interviews/focus groups  9 main student questions centering on motivation, STEM course structure, learning, instruction, and assessment  7 main faculty questions centering on course goals and objectives, pedagogical approaches, STEM course structure, forms of assessment, and institutional support for teaching

6. Analysis  Interviews digitally recorded and transcribed verbatim  NVivo 8 qualitative software  Six researchers open coded raw data for salient themes using constant comparative method  Developed first-order categories  Intercoder reliability 80-85% (Kappa coefficient)  Re-validated and developed additional categories  Developed second-order themes within individual institutions  Compared student and faculty data, looking for similarities and differences  Compared across institutional types, looking for broad themes

7. Three Main Themes: 1. In-class techniques 2. Out-of-class techniques 3. Context for learning FINDINGS

8. Southwestern Private Research University  Competitive classroom environment  Predominance of pre-health majors  Early warning system  Student centered focus  Formal classroom techniques  Real world application  Working problems on board  Skeleton notes  Class discussions  Groups projects  ELMO  Out of class techniques  Online homework  Study groups  Formal & informal SI  Office hours  Blackboard  Student Success Center StudentsFaculty Context for Learning  Formal classroom techniques  Real world application  Clickers  Out of class techniques  Online homework  Study groups  Formal & informal SI

9. Predominance of Pre-Health Students  I think in a lot of my like introductory, or at least the pre-med courses, there was like a tendency of, well, you can fail ‘cuz that means that my shot is better. It’s way more, I think, cutthroat. I mean, you still help each other, but it’s definitely like – Yeah, it’s definitely competitive like, you’re like, I’m going to be at every class. It’s, and I think that kind of helps foster education because it, you know, it’s like these are, you know, med school is competitive to get into, so it’s like, you know, you need to be going to every single class, being able to easily do the information, but I would definitely agree. It’s definitely, it’s pretty hard. (Brendan)

10. Real World Examples  I’ve had chemistry professors and calculus professors, like classes that were really dry and potentially really boring were made a lot more interesting simply because the professor absolutely loved their subject and would relate it to all sorts of things going on in the news and just, you know, relate it to all sorts of practical things that wouldn’t come up in the curriculum. (Karl)

11. Midwestern Public Research University  Formal classroom techniques  Real world application  Clickers  Out of class techniques  Grad student led discussion groups  Instructor led review sessions  Undergrad led study groups  Formal classroom techniques  Real world application  Clickers  Traditional lecture  Pair work  YouTube clips  Out of class techniques  Grad student led discussion groups  Course websites  Online quizzes  Discussion boards  Podcasts StudentsFaculty Context for Learning  Challenges of large class sizes  Highly motivated students  Collaborative culture

12. Lecture + Other Resources  The approach that Midwestern Public Research University has taken regarding lecture—you know, lots of schools have been working ways to improve lecture. We know that lecture is good for some learning goals; it’s not so good for all sorts of other learning goals. Midwestern Public Research University’s approach has been to figure out what lecture is good for, what does the research say lecture is good for, and do that, and do that as well as we can and then add other resources to the course, to try get at some of those other learning goals. (Professor Eko)  I feel like in lecture you don't really learn that much. Maybe I've learned like 10% of everything that I've learned in lecture. Everything else I get from attending like, the student led study groups, and like, on my own or with friends. (Dylan)

13. Student Study Groups  I think that for me, I really like the study groups, but my study group leader I don't think is very good. The benefit to me is that there's other students there that are taking the class, and so, I'll go over the problems with them, not the study group leader. And so, just sort of having a forced, I guess, way that you have to meet with people two hours every week, is really beneficial to me. (Timmy)

14. Southeastern Public Master’s College  Formal classroom techniques  Tablet PC’s  DYNO  Clickers  Out of class techniques  Online homework  Textbook websites  Formal classroom techniques  Tablet PC’s  DYNO  Real world application  Team teaching  In-class problem solving  Visuals  Think/Pair/Share  Animations  Action research  Out of class techniques  Online homework  Virtual labs  Formal & informal SI StudentsFaculty Context for Learning  Lack of student engagement  Lack of student preparation

15. In-class Technology  Well, my class, we had clickers and the tablet and I don’t think it was, I think it – if the professor took a different approach when he used them it could have been successful, but at the time I don’t think it was, really they made too much of a difference. And as far as the tablets go, I think that’s kind of the same. I don’t really like, I would have preferred the board, the regular white board over my teacher actually doing it on a tablet. It was, it was, I don’t see, I don’t see the benefit of it. (Franny)

16. Lack of Preparation  I find that many of the students just don’t have the preparation. They don’t have the language and so they find it difficult to cope. You can present it, PowerPoint presentation, one-on-one discussion and so on, but if you don’t know the language, it becomes difficult to operate. It means you have to go back and start from scratch. (Professor Norris)

17. Northeastern Private Master’s College  Formal classroom techniques  Workshops  Clickers  Out of class techniques  Online homework  SI sessions  Formal classroom techniques  Workshops  Clickers  Peer instructional model  Visuals  Real world application  Group projects  Out of class techniques  Online homework  SI sessions StudentsFaculty Context for Learning  Special interest housing  Cohort centered course content  Teaching focused  Collaborative culture

18. Emphasis on Experiential Learning  As I said, we are a place that has a deep tradition of experiential learning, deep tradition of quality teaching and most of us have the mindset that we’re teachers. The growth for us or the growing pains is to have a culture which is more blended, which is more the inclusion of the scholarship and the scholarship expectations. (Professor Langner)

19. Workshops  But the workshop’s really helpful ‘cuz we’re, I was in a group with four people, and we were given like 20 problems, and we’d have to finish 15 in the hour. So we kind of split up and did them, and then at the end we’d kind of look through and see which ones we had trouble with, and we’d help each other out…and then, like, everyone would look through it to make sure they had a basic understanding of how everything worked. (Irving)

20. Comparison of Institutions ThemeInstitutions Strong emphasis on teaching Southeastern Private Master’s College; Northeastern Private Master’s College; Western Private Master’s College Strong emphasis on research Western Public Research University Large classesMidwestern Pubic Research University; Southwestern Public Research University Small classesSoutheastern Private Master’s College, Northeastern Private Master’s College Extremely competitiveSouthwestern Private Research University; Southeastern Private Master’s College; Western Public Research University Extremely collaborativeMidwestern Pubic Research University; Northeastern Private Master’s College; Western Private Master’s College SI/study groups part of culture Midwestern Pubic Research University; Northeastern Private Master’s College; Western Private Master’s College

21. Discussion  Students and faculty have varying opinions about the use of innovation and technology  There is no universal technique for all students/faculty  Techniques should include multiple modalities  Professors describe more innovation and use of active learning strategies than students describe experiencing  Innovation is not standard practice  Innovation relies upon efforts of individuals  Innovation is not institutionalized

22. Discussion  Strategies should emphasize:  Student interaction and collaboration  Increased time on task outside of the classroom  Real world application for research and practice  Strategies for improving the experience of introductory STEM courses must consider differences in the context for learning  Competitive vs. collaborative environment  Size of classroom  Student characteristics  Institutional support  Institutional focus-research vs. teaching

23. Contact Info Acknowledgments: This study was made possible by the support of the American Recovery and Reinvestment Act of 2009 through the National Institute of General Medical Sciences, NIH Grant 1RC1GM090776-01. This independent research and the views expressed here do not indicate endorsement by the sponsors. Papers and reports are available for download from project website: http://heri.ucla.edu/nih Project e-mail: herinih@ucla.edu Faculty/Co-PIs: Sylvia Hurtado Mitchell Chang Tanya Figueroa Gina Garcia Felisha Herrera Postdoctoral Scholars: Kevin Eagan Josephine Gasiewski Administrative Staff: Dominique Harrison Graduate Research Assistants: Bryce Hughes Cindy Mosqueda Juan Garibay