1. Reformed Undergraduate Science Courses: A Nationwide Research Project Investigating the Impact on pK-6 Teachers Association for Science Teacher Education (ASTE) International Conference January 14-17, 2010 Sacramento, California TPC 0554594

2. Cheryl L. Mason San Diego State University Dennis W. Sunal Cynthia S. Sunal University of Alabama Dean Zollman Kansas State University *Based on the NASA Opportunities for Visionary Academics (NOVA) Professional Development Program TPC 0554594 National Study of Education in Undergraduate Science* (NSEUS)

3. Corinne H. Lardy San Diego State University Donna Turner Erika Steele Cheryl Sundberg University of Alabama Sytil Murphy Mojgan Matloob-Haghanikar Kansas State University This paper was developed under the National Science Foundation Grant TPC 0554594. The content does not necessarily represent the policy of NSF and should not be assumed as an endorsement by the Federal Government. TPC 0554594 NSEUS Research Assistants

4. Project Goals  Characterize levels of reform in NOVA versus nonNOVA undergraduate science curriculum and teaching (courses & faculty)  Investigate the impact of reformed entry level courses on undergraduate students  Determine short- and long-term impacts of undergraduate science courses on elementary teachers

6. Reformed Course Characteristics  Reflects national science standards  Emphasizes student-centered activities  Utilizes inquiry-based pedagogy  Builds on undergraduate students’ prior knowledge  Incorporates interdisciplinary learning and collaborative approaches

7. The mediocre teacher tells. The good teacher explains. The superior teacher demonstrates. The great teacher inspires. William Arthur Ward The mediocre teacher tells. The good teacher explains. The superior teacher demonstrates. The great teacher inspires. William Arthur Ward

8. Research Study Population States with study Institutions States with Institutions attending workshops but no courses were developed No institution attended workshop or developed reform course

9. NSEUS Timeline Year 1Year 2Year 3Year 4Year 5 Characterize national study population of reformed science courses o Conduct pilot studies with 3 institutions o Collect data from university science and elementary classrooms o Continue to collect data from the selected sites o Complete data collection and analyses from 30 institutions o Plan data collection instruments and protocols o Select pilot study institutions o Conduct final training in research protocols o Select 30 main study institutions o Perform data analyses o Continue to perform data analyses o Disseminate the results and conclusions on undergraduate science course reform o Develop the NSEUS website o Begin to collect data from the national sample of 30 institutions o Give presentations o Submit papers for publication o Continue to present and submit papers for publication o Conduct a national conference on undergraduate science course reform

11. Research Comparison of Entry-level Reformed and Traditional Undergraduate Science Courses v Essential characteristics for meaningful learning v Effects of varying levels of reform characteristics v Cognitive and affective learning outcomes v Long-term impacts on pK-6 teachers

12. Data Collection Instruments v QUANTITATIVE MEASURES CLES TSSI DAST STEBI-A STEBI-B CBATS v QUALITATIVE MEASURES RTOP CoRe PaP-eR Interviews Science Content Achievement (SCA)

13. Administration of Instruments v Pre-site Visit:  Undergraduate Students: DAST and On-line Demographic survey, CLES & TSSI v On-site Visit:  Undergraduate Faculty & pK- 6 Teachers: RTOPs, Interviews, CoRe, PaP-eR  Undergraduate Students: Focus Group Interviews v Post-site Visit: Undergraduate Students: DAST, On-line CLES&TSSI, and SCA  pK-6 Teachers: On-line STEBI-B, CLES, & TSSI

14. v On-site visits to university and elementary classes add richness to the data concerning reform efforts in undergraduate science. v Triangulation of results from interviews and observations provide further elaboration of the quantitative data. v On-site visits are helpful in gaining insights into the methodologies and rationale for how and why science is taught the way it is at various levels. On-site Visits

15. Results (Undergraduate Science Courses)  Differences were found between reform and traditional course instructors in coursework observed.  Reform instructors exhibit deeper understanding of how students typically think about science and modify teaching to match students’ learning needs.  Reform instructors exhibit greater proficiency in knowledge about science teaching.

16. v “The class seems to help change students’ views of science. They don’t hate science anymore and they’re not afraid of it.” v “If you just show them a drawing in the textbook or on the board it is not enough. They have to do it themselves in order to reinforce the material that is being taught.” Impact of Reformed Undergraduate Science Courses (Faculty)

17. v “Before this class I was nervous and afraid of science, but now I have more fun learning and thinking about how to teach it.” v “I failed chemistry in high school so I was nervous. But [this instructor] makes me feel very comfortable. She’s very approachable and she seems like she wants to help us learn. If I want to succeed, I can succeed. She’ll give us the tools.” Impact of Reformed Undergraduate Science Courses (Undergraduate Students)

18. Results (Elementary Science Teachers) Compared to elementary teachers experiencing only traditional science courses, reform teachers: v possess a greater depth of science content knowledge on the concepts taught; v exhibit a more expansive knowledge of how students think about science, and modify teaching to match students’ learning needs; and v demonstrate a greater knowledge of science pedagogy.

19. “You can’t just sit there and lecture as a [science] teacher. The [reform course] really showed me that the most because all of my other [science] courses were really lecture-based. If you don’t get it that way, then in a lecture class you won’t learn the material. I can learn...and really get it when I dig into it and I think that most people are that way. The [reform course] was great because it had a good mix of lecture and activity, so you got the information in multiple ways and see what those concepts actually look like.” Impact of Reformed Undergraduate Science Courses (NOVA Elementary Teacher #1)

20. “The [reform course] especially added to my understanding and made me think about why and how do these concepts work. That’s when my whole thought process about science really changed because until you really question the concepts and think about it on your own (not just memorize) it’s not meaningful. You look at the content differently when you have to teach it to others. You have to know it yourself to be able to explain it to someone else.” Impact of Reformed Undergraduate Science Courses (NOVA Elementary Teacher #2)

21. Impact of Reformed Undergraduate Science Courses (nonNOVA Elementary Teacher) “[At the undergraduate level] I just took science courses and did what I had to do. I had no idea how to relate what I was learning to how I could teach it to others. I now wish that I had more ideas about how to teach science, but I am getting there.”

22. Conclusions  Successful reform ideas are adopted by other faculty.  Collaborative teams sustain course reform over time.  Reform science courses have significantly higher positive classroom learning environments.

23. Conclusions (cont.)  Reform course faculty teach more inquiry- oriented science.  The way undergraduate students perceive former science experiences differs among individuals but not classes.  Undergraduate students’ ideas about the nature and process of science are more informed and positive in reformed classes.

24. Conclusions (cont.)  Graduates of reform courses teach more inquiry science in elementary schools.  Science PCK is more expansive in teachers who had taken reform coursework.  NOVA and nonNOVA teachers emphasize the impact of effective professional development.

26. Overall Comments  Reform efforts are sustainable with dedicated faculty & administrative support.  Professional development activities that reflect reform profoundly affect the teaching of elementary & IHE science.  Undergraduate science experiences affect how students view science on both affective and cognitive levels.

28. We would like to acknowledge the wonderful cooperation and professionalism of the elementary teachers and adminstrators, and the university faculty and adminstrators who have contributed immensely to the success of this project. http://nseus.org http://nseus.org Please visit our website http://nseus.orghttp://nseus.org National Study of Education in Undergraduate Science (NSEUS) TPC 0554594

30. Additional Information The following slides serve as a supplement to the presentation.

31. v How do the levels of reform science course characteristics - learning environments, course structure, pedagogical content knowledge, and collaboration - differ between reform courses (treatment only) and how do these differences relate to the learning outcomes of undergraduate students? (CLES, TSSI, DAST, STEBI-B, RTOP, CoRe, PaP-eR, Faculty and Undergraduate Student Interviews, SCA) v How do the science course characteristics of reform and traditional courses compare in the long-term based on graduated in-service K-6 teachers in their own science classrooms? (CLES, DAST, STEBI-A, CBATS, RTOP, Elementary Teacher Interviews, CoRe, PaP-eR) Research Questions

32. Draw A Scientist Test (DAST) v Intended to provide students with opportunity to: Picture perception of scientists Relate the scientist to their personal environment – lab, etc Consider the ways in which they view scientists relate to their own science beliefs about the nature of science v Includes an illustration plus a short personal narrative (Contributes additional information and confirms evaluator’s interpretation of drawings- Since interviews with each student are impractical) v Evaluator follows a checklist to interpret drawing. Evaluator gives one point for each item present. The higher the score, the more stereotypical the image is.

33. Constructivist Learning Environment Survey (CLES)  Survey is designed to monitor the development of constructivist approaches in the classroom, as perceived from the teachers’ and undergraduate students’ points of view.  5 key dimensions of a “critical constructivist” learning environment are ascertained: Personal relevance Uncertainty of Science Critical Voice Shared Control Student Negotiation  Six items are given for each of the 5 dimensions with possible responses of Almost Always, Often, Sometimes, Seldom, and Almost Never.

34. The Thinking about Science Survey Instrument (TSSI ) v Developed to assess preservice and inservice elementary teachers’ attitudes toward science using a Likert scale. [How well do students’ thoughts about science align with “a commonly presented image of science in the public square?” (Cobern & Loving, 2002, p. 1025)] v Examines individual’s attitudes about “the public place of science with respect to society and culture” (Cobern & Loving, 2002, p. 1020).

35. Science Teaching Efficacy (STEBI-A ) v Survey is used to measure two components of teacher efficacy beliefs(Riggs & Enochs, 1990). Teacher’s self-efficacy (level of confidence in her/his own teaching abilities) Teacher’s outcome expectancy (belief that student learning can be influenced by effective teaching) v Teachers respond to each item on this 5-point Likert scale of Strongly Agree to Strongly Disagree.

36. Science Teaching Efficacy (STEBI-B ) v Survey is designed to measure the self- efficacy of pre-service elementary science teachers. v Modified version of STEBI-A v Administered to education major undergraduate students (post-test only)

37. Reformed Teaching Observation Protocol (RTOP) v Developed by Arizona Collaborative for Excellence in the Preparation of Teachers (ACEPT). v Protocol is designed to measure quantitative characterization of the degree to which a science classroom is “reformed” based on the national standards for science education. v Observers have a list of characteristics that they rate 0- 4 (never occurred to being very descriptive). v RTOP is found to have a high inter-rater reliability. v For NSEUS, all observations take place during site visits.

38. Interviews v Faculty interviews focus on their experiences related to developing and teaching the science course. v In-service elementary school teacher interviews focus on the purpose and rationale for teaching the observed lesson and how it relates to other lessons recently taught in science. v Student focus group interviews of 5-6 students ascertain students’ opinions about the science course, science, and science education. v All interviews take place during site visits conducted during the middle of the semester.

39. Content Representation & Professional and Pedagogical Experience Repertoire ( CoRe & PaP-eR) CoPA (CoRe & PaP-eR) is used to capture and portray pedagogical content knowledge (PCK) PCK - knowledge related to ways to best formulate and present specific concepts of a subject so that they are comprehensible to others (in particular, students)

40. Context Beliefs About Teaching Science (CBATS) v Survey instrument is designed to assess teachers’ beliefs about the potential influence of specific environmental factors on their science teaching behaviors (Lumpe, Haney, & Czerniak, 2000). v For each of 26 environmental factors, teachers are asked to rate: How much they agree that the factor would enable them to be an effective teacher of science from Strongly Agree to Strongly Disagree How likely it is that the factor will occur in their school from Very Likely to Very Unlikely

41. Science Content Assessment (SCA)  Content achievement measure for undergraduate university science course students  Science content assessed as a post-test only v Criterion-based rubric measures are used to compare scores across courses and disciplines v Content questions are devised to explore students’ reasoning while they are using the scientific knowledge gained in the courses.

42. v Questions are open-ended with a distinguishing feature of applying the recently-learned concepts in a new context. v Questions require that the students recognize relevant facts and concepts and their interrelationships which should be generalized to their knowledge of theories and principles. v Students should rethink their conceptual schema, and find associations among relevant concepts by inductive and deductive reasoning. SCA (cont.)