1. National Study of Education in Undergraduate Science: 2006-2012 --What Was Learned Cheryl L. Mason, Ph.D. San Diego State University Dennis W. Sunal, Ph.D. The University of Alabama Cynthia S. Sunal, Ph.D. The University of Alabama Cheryl L. Mason, Ph.D. San Diego State University Dennis W. Sunal, Ph.D. The University of Alabama Cynthia S. Sunal, Ph.D. The University of Alabama

2. NOVA Project NASA Opportunities for Visionary Academics was a project created to develop and disseminate a national framework for enhancing education of pre- service teachers in science, mathematics, and technology. The NOVA Professional Development Model resulted in university faculty developing the skills and knowledge to reform their undergraduate science courses.

3. Why Undergraduate Science? These courses influence pre-service elementary teachers conceptions of the nature of science and how science should be taught.

4. Why Undergraduate Science? (cont.) Need to change common features that turn off students from taking additional science courses and/or considering science as a career option: o Lack of relevance to their lives o Memorization of factoids rather than conceptual understanding o Emphasis on competition rather than collaboration in the classroom o Focus on algorithmic problem-solving o Passive student roles

6. National Study of Education in Undergraduate Science (NSEUS) Multi-year National Study Goal: Investigate the impact of undergraduate course reform, as a result of the NOVA Project, on student short-term learning outcomes – all majors. student long-term learning outcomes - inservice elementary teachers of science. Research Question: How do undergraduate entry- level science courses, differing in levels of reform, affect student learning outcomes?

7. Research Sub-questions Does faculty professional development change undergraduate science teaching practices? How do reformed science course elements differ from traditional course elements? What are the essential elements of an entry level reformed undergraduate science course? How do the differing levels of course reform impact the short-term learning outcomes of undergraduate students, and long-term outcomes for practicing K-6 teachers?

8. NSEUS National Study Sample Description of Institutions (Study Sample N=20) 62% MA/MS 26% Doctoral 12% BA/BS 26% Minority

9. NSEUS Research Model

10. Data Collection Faculty, undergraduate students, and pre-service teacher survey instruments Content analyses of course materials Multiple site visits including classroom observations and interviews with faculty and teachers, along with student focus group interviews Undergraduate students science achievement and affective measures Experimental and Control groups were compared.

11. Data-gathering Measures Reformed Teaching Observation Protocol (RTOP) (Sawada, Turley, Falconer, Benford & Bloom, 2002) PCK Content Representation (CoRe) &Pedagogical and Professional experience Repertoires (PaP-ers) (Loughran, Mulhall & Berry, 2004) Classroom Learning Environment Survey (CLES) (Taylor & Fraser, 1991, 1997) Science Teaching Efficacy and Beliefs Instrument (STEBI A & B) (Riggs & Enochs, 1990) Thinking About Science Survey Instrument (TSSI) (Cobern, 2000) Student Content Achievement (SCA) (NSEUS Team, 2006)

12. Research Model Comparisons

13. Summary Findings Comparison Set 1 Initial courses vs. Planned reformed courses Analysis of NOVA funded proposals found differences in five overarching elements (themes) in course descriptions from initial to planned reformed. Learning environment Course structure and focus (lab & lecture integrated) Pedagogical content knowledge (PCK) Alternative assessments Beliefs about teaching and student learning

14. Summary Findings Comparison Set 1 Initial courses vs. Planned reformed courses (cont.) Elements of planned reformed courses reflected national science standards. emphasized student-centered activities. utilized inquiry-based pedagogy. built on students prior knowledge. integrated interdisciplinary instruction and collaborative approaches to learning. embedded assessments.

15. Summary Findings Comparison Set 2 Planned courses vs. Implemented reformed courses Courses, once reformed, continue to be offered long term. Reform clones exist within and outside of the department in ½ of the institutions. Collaborative (tenacious) teams play a vital role in developing and sustaining reformed courses. Common characteristics in the courses are related to national science standards. Inquiry-based instructional methods and learning goals dominate course descriptions.

16. Summary Findings Comparison Set 3 Reformed courses vs. Short-term student outcomes Students experiencing higher levels of reform in their undergraduate science course rated and described their classroom learning environment significantly more positive. had higher achievement on the content tests. demonstrated higher-level thinking skills. showed fewer misconceptions about the science concept(s) tested.

17. Summary Findings Comparison Set 4 Reformed courses vs. long-term teacher outcomes Elementary teachers who had experienced higher levels of reform positively contrasted in their observed teaching of science in elementary classrooms, only in specific contexts. differed in their science pedagogical content knowledge (PCK). exhibited a greater depth of science content knowledge on the concepts taught. demonstrated a knowledge of how students think about science and modified their teaching to match students learning needs.

18. NSEUS Study Answered These Questions What level of PCK is needed for faculty to be effective in reforming undergraduate classes? How is inquiry teaching at the undergraduate level demonstrated? What elements are effective in science course reform? What quality and quantity of reform is needed at the undergraduate level to show significant student achievement gains (comprehension of science concepts)?

19. What Was Learned Data analyses emerging from the project indicate that reform curriculum is possible, and has positive effects on participants. Patterns suggest common strategies for planning, implementing and sustaining reform coursework, including overcoming perceived and actual barriers.

20. What Was Learned (cont.) Reformed science courses have significantly higher positive classroom learning environments. Reformed course faculty are more likely to engage students in using inquiry-oriented science, and demonstrate a high level of PCK. Collaborative faculty teams develop and sustain course reform over time. Successful reform elements are adopted by other science faculty.

21. What Was Learned (cont.) Students experiencing higher levels of reform had greater science achievement and demonstrated higher-level thinking skills. Undergraduate students ideas about the nature and process of science differed among individuals but not always classes. Graduates of reformed courses reflect a higher level of pedagogical and science content knowledge (PCK) while teaching science in elementary classrooms.

22. Rote Memorization vs. Conceptual Understanding

23. Conclusions Faculty professional development activities that reflect reform profoundly affect the short- and long-term learning outcomes of undergraduate students. Reform efforts are sustainable with dedicated collaborative faculty & administrative support. Undergraduate science course experiences (context) affect how students comprehend science on affective and cognitive levels. Faculty professional development activities that reflect reform profoundly affect the short- and long-term learning outcomes of undergraduate students. Reform efforts are sustainable with dedicated collaborative faculty & administrative support. Undergraduate science course experiences (context) affect how students comprehend science on affective and cognitive levels.

24. Conclusions (cont.) A significantly high level of reform, both in quality and quantity, is required to develop significant gains in short- and long-term student outcomes. In-service teachers who experience reformed undergraduate science courses have a better understanding of how to teach science to elementary students. A significantly high level of reform, both in quality and quantity, is required to develop significant gains in short- and long-term student outcomes. In-service teachers who experience reformed undergraduate science courses have a better understanding of how to teach science to elementary students.

25. I want to recognize the concerted and long-dedicated efforts of: Dennis W. Sunal, Ph.D. – The University of Alabama (PI) Cynthia S. Sunal, Ph.D. – The University of Alabama (Co-PI) Dean Zollman, Ph.D.- Kansas State University (Co-PI) Corinne Lardy, Ph.D.- San Diego State University Donna Turner, Ph.D.- The University of Alabama Erica Steele, M.S.- The University of Alabama Sytil Murphy, Ph.D.- Shepherd College Mojgan Matloob-Haghanikar, Ph.D.- Winona State University Acknowledgements

26. http://nseus.org __________________________________________________________________ Work on the research project was supported by a grant from the National Science Foundation in Washington, D.C., ESI-0554594, titled Undergraduate Science Course Reform Serving Pre-service Teachers: Evaluation of a Faculty Professional Development Model. The opinions expressed in this paper are those of the authors and do not necessarily reflect those of the Foundation. Correspondence should be sent to: Dennis Sunal, dwsunal@bama.ua.edudwsunal@bama.ua.edu __________________________________________________________________ Cheryl L. Mason cmason@mail.sdsu.edu San Diego State University Dennis Sunal and Cynthia Sunal dwsunal@bama.ua.edu cvsunal@bamaed.ua.edu Dean Zollman dzollman@phys.ksu.edu Kansas State University TPC 0554594