1. Teaching and Learning Concepts of Scientific Evidence: A Design-based Research and Development Study Susan Kirch (PI) 1,3, Catherine Milne (coPI) 1, Anna Stetsenko (coPI) 2 Kara Naidoo 1, Laura Paskell-Brown 2 1 New York University - Steinhardt School of Culture, Education and Human Development, 2 The Graduate Center of the City University of New York, 3 Correspondence to: susan.kirch@nyu.edu Rationale for Teaching and Learning Concepts of Evidence The use of evidence is central to scientific knowledge and scientific thinking. Mastery of this cultural tool is essential for developing a scientific worldview. Evidence is using observations to form, support, and present an explanation (i.e. conclusion, judgment, theory), often in comparison with alternative explanations Project Goals Context, Design Decisions, and Preliminary Analysis Context K-8 Elementary and Middle School (enrlmnt. ~800) School Poverty rate: 75% (NYC-DOE) Students in temporary housing 2008-2009 (N=13, 2%) Recent immigrants 2009-2010 (N=24, 3%) Classroom participants (2009-2010) N=43 Students receiving special education, (N=20, 47%) Students receiving ESL services (13, 30%) Research interview participants (2009-2010) N=31 Students receiving special education, (12, 39%) Students receiving ESL services (9, 29%) Classroom participants (2010-2011) N=126 Students receiving special education, (22, 17%) Students receiving ESL services (36, 29%) Research interview participants (2010-2011) N=106 Students receiving special education, (16, 15%) Students receiving ESL services (24, 23%) STEME Content used to study relationships between claims and evidence Cycle 1 (3rd grade): Plant Biology and Seed Storage (historical and contemporary developments in science) Cycle 2 (3rd grade): Science of Sound (historical developments in science) Cycle 1 (4th grade): Student choice (physics, astronomy, biology, animal behavior and cognition; contemporary developments in science from EurekAlert; Henry Fountain’s Observatory column) Cycle 2 (4th grade): Colony Collapse Disorder case study (contemporary developments in science) Assessments Piloted How do students use and understand evidence? Nature of evidence interview (TOC) Use of evidence tasks (TOC) Scientific thinker identity interview (TOC) Interaction Analysis of video (O, TOC) Do students understand module materials? Are they interested and engaged? Unison Reading Protocol (O, TOC) Running records during unison reading (O) Feasibility interviews (O, TOC) Interaction Analysis of video (O, TOC) What have students learned about scientific evidence? How can module materials be improved? Pre-module assessments repeated (see 2) Interviews with teachers (O, TOC) Interviews with students (O, TOC) Quality of module survey/interview (O) Interaction Analysis of video (O, TOC) Member checking and co-analysis (O, TOC) At the level of operation (O) At the level of theory of change (TOC) Instructional Design Classroom-based analysis 2.To theorize the development of evidence as a cultural tool and utilize an Activity Theoretical framework for studying teaching-learning (e.g., Stetsenko, 2009 ). 3.To determine whether elementary students can learn concepts of scientific evidence when provided a coherent, contiguous course of study of evidence. 1. To develop instructional tools for teaching and learning concepts of scientific evidence through a design- based research process (e.g., Cobb, 2001 ). DRL#0918533 Curriculum design evolution and outcomes Preliminary Analysis Design Cycle 1a (3rd grade) Student exploration  Science Conference  Studies of claims and evidence through case studies on the problem of saving seeds in seed banks: Debate of the 2,000-year old seed and the Challenge of Storing Seeds Analysis Cycle 1a (3rd grade) Students succeeded in producing knowledge claims through investigations and defended their conclusions at the Science Conference. Although studies of claims and evidence were explicit, the next design goal was to involve students in endorsing and seeing the relevance of learning about claims and evidence. Design Cycle 1b (4th grade) Claim and evidence in everyday life  Convincing evidence scenarios  Author claims and evidence  Student wondering and questions leading to student research and debates (a major design breakthrough) Contemporary Questions in Science Analysis Cycle 1b (4th grade) Learning about claims and evidence was clearly accepted as a reasonable learning goal on the first day of activities. Overall, student interest and motivation to identify and evaluate evidence was much greater than in the 1a cycle. The next design goal was to determine how to motivate all students to see the utility of learning about evidence and involve them as researchers of knowledge. Design Cycle 2 (3rd and 4th grade) Begin by engaging students as researchers and producers of knowledge whose research questions are: (1) when we say we know something, what does that mean? (2) If someone says they know something, how could we explore how they came to know it and why they think it’s true? (a major design breakthrough) What is Knowledge? (all); History of the Science of Sound (3rd); Pollination Crisis! The Missing Honeybee Mystery (4th) Where does evidence come from?“Pre”Post An event (witnessed/inferred action)(7)*******(1)* “Us” a human-made knowledge object (7)*******(15)*************** A natural source (object)(2)**0 Everywhere0(1)* Preliminary Findings and Implications Teachers report students are more highly motivated to persevere through difficult text and challenging questions when they conduct research with a purpose. 3rd and 4th graders were able to describe relationships between observations, claims and evidence. Participants’ views of evidence as an object shifted to include evidence as a process. Participants came to appreciate that evidence is used for convincing others and strengthening a scientific explanation rather than merely for proof under the law. Participants shifted from seeing authoritative text as their primary source of information to seeing themselves as capable of conducting similar investigations. Students who produce evidence in support of or against a particular claim (from any source) can experience elementary school science in a profoundly meaningful and purposeful way compared to methods that lack reflection on the process of knowledge production in science. Current retrospective questions How do students view evidence and how did these views evolve during the project period? How do students use evidence and how did their use evolve during the project period? When do students think to use evidence? 1 Purple stars (*) show students whose responses did not shift between analytical categories in the post-interview; Blue stars (*) show students who shifted from one analytic category to another between the pre- and the post-interview (e.g., from describing evidence as a thing to describing evidence as in a relationship with a claim). Table 1. Pre/post interview cycle 1, 4th grade, N=17 How do you decide to trust what you see, hear, or read? Students’ Research Associate Responses Student Responses