Experimental Comparison of Inquiry and Direct Instruction in Science Funded by the National Science Foundation’s Interagency Education Research Initiative (IERI/NSF ) Award # Dr. William Cobern Department of Biological Sciences Dr. David Schuster and Betty Adams Department of Physics The opinions expressed in this report are the sole responsibility of the researchers

Background to the Study Inquiry-based or direct instruction in science? Long-standing educational and political debates Pendulum swings Direct Prevalent in past Inquiry National Science Education Standards Constructivism? A theory of learning – either way

We know about Experientially-based √ Active-engagement √ But these can occur in both Inquiry and Direct Both can be ‘hands-on’ and ‘minds-on’ if so designed. Research Evidence?

Our Question Is NOT whether active-engagement experiential-based learning of science is more effective than passive non-experiential learning But IS whether an inquiry approach or a direct approach to experientially-based instruction is more effective for science concept development …when both approaches are expertly designed and well executed This is the question our research addressed.

Inquiry-based Science Instruction There are now many inquiry-based curricula and lessons Their evaluations show these to be ‘successful.’ But compared to what? Assumption is that the merit/superiority of inquiry science instruction is well-established However, meta-analysis of many such studies (Educational Development Council, 2007) found Not sufficiently unconfounded to draw the inferences Research rigor has declined from

Threats to Validity of Studies Few controlled comparative studies Pitted against poor or nebulous ‘traditional’ instruction Few use randomized assignments or quasi-experimental controls for differences Evaluations not independent of developers or researchers Insufficient specification to allow replication Fidelity: implementation not compared to intended instruction

Direct Instruction Well-designed direct instruction Some relevant work: Kirschner et.al. – Why mimimally-guided instruction does not work Klahr & Nigam – ‘Direct’ vs. ‘discovery’ Ausubel – Issue is meaningful learning vs. rote learning

Instructional Context 5 experienced middle school science teachers 8 th grade students responding to flyers sent home by school district offices 2 week voluntary summer program with students randomly assigned to treatment group In-class instruction only; intrinsic motivation

Tenets Specificity Fidelity Objectivity Transparency Research Framework

Specificity Inquiry or Direct?

Specific about… A. A.The meanings of Inquiry and Direct B. B.The nature and design of instructional units C. C.The assessment

A. What do we mean by ‘Inquiry’ and ‘Direct’ Instruction? Single-word descriptors are vague, ambiguous and open to (mis)interpretation Such as “inquiry, direct, discovery, didactic, conventional/traditional, lecture, active- engagement, hands-on …” We need to specify models for each mode

1. Model of Guided-Inquiry Science Instruction PHASEDESCRIPTION Exploration Observing phenomena, posing questions, exploring, investigating … (Science-in-the- making) Concept Formation Guided formation of relevant concepts. Formulation of principles, laws or models Concept Application Applying the concepts and principles to new situations. To solve problems, explain and predict. The Learning Cycle Karplus Cycle

Karplus Cycle Diagram Epistemology: Exploration leads to concept formation Inductive aspects (Note: 5-E Learning Cycle has Karplus at its heart)

2. Model of Direct Instruction Epistemology: Presentation then illustration/confirmation Deductive PHASE DESCRIPTION Concept Presentation Presentation/reception of facts, concepts, laws etc. Ready-made-science. Explanation & clarification Illustration & Confirmation Illustrate with examples, demonstrations. Verification experiments. Concept Application Applying the concepts and principles to new situations. To solve problems, explain and predict. The Direct-Active Cycle

No Caricatures or Straw-man Comparisons Direct caricature: Pure didactic presentation with passive reception/absorption Inquiry caricature: Open discovery (unguided chaos) Hands-on alone does not make it inquiry

Lessons as Composites Lessons have many constituent parts All lessons are composites Never 100% inquiry or direct throughout To attempt this would be poor instruction generally

The essential difference? What then is the essential difference between Inquiry and Direct? “How students come to the concept” Through exploration or are they told upfront? This is the Active Agent that differs between modes Lessons may have other Common Constituents

‘Active Agent’ Example from Dynamics How do students come to the force-motion law? i. By exploring system behavior and proposing a law, OR ii. By being given the law and confirming system behavior. Added value, beyond just content knowledge? Besides acquiring knowledge of the law, what else do students learn or gain, in one mode or the other?

B. THE INSTRUCTIONAL UNITS The heart of the study  – Two important science topics – Significant conceptual development sequence Each written in Inquiry and Direct modes Student and teacher booklets for each

Two Approaches (Inquiry/Direct) Essential Similarities  Content  Objectives/goals  Equipment/materials  Practice problems  Assessment Essential Differences  Sequence  Evidence before claims  Claims before evidence  Teacher’s role  Asks…  Tells…  Student’s experience  … finds out  … confirms

INQUIRY (investigative approach) Exploration Guided Concept Development Application Student learns by experiencing scientific inquiry, guided toward developing and applying scientific concepts & laws (based on Karplus Learning Cycle) DIRECT (confirmatory approach) Delivery/Explanation Verification Application Student learns by receiving, verifying, and applying scientific concepts & laws Inquiry and Direct Instructional Models

C. THE ASSESSMENT Nature and quality of assessment is crucial The project data depends on it Tests understanding of the main science concepts Problem-based Ability to apply concepts in (relatively) new situations Bloom taxonomy levels 2 and 3 Conceptual MCQ form ‘Assessment as curriculum’ - examples as indication

Assessment: examples Prediction question

Assessment: examples Explanation question

Assessment instruments 22-question set for each topic unit 2 to 4 questions on each central concept Identical Pre- and Post-tests Ascertain gain

“Prepare and Verify” Prepare Monitor Evaluate Fidelity

Teachers blind to the assessments — no teaching to the (known) test Independent evaluators marked the assessments blind to student group assignments Independent observers rated teaching: to nature and degree of inquiry or direct Objectivity

Transparency of research: We make available details of what the research involved, thus facilitating possible replication. Our work is made available in detail at Transparency

Analysis Data aggregated over 2 years of trials (2007, 2008) N=180 students (72 Direct, 108 Inquiry… to date) Pre- and Post- assessments yielding each student’s raw percentage gain scores Student gain scores normalized Comparisons of both raw and normalized gain scores across various groups (modes of instruction, teachers) (t-test, ANOVA, α=.05)

Findings (Example of raw % gains) Total LIGHT gain 13.6%, SD=15.3, is statistically significant ( t (179)=11.934, p <.001), with an effect size (Cohen’s d) of.69 (effect size of 1.4 for normalized gain) Gain 14.1%, SD=16.4, is statistically significant ( t (107)=8.925, p <.001), effect size (Cohen’s d).67 Total DYNAMICS gain 9.7%, SD=13.5, is statistically significant ( t (179)=9.655, p <.001), with an effect size (Cohen’s d) of.54 (effect size of 1 for normalized gain)

Findings (Light Unit Summary)

Findings (Dynamics Unit Summary)

Findings LIGHT Unit – Two Trials (2007, 2008) Direct vs. Inquiry normalized gain mean difference of 3.8% was not statistically significant ( t (178)=.755, p =.451) (std. error diff. 5.1, effect size Cohen’s d=.12) Ann (Direct) and Tom (Inquiry) had a mean difference in raw gain of 7.6%, which was statistically significant ( t (73)=2.132, p =.036), but the mean difference between their normalized gain scores was not ( t (73)=1.857, p =.067) DYNAMICS Unit – Two Trials (2007, 2008) Direct vs. Inquiry mean difference of 3.1% was not statistically significant ( t (178)=.717, p =.474) (std. error diff. 4.4, effect size Cohen’s d=.11)

Conclusions Given natural class and teacher variation in realistic classroom situations, good inquiry and direct instruction led to similar understanding of science concepts and principles in comparable times. Thus advocacy of either method cannot be based on science content acquisition alone. Inquiry-based instruction offers significant potential advantages for science education by modeling scientific inquiry during concept learning: these concomitant benefits would need to be studied in research for that purpose. However for science concept understanding, expertly designed instructional units, sound active-engagement lessons, and good teaching are as important as whether a lesson is cast as inquiry or direct.

Funded by the National Science Foundation’s Interagency Education Research Initiative (IERI/NSF ) Award # The opinions expressed in this report are the sole responsibility of the researchers.

Controlled comparative study Treatment and control groups Actual classroom situations

General Design Considerations Substance Coherent development Conceptual Challenging Experiential Standards Learning objectives Clear main focus ‘Hands-on / minds-on’ Engagement Reinforcing examples Application Reflection Length?