Why SBR? Schools have largely based their practice on “tradition, superstition, professional wisdom, and anecdotal stories.”
Too much money has been invested in education and wasted in programs that do not get the intended results. –Valerie Reyna, Deputy Director Office of Educational Research and Improvement
Primary Goal of SBR Ensure that programs have been proven effective and are more likely to benefit students when used.
NCLB Legislation Goals One hundred percent proficiency for ALL students in 12 years. The method for achieving this goal is data-driven decision- making and transforming education into an evidence- based profession that utilizes SBR.
NCLB Defines Scientific Inquiry Use scientific method with an emphasis on experimental control (or comparison) groups Replication of results, using multiple studies by different investigators What makes research scientific is not the motive for carrying it out, but the manner in which it is carried out.
Before you Request a PO Know Your Research! http://www.arp.sprnet.org/ default/admin/FORMS/ma terials_requested.htmhttp://www.arp.sprnet.org/ default/admin/FORMS/ma terials_requested.htm
Ability to generalize results from one sample to others in the general population Fulfillment of rigorous standards with an emphasis on peer review (Peer Reviewed Journals) Convergence (or consistency) of results between studies.
Research 101 Two types of research 1.Collecting numbers – quantitative 2.Collecting observations – qualitative Three purposes for research 1.To assess the implementation and replicability of the reform practice or program. 2.To test a theory behind a practice or program. 3.To measure impact, effectiveness of the practice or program.
HINT The abstract will tell you the type and purpose of the study.
Different Research Designs for Different Purposes Implementation and replicability –Type – Quantitative or Qualitative Theoretical Base –Type -- Quantitative or Qualitative Evidence of Effectiveness –Type -- Quantitative
Implementation and Replicability Asks: –How was practice or program implemented? –In what setting? –Under what conditions?
Theoretical Base Asks: –What is the theory behind this practice or program? –What are the theory’s guiding principles? –What does the theory explain?
Evidence of Effectiveness Is there evidence showing that this practice or program improves student achievement? Really, the only method approved by NCLB --Core of SBR
Hierarchy of Evidence Case Studies Correlational study without statistical controls Correlational study with statistical controls Quasi- experimental Randomized Trial Valerie Reyna, OERI
Systematic Rigorous Methodology Gold Standard –Random Assignment (Experimental) Silver Standard –Control Group-Non Random (Quasi- experimental) Bronze Standard –Case Study Lead Standard –Testimonial, opinion, Intuition, Small sample, Selective criteria
Scientifically Based Research: Gold Standard Experimental study –Research Type – Quantitative Determines potential between practice or program and student achievement
Controls all of the following: –Environment –Intervention (practice or program) –Subject selection (random sampling) Shows change in outcome as a result of treatment
Promising Research: Silver Standard Quasi-experimental study –When it is not possible to control for at least one of the three elements of experimental design: Environment Intervention Assignment to experimental or control groups –Cannot determine causality
Bronze Standard Supplemental Research Reports the way things are Includes both quantitative and qualitative data
Descriptive – data summarized and simplified. Effects on student achievement cannot be attributed directly to the intervention. Hey! But this is More Fun to Read! Bronze Standard Supplemental Research
Whose Research Findings Should I Use? Findings/Conclusions Based on: –Multiple studies –Independent studies –Peer Reviewed –Published –Strong criteria for SBR
Available Evidence Become a savvy consumer of research. Only 5% or less of available research evidence is at Gold SBR Standard
Quality and Quantity Challenge Where you go to find research is important. Debate, discussion and the review process are important.
Peer Reviewed Journals Not Joe Bob’s Blog Find quality of works while using the research currently available. Context matters. Need for accumulation of studies in order to establish scientific certainty.
Components of all Research Studies Abstract Objective (problem/purpose) Research procedure/methodology Findings Discussion Section Conclusion
Professional Wisdom “…school leaders will need to rely on the best available empirical evidence and some degree of professional judgment in creating their programs.” CSR Program Guidelines
Using Professional Judgment Being a critical consumer Examining research to determine whether it is relevant to your context. Synthesizing findings across studies Incorporating the evidence into the decision-making process.
Research-Based Decision Process Step One: Gather relevant, quality research –Identify research that is: Of high quality Current (within 5 years Focused on your interest Researched in schools/programs like yours –Make sure search is extensive to capture all evidence and analysis
Research-Based Decision Process Step Two: Interpret Research for your Context Synthesize findings gathered Revisit needs Revisit and refine focus questions Narrow the range of possible solutions/interventions Make a judgment of the effectiveness of an intervention from the studies.
Research-Based Decision Process Step Three: Make your case –Make links between Identified needs Programs and practices that can address these needs directly Evidence from the highest quality research available to support the effectiveness of these interventions on student achievement.
More information National Clearinghouse for Comprehensive School Reform www.goodschools.gwu.edu What Works Clearinghouse w-w-c.org e-mail updates: firstname.lastname@example.org
Final Point Scientific research in education is about accumulating knowledge. Using the Knowledge is about Educating Successful Students. NRC, Scientific Research in Education