1. Scientific Research in Education
NRC Committee on Scientific Principles in Education Research
Editors:
Richard J. Shavelson
Lisa Towne

2. Why Are We Here? Castle bill, fall 2000
Office of Educational Research and Improvement re-authorization
OERI disbanded in 2002
Education Sciences Reform Act
Now Institute of Education Sciences
Encourage self-reflection

3. Is That All? Thought about a federal agency
Did NOT evaluate current research or techniques

4. Scientific Method “Guiding Principles”
Pose significant questions -- empirical investigation
Link research to relevant theory
Use methods w/ direct investigation
Chain of reasoning
Replicate and generalize
Professional scrutiny

5. Properties of Education
Values and Politics
Differing views on human potential
No single goal to current research
Personal values too involved
Decide on one goal, not “best” or “final” goal
Governmental changes

6. Properties of Education
Human Volition
Individual desires confound experiments
Parents are whiny and hate science
Think of the children
Think of them!
Mobility of U.S. population
10%/year ?

7. Properties of Education
Variability of Education Programs
“Tinkering towards Utopia”-Tyack & Cuban 95
State standards
Organization
Group-Class-Grade-School-District-County-State-Federal
Cultural Diversity

8. Properties of Education Research
Multidisciplinary
Integrate/correlate from different fields
Advances depend on advances in other fields
Development of training programs is hard

9. Properties of Education Research
Ethical Considerations
Cannot create education-free control
Cannot hide purpose (for double-blind)
Somebody think of the children!
Reliance on Practitioners
Recent shift to long-term partnered research

10. Designing Research Question dictates design Three important facets:
Description: what is happening?
Cause: is there a systematic effect?
Mechanism: why/how is it happening?

11. Description Collect raw valid data Random sampling (surveys and tests)
Reproducible valid results
In-depth case studies
1970s: 23 college women
Career in major, career outside major, housefrau
Schoolwork predictor

12. Causation Find causal relationships between variables
Isolate variables on random surveys/tests
Non-random tests may generalize better
Real-world education not dictated randomly
Harder to validate - “selectivity bias”
Model-fitting
Salary vs. dropouts

13. Mechanism Bring all techniques to bear on “how”
10-year study on Catholic schools
Three models:
Sector (private spiritual nature)
Compositional (composition of student body)
School (feature of school operation)
Conclusion: “coherence” (sense of community)

14. Mechanism: When Theory is Weak
College women study
Why did nontraditional majors not have careers?
Design study of ratio reasoning
“Real” problems (ramps, exchange rates)
Videotape of classroom activities
Compared to other classrooms over 3 years
Produced refined theoretical model

15. Designing An Ideal Federal Agency
An Exercise in Publicly Funded Common Sense
Staff with Scientists
Structures to Guide Agenda
Insulate from Politics
Balanced Research Portfolio
Adequate, Consistent Funding
Invest in Private Research Infrastructure