1. Engineering Education Larry D. Gruppen, Ph.D. University of Michigan Rigorous Research

2. Describe scientific research standards Compare and contrast engineering and educational research Identify a few of the peculiar challenges of educational research Objectives

3. Standards –Engineering Research (  ) –Educational Research (er) Educational Research in the Professions –Science and Application –Theoretical Frameworks –Research Areas Comparing the  er Agenda

4. ?What are the guiding principles for rigorous research in your “regular” (STEM) discipline? →Take a few moments individually to list (in your journal) the qualities and characteristics of rigorous research in your discipline. →As a group, develop a list of research standards in STEM. Engineering Research

5. 1.Question: pose significant question that can be investigated empirically 2.Theory: link research to relevant theory 3.Methods: use methods that permit direct investigation of the question 4.Reasoning: provide coherent, explicit chain of reasoning 5.Replicate and generalize across studies 6.Disclose research to encourage professional scrutiny and critique National Research Council, 2002 Guiding Principles for Scientific Research in Education

6. Curriculum DesignTeachingAssessmentOutcomes Problem-basedClinicalCompetenceExpertise learning teachingKnowledgeCognition IndependentEducationalSkillsCompetence learning technologies AttitudesEthical reasoning Professional socialization Examples of research areas in professions education 1. Significant questions that can be investigated empirically

7. Learning theories Psychometrics Adult Development Moral Development Socialization Cognition Expertise Social context of education Examples of conceptual frameworks in education 2. Link research to relevant theory

8. Quantitative methods Tests Surveys & questionnaires (defined response) Faculty or peer ratings Qualitative methods Focus groups Interviews Observations Gathering the empirical data 3. Methods for Direct Investigation (examples)

9. What makes a convincing argument Theoretical foundation - results fit with expectations Strength of observed relationships Elegance - conciseness, simplicity, explanatory power Elimination of alternative explanations Study design Confounding variables Interpreting the results 4. Reasoning

10. Common constraints and possible responses Control group - logistical and ethical limitations Delayed intervention cross-over design Non-equivalent control group design Randomization - ethical constraints Self-selection Historical group assignment Sample size - often small, limited by class size Multiple iterations of intervention Collaborative studies - multiple institutions, collaborators 4a. Study Design in Educational Research

11. Time on task Maturation Prior experience Motivation Self-selection Individual capabilities - dexterity, visualization, abstraction, communications, etc. Affective - ‘math aversion,’ computer-phobia 4b. Confounding Variables

12. 5. Replicate and Generalize Setting the results in a larger context MUST know the literature Strict replication is rare in educational research Replication with extension - to new topic, setting, learners, etc. Going beyond your individual study

13. Scholarly journals Conference presentations Peer-review is the core issue One of the few quality controls we have 6. Disclose

14. Basic ScienceLinking ScienceApplication  Physics MechanicsMechanical Engineering Civil Engineering er Psychology (Learning Theories) Instructional Theory Instructional Design Theory and Practice

15. ? Compare  and er → As a group, list the similarities and differences between research in engineering and research in education. Use the following matrix to guide your discussion. Comparing  and er

16. Engineering ResearchEducational Research Purpose Subject of Inquiry Research Context Values Theoretical Perspectives Research Methods & Tools Measurement Techniques Threats to Validity Ethical Considerations Evaluation of Research PROCESS Evaluation of Research OUTCOMES

17. How comfortable are you with the “messiness” of educational research? What is appealing about it to you? What do you find bothersome? Why? Journaling Questions