1. National Academy of Engineering Frontiers of Engineering Education
Learning Panel
Sheri Sheppard
Karl Smith
Wendy Newstetter
Barbara Olds
National Academy of Engineering
Frontiers of Engineering Education
November 2009

2. Session Objectives Participants will be able to
Describe key features of the Backward Design process – Content (outcomes) – Assessment - Pedagogy
Elaborate on why some things you do work well in promoting student learning
Apply a few new strategies for assessment and instruction to classroom practice

3. What’s up… Framing the panel: POV1: Learning takes Action (Karl)
POV2: Learning takes Feedback (Barbara)
POV3: Base Learning Systems on Design Principles (Wendy)

4. Overarching or Ancillary Goal
Framework #1: Backward Course Design
Overarching or Ancillary Goal
(for a specific group of students)
Skills
(what students will need to be able to do in order to attain this goal)
Content
Class Format
Assessment Format
Syllabus
Does your syllabus share with your students the thinking process that you followed to design the course?

5. Education as a “Cognitive Apprenticeship”
Framework #2:
Education as a “Cognitive Apprenticeship”
Modeling
“representations of practice”
Scaffolding
“providing support”
“deliberate, planned feedback”
Coaching
“removing scaffolding with growth”
Fading
Collins, Brown, Newman, 1989, pg

6. National Academy of Engineering Frontiers of Engineering Education
Learning Takes Action
Karl A. Smith
Engineering Education – Purdue University
Civil Engineering - University of Minnesota -
National Academy of Engineering
Frontiers of Engineering Education
November 2009

7. CAP Design Process Flowchart
Integrated Course Design (Fink, 2003)
1. Situational Factors
2. Learning Goals
3. Feedback and Assessment
4. Teaching/Learning Activities
5. Integration
Initial Design Phase
Context
Content
Assessment
Pedagogy
C & A & P
Alignment?
End
Start
Yes No
Backward Design 7

8. Reflect on Your Experience
(yeah! The red arrow is pointing at one of our undergrad researchers—Michelle Valeriano) 2 1

9. Student Engagement
“Perhaps the strongest conclusion that can be made is the least surprising. Simply put, the greater the student’s involvement or engagement in academic work or in the academic experience of college, the greater his or her level of knowledge acquisition and general cognitive development” - Pascarella and Terenzini (1991, 2005)
Frequency and quality of student-student and student-faculty interaction are most influential for college student’s academic development, personal development and satisfaction (Astin, 1993; Light, 1992)
High-Impact Educational Practices (Kuh, 2008)

10. The American College Teacher: National Norms for 2007-2008
Methods Used in “All” or “Most”
All – 2005
All – 2008
Assistant
Cooperative Learning 48 59 66
Group Projects 33 36 61
Grading on a curve 19 17 14
Term/research papers 35 44 47

11. Calls for evidence-based teaching practices
Engaged Pedagogy
January 13, 2009—New York Times
January 2, 2009—Science, Vol. 323
Calls for evidence-based teaching practices

15. Book Ends on a Class Session

16. Book Ends on a Class Session
Advance Organizer
Formulate-Share-Listen-Create (Turn-to-your-neighbor) -- repeated every minutes
Session Summary (Minute Paper)
What was the most useful or meaningful thing you learned during this session?
What question(s) remain uppermost in your mind as we end this session?
What was the “muddiest” point in this session?
Table summarizes my perception of the shift. A version of this table is available in New Paradigms for Engineering Education -- FIE Conf proceedings 97 (avail on the www)
One of the most significant changes that has occurred is the shift from "pouring in knowledge" to "creating a climate where learning flows among students and the professor"

17. Reflect on the session:
Session Summary
(Minute Paper)
Reflect on the session:
1. Most interesting, valuable, useful thing you learned.
2. Things that helped you learn.
3. Comments, suggestions, etc
Pace: Too slow Too fast
Relevance: Little Lots
Instructional Format: Ugh Ah 17

18. Q4 – Pace: Too slow 1 . . . . 5 Too fast (3.3)
MOT 8221 – Spring 2009 – Session 1
Q4 – Pace: Too slow Too fast (3.3)
Q5 – Relevance: Little Lots (4.2)
Q6 – Format: Ugh Ah (4.4) 18

19. MOT 8221 – Spring 2008 – Session 1
Q4 – Pace: Too slow Too fast (3.1)
Q5 – Relevance: Little Lots (3.9)
Q6 – Format: Ugh Ah (4.2)

20. Informal Cooperative Learning (Book Ends on a Class Session)
Physics
Peer Instruction - Eric Mazur - Harvard –
Richard Hake –
Chemistry
Chemistry ConcepTests - UW Madison
Video: Making Lectures Interactive with ConcepTests
ModularChem Consortium –
STEMTEC
Video: How Change Happens: Breaking the “Teach as You Were Taught” Cycle – Films for the Humanities & Sciences –
Harvard
Thinking Together, From Questions to Concepts Interactive Teaching in Physics & Interactive Teaching DVD: Promoting Better Learning Using Peer Instruction and Just-In-Time Teaching: Derek Bok Center

21. It could well be that faculty members of the twenty-first century college or university will find it necessary to set aside their roles as teachers and instead become designers of learning experiences, processes, and environments.
James Duderstadt, 1999 [Nuclear Engineering Professor; Dean, Provost and President of the University of Michigan]

22. Resources Integrated Design Approach
Pellegrino – Rethinking and Redesigning Curriculum, Instruction and Assessment
Fink, L. Dee Creating significant learning experiences: An integrated approach to designing college courses. San Francisco: Jossey-Bass. (Notes: Fink, L. Dee A Self-Directed Guide to Designing Courses for Significant Learning).
Backward Design Process (Wiggins & McTighe):
Smith, K. A., Douglas, T. C., & Cox, M Supportive teaching and learning strategies in STEM education. In R. Baldwin, (Ed.). Improving the climate for undergraduate teaching in STEM fields. New Directions for Teaching and Learning, 117, San Francisco: Jossey-Bass.
Bransford, Vye and Bateman – Creating High Quality Learning Environments
Active & Cooperative Learning
Smith, K.A., Sheppard, S. D., Johnson, D.W., & Johnson, R.T Pedagogies of engagement: Classroom-based practices. Journal of Engineering Education Special Issue on the State of the Art and Practice of Engineering Education Research, 94 (1),

23. Learning Takes Feedback
Barbara M. Olds
Sheri Sheppard
National Academy of Engineering
Frontiers of Engineering Education
November 2009

24. Types of Feedback Testing and grading Informal assessment methods
Purposes
Definitions
Guidelines
Use of learning objectives
Informal assessment methods

25. Grade Referencing
Norm-referenced – classify and rank students relative to others; a relative standard (grading on the curve)
Criterion-referenced – measure student achievement relative to a defined set of knowledge and skills; an absolute standard (competence)

26. Types of Assessment
Informal – assessment activities designed to give quick, often qualitative, feedback to students
Formal – assessment activities, often quantitative, designed to collect data for assigning a grade

27. Think/Pair/Share Exercise
Describe the worst exam situation of your academic career
List characteristics of the exam you would have changed to improve it
Share your thoughts with a neighbor
Be prepared to share with the class

28. Good Testing and Assessment Practices
Test on what you teach (learning objectives)
Test what you want your students to do
Write tests that facilitate learning as well as provide data for grades

29. Good Testing and Assessment Practices
Use non-graded formative assessments
Use other assessment measures to supplement tests
Don’t grade on the curve if you want students to learn together actively

30. Learning Objectives
Effective way to communicate course expectations to students
Measurable objectives form the basis for creating and assessing assignments, exams, projects, etc.

31. Learning Objectives
Write objectives using quantifiable action verbs (e.g. apply, describe, compute, demonstrate, analyze, evaluate)
Avoid terms which have no concrete meaning (e.g. know, learn, appreciate, understand)

32. Learning Objectives
Try to write some objectives which require higher order thinking (e.g. divergent or evaluative thinking)

33. Classroom Assessment
Small-scale assessments conducted in college classrooms by discipline-based teachers to determine what students are learning in that class.
-- Tom Angelo

34. Classroom Assessment
Is learner-centered, teacher-directed, ongoing, and context-specific
Does not require specialized training
Can be used to monitor individual student misconceptions and difficulties or overall class progress

35. Example CA Methods background knowledge survey minute paper
concept map
journal
others on the handout
More in Angelo and Cross, Classroom Assessment Techniques, 1993

36. Some Suggestions Don’t try any technique that doesn’t seem right
Don’t make CA a burden
Choose techniques that seem likely to produce useful data
Allow enough time to conduct the assessment including response to data

37. One-minute Paper
What is the most important bit of knowledge you learned in this session?
What do you still have questions about?

38. Base Learning Systems on Design Principles
Wendy C. Newstetter
Director of Learning Sciences Research
Wallace H. Coulter Department of Biomedical Engineering
Georgia Institute of Technology
National Academy of Engineering
Frontiers of Engineering Education
November 2009

39. Design principles for learning systems
Sustained apprenticeship to cognitive practice of model-based reasoning
Scaffold meta-cognitive practices-monitor, reflect, change, improve
Deep contextualization of practice in real world then textbook problems
Learning forward/”knowing with”
Thematic chaining

40. Problem-driven learning rooms

41. Distributed cognitive system

44. Design/problem solving studio

47. Learning forward through thematic chaining
Why do we model?
How do we do it?
What can it tell us
compared to other approaches?
H1N1 modeling and simulation
Year one
PBL course
Year two
PBL course
Eng CS
Conservation
DifEQ’s
Cell physiology
Year three
PBL course
Drug design for flu
Bio-systems modeling
Year four
PBL course
H1N1 cell level/Simulink
Capstone design