1. Seven Levers for Deeper Learning Research-based Guidelines and Strategies for Improving Teaching, Assessment & Learning  
An Interactive Keynote Session in the
The 2017 Celebration of Teaching and Learning
Sponsored by the Delphi Center The University of Louisville
9:30-11:00 AM on Friday 10 February 2017
Tom Angelo

2. For all our efforts . . .
Are we making any
meaningful progress on promoting learning outcomes?

3. have at least a 4-year degree.
More than 33% of US adults
have at least a 4-year degree.
About 42% have at least a 2-year degree.
Have those gains in attainment led to a more civil, just, equitable or sustainable society?
Notice I didn’t even bother mentioning the 2016 election.

4. Page 1 Middle [ 1 ] Values Affirmation
Jot down 2 or 3 educational/ professional values you hold that motivate you to invest time and energy in teaching and learning improvement efforts at U of L.
_________________________________________

5. Which value(s) did you mention?
Improving student learning
Improving teaching effectiveness
Doing scholarship of teaching & learning
Innovating in the classroom
Making teaching less burdensome
When’s the coffee break? 5 5

6. Page 1 bottom [ 2 ] Goal Ranking & Matching Exercise
What are your Learning Goals and/or Burning Questions for this session?
_________________________________________

7. My Intended Learning Outcomes (ILOs) for this Session
By the end of this workshop, you will have:
Identified at least two or three (2-3) strategies, guidelines or techniques to improve teaching, assessment, feedback, and/or grading in your work
Identified at least two (2) useful resources and/or references for follow up
Committed to experimenting with at least one small change in your practice, and
4. Talked with at least two folks you don’t normally get an opportunity to talk with about these topics

8. In other words, I’m hoping you’ll end this session with . . .
Greater conceptual clarity
Useful guidelines
Useful strategies and techniques
Useful references/resources
More connections with colleagues 8 8

9. A First “Balcony” Question
Could there be advantages to knowing something about your students’ goals? values? 9 9

10. Page 11 (The back page) Applications
Ideas/Techniques
Possible Applications

11. Background Knowledge Probe page 2
1. Of first-year students in 4Y US colleges in Fall 2015, what percentage graduated from:
A. A traditional public school? ___ %
B. A charter/magnet public school? ___%
C. A private religious/parochial school? ___ %
D. A private independent prep school? ___%
E. Were home schooled? ___%

12. Background Knowledge Probe page 2
2. Of those same students, what %-age rated the following as “very important” reasons to go to college:
A. To be able to get a better job? ___ %
B. To learn more about things that interest me? ___%
C. To get training for a specific career? ___ %
D. To gain a general ed. & appreciation for ideas? ___%
E. To be able to make more money? ___%
F. To prepare myself for grad/prof school? ___%
G. To make me a more cultured person ___%

13. Background Knowledge Probe page 2
3. Of those same students, what %-age thought they had a “very good chance of doing the following in college:
A. Socializing w/different racial/ethnic person? ___ %
B. Earning at least a ‘B’ average? ___%
C. Communicating regularly with profs? ___ %
D. Protesting or demonstrating? ___%

14. Background Knowledge Probe page 2
4. About what %-age of the 2015 FYs had:
A. First parent with at least a 4-year degree? ___ %
B. Second parent with at least a 4-year degree? ___%

15. Second “Balcony” Question If you followed directions:
Are you more interested in learning
the answers to questions on p. 2 than you would have been if I’d simply told that information to you? 15 15

16. Background Knowledge Probe page 2
1. Of FY students in 4Y US colleges in 2015, what percentage graduated from:
A. A traditional public school? 74%
B. A charter/magnet public school? 8%
C. A private religious/parochial school? 10%
D. A private independent prep school? 7%
E. Were home schooled? %

17. Background Knowledge Probe page 2
2. Of those same students, what %-age rated the following as “very important” reasons to go to college:
A. To be able to get a better job? %
B. To learn more about things that interest me? 82%
C. To get training for a specific career? %
D. To gain a general ed. & appreciation for ideas? 71%
E. To be able to make more money? 70%
F. To prepare myself for grad/prof school? %
G. To make me a more cultured person %

18. Background Knowledge Probe page 2
3. Of those same students, what %-age thought they had a “very good chance of doing the following in college:
A. Socializing w/different racial/ethnic person? 71%
B. Earning at least a ‘B’ average? 68%
C. Communicating regularly with profs? 44%
D. Protesting or demonstrating? 9%*
* Upon seeing these survey results, college presidents all over the US ordered their Admissions Offices to identify this 9%.

19. Background Knowledge Probe page 2
4. About what %-age of the 2015 FYs had:
A. First parent with at least a 4-year degree? 61%
B. Second parent with at least a 4-year degree? 60%

20. More “Balcony” Questions
Imagine that your first-year students had similar responses:
Could you see any implications for:
The courses you teach?
For your program?
For campus assessment efforts? 20 20

21. Six Dimensions of Higher Learning Outcomes
Page 3 [ 1, 2 & 4 ]
Six Dimensions of Higher Learning Outcomes
% Your Past? % Their Future?
____ Factual Learning _____
____ Conceptual Learning _____
____ Procedural Learning _____
____ Conditional Learning _____
____ Metacognitive Learning _____
____ Reflective Learning _____
100% % 21

22. 3rd Balcony Question Which of those six dimensions needs and deserves the most focus if we aim to foster: Creative thinking? Problem-solving? Innovation? Life-long learning?

23. Metacognition involves . . .
Self-Awareness
Knowing that and when you are thinking
Self-Monitoring
Noticing the quality/intensity of your thinking
Self-Regulation
Directing/Correcting your thinking
Reflection requires metacognition,
but goes beyond it to affect/change values, beliefs, actions and/or habits 23

24. A Reflection-in-Action Exercise
Rate your own learning approach thus far in the session:
1. Disengaged (I don’t see the point, or I already know all this.)
2. Semi-attentive (Checking in and out, per my interests.)
3. Engaged recipient (I’m taking it all in, but quietly.)
4. Active Participant (I’m contributing & cooperating.)
5. Reflective Synthesizer (I’m making connections to prior knowledge/experience and imagining how I might use this in my future work.)
Adapted from: Guskey, T. R. (2000). Evaluating Professional Development. Thousand Oaks, CA: Corwin Press, p. 135. 24

25. can be promoted effectively through High-Impact Practices
HOT HIPs!
Higher Order Thinking
can be promoted effectively through
High-Impact Practices 25 25

26. HIPs – High-Impact Educational Practices
First-Year Seminars and Experiences
Learning Communities
Collaborative Assignments and Projects
Undergraduate Research
Diversity/Global Learning
Service Learning/Community-Based Learning
Clinical Placements/Internships/Co-ops
Capstone Courses and Projects
Writing-Intensive Courses

27. What makes these HIPs so HOT?
High expectations
Explicit direct instruction
Metacognitive scaffolding
Deliberate practice
Effective feedback
Focused collaboration 27 27

28. Page 4 – Top – [ 1 & 3 ] The Plus-Minus-Question Mark Technique
Prior knowledge and beliefs
Formative and summative assessment
Deliberate practice
Confirmation bias
Motivated reasoning
Stereotype threat
Bus, Parrot & Parking Lot Tests
Cognitive load
Metacognition
The Dance Floor & The Balcony
Novice-Expert differences 28

29. Bloom’s Taxonomy (revised)
page 5
Bloom’s Taxonomy (revised)
Anderson & Krathwohl, 2001
Create
Design
Invent
Hypothesise
Assess
Explain
Critique
Decide
Justify
Evaluate
Determine
Induce
Question
Appraise
Analyse
Adapt
Modify
Verify
Demonstrate
Apply
Arrange
Categorise
Convert
Formulate
Generalise
Understand
Identify
Do simple
procedure
Remember

30. Page 5 – bottom [ 1, 3 & 4 ] “Blooming” – Categorizing Questions30 30

31. The Parrot Test

32. Page 5 Categorizing Questions by Bloom’s Taxonomy Levels (Blooming)
By systematically varying the elements of the task, we can better assess and promote transfer and deep learning. 32 32

33. Page 8 - top The Bondy Scale33 33

34. Bondy’s Clinical Performance (Skills) Rating Scale
Independent plus capable of assessing, instructing and supporting
[6. EXPERT]
Safe,
Very proficient,
Requires no support or direction
5. INDEPENDENT
Safe,
Proficient,
Requires only occasional support and direction
4. SUPERVISED
Safe,
Mostly proficient,
Requires frequent support and direction
3. ASSISTED
Safe only when supervised, Unskilled,
Requires continuous support and direction
2. MARGINAL
Unsafe,
Unaware &
Unskilled
1. DEPENDENT

35. A Detailed Assessment/Grading Rubric
Page 7 – [ 4 ]
A Detailed Assessment/Grading Rubric
What matters most in this course?
How do you know? 35 35

36. “From the student’s point of view, the assessment is the curriculum.”
Paul Ramsden 36

37. Handout Page 8 Why Give Learners Feedback?
To Improve performance & academic success
To increase interest & motivation to learn
To illuminate and undermine misconceptions
To promote self-assessment & self-regulation
To Develop independence as lifelong learners

38. Handout Page 8 To Use Feedback Well, Learners Need M.O.M.
Motivation – Compelling reasons to use it
Opportunities – For safe, guided practice
Means – Knowledge & skills for improvement

39. Have you heard of the “Feedback Sandwich”
Have you heard of the “Feedback Sandwich”? It’s not the most educationally “nutritious” feedback meal.

40. Handout Page 8 The Order in which We Give Feedback Matters Consider the Following five steps:
1ST - GOOD NEWS: WHAT WAS DONE WELL
2ND - BAD NEWS: WHAT STILL NEEDS IMPROVEMENT
3RD - OPTIONS: WHAT CAN BE DONE TO IMPROVE IT
4TH - PLANS: WHAT THE LEARNER INTENDS TO DO
5TH - COMMITMENTS: WHAT BOTH PARTIES AGREE TO DO, HOW, TO WHAT STANDARD, AND BY WHEN

41. A Draft Course/Teaching Feedback Form
Handout Page 9
A Draft Course/Teaching Feedback Form
What values do our course and teaching
evaluation systems convey to students?
To faculty? 41

42. “The effectiveness of teaching is best evaluated by what the students do when the teacher is not present.”

43. Page 11 (The Back Page) Applications Card – [ 1, 3 & 4 ]
Interesting Possible
IDEAS/TECHNIQUES APPLICATIONS 43 43

44. The Parking Lot Test 44 44

45. What, Why and How Choose one of your possible applications.
Prepare to answer the three questions below about that specific application:
What is it?
Why do you think it might be useful?
How do you think you might use it? 45 45