Seven Levers for Deeper Learning Research-based Guidelines and Strategies for Improving Teaching, Assessment & Learning   An Interactive Keynote Session in the The 2017 Celebration of Teaching and Learning Sponsored by the Delphi Center The University of Louisville 9:30-11:00 AM on Friday 10 February 2017 Tom Angelo

For all our efforts . . . Are we making any meaningful progress on promoting learning outcomes?

have at least a 4-year degree. More than 33% of US adults have at least a 4-year degree. About 42% have at least a 2-year degree. Have those gains in attainment led to a more civil, just, equitable or sustainable society? Notice I didn’t even bother mentioning the 2016 election.

Page 1 Middle [ 1 ] Values Affirmation Jot down 2 or 3 educational/ professional values you hold that motivate you to invest time and energy in teaching and learning improvement efforts at U of L. _________________________________________

Which value(s) did you mention? Improving student learning Improving teaching effectiveness Doing scholarship of teaching & learning Innovating in the classroom Making teaching less burdensome When’s the coffee break? 5 5

Page 1 bottom [ 2 ] Goal Ranking & Matching Exercise What are your Learning Goals and/or Burning Questions for this session? _________________________________________

My Intended Learning Outcomes (ILOs) for this Session By the end of this workshop, you will have: Identified at least two or three (2-3) strategies, guidelines or techniques to improve teaching, assessment, feedback, and/or grading in your work Identified at least two (2) useful resources and/or references for follow up Committed to experimenting with at least one small change in your practice, and 4. Talked with at least two folks you don’t normally get an opportunity to talk with about these topics

In other words, I’m hoping you’ll end this session with . . . Greater conceptual clarity Useful guidelines Useful strategies and techniques Useful references/resources More connections with colleagues 8 8

A First “Balcony” Question Could there be advantages to knowing something about your students’ goals? values? 9 9

Page 11 (The back page) Applications Ideas/Techniques Possible Applications

Background Knowledge Probe page 2 1. Of first-year students in 4Y US colleges in Fall 2015, what percentage graduated from: A. A traditional public school? ___ % B. A charter/magnet public school? ___% C. A private religious/parochial school? ___ % D. A private independent prep school? ___% E. Were home schooled? ___%

Background Knowledge Probe page 2 2. Of those same students, what %-age rated the following as “very important” reasons to go to college: A. To be able to get a better job? ___ % B. To learn more about things that interest me? ___% C. To get training for a specific career? ___ % D. To gain a general ed. & appreciation for ideas? ___% E. To be able to make more money? ___% F. To prepare myself for grad/prof school? ___% G. To make me a more cultured person ___%

Background Knowledge Probe page 2 3. Of those same students, what %-age thought they had a “very good chance of doing the following in college: A. Socializing w/different racial/ethnic person? ___ % B. Earning at least a ‘B’ average? ___% C. Communicating regularly with profs? ___ % D. Protesting or demonstrating? ___%

Background Knowledge Probe page 2 4. About what %-age of the 2015 FYs had: A. First parent with at least a 4-year degree? ___ % B. Second parent with at least a 4-year degree? ___%

Second “Balcony” Question If you followed directions: Are you more interested in learning the answers to questions on p. 2 than you would have been if I’d simply told that information to you? 15 15

Background Knowledge Probe page 2 1. Of FY students in 4Y US colleges in 2015, what percentage graduated from: A. A traditional public school? 74% B. A charter/magnet public school? 8% C. A private religious/parochial school? 10% D. A private independent prep school? 7% E. Were home schooled? 1%

Background Knowledge Probe page 2 2. Of those same students, what %-age rated the following as “very important” reasons to go to college: A. To be able to get a better job? 85% B. To learn more about things that interest me? 82% C. To get training for a specific career? 76 % D. To gain a general ed. & appreciation for ideas? 71% E. To be able to make more money? 70% F. To prepare myself for grad/prof school? 59% G. To make me a more cultured person 49%

Background Knowledge Probe page 2 3. Of those same students, what %-age thought they had a “very good chance of doing the following in college: A. Socializing w/different racial/ethnic person? 71% B. Earning at least a ‘B’ average? 68% C. Communicating regularly with profs? 44% D. Protesting or demonstrating? 9%* * Upon seeing these survey results, college presidents all over the US ordered their Admissions Offices to identify this 9%.

Background Knowledge Probe page 2 4. About what %-age of the 2015 FYs had: A. First parent with at least a 4-year degree? 61% B. Second parent with at least a 4-year degree? 60%

More “Balcony” Questions Imagine that your first-year students had similar responses: Could you see any implications for: The courses you teach? For your program? For campus assessment efforts? 20 20

Six Dimensions of Higher Learning Outcomes Page 3 [ 1, 2 & 4 ] Six Dimensions of Higher Learning Outcomes % Your Past? % Their Future? ____ Factual Learning _____ ____ Conceptual Learning _____ ____ Procedural Learning _____ ____ Conditional Learning _____ ____ Metacognitive Learning _____ ____ Reflective Learning _____ 100% 100% 21

3rd Balcony Question Which of those six dimensions needs and deserves the most focus if we aim to foster: Creative thinking? Problem-solving? Innovation? Life-long learning?

Metacognition involves . . . Self-Awareness Knowing that and when you are thinking Self-Monitoring Noticing the quality/intensity of your thinking Self-Regulation Directing/Correcting your thinking Reflection requires metacognition, but goes beyond it to affect/change values, beliefs, actions and/or habits 23

A Reflection-in-Action Exercise Rate your own learning approach thus far in the session: 1. Disengaged (I don’t see the point, or I already know all this.) 2. Semi-attentive (Checking in and out, per my interests.) 3. Engaged recipient (I’m taking it all in, but quietly.) 4. Active Participant (I’m contributing & cooperating.) 5. Reflective Synthesizer (I’m making connections to prior knowledge/experience and imagining how I might use this in my future work.) Adapted from: Guskey, T. R. (2000). Evaluating Professional Development. Thousand Oaks, CA: Corwin Press, p. 135. 24

can be promoted effectively through High-Impact Practices HOT HIPs! Higher Order Thinking can be promoted effectively through High-Impact Practices 25 25

HIPs – High-Impact Educational Practices First-Year Seminars and Experiences Learning Communities Collaborative Assignments and Projects Undergraduate Research Diversity/Global Learning Service Learning/Community-Based Learning Clinical Placements/Internships/Co-ops Capstone Courses and Projects Writing-Intensive Courses

What makes these HIPs so HOT? High expectations Explicit direct instruction Metacognitive scaffolding Deliberate practice Effective feedback Focused collaboration 27 27

Page 4 – Top – [ 1 & 3 ] The Plus-Minus-Question Mark Technique Prior knowledge and beliefs Formative and summative assessment Deliberate practice Confirmation bias Motivated reasoning Stereotype threat Bus, Parrot & Parking Lot Tests Cognitive load Metacognition The Dance Floor & The Balcony Novice-Expert differences 28

Bloom’s Taxonomy (revised) page 5 Bloom’s Taxonomy (revised) Anderson & Krathwohl, 2001 Create Design Invent Hypothesise Assess Explain Critique Decide Justify Evaluate Determine Induce Question Appraise Analyse Adapt Modify Verify Demonstrate Apply Arrange Categorise Convert Formulate Generalise Understand Identify Do simple procedure Remember

Page 5 – bottom [ 1, 3 & 4 ] “Blooming” – Categorizing Questions 30 30

The Parrot Test

Page 5 Categorizing Questions by Bloom’s Taxonomy Levels (Blooming) By systematically varying the elements of the task, we can better assess and promote transfer and deep learning. 32 32

Page 8 - top The Bondy Scale 33 33

Bondy’s Clinical Performance (Skills) Rating Scale Independent plus capable of assessing, instructing and supporting [6. EXPERT] Safe, Very proficient, Requires no support or direction 5. INDEPENDENT Safe, Proficient, Requires only occasional support and direction 4. SUPERVISED Safe, Mostly proficient, Requires frequent support and direction 3. ASSISTED Safe only when supervised, Unskilled, Requires continuous support and direction 2. MARGINAL Unsafe, Unaware & Unskilled 1. DEPENDENT

A Detailed Assessment/Grading Rubric Page 7 – [ 4 ] A Detailed Assessment/Grading Rubric What matters most in this course? How do you know? 35 35

“From the student’s point of view, the assessment is the curriculum.” Paul Ramsden 36

Handout Page 8 Why Give Learners Feedback? To Improve performance & academic success To increase interest & motivation to learn To illuminate and undermine misconceptions To promote self-assessment & self-regulation To Develop independence as lifelong learners

Handout Page 8 To Use Feedback Well, Learners Need M.O.M. Motivation – Compelling reasons to use it Opportunities – For safe, guided practice Means – Knowledge & skills for improvement

Have you heard of the “Feedback Sandwich” Have you heard of the “Feedback Sandwich”? It’s not the most educationally “nutritious” feedback meal.

Handout Page 8 The Order in which We Give Feedback Matters Consider the Following five steps: 1ST - GOOD NEWS: WHAT WAS DONE WELL 2ND - BAD NEWS: WHAT STILL NEEDS IMPROVEMENT 3RD - OPTIONS: WHAT CAN BE DONE TO IMPROVE IT 4TH - PLANS: WHAT THE LEARNER INTENDS TO DO 5TH - COMMITMENTS: WHAT BOTH PARTIES AGREE TO DO, HOW, TO WHAT STANDARD, AND BY WHEN

A Draft Course/Teaching Feedback Form Handout Page 9 A Draft Course/Teaching Feedback Form What values do our course and teaching evaluation systems convey to students? To faculty? 41

“The effectiveness of teaching is best evaluated by what the students do when the teacher is not present.”

Page 11 (The Back Page) Applications Card – [ 1, 3 & 4 ] Interesting Possible IDEAS/TECHNIQUES APPLICATIONS 43 43

The Parking Lot Test 44 44

What, Why and How Choose one of your possible applications. Prepare to answer the three questions below about that specific application: What is it? Why do you think it might be useful? How do you think you might use it? 45 45