1. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Using Technology for Content Delivery, Formative Assessment, and Reflection Michael S. Kirkpatrick JMU Computer Science Bridgewater College Annual Pedagogy Project 2015

2. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Workshop Objectives At the completion of this session, participants will demonstrate progress toward the following objectives: Summarize relevant literature for active learning Explain how to use videos for effective content delivery Describe how to incorporate formative assessment into a flipped classroom Identify practical strategies to encourage students’ day-to- day class preparation and metacognition

3. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Agenda Introductions and welcome Active learning preassessment Camtasia lessons learned Formative assessment and PI Metacognition and reflection Discussion and exploration

4. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick “Adopting instructional practices that engage students in the learning process is the defining feature of active learning.” -Michael Prince

5. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Benefits Illustrated R.R. Hake, "Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses," Am. J. Phys. 66, 64- 74 (1998). http://www.physics.indiana.edu/~sdi/ajpv3i.pdf Measure of performance gain Mechanics Diagnostic (MD) or Force Concept Inventory (FCI) 62 courses (14 trad.) at multiple institutions 6542 students (2084)

6. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Collaborative and Cooperative Learning M. Prince, “Does active learning work? A review of the research.” J. Eng. Education 93(3), 223- 241, 2004. http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Prince_AL.pdf

7. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Problem-Based Learning M. Prince, “Does active learning work? A review of the research.” J. Eng. Education 93(3), 223- 241, 2004. http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Prince_AL.pdf

8. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Active Learning in STEM Meta-analysis of 225 studies 158 studies: average 0.47 SDs better on CIs/exams 67 studies: average failure rate dropped from 33.8% to 21.8% with active learning S. Freeman et al., “Active learning increases student performance in science, engineering, and mathematics,” Proceedings of the National Academy of Sciences 111(23), 8410- 8415, 2014. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4060654/pdf/pnas.201319030.pdf

9. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick From Content Delivery to Formative Assessment

10. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick

11. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick

12. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick

13. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick

14. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Lessons Learned Be mindful of accessibility Never use red and green contrasts Create subtitles or transcripts as needed Write a script first Doing so will save you time later Makes for easy transcript creation Align videos with learning objectives Use a discussion forum for students to ask questions Incorporate an ungraded Moodle quiz for practice

15. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Formative Assessment and Peer Instruction

16. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick “Feedback is value-neutral help on worthy tasks. It describes what the learner did and did not do in relation to her goals. It is actionable information, and it empowers the student to make intelligent adjustments when she applies it to her next attempt to perform.” -Grant Wiggins

17. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Peer Instruction Created by Eric Mazur (Harvard) Augment class with ConcepTests Expose common misconceptions Think-vote-pair-revote pattern E. Mazur, Peer Instruction: A User’s Manual, 1996. http://mazur.harvard.edu/research/detailspage.php?rowid=8

18. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Physics Question 1

19. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Closing the Gender Gap PI can eliminate gender gap in physics T: traditional lectures IE: interactive lectures IE+: interactive assignments, lectures, tutorials Cooperative learning closes the gender gap Pretest scores were 10% points higher for men Gap persisted with lecture alone Posttest results for cooperative classes were almost equal E. Mazur, “The scientific approach to teaching: Research as a basis for course design,” keynote/plenary talk at the International Computing Education Research Conference (ICER), 2011. http://mazur.harvard.edu/search-talks.php?function=display&rowid=1712

20. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Closing the Gender Gap Traditional lectures leave women behind Women tend to have smaller performance gains Cooperative learning improves gains for women...but men improve as well E. Mazur, “The scientific approach to teaching: Research as a basis for course design,” keynote/plenary talk at the International Computing Education Research Conference (ICER), 2011. http://mazur.harvard.edu/search-talks.php?function=display&rowid=1712

21. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Physics Question 2

22. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Demos and Engagement Performance and understanding increase with engagement Those who only observe sometimes learn it wrong Those who discuss show clearer reasoning and provide partially correct answers

23. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Physics Question 3

24. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Confusion and Understanding “Please tell us briefly what points of reading you found most difficult or confusing.” “Nothing was difficult or confusing.” “I found the explanation inadequate. I don’t understand the reasoning that led to the conclusion.”

25. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Best Practices for Formative Assessment Identify the learning gap Space between what students know and need to know Bidirectional feedback Identify student progress and suggest corrections Actively engage students Students need to assess their own understanding Create learning progressions Break larger goal into subgoals

26. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Best Practices for Formative Assessment Ask clear questions Avoid ambiguity, use only one verb Psychological safety Positive reinforcement, accept imperfection Sequencing and balance Consider the order and type of questions, activities Wait time Come ot terms with silence Avoid pimping questions Do not try to establish intellectual superiority

27. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick

28. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Metacognition Space Race

29. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Metacognitive Activities Preassessments What do I already know about this topic? Muddiest points What am I still confused on? Exam corrections Why did I miss this question? Documented problem solving What were the steps I used to solve this problem? Learning progress journals How did my understanding of this concept change?

30. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick RSQC 2 Recall the most important points Requires student to analyze relative importance Summarize the most important points Provides practice with comprehension Construct a question you would like answered Encourages reflection and evaluation Connect this material to other concepts Establishes scaffolding from previously learned material Comment on your learning progress this week Uses metacognitive reflection to instill study habits

31. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Tools for Metacognitive Activities Socrative Good use: quick in-class polling Qualtrics Good use: offline, anonymous surveys Moodle Good use: graded and offline assessments Social media Good use: blogs as learning journals Piazza Good use: collaborative student discussions and answers

32. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Best Practices for Metacognition Develop metacognitive culture Give students freedom to be confused Integrate reflection into credited course work Model metacognitive behavior Teach the concept and language of metacognition Explicit instruction over time expands skill set Reflect the specific learning context Metacognition is NOT generic Externalize mental events Increase accurate awareness of strengths and weaknesses

33. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Bloom’s Taxonomy

34. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Discussion and Exploration

35. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Other Technology Tools Twitter Good use: announcements, highlight news stories CATME Good use: team formation, peer evaluation Top Hat Monocle Alternative to Socrative (requires paid subscription) Asynchronous MOOC Videos edX, Kahn Academy, MIT OpenCourseWare

36. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Resources Peer Instruction (PI) E. Mazur, Peer Instruction: A User’s Manual, 1996. http://mazur.harvard.edu/research/detailspage.php?rowid=8 Formative Assessment T. A. Angelo and K. P. Cross, Classroom Assessment Techniques: A Handbook for College Teachers, 1993.Classroom Assessment Techniques: A Handbook for College Teachers http://wvde.state.wv.us/teach21/ExamplesofFormativeAssessment.html http://cft.vanderbilt.edu/guides-sub-pages/cats/ Metacognition E. Cook, E. Kennedy, S. Y. McGuire, “Effect of Teaching Metacognitive Learning Strategies on Performance in General Chemistry Courses,” J. Chem. Educ., 2013, 90 (8), pp 961–967. http://www.lmu.edu/Assets/Centers+$!2b+Institutes/Center+for+Teaching+Excellence/Teach+ST EM+Students+How+to+Learn--Metacognition+is+the+Key!+Slides.pdf

37. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 Dr. Michael S. Kirkpatrick Resources Others J. D. Bransford, A. L. Brown, and R. R. Cocking, How People Learn: Brain, Mind, Experience, and School, 2000.How People Learn: Brain, Mind, Experience, and School M. Weimer, Learner-Centered Teaching: Five Key Changes to Practice, 2002. G. Wiggins and J. McTighe, Understanding by Design, 2005. N. Pinchok and W. C. Brandt, “Connecting Formative Assessment Research to Practice: An Introductory Guide for Educators,” 2009. http://www.learningpt.org/pdfs/FormativeAssessment.pdf N. Chick, Metacognition. http://cft.vanderbilt.edu/guides-sub- pages/metacognition/ T. Tofade, J. Elsner, S. T. Haines, “Best Practice Strategies for Effective Use of Questions as a Teaching Tool,” Am J Pharm Educ. 2013 Sep 12; 77(7): 155.