1. Critical Thinking Through Reflection & Self Assessment Pamela L. Stamm, PharmD, CDE, BCPS Associate Professor of Pharmacy Practice Auburn University Harrison School of Pharmacy Research conducted by Pamela L. Stamm and Kristen L. Helms November 18, 2012

2. Critical Thinking “is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.” http://www.criticalthinking.org/pages/defining-critical-thinking/410 Process of actively and skillfully analyzing and evaluating information generated by reflection as a guide to belief and action.

3. Reflection “The process of analyzing and making judgments about what has happened” Ideally, translating this to change when needed http://www.hawaii.edu/intlrel/pols382/Reflective%20Thinking%20-%20UH/reflection.html Now What? So What? What

4. Overview of Course Problem Based Learning Therapeutics course Four 8 week courses divided over 2 semesters Three facilitated sessions weekly, 3 hours each

5. Overview of Course Students expected to solve the problem AND learn optimal drug therapy approach Students expected to learn about all common drugs available for treatment Writing products: Disease SKPs, Drug SKPs, Learning issues

6. Drug SKP (Standard Knowledge Parameters) Name, doses, dosage forms Mechanism of action Relevant drug chemistry Pharmacokinetics Safety issues Special population concerns Monitoring for efficacy and safety Major counseling points Role in treating the disease Preferred agent in class Application to case

7. Assessment Methods Daily and weekly facilitator grading – Completes responsibilities, Groupwork, Learning Issue and Presentation of Learning Issues Weekly peer feedback – Strengths, Areas for Improvement, How group can help with this Quizzes Oral exam Written exam

8. Reflection Objectives Critical Thinking Require students to make connections among self, artifacts, and learning Metacognitive Thinking Require students to think of the process they used. Disciplinary Thinking & Writing Require students to find and arrange information, discriminate important from unimportant, compare & contrast, and apply information.

9. Reflection Assignment Choose drug SKP document Complete portfolio entry Upload SKP document List references List time needed to complete the SKP document Answer questions Course 1 Any drug SKP Course 2 Any drug SKP Course 3 Any drug SKP Course 4 Any drug SKP

10. Reflection Scaffold (Course 1, SKP 1) What did you find most challenging? What will you do differently? How could you have been better prepared?

11. Reflection Scaffold (Courses 2-4, SKPs 2-4) How was this SKP the same? different? What were its strengths? limitations? What will you do differently? How was this SKP the same? different? What were its strengths? limitations? How will you use this skill on rotations? As a pharmacist?

12. Data Handling Downloaded into Excel, Microsoft 2010 Data reviewed for completeness Data reviewed and categorized responses for each question Categories developed as reflections were read

13. What was the most challenging? What will you do differently? How will you use this skill? Results Critical Thinking Require students to make connections among self, artifacts, and learning

14. What was most challenging? (n= 88 ) Managing the Information Discriminating (42) Organizing (12) Formatting (7) Differentiating (5) Clinical application (4) Managing conflicting info (3) Summarizing (2) Condensing (2) Compiling (3) Plagiarism (1) Interpretation of data (1) Knowing the information Managing the Information Discriminating (42) Organizing (12) Formatting (7) Differentiating (5) Clinical application (4) Managing conflicting info (3) Summarizing (2) Condensing (2) Compiling (3) Plagiarism (1) Interpretation of data (1) Knowing the information Finding the information Resource utilization & navigation (23) Communicating information Overall process (6) Miscellaneous Time management (9) Technology challenges (2) Confidence (1) Lack of an example (2) Finding the information Resource utilization & navigation (23) Communicating information Overall process (6) Miscellaneous Time management (9) Technology challenges (2) Confidence (1) Lack of an example (2)

15. What will you do differently? (n= 88) Managing the information Consider objectives (22) Condensing (12) Discriminating (12) Organizing (12) Differentiating (11) Formatting (8) Template (7) Completeness (3) Summarizing (3) Clinical application (1) Synthesizing (1) Finding the information: Resource utilization (20) Managing the information Consider objectives (22) Condensing (12) Discriminating (12) Organizing (12) Differentiating (11) Formatting (8) Template (7) Completeness (3) Summarizing (3) Clinical application (1) Synthesizing (1) Finding the information: Resource utilization (20) Knowing the information Knowledge/understanding (10) Communicating information Clarity (2) Enhance teaching (12) Confidence Overall process Change Team Process (1) Systematic approach (5) Miscellaneous Technological Challenges (1) Time Management (9) Knowing the information Knowledge/understanding (10) Communicating information Clarity (2) Enhance teaching (12) Confidence Overall process Change Team Process (1) Systematic approach (5) Miscellaneous Technological Challenges (1) Time Management (9)

16. Challenging vs Change Discriminating (42) Resource utilization (23) Organizing (13) Time management (9) Formatting (7) Overall process (6) Differentiating (6) Discriminating (42) Resource utilization (23) Organizing (13) Time management (9) Formatting (7) Overall process (6) Differentiating (6) Consider objectives (22) Resource utilization (20) Discriminating (12) Enhance teaching (12) Organizing (12) Condensing (12) Differentiating (11) Knowledge/understanding (10) Time management (9) Consider objectives (22) Resource utilization (20) Discriminating (12) Enhance teaching (12) Organizing (12) Condensing (12) Differentiating (11) Knowledge/understanding (10) Time management (9) Discriminating (42) Resource utilization (23) Organizing (13) Time management (9) Formatting (7) Overall process (6) Differentiating (6) Discriminating (42) Resource utilization (23) Organizing (13) Time management (9) Formatting (7) Overall process (6) Differentiating (6) Change

17. How will you use this skill? Rotations (n=26)Pharmacist (n=27) Resource utilization (16) Process to follow (10) Knowledge base (4) Select appropriate drug (2) Peripheral brain (1) Discriminate (1) Evaluate of guidelines (1) Formulate a plan (1) Recognize bias (1) Self-learning skills (1) Resource utilization (20) Process to follow (7) Discriminate (6) Efficiently answer questions (4) RIT (1) Special pop knowledge (1) Understand Context (1) How will you use this skill? Resource utilization (16) Process to follow (10) Knowledge base (4) Select appropriate drug (2) Peripheral brain (1) Discriminate (1) Evaluate of guidelines (1) Formulate a plan (1) Recognize bias (1) Self-learning skills (1) Resource utilization (20) Process to follow (7) Discriminate (6) Efficiently answer questions (4) Role in therapy (1) Special pop knowledge (1) Understand Context (1)

18. Limitations Students not held accountable on assignment Lack of feedback (faculty or peers) on reflections – Current software does allow for feedback but not a conversation Reflection may not be students’ original thoughts – Daily / weekly grading – Weekly peer assessments

19. Changes to assignment Ask, “what was most challenging?” each time Asking for “strengths” did not bring much insight Asking for similarities did not bring much insight. Mostly said, content and format was similar

20. Assignment Objectives Promote self assessment & self improvement skills – Identify strengths, limitations, and plan for change Develop critical, metacognitive, and disciplinary thinking – Made connections among artifacts, learning, and self – Think about the process they were using to find information, create documents, disseminate information, and teach others – Discriminate important from unimportant, differentiate, apply, and select

21. Thank you for your time and attention