1. STUDENT LEARNING OUTCOMES ASSESSMENT Developing a rubric for critical thinking assessment. Step-by-step analysis-QEP Full Implementation training

2. Assessment of Learning Process  Student Learning  Assesses student learning as a result of the educational process The skills, knowledge, values gained Usually expressed in psychomotor, cognitive or affective verbiage –ex. demonstrates, applies, utilize, construct, classify, create, evaluate. Critical Thinking is at the root of all learning application. Rote memorization will not create a successful learner for the future.

3. Good Student Learning Outcomes are and must be:  Measurable achievements as a result of the educational experience-what the student gains in an observable/behavioral way from enrollment in the course, program, degree etc.  An ongoing process for IMPROVEMENT of elements identified.  Narrowed focus. “The tip of the funnel”.

4. Assessment is:  It is forward looking, not just where we are today but where do we want to go.  Focus is on what is accomplished and is systematic in nature.  Student centered.  To analyze the processes and results and use for improvement of learning.

5. Outcomes Assessment is NOT:  Faculty focused.  Punitive in nature (faculty or student).  Course Grades  Tests (not typically)  Broad in nature-Not the “top of the funnel”.  Not recall or memorization  Capstone courses (elements of CT may be in them but holistically speaking not CT entirely)

6. As we all know…  York Technical College has chosen to assess student learning outcomes with a specific focus in the area of Critical Thinking for our QEP.  Full Implementation is this FALL 2009.  We have some tools in place now to help you in this process.

7. Critical Thinking  Many definitions exist with some variations however, all contain primarily similar processes requiring first, foundational knowledge that is then analyzed, synthesized, compared, inferred, applied and evaluated.

8. CT is focused on teaching students.. HOW TO THINK, NOT WHAT TO THINK. Why? Consider this…

9.  http://www.platosgarden.com/id63.html http://www.platosgarden.com/id63.html

10. Adaptable Rubric template Available at www.yorktech.com/QEP

11. Model Adapted for Radiography Clinical Project

12. 4321sco re FORMULATE Using knowledge and assessment, provides in great detail, the necessary modifications/plan required for this patients condition to achieve desired outcome of exam.. Uses all medical or technical terminology Using knowledge and assessment, provides in some detail, the necessary modifications/plan required for this patients condition to achieve desired outcome of exam. Some key details omitted that influenced understanding or slight inaccuracies in design of plan. Uses most medical or technical terminology correctly. Provides in little or poor detail, the necessary modifications/plan required for this patients condition to achieve desired outcome of exam. Explanation of plan conveys a lack of understanding for modifications needed for procedure. Uses medical or technical terminology with some degree of error. Lacks explanation of or creation of a modification necessary for examination. Lack of medical or technical terminology or improper use throughout. APPLY Using formulated plan, describe in great detail, comprehensively, and factually the actions taken to achieve modifications and execute all aspects of procedures. Include film, accessories and technical factor selection utilized.. Use all medical or technical terminology correctly. Using formulated plan, describe in some detail, comprehensively, and factually the actions taken to achieve modifications and execute all aspects of procedures. Include film, accessories and technical factor selection utilized. Some key details omitted that indicated a lack of understanding for procedure or slight inaccuracies in execution. Uses most medical or technical terminology correctly. In using formulated plan, describe in little or poor detail, the actions taken to achieve modifications and execute all aspects of procedures. Limited discussion of film, accessories and technical factor selection utilized. Some key details omitted that indicated a lack of understanding for procedure or slight inaccuracies in execution. Uses most medical or technical terminology correctly. 1)DID NOT FOLLOW CONFIDENTIALLY PROCESS OR 2)DID NOT FOLLOW DIRECTIONS. OR: Unable to relate with any detail the actions taken to achieve modifications and execute all aspects of exam or failure to modify exam properly.

13. 4321 SCORE EVALUATE Able to determine in proper medical terms and great detail, the results represented in your images of: (INCLUDE ALL OF THE FOLLOWING) -1)Anatomy (bony, soft tissue, anatomical structures and alignment) 2)-Film Quality (discuss was correct positioning, centering, markers, rotation, motion represented?) 3)-Pathology represented (existence and type OR confirmation of no pathology present) 4)-Resultant technical factors (discuss film’s contrast, density, distortion-were they adequate?) -(Reflect upon your results for each step. Did you formulate the correct plan and receive the right results? What would you do differently next time or when you repeated the film?) Able to determine in most medical or technical terms and in some detail, the results represented in your images of: INCLUDE ALL OF THE FOLLOWING: -1)Anatomy (bony, soft tissue, anatomical structures and alignment) 2)-Film Quality (discuss was correct positioning, centering, markers, rotation, motion represented?) 3)-Pathology represented (existence and type OR confirmation of no pathology present) 4)-Resultant technical factors (discuss film’s contrast, density, distortion-were they adequate?) -(Reflect upon your results for each step. Did you formulate the correct plan and receive the right results? What would you do differently next time or when you repeated the film?) Able to relate with limited detail, the results represented in your images of: -1)Anatomy (bony, soft tissue, anatomical structures and alignment) 2)-Film Quality (was correct positioning, centering, markers, rotation, motion represented?) 3)-Pathology represented (existence and type OR confirmation of no pathology present) 4)-Resultant technical factors (s film’s contrast, density, distortion- were they adequate?)] OR Omission of one or more of the above topics. -(Little reflection upon your results for each step. Did you formulate the correct plan and receive the right results? What would you do differently next time or when you repeated the film?) Unable to/fails to evaluate or relate with any detail, characteristics represented Radiographically of images: OR Omission of two or more of the topics listed at the left.

14. The “SHAPE” of Assessment 

15. How to break down the process within your courses-Top down or bottom up in design.  Mission –York Technical College Mission  Goals –Established by our choice of CT for QEP “Student will be able to critically think”. Specific Outcome A specific skill or knowledge set that demonstrates your goal/competency is acheived. (Course competencies can give you insight to your course goals that are critical thinking in nature). EX “Student will be able to apply principles of critical thinking by comparing and contrasting literary works of ……” Object or Tool that demonstrates that what we said the student would be able to accomplish has been accomplished (this is the learning activity). This should be narrow in scope and specific in nature. Ex. “Assign the analysis of two literary agents to compare and contrast” Assess the expected outcome using your specific rubric template for project, set benchmark expected.

16. Some examples of activities you can think of that you ALREADY do?

17. Object examples:  Abstract, advertisement, annotated bibliography, biography, briefing, brochure, webpage, care plan, case study analysis, chart, cognitive map, court brief, debate, essay, executive summary, mathematical problem, narrative, news story, oral report, outline, presentation, review of literature, thesis, process analysis, simulations, a specific question on a test on a specific subject matter. WHAT CAN YOU THINK OF??  (Walvoord/Anderson 1998).

18. The approaches to life and living which characterize critical thinking include: * concern to become and remain well- informed, * alertness to opportunities to use critical thinking, * trust in the processes of reasoned inquiry, * self-confidence in one’s own abilities to reason, * open-mindedness regarding divergent world views, * flexibility in considering alternatives and opinions * understanding of the opinions of other people, * fair-mindedness in appraising reasoning, *honesty in facing one’s own biases, prejudices, stereotypes, or egocentric tendencies, * prudence in suspending, making or altering judgments, * willingness to reconsider and revise views where honest reflection suggests that change is warranted  http://www.insightassessment.com/pdf_files/what&why2009.pdf

19. Don’t reinvent the wheel, just reinvent the way you look at it!!  Use what you are already doing in most cases.  Take for example from real current syllabi.  ACC 101- Compute the distribution of net income/net loss between partners.  CPT 168-Entire Module 1 is Problem Solving as it relates to computer programming.  PSY 201-Take Maslow’s hierarchy of needs and write essay that reflects how the accomplishment of each level would be acheived. (Module 4 competency component).  SPC-apply basic reflective thinking procedure as it relates to argument.

20. Examples cont.  LOG 110-Solve an inventory problem. Create a solution to a scenario or explain cost analysis as it relates to marketing, procurement, or distribution.  EVT 110-Design a process chart for the analysis of wastewater, the result of contamination by bacteria or fungi.

21. Construct reporting/object thus:  GOAL:  Student applies principles of critical thinking and problem solving.  Outcome: Student will be able to __”action verb”_____ _”something”_______. (what does your course competency say they will be able to do?) OBJECT How do you show that the student achieved this? Design your rubric See your handouts for verbiage that is specific to Critical Thinking

22. Student Service areas can utilize:  Direct measures such as during counseling or tutoring.  Indirect measures by employers, trainers, tutors etc.  Surveys completed by students regarding processes on campus.  Scenarios with weighted question responses produced in a survey submission style.

23. Writing outcomes  How do you fix an unclear outcome?  Many program brochures include learning outcomes which are unclear or represent elements of curriculum rather than some action the participants will demonstrate.Note the following examples:  Participants will understand the nine reasons for conducting a needs assessment.  Participants will develop an appreciation of cultural diversity in the workplace. http://www.aallnet.org/prodev/outcomes.asp

24.  If you ask a simple question ("Can it be measured?"), you see readily that these learning outcomes have shortcomings.They are not measurable.The same outcomes can be modified by changing the action verbs.  Participants will list nine reasons for conducting a needs assessment.  Participants will summarize in writing their feelings about cultural diversity in the workplace.  Learners now have a much better idea of what is expected of them.

25. Writing Outcomes for CT-Use of higher ordered thought processing  What is the importance of action verbs?  Since the learner's performance should be observable and measurable, the verb chosen for each outcome statement should be an action verb which results in overt behavior that can be observed and measured. http://www.aallnet.org/prodev/outcomes.asp

26. Sample action verbs are:  compile, create, plan, revise, analyze, design, select, utilize, apply, demonstrate, prepare, use, compute, discuss, explain, predict, assess, compare, rate, critique

27.  Certain verbs are unclear and subject to different interpretations in terms of what action they are specifying. Such verbs call for covert behavior which cannot be observed or measured. These types of verbs should be avoided:  know, become aware of, appreciate, learn, understand, become familiar with

28. Evaluate each: Are these measurable by rubric or instrument?  “Student read poem”  “Student knows process”  Learner understands …  http://www.youtube.com/watch? v=teMlv3ripSM http://www.youtube.com/watch? v=teMlv3ripSM  Each of these can be answered by asking Is the objective clear?  How is this defined?  Are there actions verbs present in the evaluation?  Are these functions application/synthesis of learned material or recall, basic, or memorization such as definitions  Are they based upon a logically supported position with factual support of their ideas or argument? Are they based upon a logically supported position with factual support of their ideas or argument?

29. Checklist of the process…  Evaluate the results from your activity.  Were they what you expected? Yes Do you have enough data to be certain this is sustainable/ repeatable/ Or Accurate? NO What do you see as the potential cause for these results?  What can you do next time in this course to improve the outcome related to the cause? Implement Changes NO Re-evaluate Data YES New activity/ different look ????

30.  You have to know WHAT the problem is in order to improve or correct it!! 

31. What if they do it very well?  You look at the results and they ARE able to critically think well in the ONE area.  Then move on to another area or tool to look at this outcome in another way to see how that is going.  REMEMEBER: This is annually at a minimum, semester by semester if the same course is taught each semester. This is ongoing This is for improvement in student learning Keep it simple and narrow in WHAT you are measuring Be specific in the CRITERIA A good tool/rubric removes ambiguity Use these SAME activities and measures for QEP requirement AND Student Learning Outcomes-NO NEED TO DUPLICATE!

32. Modify the rubric for your use by using BEST outcome (4) first. Use action verbs.

33. Summary Template & Website  http://www.yorktech.com/qep/ http://www.yorktech.com/qep/