1. Creating SLO Assessment Plans Presented by ORIE Team Summer 2013 Academy for Planning, Assessment, and Research

2. Table of Contents I.Defining Student Learning Outcomes II.Creating PLO or CLO Statements A. Bloom’s Taxonomy B. Habits of Mind III. Selecting Measures A. Direct vs. Indirect Measures B. Program Level Measures vs. Course Level Measures IV. Selecting Evaluation Methods A. Score Weighting Guide B. Rubrics VI. Setting Performance Criteria for Individual Students VII. Specifying Target Outcome for Group Success in meeting the SLO

3. Defining Student Learning Outcomes The student learning outcome is a statement that explains what the student is learning, including the accumulated and demonstrated knowledge, skills, abilities, behaviors, and habits of mind, as a result of actively participating in the course or program of study. Program Learning Outcomes answer the question: What will students know or be able to do when they exit the program? Course Learning Outcomes answer the question: What will students know or be able to do when they complete the course?

4. Creating Student Learning Outcome Statements Student Learning Outcome Statements  Should align with the purpose of the program or the course;  Should be specific and measurable, using action verbs. Bloom’s Taxonomy-- Three Learning Domains: Cognitive Affective Psychomotor

5. Bloom’s Taxonomy Knowledge: Recall data or information cite, count, define, describe, draw, identify, know, label, list, match, name, outline, point, quote, read, recall, recite, record, recognize, repeat, reproduce, select, state, tabulate, tell, trace, underline. Knowledge: Recall data or information cite, count, define, describe, draw, identify, know, label, list, match, name, outline, point, quote, read, recall, recite, record, recognize, repeat, reproduce, select, state, tabulate, tell, trace, underline. SLO Examples: Students will be able to Describe history, purpose, and scope of physical therapy Identify the social, political, economic and cultural influences and differences that affect the development process of the individual. SLO Examples: Students will be able to Describe history, purpose, and scope of physical therapy Identify the social, political, economic and cultural influences and differences that affect the development process of the individual.

6. Bloom’s Taxonomy Comprehension: Understand the meaning, translation, interpolation, and interpretation of instructions and problems. State a problem in one's own words. comprehend, compute, convert, defend, differentiate, discuss, distinguish, estimate, explain, express, extrapolate, extend, generalize, give examples, infer, interpret, paraphrase, predict, report, restate, review, rewrite, summarize, translate. Comprehension: Understand the meaning, translation, interpolation, and interpretation of instructions and problems. State a problem in one's own words. comprehend, compute, convert, defend, differentiate, discuss, distinguish, estimate, explain, express, extrapolate, extend, generalize, give examples, infer, interpret, paraphrase, predict, report, restate, review, rewrite, summarize, translate. Examples of SLO: Students will be able to Distinguish important aspects of the western moral theories, from the virtue theory of ancient philosophers to the modern theories of act ethics. Explain georeferencing of photos or images to maps. Summarize the principles of magnetism. Examples of SLO: Students will be able to Distinguish important aspects of the western moral theories, from the virtue theory of ancient philosophers to the modern theories of act ethics. Explain georeferencing of photos or images to maps. Summarize the principles of magnetism.

7. Bloom’s Taxonomy Application: Use a concept in a new situation or applies what was learned in the classroom into novel situations in the work place. apply, calculate, change, compute, construct, demonstrate, determine, discover, dramatize, employ, examine, illustrate, interpret, locate, manipulate, modify, operate, predict, prepare, produce, relate, show, solve, use. Application: Use a concept in a new situation or applies what was learned in the classroom into novel situations in the work place. apply, calculate, change, compute, construct, demonstrate, determine, discover, dramatize, employ, examine, illustrate, interpret, locate, manipulate, modify, operate, predict, prepare, produce, relate, show, solve, use. SLO Examples Students will be able to Implement plan of care for patients and families within the legal, ethical, and regulatory parameters. Apply appropriate organizational skills for selected types of speeches Conduct basic laboratory experiments involving classical mechanics. SLO Examples Students will be able to Implement plan of care for patients and families within the legal, ethical, and regulatory parameters. Apply appropriate organizational skills for selected types of speeches Conduct basic laboratory experiments involving classical mechanics.

8. Bloom’s Taxonomy Analysis: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences. analyze, appraise, break down, compare, contrast, diagram, deconstruct, differentiate, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate. Analysis: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences. analyze, appraise, break down, compare, contrast, diagram, deconstruct, differentiate, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate. SLO Examples: Students will be able to Analyze and design electrical and electronic circuits and systems, using Knowledge of mathematics and basic sciences. Analyze self-practice in relation to the roles of the professional nurse. SLO Examples: Students will be able to Analyze and design electrical and electronic circuits and systems, using Knowledge of mathematics and basic sciences. Analyze self-practice in relation to the roles of the professional nurse.

9. Bloom’s Taxonomy Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure. categorize, combine, compile, compose, create, devise, design, explain, generate, integrate, modify, organize, plan, rearrange, reconstruct, relate, reorganize, revise, rewrite, summarize, tell, write Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure. categorize, combine, compile, compose, create, devise, design, explain, generate, integrate, modify, organize, plan, rearrange, reconstruct, relate, reorganize, revise, rewrite, summarize, tell, write SLO Examples: Students will be able to Integrate the pathophysiological assessment findings to formulate a field impression. Coordinate human information and material resources in providing care for patients and their families. SLO Examples: Students will be able to Integrate the pathophysiological assessment findings to formulate a field impression. Coordinate human information and material resources in providing care for patients and their families.

10. Bloom’s Taxonomy Evaluation: Make judgments about the value of ideas or materials. appraise, compare, conclude, contrast, criticize, critique, defend, describe, discriminate, evaluate, explain, interpret, justify, relate, summarize, support Evaluation: Make judgments about the value of ideas or materials. appraise, compare, conclude, contrast, criticize, critique, defend, describe, discriminate, evaluate, explain, interpret, justify, relate, summarize, support SLO Examples: Students will be able to Use critical thinking and a systematic problem-solving process for providing comprehensive care. Analyze and evaluate economic events that apply to the preparation of financial statements. SLO Examples: Students will be able to Use critical thinking and a systematic problem-solving process for providing comprehensive care. Analyze and evaluate economic events that apply to the preparation of financial statements.

11. Habits of Mind Arthur Costa and Bena Kallick (2000)’s Habits of mind: Persisting, thinking flexibly, and striving for accuracy Examples of Habits of Mind:  Develop and use effective time management skills  Follow directions correctly  Appreciate negatives and failures as opportunities to learn and discover choices Arthur Costa and Bena Kallick (2000)’s Habits of mind: Persisting, thinking flexibly, and striving for accuracy Examples of Habits of Mind:  Develop and use effective time management skills  Follow directions correctly  Appreciate negatives and failures as opportunities to learn and discover choices SLO Examples: Students will demonstrate time management skills, adhering to all deadlines for assignments, tests, and projects. Students will demonstrate effective therapeutic patient communication/relationship. SLO Examples: Students will demonstrate time management skills, adhering to all deadlines for assignments, tests, and projects. Students will demonstrate effective therapeutic patient communication/relationship.

12. Selecting Measures A. Direct vs. Indirect Measures To measure student learning outcomes, use the direct measures primarily, but also use indirect measures for triangulation of data Direct Measures: Refer to the type of data that can be used to directly measure students’ knowledge and skills Examples: Pre-test, post-test, comprehensive subject matter exam, licensure exam, portfolio, thesis, writing assignment, internship evaluation, capstone assignment, and performance demonstration (recital, art exhibit, or science project)

13. Selecting Measures Indirect Measures: Refer to the type of data that can be used to infer student learning or achievement Examples: Surveys, Interviews, Focus Group Studies, Document Analyses, or Audit Measures (course grades, retention/graduation/job placement rates)

14. Selecting Measures B. Program Level Measures vs. Course Level Measures CLO Measures: Exams, Tests Rubric-Based Assignments: Projects Essays Portfolios Presentations Performance Demo. PLO Measures: Exit exam (standardized, licensure/professional exams) Thesis, Dissertation Capstone Course Paper or Project Portfolios PLO Measures: Exit exam (standardized, licensure/professional exams) Thesis, Dissertation Capstone Course Paper or Project Portfolios

15. Selecting Measures C. How to Select Measures? Measuring Knowledge, understanding Tests, Exams, Essays, Portfolios, Presentations Measuring SkillsPerformance Demonstrations, Projects, Presentations, Internship Evaluation Measuring Habits of MindSurveys, Interviews, Behavior Observations, Focus Groups, Document Audit

16. Examples of Measures Aligned with SLOs SLOsMeasures (Data Related to Student Work /Performance /Opinion) Students will be able to describe history, purpose, and scope of physical therapy (Knowledge) Exam on the history and purpose of Physical Therapy Essay describing the scope of physical therapy Students will be able to distinguish important aspects of the western moral theories, from the virtue theory of ancient philosophers to the modern theories of act ethics. (Understanding) Exam or Essay Questions on the Exam summarizing the principles of ethical theories Essay comparing the similarities and differences between the ancient and modern theories

17. Examples of Measures Aligned with SLOs SLOsMeasures (Data Related to Student Work /Performance /Opinion) Students will be able to apply appropriate organizational skills for selected types of speeches. (Skills) Demonstration of public speaking Speech in writing Students will demonstrate time management skills, adhering to all deadlines for assignments, tests, and projects. (Habit of Mind) Grade Book Audit Survey on student time management skills

18. Selecting Evaluation Methods Aligned with Measures Measures (Data Related to Student Work /Performance /Opinion) Evaluation Methods (How do instructors assess students’ work or performance?) Exam on the history and purpose of Physical Therapy Essay describing the scope of physical therapy Score Weighting Guide with scores assigned to each exam question Grading Rubric specifying levels of quality in identified areas Essay Questions on the Exam summarizing the principles of ethical theories Essay comparing the similarities and differences between the ancient and modern theories Grading Rubric specifying levels of quality in identified areas

19. Selecting Evaluation Methods Aligned with Measures Measures (Data Related to Student Work /Performance /Opinion) Evaluation Methods (How do instructors assess students’ work or performance?) Demonstration of public speaking Speech in writing Grading Rubric Grade Book Audit Survey on student time management skills Grade Book Record Survey Instrument

20. Creating a Rubric A rubric is a systematic scoring guide to evaluate student performance. Three Essential Parts of a Rubric: 1)A Set of Indicators of Learning 2)A Scale 3)A Set of Descriptors

21. Creating a Rubric Indicators for a rubric-based assignment (essay, project, speech, performance) should be aligned with the instructional objectives. If an instructor’s objectives of teaching an essay are related to helping students compose an essay that is focused, coherent, organized, etc., then these features should serve as indictors of learning. Example of Indicators for an Essay for an English Class:  Focus  Coherence  Organization  Sentence Structure  Word Choice

22. Creating a Rubric A scale indicates points to be assigned in scoring a piece of work on a continuum of quality. High numbers are typically assigned to the best work. Scale examples: Needs Improvement (1)... Satisfactory (2)... Good (3)... Exemplary (4) Beginning (1)... Developing (2)... Accomplished (3)... Exemplary (4) Needs work (1)... Good (2)... Excellent (3) Novice (1)... Apprentice (2)... Proficient (3)... Distinguished (4)

23. Creating a Rubric Descriptors are used to describe signs of performance at each level; the description needs to include both strengths and weaknesses (Weaknesses should be described particularly in lower levels of performance). Examples of Descriptors: Word Choice 4-- Vocabulary reflects a thorough grasp of the language appropriate to the audience. Word choice is precise, creating a vivid image. Metaphors and other such devices may be used to create nuanced meaning. 3-- Vocabulary reflects a strong grasp of the language appropriate to the audience. Word choice is accurate, but may be inappropriate in a couple of places. 2-- Vocabulary reflects an inconsistent grasp of the language and may be inaccurate or inappropriate to the audience. 1-- Vocabulary is typically inaccurate and inappropriate to the audience. Word choice may include vague, non-descriptive, and/or trite expressions.

24. Example of a Rubric (A Snapshot) Critical Thinking VALUE Rubric (AACC) Indicators Capstone 4 Milestones 3 2 Benchmark 1 Explanation of IssuesIssue/problem to be considered critically is stated clearly and described comprehensively, delivering all relevant information necessary for full understanding. Issue/problem to be considered critically is stated, described, and clarified so that understanding is not seriously impeded by omissions. Issue/problem to be considered critically is stated but description leaves some terms undefined, ambiguities unexplored, boundaries undetermined, and/or backgrounds unknown. Issue/problem to be considered critically is stated without clarification or description. EvidenceInformation is taken from source(s) with enough Interpretation/ evaluation to develop a comprehensive analysis or synthesis. Information is taken from source(s) with enough interpretation /evaluation to develop a coherent analysis or synthesis. Information is taken from source(s) with some interpretation/ evaluation, but not enough to develop a coherent analysis or synthesis. Information is taken from source(s) without any interpreta- tion/evaluation.

25. Setting Criteria for Meeting SLO To determine whether students have successfully met the learning outcomes, we need to specify what score or rating an individual student needs to get in order to be considered as meeting the learning outcome. Examples: Measure: Exam on the history and purpose of Physical Therapy Evaluation Method: Score Weighting Guide with scores assigned to each exam question Criterion for Meeting the SLO: A score of 70 or above on the exam Measure: Essay describing the scope of physical therapy Evaluation Method: Grading Rubric specifying levels of quality in identified areas Criterion for Meeting the SLO: A score of 3 or above on a scale of 1-4

26. Specifying Target Outcomes To determine whether students as a group have successfully met the learning outcomes, we need to specify what % from the group should meet the learning outcome. Example I: Measure: Exam on the history and purpose of Physical Therapy Evaluation Method: Score Weighting Guide with scores assigned to each exam question Criterion for Meeting the SLO: A score of 70 or above on the exam Target Outcome: 85% of the students will score 70 or above on the exam.

27. Specifying Target Outcomes To determine whether students as a group have successfully met the learning outcomes, we need to specify what % from the group should meet the learning outcome. Example II: Measure: Essay describing the scope of physical therapy Evaluation Method: Grading Rubric specifying levels of quality in identified areas Criterion for Meeting the SLO: A score of 3 or above on a scale of 1-4 Target Outcome: 70% of the students will score a 3 or above on their essays

28. Recap: SLO Assessment Plan Student Learning Outcome Assessment Plan Mission: PLO or CLO Statement Measure (Student Work/Performanc e) Teacher's Evaluation Method Criterion (What cut-off score is considered as meeting PLO or CLO for each student?) Target Outcome (Group Success: % of the group meeting the PLO or CLO)

29. PLO Assessment Plan: (Example) Associate Degree in Political Science Program Learning Outcomes Mission: The Political Science Department is committed to producing students who are knowledgeable about the operation of Texas and national government. Students should also have the ability to think critically about important political topics that will prepare them for transfer to higher institutions of learning and provide a framework for lifetime learning skills. PLO StatementMeasure (Student Work/Performance) Teacher's Evaluation Method Criterion (What cut-off score is considered as meeting PLO for each student?) Target Outcome (Group Success: % of the group meeting the PLO) Students will understand the American politics subfield of political science. GOVT2304 (Intro to Political Science) Essay Question on American Politics Essay Assessment RubricA score of 3 or above on essay (on a scale of 1-5) 80% of the students answering the essay question will get a score 3 or above. Students will understand the political theory subfield of political science. GOVT2304 Essay Question on Political Theory Essay Assessment RubricA score of 3 or above on essay (on a scale of 1-5) 80% of the students answering the essay question will get a score 3 or above. Students will demonstrate a level of civic engagement Internship ProjectComments on strengths and weaknesses of the students’ internship performance in an evaluation letter 70% of the comments in the evaluation letter are positive. 95% of the students will get a positive evaluation letter from their internship supervisor.

30. CLO Assessment Plan: (Example) Engl 1302 Rhetoric and Composition Course Learning Outcome Assessment Plan CLO StatementMeasure (Student Work/Performance) Teacher's Evaluation Method Criterion (What cut-off score is considered as meeting CLO for each student?) Target Outcome (Group Success: % of the group meeting the CLO) Students will be able to present a valid argument using convincing evidence. Persuasive EssayEssay Grading RubricA score of 3 or above (1-5 scale) 80% of the students taking the course will receive a score of 3 or above. Students will be able to demonstrate appropriate use of standard English. Grammar Quiz Writing Portfolio a. Quiz grading standard b. Portfolio Grading Rubric a. A score of 70 on the quiz b. A score of 3 or above (1-5 scale) 75% of the students taking the Engl1302 course will receive a score of 70 for the quiz and a score of 3 or above for the portfolios. Student will apply researched information in writing a research paper. Summary of Readings Research Paper a. Summary Grading Rubric b. Research Paper Grading Rubric a. A score of 3 or above (1-5 scale) b. A score of 3 or above (1-5 scale) 75% of the students taking the Engl1302 course will receive a score of 3 or above for the summary assignment and for the research papers.

31. Do’s and Don’ts When Creating an SLO Assessment Plan Do’s 1.When creating PLO or CLO statements, DO include both lower level and higher level of cognitive skills: not only knowledge, comprehension, but also application, analysis, evaluation, and synthesis. 2.When selecting assessment measures for PLOs, DO include both course-embedded assignments and program level measures (thesis, exit exams, or national exams, or graduate portfolios, or internship project, or capstone course project). 3.Encourage faculty participation in the design of the Assessment Plan. 4.Try to use common assessment assignments or tools, but allow for flexibility.

32. Do’s and Don’ts When Creating an SLO Assessment Plan Don’ts Don’t be overly ambitious—Create only 5 to 7 PLOs or CLOs, and use only 7-10 pieces of data!

33. Academy for Planning, Assessment, and Research