1. Writing Outcomes & Measures Adapted from Susan Hatfield Presented to DC faculty InterCampus Day Spring 2012

2. Designing academic programs 1. Identify desired outcomes. 2. Determine acceptable evidence. 3. Plan learning experiences & instruction.

3. Doing assessment Nitsáhákees: establish learning goals Nahat’a: plan learning opportunities Íiná: teach Siihasin: assess student learning and use the results

4. The most important thing Start with good student learning outcomes.

5. Avoid… Students should be able to comprehend, interpret, analyze and critically evaluate material in a variety of written and visual formats. Students will demonstrate creative and evaluative thinking in the analysis of theoretical and practical issues in the areas of politics, the economy, and the environment.

6. Models of understanding Bloom’s taxonomy Understanding by Design Linda Suskie KnowledgeExplanation Familiarity ComprehensionInterpretation Application AnalysisPerspective SynthesisEmpathy Habit of mind (or disposition) EvaluationSelf-knowledge

7. SLO formatting FORMAT: Students should be able to + RULE OF THUMB: If you have more than one action verb, keep the one that represents the highest order of thinking.

8. KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Su rize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Lower-level outcomes

9. KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Upper-level outcomes

10. Good SLO’s Locate online resources on a particular issue or topic. Apply scientific principles to everyday life. Analyze errors. Compare and contrast perspectives and values. Justify potential careers. Use an appropriate mathematic procedure to solve a problem. Conduct a research study. Design a community service project. Demonstrate intellectually curiosity. Appreciate the perspective of people from backgrounds other than one’s own. Choose ethical courses of action.

11. Direct measures of learning FamiliarityApplicationDisposition Locally developed tests Essay questions scored blindly by faculty Standardized tests National certifica- tion or professional exams Performance- based assess- ments Juried review of projects Externally review- ed exhibitions and performances Evaluation of internships based upon program learning outcomes Capstone experience Observations of student perfor- mance Student reflections Student self- assessment

12. More performance measures Abstract or executive summary Advertisement of commercial Annotated bibliography Autobiography from an historical period Briefing paper Brochure or pamphlet Campaign speech Case study Client report Collaborative group activity Database Debate or discussion Debriefing interview preparation Dramatization of an event or scenario Revising a poorly written paper Evaluation of opposing points of view Experiment or other lab experience Field notes Game invention Graph, chart, or diagram Graphic organizer or taxonomy Handbook or instructional manual Journal or log Letter to an editor or business Model, simulation, or illustration Narrative Newspaper story or news report Oral history recording of an event Plan for researching or solving a problem Portfolio Poster, display, or exhibit Presentation or demonstration Process description Proposal for solution to a problem Reflection on what was learned Review and critique of one’s own work Selected portions of an essay Survey with analysis of results Video or audio recording Web site

13. Kinds of rubrics Checklist rubrics Rating scale rubrics Descriptive rubrics Holistic scoring guides Structured observation guides

14. Designing rubrics Create the rating scale List what you are looking for (refer back to your student learning outcomes). Create at least 3 levels. Label each level with names, not just numbers, e.g., “complete evidence, partial evidence, minimal evidence, no evidence” or “exceeds standard, meets standard, approaching standard, or below standard.” If developing a descriptive rubric, describe in the boxes the different levels of performance. Try out the rubric! Adjust as needed.

15. DIRECT MEASURES INDIRECT MEASURES 19902010 2000 Assessment measures – what the HLC expects

16. Indirect performance measures Student, employer, and alumni satisfac- tion survey Exit interviews and focus group inter- views of graduates Retention and transfer studies Length of time to degree Graduation and transfer rates Job placement rates