Gathering, Analyzing, Using & Reporting Assessment Data University of Wisconsin Parkside January 20, 2012 Susan Hatfield Winona State University

1. Gathering Information

Learning Objects Standardized Exam, abstract, advertisement, annotated bibliography, biography, briefing, brochure, budget, care plan, case analysis, chart, cognitive map, court brief, debate, definition, description, diagram, dialogue, diary, essay, executive summary, exam, flow chart, group discussion, instruction manual, inventory, lab notes, letter to the editor, matching test, mathematical problem, memo, micro theme, multiple choice test, narrative, news story, notes, oral report, outline, performance review, plan, presentation, process analysis, proposal, regulation, research proposal, review of literature, taxonomy, technical report, term paper, thesis, word problem, work of art. (Walvoord Anderson 1998).

Learning Objects Test Project Assignment Portfolio Recital Performance Presentation Exhibit Internship

Student Learning Outcome ComponentsTest Questions

Student Learning Outcome ComponentsAssignment

Student Learning Outcome ComponentsAssignment 1 2

Student Learning Outcome ComponentsExperience

Assessing Learning Test Project Case Study Portfolio Recital Performance Presentation Exhibit Internship Number or percentage correct

Student Learning Outcome Test Questions % 82% 64% 87% % correct

Assessing Learning Test Project Case Study Portfolio Recital Performance Presentation Exhibit Internship Number or percentage correct Scale

Semantic Differential Repeat mode effective / ineffective

Student Learning Outcome 2 1 Components effective ineffective

Scale Semantic Differential Repeat mode effective / ineffective unique descriptors enhances / distracts original / common always / never engaged / disconnected

Student Learning Outcome 2 1 Components

Assessing Learning Test Project Case Study Portfolio Recital Performance Presentation Exhibit Internship Number or percentage correct Scale Rubric

Rubrics Analytical Numerous components which you want to assess individually Summative

Components Analytical / Summative Rubric Does not meet expectations Outcome:

Components Meets expectations Analytical / Summative Rubric Outcome:

Components Exceeds expectations Outcome: Analytical / Summative Rubric

Components Analytical / Developmental Rubric Outcome: complexity

Components Analytical / Developmental Rubric Outcome: detail

Components Analytical / Developmental Rubric Outcome: sophistication

Components Analytical / Developmental Rubric Fourth year Outcome:

Rubrics Analytical Holistic - generally smaller assignments with fewer criteria, shorter turn around time.

Exceptional component Level Holistic Rubric Above Average component Average component Below Average component

Resource

PRACTICAL ADVICE: Pilot test your rubrics!

Verbal Delivery Nonverbal Delivery Organization Does not meet MeetsExceeds 1% 3% 0% 15% 5% 3% 84% 92% 97% Evidence 1%6% 93% Transitions 2%8% 90%

2. Analyzing Data

Interpreting Data  Consistency - over time  Consensus - different populations  Distinctiveness - different situations/ variables / items

Consistency Examines the same practice of and individual or group over time Key question: Has this person or group acted, felt, or performed this way in the past / over time?

Consistency How well are students performing on the learning outcome? High performance Low performance

Consensus Comparison to or among groups of students Variation between disciplines, gender, other demographic variables Key questions: What is the general feeling, outcome, attitude, behavior? Do other groups of people act, perform or feel this way?

Consensus How well are students performing on the learning outcome? FemalesMalesTransfersOTA High performance Low performance

Distinctiveness Examines individual or cohort perspectives across different situations, categories Key Question : Does a person or group respond differently based upon the situation, item, issue?

Distinctiveness How well are our students achieving our Core Learning Outcomes? High Performance Low Performance

3. Using Data

How Assessment Works component BASELINE Compare Against Benchmarks, Standards, Targets, Past Performance New / Revised learning event New / Revised learning event New / Revised learning event Year 2

Acting on Results Development Faculty, Staff, Student Infrastructure Policy, Process, Planning Curriculum Learning Opportunities

The Seven Principles for Good Practice in Undergraduate Education 1. Student-Faculty Contact 2. Cooperative Learning 3. Active Learning 4. (Prompt) Feedback 5. Time on Task 6. High Expectations 7. Respect for Diverse Talents and Ways of Learning

Increase Reduce Determine Maintain the Number Amount Percentage Ratio Level of who that with Someone Majors Faculty Staff Community members Employers > Something Can analyze Performance data

Increase Reduce Improve Determine Maintain the Someone Students Faculty Staff Community members Employers Something applications grants Interviews request workshops reports satisfaction engagement by of who that with Action Number Amount Percentage Ratio Level > Something

Educating….. Assisting ….. Explaining ….. Assisting ….. Developing ….. Consulting ….. Delivering ….. Facilitating ….. Creating ….. Nurturing ….. Negotiating ….. Sponsoring ….. Training ….. Translating ….. Validating ….. Updating ….. Enforcing ….. Testing ….. Assessing ….. Reporting ….. Issuing ….. Communicating ….. Integrating ….. Responding ….. Arranging ….. Coordinating ….. Providing ….. Organizing ….. Producing ….. Training ….. Maintaining ….. Supporting ….. Directing ….. Designing ….. Collecting ….. Developing ….. Responding ….. Reviewing ….. Studying ….. Distributing ….. Executing ….. Recruiting ….. Generating ….. Implementing ….. Investigating ….. Designing ….. Collaborating ….. Revising ….. Scheduling ….. Sharing ….. Increase Reduce Improve Determine Maintain the Number Amount Percentage Ratio Level Something applications grants Interviews request workshops reports satisfaction engagement by of who that with Someone Students Faculty Staff Community members Employers > Something

Increase Reduce Determine Maintain the Number Amount Percentage Ratio Level of who that with Someone Second year students Faculty Staff Community members Employers > Something Demonstrate competence in critical thinking Something Action by

Increase Reduce Determine Maintain the Number Amount Percentage Ratio Level of who that with Someone Second year students Faculty Staff Community members Employers > Something Demonstrate competence in critical thinking Something Action by

4. Reporting Data

Outcome outcome What did you look at? object What did you find? results What does it mean? analysis What are you going to do about it? action What happened? reaction

Spring 2012 Summer 2012 Fall 2012 Spring 2013 Summer 2013 After That

Gathering, Analyzing, Using & Reporting Assessment Data University of Wisconsin Parkside January 20, 2012 Susan Hatfield Winona State University