1 Aligning Assessment Methods with Learning Outcome Statements and Curricular Design Presented at CCRI April 8, 2005 Material from Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing and AAHE.

2 Alignment—degree to which learning outcome statements match how and what we teach; degree to which assessment methods match or build upon our collective educational practices What methods of assessment capture desired student learning--methods that align with pedagogy, content, and curricular design?

3 “The tasks to which students are asked to respond on an assessment are not arbitrary. They must be carefully designed to provide evidence that is linked to the cognitive model of learning and to support the kinds of inferences and decisions that will be based on the assessment results.” National Research Council. Knowing what students know: The science and design of educational assessment. Washington, D.C.: National Academy Press, 2001, p. 47.

4 Assumptions Underlying Teaching Actual Practices Assumptions Underlying Assessment Tasks Actual Tasks

5 When Will You Seek Evidence ? Formative—along the way? For example, to ascertain progress or development Summative—at the end? For example, to ascertain mastery level of achievement

6 What Tasks Elicit Learning You Desire? Tasks that require students to select among possible answers (multiple choice test)? Tasks that require students to construct answers (students’ problem- solving and thinking abilities)?

7 Direct Methods Focus on how students represent or demonstrate their learning (meaning making) Align with students’ learning and assessment experiences Align with curricular-and co-curricular design verified through mapping

8 Invite collaboration in design (faculty, students, tutors)

9 Standardized Instruments Psychometric approach—values quantitative methods of interpretation History of validity and reliability Quick and easy adoption and efficient scoring One possible source of evidence of learning

10 Do Not Usually Provide Evidence of strategies, processes, ways of knowing, understanding, and behaving that students draw upon to represent learning Evidence of complex and diverse ways in which humans construct and generate meaning Highly useful results that relate to pedagogy, curricular design, sets of educational practices

11 Authentic, Performance-based Methods Focus on integrated learning Directly align with students’ learning and previous assessment experiences Provide opportunity for students to generate responses as opposed to selecting responses

12 Provide opportunity for students to reflect on their performance—strengths, weaknesses, repositioned learning

13 Do Not Provide Immediate reliability and validity (unless there has been a history of use) Usually do not provide easy scoring unless closed-ended questions are used.

14 Some Options for Alternative Methods E-Portfolios Capstone projects (mid-point and end- point) Performances, productions, creations Visual representations (mind mapping, charting, graphing)

15 Case studies Disciplinary or professional practices Agreed upon embedded assignments Selection of assignments students hand in Writing to speaking to visual presentation

16 Team-based or collaborative projects Internships and service Projects Oral examinations/questions Critical incidents

17 Externally or internally juried review of student projects Externally reviewed internship Performance on a case study/problem Performance on case study accompanied with students’ analysis

18 Locally developed tests Pre-and post-tests Learning Logs or Journals Videotaping over time

19 Simulations—virtual labs, scenarios that track decision making and actions Magic box—problem solving over time

20 Indirect Methods-- May Be Combined with Direct Methods Focus group (representative of the population) Interviews (representative of the population) Surveys Transcript analyses

21 Other sources of information that contribute to your inference making: CCSSE results, grades, participation rates or persistence in support services, course-taking patterns, majors

22 Identify Methods to Assess Outcomes Using the handout, identify both direct and indirect methods you might use to assess several of your outcomes. Determine the kinds of inferences you will be able to make based on each method.

23 Examples of Changes: Increased attention to weaving experiences across the institution, a program, or a department to improve student achievement Changes in advising based on assessment results Closer monitoring of student achievement--tracking

24 Faculty and staff development to learn how to integrate experiences that contribute to improved student learning Changes in pedagogy and curricular and co-curricular design Development of modules to assist learning; use of technology; self-paced learning, supplemental learning

25 Mission/Purposes Learning Outcomes How well do we achieve our outcomes? Gather Evidence Interpret Evidence Enhance teaching/ learning; inform institutional decision- making, planning, budgeting

26 “What and how students learn depends to a major extent on how they think they will be assessed.” John Biggs, Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p. 141.

27 Works Cited Biggs, J. (1999). Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p Maki, P. (forthcoming, 2004., May). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Sterling, VA: Stylus Publishing, LLC, and the American Association for Higher Education. National Research Council Knowing What Students Know: The Science and Design of Educational Assessment. Washington, D.C.: National Academy Press