Presentation is loading. Please wait.

Presentation is loading. Please wait.

Aligning Assessment with Course Objectives

Similar presentations


Presentation on theme: "Aligning Assessment with Course Objectives"— Presentation transcript:

1 Aligning Assessment with Course Objectives
Presented on 1/30/15 at MEC new faculty orientation session. Sharon Karkehabadi Student Learning Outcomes Specialist Office of Institutional Effectiveness and Student Success Initiatives Northern Virginia Community College New Faculty Orientation Spring 2015

2 Objectives for today’s workshop
Distinguish different types of learning objectives Define purposes of assessment Align assessment methods with learning objectives Analyze an assessment method for content validity Evaluate your assessment methods and make improvements Integrate different assessment measures into your courses.

3 Choose Method(s) of Assessment Gather Evidence Use Results
Identify Student Learning Outcomes Map to Curriculum Choose Method(s) of Assessment Gather Evidence Use Results Assessment at NOVA Assessment Loop, NOVA OIRPA

4 What is Learning?

5 Three Critical Components of Learning:
Learning is a process, not a product. However, because this process takes place in the mind, we can only infer that it occurred from the students’ products or performances. Learning involves change in knowledge, beliefs, behaviors, or attitudes. This change unfolds over time; it is not fleeting but rather has a lasting impact on how students think and act. Learning is not something done to students, but rather something students themselves do. It is the direct result of how students interpret and respond to their experiences-conscious and unconscious, past and present. Ambrose, Susan & al How Learning Works: 7 Research-Based Principles for Smart Teaching. SF: Jossey-Bass.

6 Why Align Assessments? Increases the probability that we will provide students with opportunities to learn and practice knowledge and skills to be assessed. “Good grades” are more likely to translate into “good learning.” “Students learn more effectively when their learning experiences are purposeful, coherent, and integrated.” Suskie, 2004, p. 127

7 Learning Goals vs. Performance Goals vs. Work-Avoidant Goals
Goals serve as the basic organizing feature of motivated behavior…they act as the compass that guides and directs a broad range of purposeful actions… Learning Goals vs. Performance Goals vs. Work-Avoidant Goals Learning Goals: Students try to gain competency and truly learn what an activity or task can teach them. Performance Goals: Students do what is necessary to get a good grade. Work-Avoidant Goals : Students desire to finish work as quickly as possible with as little effort as possible. Ambrose, Susan & al How Learning Works: 7 Research-Based Principles for Smart Teaching. SF: Jossey-Bass.

8 So…how can teachers motivate students to learn
When students find positive value in a learning goal or activity, expect to successfully achieve a desired learning outcome, and perceive support from their environment, they are likely to be strongly motivated to learn. Ambrose, Susan & al How Learning Works: 7 Research-Based Principles for Smart Teaching. SF: Jossey-Bass. So… if learning is a process and we can’t just dump the knowledge into students heads, how do we motivate them to learn And… how can assessment help

9 Student Learning Outcomes
Difference between… Learning Objectives Student Learning Outcomes Course-level statements describing the knowledge, skills, attitudes, and values that students gain from a course. More detailed and course content-specific Program-level statements describing the knowledge, skills, attitudes, and values that students gain from the program. More overarching and often encompass multiple courses Criterion referenced, based on faculty standards of acceptable student work (competency based) Not norm referenced- compared to other students/peers From Roberts, Jennifer. (January 2013). Aligning Test Items with Course Learning Objectives. PUP Conference. Northern Virginia Community College

10 Categories of Learning Objectives
Bloom’s Taxonomy Original 1956 by education committee chaired by Benjamin Bloom Anderson and Krathwohl revised 2001 Taxonomy for Learning, Teaching and Assessing Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York, USA: Addison-Wesley Longman.

11 Categories of Learning Objectives: Fink’s
Fink’s approach switches the emphasis away from content toward the goals and skills the instructor wants his or her students to retain after the course is completed. 1. Foundational Knowledge. Foundational Knowledge includes all of the content, ideas, and information that you want your students to know at the end of the semester. 2. Application. The Application taxon encompasses critical, creative, and practical thinking, as well as additional skill sets that may be beneficial to students. 3. Integration. Integration includes connecting different ideas that might appear in different disciplines or across the lifespan. 4. Human Dimension. The Human Dimension taxon helps assess if students learn more about themselves and others. It stresses the human factor and gives human significance to learning. 5. Caring. Caring is the taxon that provides the motivation and energy for learning by developing new interests, feelings, and values associated with the course material. 6. Learning How to Learn. The Learning How to Learn taxon provides the ability for long-term learning by teaching students to become self-directed learners. Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco: Jossey-Bass.

12 Reflection What is the purpose of assessment?

13 Formative: To inform teaching and improve learning
Purpose of Assessment Formative: To inform teaching and improve learning Summative: “To provide evidence of how students have learned what we intended them to learn.” When instructor/program/institution is committed to student learning, then assessment is a high priority. Here we are not just talking about high stakes testing and evaluation (summative assessment) we are more importantly talking about everyday formative assessment.

14 Assessment is a diagnostic tool for students and teachers.
For Teacher For Student Pre-assessment (finding out) Reveals what students know/think/are interested in Engages; helps connect (or integrate) previous knowledge/skills/ attitude Formative (keeping track/ checking-up) Reveals what students understand/apply; helps with adjusting curriculum and teaching style Helps self-assess; critique the quality of their work Summative (making sure) Determines student mastery; helps with course and program design Provides integration; Builds confidence 3 ways that assessment helps. 3 categories of assessment

15 Summative Assessment: Making Sure Purpose
Means to determine student mastery, understanding of information, skills, concepts, or processes. Should align with formative assessments and instruction that preceded it. May be tied to a grade.

16 Summative Assessment: Making Sure Benefits
Should determine student’s exit achievement/ability Can provide useful program-wide data

17 Reflection What types of summative assessment do I use in my classes?

18 Summative Assessment: Examples
Making Sure Summative Assessment: Examples Standardized/ National Tests Portfolios Performance task Product/exhibit  Presentation Demonstration Unit Test Mid-term/Final 

19 Challenge

20 Exemplary Assessment:
Authentic assessment Or Exemplary assessment. Evaluates student learning at the same time that it promotes it (Huba and Freed) Requires student to perform a task rather than take a test. Grant Wiggins, Educative Assessment: Designing Assessments to Inform and Improve Student Performance (San Francisco: Jossey-Bass Publishers, 1998).

21 8 Characteristics of Exemplary Assessment
Valid Yields useful information to guide learning Coherent Is structured so that the activities lead to desired performance Authentic Addresses ill-defined problems/ issues that are enduring or emerging Rigorous Requires use of declarative, procedural, and metacognitive knowledge Huba and Freed, 2000 Or Exemplary assessment. Evaluates student learning at the same time that it promotes learning (Huba and Freed, Requires student to perform a task rather than take a test. Grant Wiggins, Educative Assessment: Designing Assessments to Inform and Improve Student Performance (San Francisco: Jossey-Bass Publishers, 1998). Realistic, replicates the ways in which a person's knowledge and abilities are "tested" in the real-world Requires judgment and innovation by requiring the student to use knowledge and skills wisely and effectively to solve unstructured or ill-defined problems Simulates contexts that mirror the workplace or other real-life contexts Assesses the student's ability to efficiently and effectively use a repertoire of knowledge and skills to negotiate a complex task. (Wiggins, pp )

22 8 Characteristics of Exemplary Assessment
5. Engaging Provokes student interest and persistence 6. Challenging Provokes, as well as evaluates, student learning 7. Respectful Allows students to reveal their uniqueness as learners 8. Responsive Provides feedback to students leading to improvement Huba and Freed, 2000 Authentic assessment Or Exemplary assessment. Evaluates student learning at the same time that it promotes it (Huba and Freed, Requires student to perform a task rather than take a test. Grant Wiggins, Educative Assessment: Designing Assessments to Inform and Improve Student Performance (San Francisco: Jossey-Bass Publishers, 1998).

23 “What is it that I hope that students would have learned, that will be there and have value, several years after the course is over?” What would students have to do to convince me that they have achieved those learning goals?” Grant Wiggins, 1998 from Fink, 2003, p. 63.

24 Creating Exemplary Assessment
What declarative knowledge [knowing the facts and concepts in the discipline] do I expect students to draw upon in this task? What procedural knowledge [knowing how to reason, inquire, and present knowledge in the discipline] do I expect students to use? What metacognitive knowledge [e.g., setting goals, determining when additional information is needed, and assessing the fruitfulness of a line of inquiry] do I expect student to develop and reveal? In what real-life settings do individuals use the knowledge that I identified and what ill-defined problems do they typically address? Huba and Freed, 2000

25 Aligning Course Objectives with Assessment Methods

26 Assessment Methods for “Remembering” Learning Objectives
Type of Learning Objective Examples of Types of Assessment How to Measure Remember Students will be able to: recall recognize identify list Objective Test items that require students to recall or recognize information: Fill-in the Blank Multiple Choice items with question stems such as, “what is a…”, or “which of the following is the definition of…” Labeling diagrams Reciting (orally, musically, or in writing) Accuracy – correct vs. number of errors Item Analysis (at the class level, are there items that had higher error rates? Did some items result in the same errors?)

27 Assessment Methods for “Understanding” Learning Objectives
Type of Learning Objective Examples of Types of Assessment How to Measure Understand Students will be able to: interpret exemplify classify summarize infer compare explain Papers, oral/written exam questions, problems, class discussions, concept maps, homework assignments that require (oral or written): Summarizing readings, films, speeches, etc. Comparing and/or contrasting two or more theories, events, processes, etc. Classifying or categorizing cases, elements, events, etc., using established criteria Paraphrasing documents or speeches Finding or identifying examples or illustrations of a concept, principle Scoring or performance rubrics that identify critical components of the work and discriminates between differing levels of proficiency in addressing the components

28 Using Rubrics to Assess Learning Objectives
What is a rubric? A scoring guide composed of Criteria that you are looking for and Guidelines for evaluating each of those things (Suskie, 2004) Jennifer Roberts, Methods for Assessing Student Learning Outcomes (Workshop 3), NOVA

29 Why Use Rubrics? If the assignment requires an answer more complicated than one that could be corrected with an answer key. When “complex products or behaviors” are being evaluated, which require more than a right or wrong answer. Rubrics are often used to assess how well students perform a task (speaking, writing, performing, etc.) as opposed to whether an answer is right or wrong “quality continuum” from exceptional to not meeting expectations Jennifer Roberts, Methods for Assessing Student Learning Outcomes (Workshop 3), NOVA

30 Assessment Methods for “Applying” Learning Objectives
Type of Learning Objective Examples of Types of Assessment How to Measure Apply Students will be able to: execute implement perform produce solve Activities that require students to use procedures to solve or complete familiar or unfamiliar tasks; may also require students to determine which procedure(s) are most appropriate for a given task.  Labs Performances Problem sets Prototyping Simulations Accuracy scores Check lists Rubrics

31 Assessment Methods for “Analyzing” Learning Objectives
Type of Learning Objective Examples of Types of Assessment How to Measure Analyze Students will be able to: differentiate organize attribute Activities that require students to discriminate or select relevant from irrelevant parts, determine how elements function together, or determine bias, values or underlying intent in presented materials. These might include: Case studies, Critiques, Labs, Papers, Projects, Debates, Concept Maps,   Rubrics Scored by instructor Juries External clients Employers Internship Supervisor Etc.

32 Assessment Methods for “Evaluating” Learning Objectives
Type of Learning Objective Examples of Types of Assessment How to Measure Evaluate Students will be able to: check critique debate justify A range of activities that require students to test, monitor, judge or critique readings, performances, or products against established criteria or standards.  These activities might include: Journals, Diaries, Critiques, Problem Sets, Product Reviews, Case Studies. Rubrics, scored by instructor, juries, external clients, employers, internship supervisor, etc.

33 Assessment Methods for “Creating” Learning Objectives
Type of Learning Objective Examples of Types of Assessment How to Measure Create Students will be able to: generate plan produce Research projects, musical compositions, performances, essays, business plans, website designs, prototyping, set designs Rubrics, scored by instructor, juries, external clients, employers, internship supervisor, etc.

34 Choosing Appropriate Types of Activities
After determining which type of test item would be appropriate for a given learning objective (limited-choice items and open-ended items), one must then decide which type of activity would effectively evaluate student learning. Learning Objectives Most Suitable Test Item Type of Activity The student will be able to name the parts of the human skeletal system. The student will be able to identify common characteristics of various genres of literature. The student will explain the processes and outcomes of communication and miscommunication within groups, teams, and leadership. The student will describe the differences between translating, transliterating, and interpreting. The student will exhibit appropriate laboratory safety skills. Limited- choice or open-ended. Fill-in-the-blanks, Matching, Essay, Short answers, Performance Example from Online Faculty Training on Aligning Test Items with Course Learning Objectives Certain types of exam items (limited-choice vs. open-ended) are better suited than others for measuring different types of learning objectives. The verb in the learning objective can often limit the type of item that can be used. The goal is to write essay items that measure higher cognitive processes. The question should represent a problem situation that tests the student's ability to use knowledge in order to analyze, justify, explain, contrast, evaluate, and so on. Try to use verbs that elicit the kind of thinking you want them to demonstrate. However, while essays can be used to evaluate lower levels of understanding (e.g., knowledge), if you want to measure only the lower levels, limited-choice items can be more efficient.

35 Sample Test Blueprint or Matrix (Table of Specifications)
% of period being tested devoted to topic Level of Knowledge (from Bloom’s Taxonomy) Topic to be tested Questions measuring recall/ comprehension Questions measuring application/ analysis Questions measuring synthesis/ evaluation # of quest-ions % of test devoted to topic Number of Questions % of test devoted to each level of understanding Using a testing blueprint also helps an instructor to address all levels of learning and therefore avoid a very common mistake in tests (i.e., writing all the items at the knowledge level). Example from Online Faculty Training on Aligning Test Items with Course Learning Objectives

36 Course Adjustments CATs are a simple way to assess how students are doing before giving larger assessments.

37 Formative Assessment: Keeping Track and Checking-Up Purpose
Provides information about what a student understands of the concept, knowledge, skills, attitudes/values in order to make instructional decisions that will improve student learning. Good assessment is good learning instruction.

38 Formative assessment answers…
Are we teaching our course objectives? What are our students actually learning? What can we do to help our students learn? What types of changes can we make (to assignments, activities, materials) to increase actual student learning? Jennifer Roberts, Methods for Assessing Student Learning Outcomes (Workshop 3), NOVA

39 Formative Assessment: Keeping Track and Checking-Up Benefits
Alerts the teacher about student misconceptions Allows students to build on previous experiences Engages students actively in the learning process Jennifer Roberts, Methods for Assessing Student Learning Outcomes (Workshop 3), NOVA

40 Formative Assessment: Benefits continued
Provides regular feedback, and adapted instruction to meet identified needs  Feedback: key to learning, reaching goals Speeds up the learning process Uses varied instructional methods and approaches to assessing students understanding and needs

41 Formative Assessment: Benefits continued
Establishes a classroom culture that encourages interaction and discussion of learning Teaches students to monitor and actively evaluate their own learning Provides a challenging yet supportive environment Students realize and have a say in their own role in learning Students learn from other students

42 Formative Assessment: Benefits continued
Provides an opportunity for trial and error without fear of consequences (allows students to make mistakes and learn from them) Provides evidence of progress Addresses both cognitive and motivational factors (teachers “seek to understand not just what students know, but also how they know it” (Huba and Freed 2000)

43 Formative Assessment: Benefits continued
Contributes to general education goals: communication skills, self-awareness, critical judgment, diversity of human culture and experience, awareness of the process of acquiring knowledge (or learning process) “Assessments are learning activities in their own right.” (Suskie, 2009)

44 Reflection What types of formative assessment do I use in my courses?

45 Formative Assessment Examples
Self –assessment Peer-assessment Portfolio review Conference Discussion Quiz Exit card Journal entry Group work Clickers

46 with Student Learning and Attitudes:
Keeping in Tune with Student Learning and Attitudes: Classroom Assessment Techniques (CATs) “Simple, non-graded, in-class activities designed to give you and your students useful feedback on the teaching-learning process.”   CATs are a simple way to assess how students are doing before giving larger assessments. Angelo, T. A., & Cross, P.H. Classroom Assessment Techniques: A Handbook for College Teachers. (2nd ed.) San Francisco: Jossey-Bass, 1993

47 Classroom Assessment Techniques
Benefits Provides short-term feedback about the day-to-day learning and teaching process at a time when it is still possible to make mid-course corrections.

48 Classroom Assessment Technique Examples
Type of Learning Content Question CATs Content/ Material What are students learning? Focused Listing, Empty outlines, Memory Matrix Process/ Barriers How are students learning? Minute Papers, Muddiest Point, Punctuated Pauses, Fish Bowl Application How do students use knowledge/skills? Directed Paraphrasing, Application Cards, Student-Generated Test Questions Study Skills Do students have the tools needed? Course-related self-confidence survey Attitudes What do students think/feel/value? Reading rating sheet, group work evaluation, Minute papers (Palomba, C and Banta, T, "The Essentials of Successful Assessment" in Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education, Jossey-Bass ) (Suskie, L. “Assessing Student Learning: A Common Sense Guide,” Anker Publishing, 2004)

49 Classroom Assessment Techniques (CATs)
Muddiest Point: At the end of class, ask students to jot down a quick response to one question: "What was the muddiest point in the … [class meeting, readings, homework assignment, lecture, etc.] ? (Angelo & Cross, 1993) Additional question (Bateman & Roberts, 1992): What percent of mud was due to: Unclear presentation by instructor? Lack of opportunity to ask questions? Your lack of preparation? Your lack of participation in class instruction? Other? With this feedback Faculty can discover which points are most difficult for students to learn and this can guide their teaching decisions about which topics to emphasize and how much time to spend on each.

50 Pre-Assessment: Finding Out
Engages students and models the integrative process of learning. Research shows that students learn best connecting new knowledge with what they already know. A type of formative assessment To find out what students know So that you can adjust curriculum

51 Pre-Assessment: Finding Out
To determine students: Readiness: skills, concepts, content knowledge Interests Learning Profile: Strengths/weaknesses. Work preferences Self-awareness …in order for instructor to plan/adjust options for students Engages students and models the integrative process of learning. Research shows that students learn best connecting new knowledge with what they already know. A type of formative assessment To find out what students know So that you can adjust curriculum

52 Reflection What types of pre-assessments do you use?

53 Pre-Assessment: Finding Out
Examples: Portfolios Pretests Auditions Concept Map CATs: Prior (Concept) Knowledge, Misconception/Preconception, Journal Prompt Based on your learning objectives/goals?

54 Pre-Assessment: Prior Knowledge
For each of the following Shakespearean plays, place a check mark in the cell if it describes your experience. Play Have Read it Have seen a live performance Have seen a TV or movie production Have written a college-level paper on it Hamlet King Lear Henry IV Othello

55 Pre-Assessment: Concept map example
Graphic representation of students’ understandings

56 Misconception/Preconception
Pre-Assessment: Misconception/Preconception Classroom Assessment Technique (CAT) Native Americans who stay on the reservation are better off. (Anthropology) Strongly Disagree –Disagree- Don’t Know- Agree -Strongly Agree

57 Pre-Assessment: Journal Prompts
How do you feel about __________________? What do you know about __________________? How does ____________________relate to you? When we mention _______________, what do you want to learn about it? ________________________ What learning experiences have you had with __________________? List all the words you know that best explains _____________________. If I asked you to tell me about __________________, you would say ____________________. Quick Write Example: Tell me everything you know about the events that led to the United States entering World War II. Not yes or no questions.

58 Assessment: An On-going Process
Assessment is an on-going process, iterative process Each assessment is a “pilot test” for the next one Keep a record of what worked…and what didn’t There’s always error Each iteration brings you: The benefit of experience A more accepting environment Baselines for future measurements Goal: A valid and reliable assessment Student work: Does it reflect the learning objective? Assessment method: Does it measure the learning objective? Does it measure what you intend it to measure? Assessment: Is scoring consistent? From “The “Nuts and Bolts” of Implementing An Assessment Plan” at From Jennifer Roberts, Methods for Assessing Student Learning Outcomes (Workshop 3), NOVA

59 Objectives for today’s workshop
Distinguish different types of learning objectives Define purposes of assessment Align assessment methods with learning objectives Analyze an assessment method for content validity Evaluate your assessment methods and make improvements Integrate different assessment measures into your courses.

60

61 Further Resources Ambrose, Susan & al How Learning Works: 7 Research-Based Principles for Smart Teaching. SF: Jossey-Bass. Angelo, T. A., & Cross, P.H. Classroom Assessment Techniques: A Handbook for College Teachers. (2nd ed.) San Francisco: Jossey-Bass, 1993. Fink, D. L. (2003). SF: Jossey-Bass. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. Huba, M & J. Freed. (2000). Learner-Centered Assessment on College Campuses. Boston: Allyn & Bacon. NILOA (National Institute for Learning Outcomes Assessment) Roberts, J. Aligning Test Items with Course Learning Objectives (Workshop 7). Roberts, J. Methods for Assessing Student Learning Outcomes (Workshop 3).http://www.nvcc.edu/about-nova/directories--offices/administrative-offices/assessment/resources/index.html Suskie, L. (2009). Assessing student learning: A common sense guide. 2nd Edition. Jossey-Bass: SF. Walvoord, B. (2010). Assessment Clear and Simple. SF: Jossey-Bass. Wiggins, G. & J. McTighe. (2005). Understanding by design. 2nd Edition. Alexandria, VA: Pearson Education.

62 Questions? Contact: Sharon Karkehabadi
Student Learning Outcomes Specialist Office of Institutional Effectiveness and Student Success Initiatives (2-7390)


Download ppt "Aligning Assessment with Course Objectives"

Similar presentations


Ads by Google