What is assessment of student learning? The systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning (Barbara E. Walvoord, Assessment Clear and Simple)
In other words… We identify learning outcomes We design ways to measure the learning We provide learning opportunities We gather, analyze, and interpret evidence to determine how well learning matches our expectations We use the results to understand and improve student learning
Good learning outcomes… Provide a framework for course development Communicate expectations to students Enable assessment of student learning Help evaluate student work Provide information to non-student stakeholders
Learning outcomes… What a student should be able to do or know as a result of having completed a particular course or program Sample learning outcome: Students who successfully complete Tennis 101 will demonstrate skill proficiency in accurately serving a tennis ball
Identifying learning outcomes Upon successful completion of the course, students will appreciate the pig (Example from Barbara Walvoord and Ginny Anderson’s book, Effective Grading — Swine Management Class)
A learning outcome should contain 1. A description of what the student should be able to do or know 2. The conditions under which the student will be able to do the task 3. The measurable standards for evaluating student performance of the task After completing this course you will be able to: operate your phone know how to greet callers understand the procedure for transferring a call
After completing this course you will be able to: place a caller on hold activate the speaker phone play new messages on the voice mail system list the three elements of a proper phone greeting transfer a call to a requested extension After completing this course you will be able to: operate your phone know how to greet callers understand the procedure for transferring a call
Student learning outcomes expressed using verbs related to Bloom’s taxonomy can become the foundation for the selection and design of classroom activities, assignments and tests
Psychomotor Manual or physical skills Cognitive Mental knowledge and skills Learning information and processes for dealing with that information Affective How we deal with emotions Learning beliefs, attitudes, and values Bloom’s Taxonomy and the 3 Domains of Learning
Steps for writing learning outcomes using the domains of learning 1. Decide which domain applies 2. Select an action verb that specifies what the student should be able to know or do and that can be assessed 3. Make sure that the statement includes the conditions under which the student will be able to do the task as well as criteria for evaluation
Knowledge is the ability to memorize and recall existing terminology, facts, and methods Comprehension is the ability to understand the meaning of conceptual information Application is the ability to apply or use previously learned information in a new situation Analysis is the ability to examine a concept and break it down into its parts Synthesis is the ability to put information together in a unique or novel way to solve a problem Evaluation is the ability to judge the relative value of information based on prior knowledge Low level High level
Recall Restate Sequence Investigate Systematize Recite Review Sketch Observe Theorize Recognize Select Solve Order Write Record Transform Use Outside Repeat Translate Recognize Reproduce Select State Tabulate Tell Trace Underline Knowledge Comprehension Application Analysis Synthesis Evaluation Acquire Abstract Apply Advertise Argue Appraise Arrange Associate Calculate Analyze Build Assess Assemble Classify Carry out Breakdown Compose Check Choose Comprehend Compute Catalog Conclude Choose Cite Convert Demonstrate Classify Construct Criticize Count Demonstrate Determine Compare Derive Defend Define Discuss Dramatize Contrast Design Determine Distinguish Draw Detect Discuss Evaluate Draw Estimate Employ Determine Formulate Judge Find Explain Differentiate Generalize Justify Identify Extrapolate Generalize Discover Integrate Measure Indicate Formulate Implement Discriminate Organize Rank Itemize Illustrate Instruct Dissect Plan Select Label Indicate Operate Distinguish Produce Support List Interpret Perform Estimate Propose Test Name Locate Plan Examine Relate Verify Order Predict Explore Restate Weigh Point Report Prepare Identify Specify Quote Represent Repair Interpret Summarize Verbs used for cognitive domain
Receiving relates to getting, holding, and directing the student’s attention and whether the student is “open” to learning Responding involves active participation on the part of the learners Valuing involves the student seeing worth or value in the learning--Clues to valuing are expressed in the student’s overt behavior Organizing involves arranging values into priorities by contrasting different values, resolving conflicts between them, and creating a unique value system (comparing, relating, and synthesizing values) Internalizing values is demonstrated when the student has a value system that controls his/her behavior as a learner Low level High level
Verbs used for affective domain
Psychomotor domain This domain includes physical movement, coordination, and use of large and small motor skills Development of these skills requires practice and is measured in terms of speed, precision, distance, procedures, or techniques in execution Note: Bloom never completed work on this domain Many models have been developed This section is based on Dave’s model (Dave, R. H. (1975). Developing and Writing Behavioural Outcomes. (R J Armstrong, ed.) Educational Innovators Press)
Imitation is the ability to observe the action of another person, such as the teacher or trainer, and copy the action Manipulation is the ability to follow instructions and reproduce an activity from instruction or from memory Precision is the ability to execute a skill reliably, independent of help from the instructor or demonstrate an activity or skill to other learners Articulation is the ability to combine or integrate related skills Naturalization occurs when the skill becomes automatic or internalized Low level High level
Verbs used for psychomotor domain
We design or select learning activities to achieve our outcomes Students should be able to reasonably achieve the learning outcomes by completing these activities. Choose activities that align with and support the learning outcomes.
Course grades versus assessment I grade, therefore I assess Course grades tell students how they did in the class relative to other students and convey to students how well they have met their teachers’ expectations The assignment of a grade to an individual student provides a summary measure about the student’s performance in the class May tell something about the standards of the teacher Course grades do NOT usually Convey direct information about which of the course’s goals and outcomes for learning have been met or how well they have been met
You can use the grading process to measure learning Ensure that the activity, exam or assignment actually measures the learning goal(s) The outcome is to be able to "write a persuasive essay" but the assessment is a multiple choice test State explicitly in writing the criteria for evaluating student work in sufficient detail to identify students’ strengths and weaknesses Use rubrics
The advantages of using rubrics… Allow assessment to be more outcome and consistent Focus the teacher to clarify his/her criteria in specific terms Clearly show the student how their work will be evaluated and what is expected Promote student awareness of about the criteria to use in assessing peer performance Provide useful feedback regarding the effectiveness of the instruction Provide benchmarks against which to measure and document progress
Scoring rubric for tennis serve Excellent (2 points) Acceptable (1 point) Poor (0 points) Accuracy Hits target area at least 80% of the time Hits target area about 50%-80% of the time Hits target area less than 50% of the time Effort Puts forth good effort and tries to serve correctly Shows moderate interest in learning how to serve Shows little or no interest in learning the correct way of serving
321 Participates in class discussions Almost Always Occasionally Almost Never Asks relevant questions in class Almost Always Occasionally Almost Never Participates in online discussion Almost Always Occasionally Almost Never Offers questions or comments via Almost Always Occasionally Almost Never Attends class Almost Always Occasionally Almost Never Arrives on time/stays for entire class Almost Always Occasionally Almost Never Class participation rubric
Evidence and the feedback loop are very important Develop systematic ways of feeding information about student learning back to decision makers to use for program improvement Evidence via assessment is more fundamental than ever to knowing students are learning what they need to learn, to ensuring student learning is central at our institutions, and to demonstrating higher education’s effectiveness to the public and others Assessment is not about amassing data, but rather about analyzing and using it to make a difference in student learning
The Higher Learning Commission cares about… Are students learning what you intended for them to learn? What is the evidence that teaching is effective? The organization’s goals for student learning outcomes are clearly stated for each educational program and make effective assessment possible The organization’s ongoing evaluation and assessment processes provide reliable evidence of institutional effectiveness that clearly informs strategies for continuous improvement What do you most want your students to learn during their time with you? To get from their whole experience at your institution? What are they actually learning? Is it the right learning? Right level? (effective for what and for whom?) What difference do you want to make in their lives? What difference does that make to society? To their profession? What evidence do you have that you’re worth the investment? That they achieve the learning intended?
How are your stated student learning outcomes appropriate to your mission, programs, and degrees? What evidence do you have that students achieve your stated learning outcomes? In what ways do you analyze and use evidence of student learning? How do you ensure shared responsibility for student learning & assessment of student learning? How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
Minute Paper (Angelo, T. A. & Cross, K.P. Classroom Assessment Techniques: A Handbook for College Teachers, 2 nd edition. San Francisco: Jossey-Bass, 1993, pp ) What was the most useful or meaningful thing you learned during this session? What question(s) remain uppermost in your mind as we end this session?