2Outcomes for today’s Session At the end of today’s session, participants will be able toExplain what outcomes assessment is and why we do itIdentify the requirements and expectations for the MC OA processInform disciplines about student learning outcomes assessment at MCDevelop a meaningful assessment planInvolve disciplines in the development of an OA plan
3OA- Overview and Goals What is outcomes assessment? Why do we do it? What do we assess?What do we do with the assessment information?
4What is Outcomes Assessment? Outcomes Assessment is an ongoing, systematic process ofidentifying student learning outcomes,assessing student achievement of those outcomes,and using the results to improve student learning.
5The Ultimate GoalDevelop a series of recommendations that lead to the improved student learning.Recommendations should be:Action orientatedRelated to student learningBased on data and OA process
6Reaching Meaningful Recommendations Ask important, meaningful questionsCreate an authentic, realistic assessmentHave meaningful, honest discussions among faculty to analyze instructional practices, performance expectations, student preparation and grading practices.Use OA to make positive changes happen for students.
7Outcomes Assessment at Montgomery College - What We Want to Achieve? Improve Student LearningUse college-wide assessment to help set academic standardsUse assessment to stimulate faculty discussion on:Course contentCourse designInstructional methodologyStandardsGeneral Education
8Outcomes Assessment at Montgomery College - What We Want to Achieve (Cont.) Assessment of General Education competenciesWritten & oral communicationInformation literacyCritical thinkingQuantitative & scientific reasoningTechnological competencyAssessment of course content in our most highly enrolled coursesCreate a community of practitioners rather than independent agents
9An OA Process Designed to Achieve Our Goals Course based and course embedded assessment of three outcomes selected by discipline facultyFaculty work groups lead disciplines to choose outcomes to assess and develop assessment toolsA two-year assessment cycle with 4 phases:Planning & pilotingFull-scale assessmentData analysisRecommendation & follow-upActive participation of all facultyCommon assessments & scoring within each course, every sectionGen Ed courses select and assess at least one outcome that aligns/embeds with a Gen Ed competency
10What do we do with the assessment information? We don’t do anything with the information.Your disciplines should use the informationto improve student learning by aligning instructional activities with student performance.the information to celebrate the success of our courses and programs by identifying strong practices.the information to demonstrate accountability for student learning.
11What don’t we do with the assessment information? We don’t use information obtained through this process to evaluate faculty performance. Outcomes assessment is about evaluating the effectiveness of programs, not individuals.
132-year Time Frame Semester 1- Planning Semester 2- Pilot and Feedback OA PlanSemester 2- Pilot and FeedbackDiscipline Pilots OA PlanSemester 3- Full ImplementationFull Discipline participates in the OA PlanSemester 4 –Recommendations and ObservationsData Analysis conducted by OA Data Resource TeamDiscipline reviews results/data and makes observations about strengths and weaknesses, and makes recommendations.
14What is MC’s OA process? College-wide Course embedded Common task all sections and instructors teaching the course participate.Course embeddedAssessment instrument is developed as part of the course, not separate or additionalCommon taskAll instructors use the same discipline developed assessment instrument and scoring tools.Faculty drivenDisciplines determine outcomes to assess, develop assessment tools and come up with observations and recommendations.
15Who is involved? Leading Courses Other Selected based on number of sections(10 or more in a fall or spring semester) and included in a 5 year rotating cycleOtherDisciplines or course groups can elect to create an assessment plan for various purposes.
17Workgroup Responsibilities Develop and implement an OA plan for the course. OA Plan consists of:Identified Student Learning Outcomes that are meaningful for the discipline and aligned with General Education Competencies where appropriateSupporting Student Activities WorksheetAssessment InstrumentScoring ToolsData Collection InformationFacilitate discipline discussion about OA Plan and incorporate discipline feedbackCoordinate the implementation of the OA plan in the discipline
18Discipline Faculty Responsibilities Participate in the development of the OA PlanCommunicate collegewide, common, core, course outcomes to studentsCommunicate assessment expectations (detailed instructions, scoring rubrics, etc.) to studentsParticipate in the assessment
19College-wide Outcomes Assessment Team Responsibilities Supporting the development of the OA PlanGiving feedback on the planFacilitating the collection of dataReporting resultsProviding training and leadership about OA
20How do the Gen Ed Competencies fit in? General EducationHow do the Gen Ed Competencies fit in?
21What are the Gen Ed assessment expectations? Review the primary Gen Ed competencies selected for the course.Develop an assessment instrument that gives student the opportunity to demonstrate those competencies.Collect student performance information on the selected competencies.
22Embedding Gen Ed. Competencies SLO- Students will be able to discuss the extent and impact of crime in America today (CJ 110)Gen Ed Competency- Effective Written and Oral communicationAssessment- Oral presentation or written report that is scored on both the content and elements of either oral or written communication.
25Part One: Asking a Meaningful Question OA is not hard science as we are doing it- it is more closely aligned to action research.Data that you collect is only as meaningful as you make it.
26Exercise #1- Part AImagine you are at a discipline conference and you meet a colleague teaching your course at a similar institution.How would you describe this course to your colleague?What are the most important skills/knowledge/attitude that students take away from the course?
27Exercise #1- Part B Consider how your students perform in your course. Complete the following statements relating to course content:I wish my students would/could :I am impressed that many of my students can:How does the performance of students in your classroom reflect the performance in your colleagues classrooms? National trends?
28Exercise #1- Part CAs a group, using your course outcomes and your thoughts about student performance, decide on one area of student performance that would useful for your discipline to examine and document.Align that area to a course outcome if it is not already clear.For General Education courses, consider whether the selected outcome aligns with a general education competency that is core to the course
30Designing Assessment Instruments Imagine you are meeting a student who took your course several years ago, if they successfully mastered the outcome you have selected, how would it be reflected in this student’s life?Consider what has changed in a student who has mastered this outcome- what can they do now that they couldn’t do before?
31Supporting Student Activities Consider what you and your colleagues do in your courses now to give students the opportunity to learn this outcome. What activities or experiences are good strategies to give students opportunities to practice and master this outcome.How do you currently know whether students achieve this outcome? What do you currently use to measure this outcome?For Gen Ed- how do you currently help students become more proficient in the competency?
32Assessment Instrument Exercise In your groups, share the lists you have made that reflect your current instructional activities and your views on how these outcomes are reflected in students who have mastered the course.As a group, select an activity/assignment that can be used to measure the outcome and that reflects what you expect students to be able to do as a result of this outcome.
33Assessment Instrument tips Alignment-Assessment should measure the outcome it is designed to measure. The assessment tool aligns with the course and SLO.Communication/Clarity-The instructions for students and instructors are easily understood and lead to consistent use of the assessment tool. Students are informed of the assessment, the expectations and the way the assessment will be scored in advance.Course Embedded-The assessment tool can easily be given in all classrooms. The assessment can be graded as well as scored as the individual instructor/department sees fit.
34Assessment Instrument Tips (con.) Accommodation/Fairness-The assessment tool allows for student accommodations as appropriate. Difficulty- The assessment should reflect the level of the course.Reliability-The assessment tool provides for reasonable reliability. The assessment will provide information which is useful.Feasibility-The assessment is relatively easy to implement and doesn’t require excessive time for faculty to score, report or implement. Assessments should be rolled into one assessment when practical
35Assessment Instrument Tip (con.) AvoidAssessments that are developed for ease of delivery.Avoid making students think the assessment is unimportant.Assessments that are overly complicated or burdensome.Assessments that will not provide useful information
36Part Three: Developing a Scoring Tool and Data Collection information
37Scoring vs Grading Scoring Grading Determining student achievement of certain outcomesGradingLooking at the whole product and assigning it a grade in the -context of the class.
38Alignment/Task Analysis To develop a scoring tool, first think about all the things a student must do in order to successfully complete this assessment.If you were completing this assignment, what would you do to complete it?Think about how the outcome is reflected in the assessment instrument.How will you know that students have completed each step or element?
39Scoring Tool ExerciseMake a list of all the steps/activities that a student must complete or that you would be looking for in a successful completion of the assignment.Determine how you will collect that information- will you use a rubric? Will you identify specific questions or parts of answer to score?
40Scoring Tool Advice- Rubric Identify the areas you want to collect information about- (Create dimensions)Develop descriptions that characterize student performance on that dimension (Develop Criteria)Assign values to various levels of performance (Scale)
41Data CollectionOnce you have your scoring tool developed and aligned with the SLO, you will need to determine what headings should be used for each score the instructor will enter.See examples and work with your cadre person on this.Determine what is considered satisfactory on each scoring element and holistically.
42Final ThoughtsMake sure to involve the discipline in the selection of outcomes and development of the assessment instrument.Work with your Cadre person and meet deadlines so we can give you good feedback.Use the handbook as a resource.Ask for help!