Presentation on theme: "Assessment Peer Consultant Training"— Presentation transcript:
1 Assessment Peer Consultant Training St. Cloud State UniversityJanuary 28 and 29, 2008Trainers: Patricia Aceves, Elaine Ackerman, Wendy Bjorklund, Lisa Foss, Sandra Johnson, Jim Knutson-Kolodzne, Joe Melcher, Jim Sherohman
3 Introductions Name and current work assignment Why you want to be an assessment peer consultantA concern, question, or apprehension you have about being an assessment peer consultantYour impressions of Walvoord’s assessment workshopEveryone responds to the first two questions, then open it up for discussion of the third, then the fourth question.
5 HLC Assessment Academy 4-year commitmentExempts institutions from accreditation follow- up visits on assessmentCenters on completion of student learning projects
6 SCSU’s Assessment Academy Projects University Assessment SystemAssessment of Student Learning in ProgramsGeneral Education AssessmentRefer to Handbook for goals and tasks associated with each of these projects.
7 Goals of University Assessment System Project Refine institutional assessment policiesCommunicate institutional assessment policiesBuild institutional assessment capacityIntegrate assessment into work life of institutionImplement assessment of assessment
8 Goals of Project on Assessment of Student Learning in Programs Implement institutional assessment plan at level of major programsImplement institutional assessment reporting system at program level
9 “Train the Trainers” Workshop, May 2007 Did initial planning for peer consultant trainingFacilitated by Academy mentor Elaine Klein20 participants from across the campus
10 Goals of General Education Assessment Project Approve the programImplement program structureCollect and analyze dataUse data for improvement
11 Goals of the Peer Consulting Program Improve program assessment across the institutionBuild assessment capacityIncrease acceptance of assessment of student learning at the program level
12 Learning Outcomes for Peer Consultant Training Those who complete this training will be able to:Articulate the basic principles of assessmentPresent the SCSU model of assessmentFacilitate the assessment process in departments, programs, and unitsComment on correspondence of these goals to outline of training
13 Consultant Expectations and the Assessment Peer Consulting Program
14 Assessment Peer Consultants Assist with any aspect of program-level assessmentRespond to requests from programsWork with at least one other peer consultantAre familiar with the resources available at SCSU and use these when appropriateRepresent the SCSU Assessment Peer Consulting Program and the Assessment Steering Committee
15 Peer Consulting Process Request submitted to Assessment OfficeAssessment Director contacts program to obtain additional information about the requestAssessment Office selects two (or more) peer consultants with complementary backgroundsPeer consultants meet, then contact the programPeer consultants assist the programPeer consultants provide brief descriptive report to the Assessment OfficeProgram evaluates the peer consulting service
16 Peer Consulting Requests Peer Consulting Request formHow will peer consultants be assigned?Can peer consultants decline assignments? What are some good reasons for doing so?Refer to the peer consultant request form.
17 Some Tips for Peer Consultants Learn about the program before you visit.Focus on the problem as defined by program faculty/staff.Early on, ask questions that will help you understand the situation from their perspective.If you are not actively facilitating, observe and take notes.Help programs find a way to do assessment work themselves; don’t do it for them.Nudge the program in the direction of recommended policies, to the extent this is appropriate.Request help when needed.
18 Peer Consultants as a Team Should there be a “lead” consultant?How will you share responsibility—drawing upon each other’s strengths, while allowing opportunities for each to learn?How will you communicate in the presence of program faculty/staff?Support each otherAvoid the appearance of “ganging up” on program faculty/staffComment on partner’s line of action without offending program faculty/staff or the partner
20 DefinitionAssessment is an ongoing process aimed at understanding and improving student learning. It involvesmaking our expectations explicit and public;setting appropriate criteria and high standards for learning quality;systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards;using the resulting information to document, explain, and improve performance.
21 AssessmentAssessment is an ongoing process aimed at understanding and improving student learning. It involvesmaking our expectations explicit and public;setting appropriate criteria and high standards for learning quality;systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards;using the resulting information to document, explain, and improve performance.
22 Walvoord’s Definition the systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning(Barbara Walvoord, Assessment Clear and Simple)
23 Assessment’s core questions What do you want students to know/do/value as a result of completing your program?How do you know whether your students know/do/value these things?If they know/do/value these things…you’ve got something to tell!If NOT, you have useful information to guide changes to improve learning.
24 Where do programs begin in developing an Assessment Plan? Consultants may assist:Agree on MissionIdentify Program Goals
25 ASSESS Department Goals Program Educational Objectives Standards and ElementsMnSCU Work PlanMnSCU, SCSU, College Mission and GoalsProfessional/Accrediting CriteriaConstituencies RecommendationsDepartment MissionASSESSStudent LearningOutcomesChanges to Enhance Student LearningStrategies to Accomplish OutcomesData Collection and InterpretationAreas and Methods of Assessment
26 ASSESSMENT – THE BOTTOM LOOP STUDENT Outcomes Student LearningOutcomesChanges to Enhance Student LearningSTUDENTStrategies to Accomplish OutcomesData Collection and InterpretationAreas and Methods of Assessment
27 Walvoord’s Three Steps of Assessment 1. Articulate your goals for student learning“When they complete our program, students will be able to . . .”(Barbara Walvoord, Assessment Clear and Simple)What should consultants know?
28 Identify your student learning outcomes At SCSU we say:Identify your student learning outcomesStudents will <<action verb>> <<something>>
29 Student Learning Outcomes Learner CenteredSpecificAction orientedCognitively appropriate at the program level
30 Possible Outcomes Students will appreciate…. Students will be exposed to….Students will demonstrate knowledge of the history, theories and applications of…Students will attend….Faculty will provide students with opportunities to….
31 Possible Outcomes Students will appreciate… Students will be exposed to….Students will demonstrate knowledge of the history, theories and applications of…Students will attend….Faculty will provide students with opportunities to….
35 Outcome: Reason Scientifically – Geology What is asbestos?Explain how the characteristics of amphibole asbestos make it more conducive to producing lung damage than other fibrous minerals.Given the formula Mg3Si2O5(OH)4, calculate the weight percent of magnesium in chrysotile.
36 Outcome: Reason Scientifically – Geology Two controversies surround the asbestos hazard: (1) it is nothing more than a very costly fabrication all out of proportion to actual risk, or (2) it is a serious hazard that accounts for tens of thousands of deaths annually. What is the basis for each argument?What is your role as a consultant?
37 ASSESSMENT – THE BOTTOM LOOP STUDENT Outcomes Student LearningOutcomesSTUDENTStrategies to Accomplish OutcomesAreas and Methods of Assessment
38 Strategies to accomplish Identify the assignments and activities that demonstrate achievement of each learning outcomeConsulting at this stage?
39 Program Matrix Student Learning Outcomes x x x x x x x x x x x x x x x Course1Course2Course3Course4OtherxxxxxxxxxxxxxxxProgram Matrix
40 Walvoord’s Three Steps of Assessment 2. Gather evidence about how well students are meeting the goals.Direct measuresIndirect measures“Evidence includes qualitative as well as quantitative information.”(Barbara Walvoord, Assessment Clear and Simple)
41 Gather Evidence Using Direct Measures Directly evaluate student work“Exams, papers, projects, computer programs,interaction with a client, musical performance”(Barbara Walvoord, Assessment Clear and Simple)
42 Gather Evidence Using Indirect Measures Asking students or alumni how well they thought they learnedTracking graduate school or job placement rates(Barbara Walvoord, Assessment Clear and Simple)
43 Challenges in Gathering Evidence Recognize appropriate measures to useData from multiple courses and instructors - interpret, report, and utilize in decision makingRelate to outcomes when course or instructor- specific assignments and activities are used
44 Possible Consulting Requests Identify the traits of successful achievement ofthe outcomeWhat are we looking for to be able to tell whether or not student performance is acceptable?
45 ExampleOUTCOME: Gather factual information and apply it to a given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected BETTER: Students will apply factual information to a problem. TRAITS: Relevance Clarity Comprehensiveness Awareness of Bias
46 Possible Consulting Requests Identify characteristics ofeffective/accurate/successfulperformance for each traitScale or description for assessing each of the traitsTwo to five-point scale for each trait
47 Performance Characteristics Performance RubricPerformance CharacteristicsDoes not meetExpectationsMeetsExpectationsExceedsExpectationsTraitsDescriptionsofUnacceptable,Acceptable,or ExcellentPerformance
48 Performance Rubric Business Management Performance CharacteristicsDoes not meetExpectationsMeetsExpectationsExceedsExpectationsTeam’s CustomerSatisfaction SkillsAll team membersarrived on time forappointmentsand returned all phonecalls promptly.All teammemberswere alwaysearly.Some team membersmissed appointments ordid not return phone calls.PunctualityAll employees felt thatThe team memberswere very courteousand respectful and fullyelicited their ideas.All team memberswere always courteousand respectfulof all firm employees.Some team memberswere not respectful offirm employeesCourtesyThe team membersalways communicatedclearly during meetingsand phone calls.The team membersalways made an extraeffort to make surethat they understoodus and that weunderstood them.Some team membersdid not communicateclearly during meetingsand phone calls.Communication
49 ASSESSMENT – THE BOTTOM LOOP STUDENT Outcomes Student LearningOutcomesChanges to Enhance Student LearningSTUDENTStrategies to Accomplish OutcomesData Collection and InterpretationAreas and Methods of Assessment
50 Walvoord’s Three Steps of Assessment 3. Use the information for improvementPitfall: Gathering data that no one will use“It is not enough to gather data about student strengthsand weaknesses; you need information and hypothesesabout the causes of student weaknesses.”(Barbara Walvoord, Assessment Clear and Simple)
51 The Final Challenge Close the loop Use data – develop hypotheses Review assessment methodsChange curriculum and/or instructionPossible consulting issues?
52 Thinking Outside the Loop Accredited ProgramsSpecial Consideration?Programs?Departments?Colleges?
53 Thinking Outside the Loop More Pitfalls:Mere compliance with external demandsTriggering resistance and hostility of facultyLetting administrators do itMaking the process too complicated(Barbara Walvoord, Assessment Clear and Simple)
54 Thinking Outside the Loop It’s a passing fadIt’s not my job or concernIt violates academic freedomIt violates student privacyYou can’t assess what we do in academeI already assess - it’s called ‘grading’I don’t have time to do assessment – workload!I don’t have the expertiseAssessment might reveal “bad” news
56 SCSU Assessment Structure Organization chartAssessment Steering CommitteeAssessment OfficeInstitutional EffectivenessContinuing StudiesStudent Life and DevelopmentOther areas
57 SCSU Recommended Policies Assessment PlansAnnual Assessment ReportsThese policies apply to all undergraduate major programs, graduate programs, and free-standing minor programsThese policies have not been approved by the Faculty Senate, but they are recommended by the Assessment Steering Committee
58 Assessment Plan Mission Student Learning Outcomes Program Matrix – shows which courses or outside-of- course activities will be used to assess each student learning outcomeTimeline – identifies when (which year) these assessments will take placeAssessment plans that do not fit our definition (omit some components, include additional components, use different terminology)
59 Annual Assessment Report Student learning outcomes assessedMethods and tools used to assess these outcomesSummary of findings for these outcomesDiscussion of findingsUse of findings for improvementPlan for next yearProblems stemming from use of template (accredited programs, assessment activities that don’t fit, including more detail than is needed, use of reports by college ADs, concerns about use of reports for faculty evaluation)
60 Other Institutional Policies General educationUpper division writingpperDivisionWritingRequirement.aspInstitutional learning outcomes?oups/outcomes.asp
62 Office of Institutional Effectiveness (OIE) The Office of Institutional Effectiveness strives to facilitate the integration of planning, institutional research, and assessment to encourage the continuous and dynamic improvement of the University’s programs and services. The goals of the office include:Create a more balanced and transparent planning processEncourage broad discussion and understanding of university prioritiesEnhance accountability to internal and external stakeholdersExpand use of data in decision-making & budgetingEncourage and support evaluation and assessment that leads to continuous organizational improvement
63 OIE areas of responsibility Institutional Research:SCSU Strategic Planning:SCSU Action Planning:Minnesota State Colleges & Universities reporting
64 Data & reports from OIE Enrollment reports Common Data Set Retention and graduation10th & 30th day enrollmentsStudent profileCommon Data SetFact Book and Department Data ReportsGraduating Senior SurveyCustom data requests (http://www.stcloudstate.edu/oie/research/datarequest.asp)
65 Nationally normed reports from OIE National Survey of Student EngagementSelf-reported data on student experience of freshmen and seniorsAdministered spring of odd years (‘01, ‘03, ‘05, ‘07)Collegiate Assessment of Academic Proficiency (CAAP)Measures critical thinking and writingAdministered spring 2007Collegiate Learning Assessment (CLA) Measures critical thinking and writingAdministered fall 2007 (freshmen); spring (seniors)
66 University Data System The University Data System (UDS) is being created to enhance the use of data in university planning and budgeting, to streamline access to disparate data sources for institutional decision makers, and to increase accountability to internal and external stakeholders. The development of the UDS will contribute to evidence-based planning and resource allocation, and to a culture of continuous improvement at SCSU.Cubes currently under developmentInstructional Workload (May 2008)Assessment (November 2008)Enrollment Management & Retention (May 2009)
67 OIE staff & contact informationLisa Helmin Foss, AVP for Institutional EffectivenessDirector of Institutional Research (vacant)Deborah Bechtold, Research AnalystDavid Sikes, Research Analyst
68 Links to data and reports and information on the Office of Institutional Effectiveness
70 Student Life and Development The Division of Student Life and Developmenthas established learning outcomes and translated them into understandable outcomes for our students.
71 Student Life and Development Learning Outcomes Diversity and Global Environment.The world we live and interact in is diverse and requires us to interact globally. To effectively engage in today’s world students will learn to…
72 Student Life and Development Learning Outcomes Understand uses of power and nature of oppression, be aware of cultural and personal differencesTrust and respect others, empathize, access culturally appropriate resourcesBe committed to cross-cultural communication, value social responsibility, be committed to social justice
73 Student Life and Development Learning Outcomes Inter and Intra-Personal Competence. Developing awareness of ourselves and the way we interact with others are essential characteristics. The more we know about our strengths and weaknesses and the impact of our behavior on others, the better we are able to develop healthy relationships. To facilitate this growth students will learn to…
74 Student Life and Development Learning Outcomes Gain knowledge about themselves, develop consciousness of self, self- confidence, feelings of mattering, manage personal emotions, value cultural heritageApply self-knowledge, practice self-worth , congruence, commitment, identify passions, discuss cultural differences and issuesWork with others, practice collaborations, controversy with civility, engage across difference, be committed to ethical action
75 Student Life and Development Learning Outcomes Engagement/Social Responsibility and Community Building.it is essential that we learn what it means to work together toward a common good. These skills are important whether on campus now or in our future communities. It is also imperative that we understand our responsibility as educated citizens in our community. Toward this end students will learn to…
76 Student Life and Development Learning Outcomes Create effective change and practice collective efficacyDevelop common purposeDevelop civic awareness, value civic responsibility, practice engaged citizenshipDevelop a sense of connectedness within one’s communities
77 Student Life and Development Learning Outcomes Cognitive and Intellectual Growth.The intellectual and practical skills we need are extensive, sophisticated and expanding with the explosion of new technologies and increasingly complex societies. As we progress through the educational system it is imperative to develop competencies that synthesize specific skills into higher level reasoning capabilities. To develop these qualities students will learn to…
78 Student Life and Development Learning Outcomes Engage with others in constructive ways, engage in principled dissent; accept and appreciate other world views; manage conflict constructivelyDevelop critical thinking/ decision making skills.Be open to changeFoster lifelong learning.Integrate academic knowledge into all aspects of living.
79 Student Life and Development Learning Outcomes Leadership Exploration and Development.In today’s world there are many situations in which we will find ourselves in roles requiring leadership skills. While these skills ultimately develop through training, practice and mentoring, understanding what it means to be a leader is multifaceted. To gain these skills students will learn to…
80 Student Life and Development Learning Outcomes Develop personal leadership skills, set individual goals, practice risk-taking, delegate, serve as a role model, manage people and tasks, facilitate group processesView leadership as a process, not a position
81 Student Life and Development Learning Outcomes Critically evaluate leadership models, develop group leadership skills, effectively lead change, develop team building skills, develop leaderships in peers, identify common purpose in groups, help groups set goals, apply problem solving strategies, value recognition and organizational sustainability
82 Student Life and Development Learning Outcomes Life Skills Competence. Life teaches us many things, though learning to live with purpose and responsibility toward self and others can be quite challenging. Despite these challenges, living a satisfied and competent life provides fulfillment and direction for ourselves and those with whom we come in contact. In order to gain these skills students will learn to…
83 Student Life and Development Learning Outcomes Develop effective communication skills and practice effective written and oral communicationDevelop wellness skillsDevelop life and career pathUnderstand and develop professionalism and self responsibility
84 As an SCSU student I will: Be globally-aware and support diversity.Be self-aware and strive for meaningful relationships.Be an engaged member of my various communities.Take full advantage of opportunities for learning.Explore leadership.Live a balanced and purposeful life.
85 Student Life and Development Each department of the division is currently in the process of establishing departmental and programmatic learning outcomes and identifying methods of assessment.
86 Student Life and Development The SLD assessment plan includes:a process for reviewing department assessment plansan Assessment Skills surveyan audit of assessment practicesdepartment feedback processtraining and professional development to help support the faculty and staff skill sets
87 Student Life and Development Essentially, the challenge is to provide support for the division to develop the competencies inherent to a culture of assessment.
88 Student Life and Development The division must plan proper support for competency development and resources to inspire confidence and move the division forward with regard to assessment, measurement, and effective methodologies.
89 Student Life and Development It is our intent to secure resources to jumpstart the required competency development; identify needs and assess our competency as a division; and provide ‘in house’ support to help each member of the division.
90 Student Life and Development The division is eager to move forward, but has a limited number of personnel who are at a sufficient level of competencyThe motivation and expectation is present, but the need to measure who needs what support and to provide competency development is critical to an ‘all hands” effort.
91 Student Life and Development This; in turn, sets the expectation and movement to make assessment principles and practices a mainstream part of all program entities in the SLD division and also provides leadership in leading SCSU in a integrated learning direction for the benefit of the students.
92 Tools and ResourcesACPA ASK Standards Needs AssessmentTen-Step Matrix in Student Affairs AssessmentTen-Step Matrix in Student Outcomes AssessmentAssessment Practice in Student Affairs: An Applications ManualStudent Life & Development Assessment website
94 SWOT ExerciseSWOT is a planning tool used to analyze the strengths, weaknesses, opportunities, and threats associated with meeting a particular strategic objectiveStrengths and Weaknesses are internal to the organization or unitOpportunities and Threats are external to the organization or unit
95 SWOT Discussion Form groups of four. Your group should not include anyone else from your department, program, or unit.Describe to the others in your group one weakness of or threat to assessment in the program you analyzed.Lead a discussion of how peer consultants might respond to this weakness or threat.Allow an equal amount of time for each person (about 5 minutes each).
96 Group ReportsWhich weaknesses and threats did you discuss in your small group?Which of these were hardest to deal with? Why?Do you see any interesting patterns that you would like to discuss?
97 ConclusionProcedure for obtaining professional development funds from the Assessment OfficeBiographical information for website?Evaluation of the trainingWhat do you feel most and least confident about as an peer assessment consultant?Would you like additional professional development on any topic related to assessment consulting?