Presentation on theme: "ASSESSMENT PEER CONSULTANT TRAINING St. Cloud State University January 28 and 29, 2008 Trainers: Patricia Aceves, Elaine Ackerman, Wendy Bjorklund, Lisa."— Presentation transcript:
ASSESSMENT PEER CONSULTANT TRAINING St. Cloud State University January 28 and 29, 2008 Trainers: Patricia Aceves, Elaine Ackerman, Wendy Bjorklund, Lisa Foss, Sandra Johnson, Jim Knutson-Kolodzne, Joe Melcher, Jim Sherohman
INTRODUCTIONS OF PARTICIPANTS AND TRAINERS
Introductions Name and current work assignment Why you want to be an assessment peer consultant A concern, question, or apprehension you have about being an assessment peer consultant Your impressions of Walvoords assessment workshop
INTRODUCTION TO PEER CONSULTANT TRAINING
HLC Assessment Academy 4-year commitment Exempts institutions from accreditation follow- up visits on assessment Centers on completion of student learning projects
SCSUs Assessment Academy Projects University Assessment System Assessment of Student Learning in Programs General Education Assessment
Goals of University Assessment System Project 1) Refine institutional assessment policies 2) Communicate institutional assessment policies 3) Build institutional assessment capacity 4) Integrate assessment into work life of institution 5) Implement assessment of assessment
Goals of Project on Assessment of Student Learning in Programs 1) Implement institutional assessment plan at level of major programs 2) Implement institutional assessment reporting system at program level
Train the Trainers Workshop, May 2007 Did initial planning for peer consultant training Facilitated by Academy mentor Elaine Klein 20 participants from across the campus
Goals of General Education Assessment Project 1) Approve the program 2) Implement program structure 3) Collect and analyze data 4) Use data for improvement
Goals of the Peer Consulting Program Improve program assessment across the institution Build assessment capacity Increase acceptance of assessment of student learning at the program level
Learning Outcomes for Peer Consultant Training Those who complete this training will be able to: Articulate the basic principles of assessment Present the SCSU model of assessment Facilitate the assessment process in departments, programs, and units
CONSULTANT EXPECTATIONS AND THE ASSESSMENT PEER CONSULTING PROGRAM
Assessment Peer Consultants Assist with any aspect of program-level assessment Respond to requests from programs Work with at least one other peer consultant Are familiar with the resources available at SCSU and use these when appropriate Represent the SCSU Assessment Peer Consulting Program and the Assessment Steering Committee
Peer Consulting Process Request submitted to Assessment Office Assessment Director contacts program to obtain additional information about the request Assessment Office selects two (or more) peer consultants with complementary backgrounds Peer consultants meet, then contact the program Peer consultants assist the program Peer consultants provide brief descriptive report to the Assessment Office Program evaluates the peer consulting service
Peer Consulting Requests Peer Consulting Request form How will peer consultants be assigned? Can peer consultants decline assignments? What are some good reasons for doing so?
Some Tips for Peer Consultants 1) Learn about the program before you visit. 2) Focus on the problem as defined by program faculty/staff. 3) Early on, ask questions that will help you understand the situation from their perspective. 4) If you are not actively facilitating, observe and take notes. 5) Help programs find a way to do assessment work themselves; dont do it for them. 6) Nudge the program in the direction of recommended policies, to the extent this is appropriate. 7) Request help when needed.
Peer Consultants as a Team Should there be a lead consultant? How will you share responsibilitydrawing upon each others strengths, while allowing opportunities for each to learn? How will you communicate in the presence of program faculty/staff? Support each other Avoid the appearance of ganging up on program faculty/staff Comment on partners line of action without offending program faculty/staff or the partner
CONSULTANT TRAINING: ASSESSMENT PRINCIPLES
Definition Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; using the resulting information to document, explain, and improve performance.
Assessment Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; using the resulting information to document, explain, and improve performance.
Walvoords Definition the systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning (Barbara Walvoord, Assessment Clear and Simple)
Assessments core questions What do you want students to know/do/value as a result of completing your program? How do you know whether your students know/do/value these things? If they know/do/value these things…youve got something to tell! If NOT, you have useful information to guide changes to improve learning.
W here do programs begin in developing an Assessment Plan? Consultants may assist: Agree on Mission Identify Program Goals
Department Goals Program Educational Objectives Standards and Elements ASSESS Student Learning Outcomes Data Collection and Interpretation Areas and Methods of Assessment Changes to Enhance Student Learning MnSCU Work Plan MnSCU, SCSU, College Mission and Goals Professional/Accrediting Criteria Constituencies Recommendations Department Mission Strategies to Accomplish Outcomes
ASSESS Changes to Enhance Student Learning Student Learning Outcomes Strategies to Accomplish Outcomes Areas and Methods of Assessment Data Collection and Interpretation ASSESSMENT – THE BOTTOM LOOP STUDENT
Walvoords Three Steps of Assessment 1. Articulate your goals for student learning When they complete our program, students will be able to... (Barbara Walvoord, Assessment Clear and Simple) What should consultants know?
At SCSU we say: Identify your student learning outcomes Students will > >
Student Learning Outcomes Learner Centered Specific Action oriented Cognitively appropriate at the program level
Possible Outcomes Students will appreciate…. Students will be exposed to…. Students will demonstrate knowledge of the history, theories and applications of… Students will attend…. Faculty will provide students with opportunities to….
Possible Outcomes Students will appreciate… Students will be exposed to…. Students will demonstrate knowledge of the history, theories and applications of… Students will attend…. Faculty will provide students with opportunities to….
KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate
KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Lower division course outcomes
KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Upper division Course / Program outcomes
Outcome: Reason Scientifically – Geology What is asbestos? Explain how the characteristics of amphibole asbestos make it more conducive to producing lung damage than other fibrous minerals. Given the formula Mg3Si2O5(OH)4, calculate the weight percent of magnesium in chrysotile.
Outcome: Reason Scientifically – Geology Two controversies surround the asbestos hazard: (1) it is nothing more than a very costly fabrication all out of proportion to actual risk, or (2) it is a serious hazard that accounts for tens of thousands of deaths annually. What is the basis for each argument? What is your role as a consultant?
ASSESS Student Learning Outcomes Strategies to Accomplish Outcomes Areas and Methods of Assessment ASSESSMENT – THE BOTTOM LOOP STUDENT
Strategies to accomplish Identify the assignments and activities that demonstrate achievement of each learning outcome Consulting at this stage?
Student Learning Outcomes Course 1 x x Course 2 x x x x Course 3 x x Course 4 x x x x Other x x x Program Matrix
Walvoords Three Steps of Assessment 2. Gather evidence about how well students are meeting the goals. Direct measures Indirect measures Evidence includes qualitative as well as quantitative information. (Barbara Walvoord, Assessment Clear and Simple)
Gather Evidence Using Direct Measures Directly evaluate student work Exams, papers, projects, computer programs, interaction with a client, musical performance (Barbara Walvoord, Assessment Clear and Simple)
Gather Evidence Using Indirect Measures Asking students or alumni how well they thought they learned Tracking graduate school or job placement rates (Barbara Walvoord, Assessment Clear and Simple)
Challenges in Gathering Evidence Recognize appropriate measures to use Data from multiple courses and instructors - interpret, report, and utilize in decision making Relate to outcomes when course or instructor- specific assignments and activities are used
Possible Consulting Requests Identify the traits of successful achievement of the outcome What are we looking for to be able to tell whether or not student performance is acceptable?
Example OUTCOME: Gather factual information and apply it to a given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected BETTER: Students will apply factual information to a problem. TRAITS: Relevance Clarity Comprehensiveness Awareness of Bias
Possible Consulting Requests Identify characteristics of effective/accurate/successful performance for each trait Scale or description for assessing each of the traits Two to five-point scale for each trait
Performance Rubric Exceeds Expectations Meets Expectations Does not meet Expectations Traits Performance Characteristics Descriptions of Unacceptable, Acceptable, or Excellent Performance
Performance Rubric Business Management Exceeds Expectations Meets Expectations Punctuality Teams Customer Satisfaction Skills Performance Characteristics Does not meet Expectations Courtesy Communication Some team members missed appointments or did not return phone calls. Some team members were not respectful of firm employees All team members were always courteous and respectful of all firm employees. All team members were always early. All team members arrived on time for appointments and returned all phone calls promptly. All employees felt that The team members were very courteous and respectful and fully elicited their ideas. Some team members did not communicate clearly during meetings and phone calls. The team members always communicated clearly during meetings and phone calls. The team members always made an extra effort to make sure that they understood us and that we understood them.
ASSESS Changes to Enhance Student Learning Student Learning Outcomes Strategies to Accomplish Outcomes Areas and Methods of Assessment Data Collection and Interpretation ASSESSMENT – THE BOTTOM LOOP STUDENT
Walvoords Three Steps of Assessment 3. Use the information for improvement Pitfall: Gathering data that no one will use It is not enough to gather data about student strengths and weaknesses; you need information and hypotheses about the causes of student weaknesses. (Barbara Walvoord, Assessment Clear and Simple)
The Final Challenge Close the loop Use data – develop hypotheses Review assessment methods Change curriculum and/or instruction Possible consulting issues?
Thinking Outside the Loop Accredited Programs Special Consideration? Programs? Departments? Colleges?
Thinking Outside the Loop More Pitfalls: Mere compliance with external demands Triggering resistance and hostility of faculty Letting administrators do it Making the process too complicated (Barbara Walvoord, Assessment Clear and Simple)
Thinking Outside the Loop Its a passing fad Its not my job or concern It violates academic freedom It violates student privacy You cant assess what we do in academe I already assess - its called grading I dont have time to do assessment – workload! I dont have the expertise Assessment might reveal bad news
INSTITUTIONAL POLICIES AND RESOURCES
SCSU Assessment Structure Organization chart Assessment Steering Committee Assessment Office Institutional Effectiveness Continuing Studies Student Life and Development Other areas
SCSU Recommended Policies Assessment Plans Annual Assessment Reports These policies apply to all undergraduate major programs, graduate programs, and free-standing minor programs These policies have not been approved by the Faculty Senate, but they are recommended by the Assessment Steering Committee
Assessment Plan Mission Student Learning Outcomes Program Matrix – shows which courses or outside-of- course activities will be used to assess each student learning outcome Timeline – identifies when (which year) these assessments will take place
Annual Assessment Report Student learning outcomes assessed Methods and tools used to assess these outcomes Summary of findings for these outcomes Discussion of findings Use of findings for improvement Plan for next year
Other Institutional Policies General education Upper division writing pperDivisionWritingRequirement.asp pperDivisionWritingRequirement.asp Institutional learning outcomes? oups/outcomes.asp oups/outcomes.asp
OFFICE OF INSTITUTIONAL EFFECTIVENESS (OIE)
Office of Institutional Effectiveness (OIE) The Office of Institutional Effectiveness strives to facilitate the integration of planning, institutional research, and assessment to encourage the continuous and dynamic improvement of the Universitys programs and services. The goals of the office include: Create a more balanced and transparent planning process Encourage broad discussion and understanding of university priorities Enhance accountability to internal and external stakeholders Expand use of data in decision-making & budgeting Encourage and support evaluation and assessment that leads to continuous organizational improvement
OIE areas of responsibility Institutional Research: SCSU Strategic Planning: SCSU Action Planning: Minnesota State Colleges & Universities reporting
Data & reports from OIE Enrollment reports Retention and graduation 10 th & 30 th day enrollments Student profile Common Data Set Fact Book and Department Data Reports Graduating Senior Survey Custom data requests (http://www.stcloudstate.edu/oie/research/datarequest.asp)
Nationally normed reports from OIE National Survey of Student Engagement Self-reported data on student experience of freshmen and seniors Administered spring of odd years (01, 03, 05, 07) Collegiate Assessment of Academic Proficiency (CAAP) Measures critical thinking and writing Administered spring 2007 Collegiate Learning Assessment (CLA) Measures critical thinking and writing Administered fall 2007 (freshmen); spring 2008 (seniors)
University Data System The University Data System (UDS) is being created to enhance the use of data in university planning and budgeting, to streamline access to disparate data sources for institutional decision makers, and to increase accountability to internal and external stakeholders. The development of the UDS will contribute to evidence-based planning and resource allocation, and to a culture of continuous improvement at SCSU. Cubes currently under development Instructional Workload (May 2008) Assessment (November 2008) Enrollment Management & Retention (May 2009)
OIE staff & contact information Lisa Helmin Foss, AVP for Institutional Effectiveness Director of Institutional Research (vacant) Deborah Bechtold, Research Analyst David Sikes, Research Analyst
Links to data and reports and information on the Office of Institutional Effectiveness
STUDENT LIFE AND DEVELOPMENT ASSESSMENT 2007
Student Life and Development The Division of Student Life and Development has established learning outcomes and translated them into understandable outcomes for our students.
Student Life and Development Learning Outcomes Diversity and Global Environment. The world we live and interact in is diverse and requires us to interact globally. To effectively engage in todays world students will learn to…
Student Life and Development Learning Outcomes Understand uses of power and nature of oppression, be aware of cultural and personal differences Trust and respect others, empathize, access culturally appropriate resources Be committed to cross-cultural communication, value social responsibility, be committed to social justice
Student Life and Development Learning Outcomes Inter and Intra-Personal Competence. Developing awareness of ourselves and the way we interact with others are essential characteristics. The more we know about our strengths and weaknesses and the impact of our behavior on others, the better we are able to develop healthy relationships. To facilitate this growth students will learn to…
Student Life and Development Learning Outcomes Gain knowledge about themselves, develop consciousness of self, self- confidence, feelings of mattering, manage personal emotions, value cultural heritage Apply self-knowledge, practice self-worth, congruence, commitment, identify passions, discuss cultural differences and issues Work with others, practice collaborations, controversy with civility, engage across difference, be committed to ethical action
Student Life and Development Learning Outcomes Engagement/Social Responsibility and Community Building. it is essential that we learn what it means to work together toward a common good. These skills are important whether on campus now or in our future communities. It is also imperative that we understand our responsibility as educated citizens in our community. Toward this end students will learn to…
Student Life and Development Learning Outcomes Create effective change and practice collective efficacy Develop common purpose Develop civic awareness, value civic responsibility, practice engaged citizenship Develop a sense of connectedness within ones communities
Student Life and Development Learning Outcomes Cognitive and Intellectual Growth. The intellectual and practical skills we need are extensive, sophisticated and expanding with the explosion of new technologies and increasingly complex societies. As we progress through the educational system it is imperative to develop competencies that synthesize specific skills into higher level reasoning capabilities. To develop these qualities students will learn to…
Student Life and Development Learning Outcomes Engage with others in constructive ways, engage in principled dissent; accept and appreciate other world views; manage conflict constructively Develop critical thinking/ decision making skills. Be open to change Foster lifelong learning. Integrate academic knowledge into all aspects of living.
Student Life and Development Learning Outcomes Leadership Exploration and Development. In todays world there are many situations in which we will find ourselves in roles requiring leadership skills. While these skills ultimately develop through training, practice and mentoring, understanding what it means to be a leader is multifaceted. To gain these skills students will learn to…
Student Life and Development Learning Outcomes Develop personal leadership skills, set individual goals, practice risk-taking, delegate, serve as a role model, manage people and tasks, facilitate group processes View leadership as a process, not a position
Student Life and Development Learning Outcomes Critically evaluate leadership models, develop group leadership skills, effectively lead change, develop team building skills, develop leaderships in peers, identify common purpose in groups, help groups set goals, apply problem solving strategies, value recognition and organizational sustainability
Student Life and Development Learning Outcomes Life Skills Competence. Life teaches us many things, though learning to live with purpose and responsibility toward self and others can be quite challenging. Despite these challenges, living a satisfied and competent life provides fulfillment and direction for ourselves and those with whom we come in contact. In order to gain these skills students will learn to…
Student Life and Development Learning Outcomes Develop effective communication skills and practice effective written and oral communication Develop wellness skills Develop life and career path Understand and develop professionalism and self responsibility
Be globally-aware and support diversity. Be self-aware and strive for meaningful relationships. Be an engaged member of my various communities. Take full advantage of opportunities for learning. Explore leadership. Live a balanced and purposeful life. As an SCSU student I will:
Student Life and Development Each department of the division is currently in the process of establishing departmental and programmatic learning outcomes and identifying methods of assessment.
Student Life and Development The SLD assessment plan includes: a process for reviewing department assessment plans an Assessment Skills survey an audit of assessment practices department feedback process training and professional development to help support the faculty and staff skill sets
Student Life and Development Essentially, the challenge is to provide support for the division to develop the competencies inherent to a culture of assessment.
The division must plan proper support for competency development and resources to inspire confidence and move the division forward with regard to assessment, measurement, and effective methodologies. Student Life and Development
It is our intent to secure resources to jumpstart the required competency development; identify needs and assess our competency as a division; and provide in house support to help each member of the division.
Student Life and Development The division is eager to move forward, but has a limited number of personnel who are at a sufficient level of competency The motivation and expectation is present, but the need to measure who needs what support and to provide competency development is critical to an all hands effort.
Student Life and Development This; in turn, sets the expectation and movement to make assessment principles and practices a mainstream part of all program entities in the SLD division and also provides leadership in leading SCSU in a integrated learning direction for the benefit of the students.
Tools and Resources ACPA ASK Standards Needs Assessment Ten-Step Matrix in Student Affairs Assessment Ten-Step Matrix in Student Outcomes Assessment Assessment Practice in Student Affairs: An Applications Manual Student Life & Development Assessment website
SWOT Exercise SWOT is a planning tool used to analyze the strengths, weaknesses, opportunities, and threats associated with meeting a particular strategic objective Strengths and Weaknesses are internal to the organization or unit Opportunities and Threats are external to the organization or unit
SWOT Discussion Form groups of four. Your group should not include anyone else from your department, program, or unit. Describe to the others in your group one weakness of or threat to assessment in the program you analyzed. Lead a discussion of how peer consultants might respond to this weakness or threat. Allow an equal amount of time for each person (about 5 minutes each).
Group Reports Which weaknesses and threats did you discuss in your small group? Which of these were hardest to deal with? Why? Do you see any interesting patterns that you would like to discuss?
Conclusion Procedure for obtaining professional development funds from the Assessment Office Biographical information for website? Evaluation of the training What do you feel most and least confident about as an peer assessment consultant? Would you like additional professional development on any topic related to assessment consulting?