Assessment Peer Consultant Training

Assessment Peer Consultant Training
St. Cloud State University January 28 and 29, 2008 Trainers: Patricia Aceves, Elaine Ackerman, Wendy Bjorklund, Lisa Foss, Sandra Johnson, Jim Knutson-Kolodzne, Joe Melcher, Jim Sherohman

Introductions of Participants and Trainers

Introductions Name and current work assignment
Why you want to be an assessment peer consultant A concern, question, or apprehension you have about being an assessment peer consultant Your impressions of Walvoord’s assessment workshop Everyone responds to the first two questions, then open it up for discussion of the third, then the fourth question.

Introduction to Peer Consultant Training

HLC Assessment Academy
4-year commitment Exempts institutions from accreditation follow- up visits on assessment Centers on completion of student learning projects

SCSU’s Assessment Academy Projects
University Assessment System Assessment of Student Learning in Programs General Education Assessment Refer to Handbook for goals and tasks associated with each of these projects.

Goals of University Assessment System Project
Refine institutional assessment policies Communicate institutional assessment policies Build institutional assessment capacity Integrate assessment into work life of institution Implement assessment of assessment

Goals of Project on Assessment of Student Learning in Programs
Implement institutional assessment plan at level of major programs Implement institutional assessment reporting system at program level

“Train the Trainers” Workshop, May 2007
Did initial planning for peer consultant training Facilitated by Academy mentor Elaine Klein 20 participants from across the campus

Goals of General Education Assessment Project
Approve the program Implement program structure Collect and analyze data Use data for improvement

Goals of the Peer Consulting Program
Improve program assessment across the institution Build assessment capacity Increase acceptance of assessment of student learning at the program level

Learning Outcomes for Peer Consultant Training
Those who complete this training will be able to: Articulate the basic principles of assessment Present the SCSU model of assessment Facilitate the assessment process in departments, programs, and units Comment on correspondence of these goals to outline of training

Consultant Expectations and the Assessment Peer Consulting Program

Assessment Peer Consultants
Assist with any aspect of program-level assessment Respond to requests from programs Work with at least one other peer consultant Are familiar with the resources available at SCSU and use these when appropriate Represent the SCSU Assessment Peer Consulting Program and the Assessment Steering Committee

Peer Consulting Process
Request submitted to Assessment Office Assessment Director contacts program to obtain additional information about the request Assessment Office selects two (or more) peer consultants with complementary backgrounds Peer consultants meet, then contact the program Peer consultants assist the program Peer consultants provide brief descriptive report to the Assessment Office Program evaluates the peer consulting service

Peer Consulting Requests
Peer Consulting Request form How will peer consultants be assigned? Can peer consultants decline assignments? What are some good reasons for doing so? Refer to the peer consultant request form.

Some Tips for Peer Consultants
Learn about the program before you visit. Focus on the problem as defined by program faculty/staff. Early on, ask questions that will help you understand the situation from their perspective. If you are not actively facilitating, observe and take notes. Help programs find a way to do assessment work themselves; don’t do it for them. Nudge the program in the direction of recommended policies, to the extent this is appropriate. Request help when needed.

Peer Consultants as a Team
Should there be a “lead” consultant? How will you share responsibility—drawing upon each other’s strengths, while allowing opportunities for each to learn? How will you communicate in the presence of program faculty/staff? Support each other Avoid the appearance of “ganging up” on program faculty/staff Comment on partner’s line of action without offending program faculty/staff or the partner

Consultant Training: Assessment Principles

Definition Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; using the resulting information to document, explain, and improve performance.

Assessment Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; using the resulting information to document, explain, and improve performance.

Walvoord’s Definition
the systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning (Barbara Walvoord, Assessment Clear and Simple)

Assessment’s core questions
What do you want students to know/do/value as a result of completing your program? How do you know whether your students know/do/value these things? If they know/do/value these things…you’ve got something to tell! If NOT, you have useful information to guide changes to improve learning.

Where do programs begin in developing an Assessment Plan?
Consultants may assist: Agree on Mission Identify Program Goals

ASSESS Department Goals Program Educational Objectives
Standards and Elements MnSCU Work Plan MnSCU, SCSU, College Mission and Goals Professional/Accrediting Criteria Constituencies Recommendations Department Mission ASSESS Student Learning Outcomes Changes to Enhance Student Learning Strategies to Accomplish Outcomes Data Collection and Interpretation Areas and Methods of Assessment

ASSESSMENT – THE BOTTOM LOOP STUDENT Outcomes
Student Learning Outcomes Changes to Enhance Student Learning STUDENT Strategies to Accomplish Outcomes Data Collection and Interpretation Areas and Methods of Assessment

Walvoord’s Three Steps of Assessment
1. Articulate your goals for student learning “When they complete our program, students will be able to . . .” (Barbara Walvoord, Assessment Clear and Simple) What should consultants know?

Identify your student learning outcomes
At SCSU we say: Identify your student learning outcomes Students will <<action verb>> <<something>>

Student Learning Outcomes
Learner Centered Specific Action oriented Cognitively appropriate at the program level

Possible Outcomes Students will appreciate….
Students will be exposed to…. Students will demonstrate knowledge of the history, theories and applications of… Students will attend…. Faculty will provide students with opportunities to….

Possible Outcomes Students will appreciate…
Students will be exposed to…. Students will demonstrate knowledge of the history, theories and applications of… Students will attend…. Faculty will provide students with opportunities to….

COMPREHENSION EVALUATION APPLICATION ANALYSIS SYNTHESIS KNOWLEDGE
Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write

Lower division course outcomes COMPREHENSION EVALUATION APPLICATION
ANALYSIS SYNTHESIS KNOWLEDGE Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Lower division course outcomes

Upper division Course / Program outcomes COMPREHENSION EVALUATION
APPLICATION ANALYSIS SYNTHESIS KNOWLEDGE Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Upper division Course / Program outcomes

Outcome: Reason Scientifically – Geology
What is asbestos? Explain how the characteristics of amphibole asbestos make it more conducive to producing lung damage than other fibrous minerals. Given the formula Mg3Si2O5(OH)4, calculate the weight percent of magnesium in chrysotile.

Outcome: Reason Scientifically – Geology
Two controversies surround the asbestos hazard: (1) it is nothing more than a very costly fabrication all out of proportion to actual risk, or (2) it is a serious hazard that accounts for tens of thousands of deaths annually. What is the basis for each argument? What is your role as a consultant?

Student Learning Outcomes STUDENT Strategies to Accomplish Outcomes Areas and Methods of Assessment

Strategies to accomplish
Identify the assignments and activities that demonstrate achievement of each learning outcome Consulting at this stage?

Program Matrix Student Learning Outcomes x x x x x x x x x x x x x x x
Course 1 Course 2 Course 3 Course 4 Other x x x x x x x x x x x x x x x Program Matrix

2. Gather evidence about how well students are meeting the goals. Direct measures Indirect measures “Evidence includes qualitative as well as quantitative information.” (Barbara Walvoord, Assessment Clear and Simple)

Gather Evidence Using Direct Measures
Directly evaluate student work “Exams, papers, projects, computer programs, interaction with a client, musical performance” (Barbara Walvoord, Assessment Clear and Simple)

Gather Evidence Using Indirect Measures
Asking students or alumni how well they thought they learned Tracking graduate school or job placement rates (Barbara Walvoord, Assessment Clear and Simple)

Challenges in Gathering Evidence
Recognize appropriate measures to use Data from multiple courses and instructors - interpret, report, and utilize in decision making Relate to outcomes when course or instructor- specific assignments and activities are used

Possible Consulting Requests
Identify the traits of successful achievement of the outcome What are we looking for to be able to tell whether or not student performance is acceptable?

Example OUTCOME: Gather factual information and apply it to a given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected BETTER: Students will apply factual information to a problem. TRAITS: Relevance Clarity Comprehensiveness Awareness of Bias

Possible Consulting Requests
Identify characteristics of effective/accurate/successful performance for each trait Scale or description for assessing each of the traits Two to five-point scale for each trait

Performance Characteristics
Performance Rubric Performance Characteristics Does not meet Expectations Meets Expectations Exceeds Expectations Traits Descriptions of Unacceptable, Acceptable, or Excellent Performance

Performance Rubric Business Management
Performance Characteristics Does not meet Expectations Meets Expectations Exceeds Expectations Team’s Customer Satisfaction Skills All team members arrived on time for appointments and returned all phone calls promptly. All team members were always early. Some team members missed appointments or did not return phone calls. Punctuality All employees felt that The team members were very courteous and respectful and fully elicited their ideas. All team members were always courteous and respectful of all firm employees. Some team members were not respectful of firm employees Courtesy The team members always communicated clearly during meetings and phone calls. The team members always made an extra effort to make sure that they understood us and that we understood them. Some team members did not communicate clearly during meetings and phone calls. Communication

Student Learning Outcomes Changes to Enhance Student Learning STUDENT Strategies to Accomplish Outcomes Data Collection and Interpretation Areas and Methods of Assessment

3. Use the information for improvement Pitfall: Gathering data that no one will use “It is not enough to gather data about student strengths and weaknesses; you need information and hypotheses about the causes of student weaknesses.” (Barbara Walvoord, Assessment Clear and Simple)

The Final Challenge Close the loop Use data – develop hypotheses
Review assessment methods Change curriculum and/or instruction Possible consulting issues?

Thinking Outside the Loop
Accredited Programs Special Consideration? Programs? Departments? Colleges?

More Pitfalls: Mere compliance with external demands Triggering resistance and hostility of faculty Letting administrators do it Making the process too complicated (Barbara Walvoord, Assessment Clear and Simple)

It’s a passing fad It’s not my job or concern It violates academic freedom It violates student privacy You can’t assess what we do in academe I already assess - it’s called ‘grading’ I don’t have time to do assessment – workload! I don’t have the expertise Assessment might reveal “bad” news

Institutional Policies and Resources

SCSU Assessment Structure
Organization chart Assessment Steering Committee Assessment Office Institutional Effectiveness Continuing Studies Student Life and Development Other areas

SCSU Recommended Policies
Assessment Plans Annual Assessment Reports These policies apply to all undergraduate major programs, graduate programs, and free-standing minor programs These policies have not been approved by the Faculty Senate, but they are recommended by the Assessment Steering Committee

Assessment Plan Mission Student Learning Outcomes
Program Matrix – shows which courses or outside-of- course activities will be used to assess each student learning outcome Timeline – identifies when (which year) these assessments will take place Assessment plans that do not fit our definition (omit some components, include additional components, use different terminology)

Annual Assessment Report
Student learning outcomes assessed Methods and tools used to assess these outcomes Summary of findings for these outcomes Discussion of findings Use of findings for improvement Plan for next year Problems stemming from use of template (accredited programs, assessment activities that don’t fit, including more detail than is needed, use of reports by college ADs, concerns about use of reports for faculty evaluation)

Other Institutional Policies
General education Upper division writing pperDivisionWritingRequirement.asp Institutional learning outcomes? oups/outcomes.asp

Office of Institutional Effectiveness (OIE)

Office of Institutional Effectiveness (OIE)
The Office of Institutional Effectiveness strives to facilitate the integration of planning, institutional research, and assessment to encourage the continuous and dynamic improvement of the University’s programs and services. The goals of the office include: Create a more balanced and transparent planning process Encourage broad discussion and understanding of university priorities Enhance accountability to internal and external stakeholders Expand use of data in decision-making & budgeting Encourage and support evaluation and assessment that leads to continuous organizational improvement

OIE areas of responsibility
Institutional Research: SCSU Strategic Planning: SCSU Action Planning: Minnesota State Colleges & Universities reporting

Data & reports from OIE Enrollment reports Common Data Set
Retention and graduation 10th & 30th day enrollments Student profile Common Data Set Fact Book and Department Data Reports Graduating Senior Survey Custom data requests (

Nationally normed reports from OIE
National Survey of Student Engagement Self-reported data on student experience of freshmen and seniors Administered spring of odd years (‘01, ‘03, ‘05, ‘07) Collegiate Assessment of Academic Proficiency (CAAP) Measures critical thinking and writing Administered spring 2007 Collegiate Learning Assessment (CLA) Measures critical thinking and writing Administered fall 2007 (freshmen); spring (seniors)

University Data System
The University Data System (UDS) is being created to enhance the use of data in university planning and budgeting, to streamline access to disparate data sources for institutional decision makers, and to increase accountability to internal and external stakeholders. The development of the UDS will contribute to evidence-based planning and resource allocation, and to a culture of continuous improvement at SCSU. Cubes currently under development Instructional Workload (May 2008) Assessment (November 2008) Enrollment Management & Retention (May 2009)

OIE staff & contact information Lisa Helmin Foss, AVP for Institutional Effectiveness Director of Institutional Research (vacant) Deborah Bechtold, Research Analyst David Sikes, Research Analyst

Links to data and reports and information on the Office of Institutional Effectiveness

Student Life and Development Assessment 2007

Student Life and Development
The Division of Student Life and Development has established learning outcomes and translated them into understandable outcomes for our students.

Student Life and Development Learning Outcomes
Diversity and Global Environment. The world we live and interact in is diverse and requires us to interact globally. To effectively engage in today’s world students will learn to…

Understand uses of power and nature of oppression, be aware of cultural and personal differences Trust and respect others, empathize, access culturally appropriate resources Be committed to cross-cultural communication, value social responsibility, be committed to social justice

Inter and Intra-Personal Competence. Developing awareness of ourselves and the way we interact with others are essential characteristics. The more we know about our strengths and weaknesses and the impact of our behavior on others, the better we are able to develop healthy relationships. To facilitate this growth students will learn to…

Gain knowledge about themselves, develop consciousness of self, self- confidence, feelings of mattering, manage personal emotions, value cultural heritage Apply self-knowledge, practice self-worth , congruence, commitment, identify passions, discuss cultural differences and issues Work with others, practice collaborations, controversy with civility, engage across difference, be committed to ethical action

Engagement/Social Responsibility and Community Building. it is essential that we learn what it means to work together toward a common good. These skills are important whether on campus now or in our future communities. It is also imperative that we understand our responsibility as educated citizens in our community. Toward this end students will learn to…

Create effective change and practice collective efficacy Develop common purpose Develop civic awareness, value civic responsibility, practice engaged citizenship Develop a sense of connectedness within one’s communities

Cognitive and Intellectual Growth. The intellectual and practical skills we need are extensive, sophisticated and expanding with the explosion of new technologies and increasingly complex societies. As we progress through the educational system it is imperative to develop competencies that synthesize specific skills into higher level reasoning capabilities. To develop these qualities students will learn to…

Engage with others in constructive ways, engage in principled dissent; accept and appreciate other world views; manage conflict constructively Develop critical thinking/ decision making skills. Be open to change Foster lifelong learning. Integrate academic knowledge into all aspects of living.

Leadership Exploration and Development. In today’s world there are many situations in which we will find ourselves in roles requiring leadership skills. While these skills ultimately develop through training, practice and mentoring, understanding what it means to be a leader is multifaceted. To gain these skills students will learn to…

Develop personal leadership skills, set individual goals, practice risk-taking, delegate, serve as a role model, manage people and tasks, facilitate group processes View leadership as a process, not a position

Critically evaluate leadership models, develop group leadership skills, effectively lead change, develop team building skills, develop leaderships in peers, identify common purpose in groups, help groups set goals, apply problem solving strategies, value recognition and organizational sustainability

Life Skills Competence. Life teaches us many things, though learning to live with purpose and responsibility toward self and others can be quite challenging. Despite these challenges, living a satisfied and competent life provides fulfillment and direction for ourselves and those with whom we come in contact. In order to gain these skills students will learn to…

Develop effective communication skills and practice effective written and oral communication Develop wellness skills Develop life and career path Understand and develop professionalism and self responsibility

As an SCSU student I will:
Be globally-aware and support diversity. Be self-aware and strive for meaningful relationships. Be an engaged member of my various communities. Take full advantage of opportunities for learning. Explore leadership. Live a balanced and purposeful life.

Each department of the division is currently in the process of establishing departmental and programmatic learning outcomes and identifying methods of assessment.

The SLD assessment plan includes: a process for reviewing department assessment plans an Assessment Skills survey an audit of assessment practices department feedback process training and professional development to help support the faculty and staff skill sets

Essentially, the challenge is to provide support for the division to develop the competencies inherent to a culture of assessment.

The division must plan proper support for competency development and resources to inspire confidence and move the division forward with regard to assessment, measurement, and effective methodologies.

It is our intent to secure resources to jumpstart the required competency development; identify needs and assess our competency as a division; and provide ‘in house’ support to help each member of the division.

The division is eager to move forward, but has a limited number of personnel who are at a sufficient level of competency The motivation and expectation is present, but the need to measure who needs what support and to provide competency development is critical to an ‘all hands” effort.

This; in turn, sets the expectation and movement to make assessment principles and practices a mainstream part of all program entities in the SLD division and also provides leadership in leading SCSU in a integrated learning direction for the benefit of the students.

Tools and Resources ACPA ASK Standards Needs Assessment Ten-Step Matrix in Student Affairs Assessment Ten-Step Matrix in Student Outcomes Assessment Assessment Practice in Student Affairs: An Applications Manual Student Life & Development Assessment website

SWOT Exercise

SWOT Exercise SWOT is a planning tool used to analyze the strengths, weaknesses, opportunities, and threats associated with meeting a particular strategic objective Strengths and Weaknesses are internal to the organization or unit Opportunities and Threats are external to the organization or unit

SWOT Discussion Form groups of four.
Your group should not include anyone else from your department, program, or unit. Describe to the others in your group one weakness of or threat to assessment in the program you analyzed. Lead a discussion of how peer consultants might respond to this weakness or threat. Allow an equal amount of time for each person (about 5 minutes each).

Group Reports Which weaknesses and threats did you discuss in your small group? Which of these were hardest to deal with? Why? Do you see any interesting patterns that you would like to discuss?

Conclusion Procedure for obtaining professional development funds from the Assessment Office Biographical information for website? Evaluation of the training What do you feel most and least confident about as an peer assessment consultant? Would you like additional professional development on any topic related to assessment consulting?

Assessment Peer Consultant Training

Similar presentations

Presentation on theme: "Assessment Peer Consultant Training"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Assessment Peer Consultant Training

Similar presentations

Presentation on theme: "Assessment Peer Consultant Training"— Presentation transcript:

Similar presentations

About project

Feedback