The “Self Regarding Institution” Using Assessment for Accountability and Improvement Douglas R. Davenport Dean, College of Arts and Sciences Truman State University Kirksville, Missouri

Indiana University South Bend Assessment Workshop March 28, 2008

What are your goals? List three things you hope to gain by attending this workshop. Rank them in terms of importance.

Questions to Consider… Why does higher education engage in assessment? assessment What motivates YOU to do assessment?

Assessment at IUSB… What are the best examples of assessment you know of on campus? What might you learn from them? What are the key challenges facing the implementation of assessment on campus? Who is responsible for assessment at IUSB? (Everybody!)

Where Do We Begin? “Begin with the end in mind.” (Covey)

What are our Learning Goals?” Write down the student learning goals for…  Your favorite course to teach  Your academic program  The university Can they be assessed?

Writing Student Learning Outcomes Learning outcome statements should: reflect essential knowledge, skills or attitudes; focus on results of the learning experiences; reflect the desired end of the learning experience, not the means or the process; Learning outcomes statements may be considered to be exit behaviors. (Adapted from Lesch, 2008)

Writing Student Learning Outcomes Key Components: Action word (performance) Learning statement (the learning) Criterion (the conditions of the performance) (Adapted from Lesch, 2008)

Writing Student Learning Outcomes Action words to avoid: Appreciate Enjoy Know Realize Be aware of Perceive Appropriate action words: Use Develop Analyze Express Evaluate Organize Create Write Plan Apply (Adapted from Lesch, 2008)

Exercise… Write one or more learning outcome statements for the following goals… 1. Graduates will be able to communicate effectively in writing. 2. Graduates will have an understanding of cultures beyond the United States. 3. Graduates will appreciate the fine and performing arts.

Time for a Temperature Check… “What is the most important thing you have learned thus far?” “What important questions remain unanswered?”

How Do We Assess Outcomes? General design principles… Know your purposes (accountability and improvement)accountability and improvement Use multiple measures and mixed-methodsmultiple measures Make it your own Don’t reinvent the wheel Start slowly Don’t assess every outcome every year Involve faculty Use authentic assessment wherever possible (e.g., embed assessment in courses, etc)

The Truman Experience Began assessment program in 1972 Started with senior tests Added a writing assessment (SWE) and portfolio (1980s) Emphasized value-added assessment (1990s) Added self-reports (CSEQ, GSQ, CIRP) (2000s)

Truman Assessment Today Nationally-normed senior tests in every major CLA MAPP and CAPP CSEQ CIRP Portfolio Project Student Interview Project Graduating Student Questionnaire

Using Portfolios for Multiple Assessments at Truman Writing Critical Thinking “Modes of Inquiry” Interdisciplinary Thinking ortfolio/index.asp ortfolio/index.asp

Writing Assessment & Portfolios: Rationale “Leverage” current assessment activities A natural location Retains the benefit of embedded assessment Minimizes additional student work

Writing Assessment & Portfolios: Putting it into Practice Tested in 2004, using Critical Thinking Submissions Incorporated in 2005, using a new “Critical Thinking and Writing” prompt

Portfolio Findings The vast majority of students demonstrated competence in all three areas Organization – 81% Style – 81% Mechanics – 84%

Portfolio Findings

Questions?

Why Assessment? - Accountability External Forces…  Spellings Report, A Test of Leadership (NCLB, Part 2)  Public Funding Priorities  Rising Cost of Higher Education  Call for Objective, Comparative Measures  “It's proper for the government to define ‘legally drunk,’ but not ‘legally intelligent.’” (Eubanks, 2006)

College Portrait: Voluntary System of Accountability (VSA) Sponsored by NASULGC and AASCU Provides consumer information regarding institutional measures, student experiences and perceptions, and student learning outcomes. Learning outcomes– CLA, MAPP, and CAPP Experiences and perceptions – CSEQ, CIRP, NSSE, CSS

Why Assessment? - Improvement Assessment for Learning The Learning Organization (Senge) Turbulent Times Turbulent “It is knowledge creation, not knowledge reproduction, that creates competitive advantage.” (Banta, 2007)

Are These Motivations Mutually Exclusive? Not at all… Accountability is a hallmark of democratic institutions. Higher education requires transparency. However, assessment to merely pacify is problematic – it reinforces shallow learning. So…

Characteristics of Assessment by PurposePurpose AccountabilityImprovement AudienceExternalInternal Stakes – InstitutionHigh Stakes – ProgramsLowHigh Stakes – StudentsLowVaries MethodsStandardized Nationally-Normed Criterion Referenced Customized Locally-developed Unit of AnalysisInstitutionProgram or Student Assessment TypeSummativeFormative

Turbulent Times Did you know?…know  Today’s learner will have jobs by age 38.  The top 10 jobs that will be demand in 2010 did not exist in  We are preparing students for jobs that do not yet exist, using technologies that have not been invented in order to solve problems that we have not yet recognized.

Nationally Normed/Criterion Referenced Assessment Instruments – General Collegiate Learning Assessment (CLA - CAE) College Basic Academic Subjects Examination (C BASE – Univ. of MO) Collegiate Assessment of Academic Proficiency (CAAP - ACT) Measure of Academic Proficiency and Progress (MAPP - ETS) iSkills (formerly ICT Literacy Assessment - ETS) Standardized Assessment of Information Literacy Skills (SAILS – Kent State) WorkKeys (ACT) Components of Truman's Assessment Program. Click on an instrument in the left column to learn more.

Nationally Normed/Criterion Referenced Assessment Instruments – Subject Specific Area Concentration Achievement Test (ACAT) Major Field Test (MFT – ETS) Praxis II (ETS) A vast array of subject specific exams are administered through accrediting bodies. Components of Truman's Assessment Program. Click on an instrument in the left column to learn more.

Nationally Normed Assessment Instruments – Self Report National Survey of Student Engagement (NSSE) College Student Experiences Questionnaire (CSEQ) Cooperative Institutional Research Program (CIRP) Freshman Survey (HERI) College Senior Survey (CSS – HERI) Components of Truman's Assessment Program. Click on an instrument in the left column to learn more.

Local Instruments – Some Examples Portfolios Exit Surveys Capstone Projects Juried Theses, Exhibits, Performances Components of Truman's Assessment Program. Click on an instrument in the left column to learn more.

What Do Our Constituents Think? Employers deem both multiple-choice tests of general content knowledge and institutional assessment that show how colleges compare in advancing critical- thinking skills of limited value for evaluating applicants’ potential for success in the workplace …faculty-assessed internships, community-based projects, and senior projects would be the most useful in gauging graduates’ readiness for the workplace. P. Hart Research Associates, AAC&U (January, 2008)January, 2008