Making Good Use of Existing Information about Student Learning to Inform and Support Curricular Reform Dr. Diane M. Enerson Professor of Psychology & Director of the Center for Teaching Excellence Dr. Heidi Northwood Professor of Philosophy & Director of the Core Curriculum

The Old Core Had become calcified into a set of seemingly unconnected requirements Politics… But no comprehensive data analysis-- dissatisfaction with old Core based on faculty perceptions.

The Union of Insufficiencies What we need therefore is a union of insufficiencies, a marriage of complements, in which the flaws of individual approaches to assessment are offset by the virtues of their fellows. Shulman, L.S. (1988)

The New Core Oversight Faculty endorsed Core SLOs (Skills & Whole Person) Intentionally integrated curriculum (Enduring Questions, Multidisciplinary exploration of central topic, Experiential Learning, etc.) Emphasis on student responsibility & ownership of learning BUT… how do we know if its any better?

Our SLOs SKILLS Read and write with clarity and precision Speak with clarity and precision and actively listen to formal and informal discourse Demonstrate ability to discern the formal, historical and cultural aspects of fine arts Effectively confront and solve problems using quantitative and qualitative methods Identify, evaluate and engage in scientific and empirical modes of inquiry Communicate information symbolically, visually, numerically, and verbally Rationally and objectively apply criteria in order to evaluate and question information or ideas Individually and collaboratively articulate multiple solutions to problems or questions, and evaluate those solutions in both disciplinary and multidisciplinary contexts Identify, access, evaluate, manipulate & use information effectively from a variety of sources Demonstrate the technological skills necessary for personal and scholarly activities Describe and evaluate, from various perspectives, the historical construction of cultures, including ones own Interact effectively in various social and cultural settings WHOLE PERSON Describe and reflect on the learning process Identify, reflect on, and compare different worldviews, philosophies and spiritualities including ones own Evaluate the ethical implications of decisions Achieve an appreciation of and ability to analyze aesthetic qualities of fine arts Explore, evaluate and articulate personal values Integrate and apply academic skills to understanding practical experiences and problems found in our world Demonstrate knowledge of different disciplinary perspectives used to understand and explain the human experience Identify and pursue life choices that promote physical and psychological health and well-being.

Other Things We Thought About Mission of Institution Middle States College of Arts & Science SLOs The data that had been collecting dust on our shelves… NSSE, HERI, LAST and IDEA. The result…

Our SLO Mapping Document Core Skills SLOsMiddle States SLOs College of Arts & Sciences SLOs Possible RubricDirect in Core EARLY Direct in Core LATE Indirect Assessments 1. Read & WriteWritten Communication Communicate eff. through writing AAC&U Written Communication ENGW coursesCore Milestone Exp, LAST (partial) HERI Q14, NSSE Q1c, NSSE Q11c, IDEA Q28 2. Speak & ListenOral Communication Communicate eff. though oral presentation skills AAC&U Oral Communication First Year Seminar & Academic College Success (ACS) CMENSSE NAZ Q11a, NSSE Q1b, NSSE Q11d, IDEA Q28 3. Fine Arts discernOur V/P ArtsTHA, ART, MUS 4. Quan/QualQuantitative Reasoning Develop quant. abilities AAC&U Quantitative Lit MTH, SCILAST (partial) Some Capstones NSSE Q11f 5. ScientificScientific Reasoning Our Scientific Method BIO, CHM, SCILAST (partial) Some Capstones 6. Information ComAAC&U Quan LitMTHLAST (partial) 7. Eval ideasCritical Anaysis/ Reasoning Develop analyt abil and crit thinking Our Crit ThinkingPHL, PSC, ENG, PSYCME Some Capstones HERI Q1, NSSE Q2b, NSSE Q2d, NSSE Q11e, IDEA Q31 8. Problem SolvingCritical Anaysis/ Reasoning Problem solving abilities Our Crit Thinking?All P-EQ coursesSome CapstonesNSSE Q1g, NSSE Qih, NSSE Q1i, NSSE Q2c, NSSE Q11h, XEEW Q11m, IDEA Q23 9. Use InformationCritical Anaysis/ Reasoning & Info Literacy Communicate eff through critical reading skills Our Crit Thinking?HIS, ECOCME, LAST (partial) HERI Q15, NSSE 1d, NSSE Q11g 10. Tech SkillsTechnological Competence To be developedCMENSSE Q1l, NSSE Q10g, NSSE Q11g 11. Hist constructCultural & Global Awareness & Sens Cultural compet & civic engaged AAC&U Intercul Kn & Competence HIS, RES, SOC, ANT, ENG NSSE Q1e 12. Interact socialPrep for Enlightened Judg. Cultural compet & civic engaged AAC&U Intercul Kn & Competence ACSHERI Q10, HERI Q16, HERI Q17, NSSE Q6e, NSSE Q10c, NSSE Q11l

Nazareth College Core Baseline Report INTRODUCTION The following is a compilation of the various indirect assessment data (HERI, NSSE, IDEA) that are related to the new Core Curriculums Student Learning Outcomes. Each question on each survey was judged to be either relevant or not as indirect evidence of an SLO, and further, whether it captured students beliefs about their own progress towards a particular SLO or indicated a behavior or activity that was judged to be relevant to the SLO (indicated below as Related Activities). Data for each question were grouped under each Core SLO (both Core Skills SLOs and Core Whole Person SLOs). In addition, there is a Highlights section for each Core SLO. For the most part, these highlights are only descriptive. The intent of this document is to provide a baseline set of data that can be compared to answers to the same questions on the same survey instruments regularly during the implementation process of the new core, and once it is fully implemented. Starting Fall 2011 we will also have (in addition to the LAST data discussed below) direct measures for some of these SLOs through the use of AAC&U rubrics in our pilots of both Perspective-Enduring Questions courses and First Year Seminar. P.S. The NSSE results for 2011 is very different from that of 2004 and We need to study this. ============================================================================================================================== CORE SKILLS SLO#1: READ AND WRITE WITH CLARITY AND PRECISION (HERI Q14, NSSE Q11c, IDEA Q28, NSSE Q1c; LAST (Written Analytic Expression)) Highlights: 95% of faculty think that it is essential or very important to promote students ability to write effectively. (HERI) By the time they are seniors, 77% (2004) and 76% (2007) and 83% (2011) of students report that their experience at Nazareth has promoted their writing clarity and effectiveness very much or quite a bit. If one adds some to this, the percentages are 92% (2004) and 90% (2007) and 98% (2011). But first year and seniors reports are surprisingly similar, or, in the cases of both 2007 and 2011, a higher percentage of first year students than seniors report that Nazareth has promoted this skill). (NSSE) At the individual course level, the percentages of students who report that they have made substantial or exceptional progress towards developing skill in oral or written expression is much lower (59%-63% range, ). (IDEA) There are significant differences between 2004/2007 and 2011 with regard to student reports about how often they write multiple drafts of a paper or assignment between turning it in. The main difference is that in 2011, students report engaging in this activity a lot in their first year but not so much in their senior year (74% of first year students report that they do this often or very often, whereas 43% of seniors say the same). The opposite trend is found in both 2004 and In their senior year, approximately 60% of seniors (in both 2004 and 2007) report that they often orvery often write multiple drafts of a paper or assignment before turning it in (as opposed to 40% of first year students in 2004, and 33% in 2007; only 17% of seniors in 2007 reported that they never did so. (NSSE Q1c) Also, while the data are not included below, there is a clear trend to shorter paper assignments between 2004 and 2007 (see NSSE Q3c-e). The significance and consequences of this should be more fully studied. Of the five sub-tests in LAST, Written Analytic Expression had the lowest success rate (86%) (cf. Science, Math & Technology: 98%). ______________________________________________________________________________________________________________________________ …and so on…

Where We Are Now Continuing to collect Indirect Assessment Data Faculty beginning to use standard rubrics in new core pilot courses Much discussion & tweaking of rubrics Much discussion & tweaking of who is responsible for what (SLOs, rubrics, etc.) Assessment of pilots (IDEA, rubrics, student evaluation, faculty focus groups)

Your Task Today 1)Select a Student Learning Outcome 2)Choose an approach (i.e., forwards vs. backwards) 3)Using the IDEA data from Utopian U, answer the following: a) What information do we have that might apply? b) How can we use it? c) What inferences can we draw? d) What else do we need to know?

Take Aways DIANE: Persistence pays Together blind men CAN describe an elephant HEIDI: Stop and smell the data Rome was not built in a day