Presentation on theme: "Assessing for Learning: Focusing an Interdisciplinary General Education Revision on Student Outcomes Kate Marley, Associate Vice President for Academic."— Presentation transcript:
Assessing for Learning: Focusing an Interdisciplinary General Education Revision on Student Outcomes Kate Marley, Associate Vice President for Academic Affairs John M. Burney, Vice President for Academic Affairs Doane College Ida Asner, Director of Consulting Services LiveText
Presentation Overview Participants will Understand the value of sequencing learning skills and outcomes across four years. Review a curriculum design that reduces broad distribution requirements and provides three levels of liberal arts seminars Learn about a focus on outcomes both for faculty development and for assessment of student learning Understand issues of implementing and sustaining an interdisciplinary curriculum that builds authentic assessment within the course structure
Your issues In a one minute paper jot down the biggest obstacles to designing programs based on LEAP principles and using authentic assessment at your own institutions. Then share those obstacles with a colleague.
Timeline for developing a new general education program : Mission reviewed and revised 2012: Initial writing of new mission-driven general learning outcomes : Slowly built new Gen Ed curriculum Focus on scaffolding student development toward the outcomes Process of draft and comment in faculty forums Ongoing faculty education on active learning through workshops and shareshops May 2013: Revised curriculum passed by faculty : Planning and development Faculty development Design of new courses Revision of existing courses and new approval processes Development of assessment plan, rubric building and testing Review of assessment management software systems : Implementation of the first year of the new curriculum and assessment
Doane’s Mission Doane College’s mission is to provide an exceptional liberal arts education in a creative, inclusive, and collaborative community where faculty and staff work closely with undergraduate and graduate students preparing them for lives rooted in intellectual inquiry, ethical values, and a commitment to engage as leaders and responsible citizens in the world.
Doane Core Connections (DCC) Foundational Areas of Knowledge – 1 course in each of 7 interdisciplinary areas Community and Identity Mathematical Reasoning Rhetorical Communication Global and Cultural Contexts Scientific Perspectives Human Creativity In Search of Meaning and Values Liberal Arts Seminars (LAR) LAR 101 Inquiry Seminar: Learning the Art of Inquiry (freshmen) LAR 202 Integrative Seminar: Democracy and Diversity (sophomores) LAR 303 Impact Seminar: Connecting Knowledge to Choices and Actions (juniors and seniors) Experiential Studies credits
Traditional outcomes Through the liberal arts, students will: Understand foundational areas of knowledge. Community and Identity Mathematical Reasoning Rhetorical Communication Global and Cultural Contexts Scientific Perspectives Human Creativity In Search of Meaning and Values Develop crucial intellectual skills Communicate effectively Use information wisely Pursue a healthy lifestyle
Advanced outcomes Build connections of knowledge across various disciplines. Students will learn to: synthesize knowledge across foundational areas and specialized studies; develop creative and imaginative insights and expressions; and apply and integrate knowledge collaboratively to solve complex problems. Adapt their liberal education to serve and to lead at all levels of citizenship. Students will learn to: pursue a refined, empathetic understanding of a multifaceted world; orient their own ethical compasses to act accordingly; and engage with people of varying perspectives to build just societies.
Hallmark: Progressively building to advanced skills across four years through the Liberal Arts Seminars Traditional skills introduced in LAR 101 Traditional skills practiced, advanced outcomes introduced in LAR 202 Traditional and advanced skills applied in LAR 303 LAR 101LAR 202LAR 303 First Year Second Year Third Year Fourth Year Fall Spring Fall SpringFall Spring Fall Spring
LAR 101 Inquiry Seminar: Learning the Art of Inquiry – First Year A course designed to introduce students to college-level writing, discussion, critical thinking, and critical reading. Faculty will choose a topic for each section in order to help students learn information research skills, to work collaboratively, and to gain an appreciation for interdisciplinary study and multiple perspectives.
LAR 202 Integrative Seminar: Democracy and Diversity – Sophomore Year In a collaborative setting, students will apply and integrate knowledge and experiences to examine complex questions related to democracy and diversity from multiple perspectives. This course can address far-reaching issues that are enduring or contemporary in areas such as culture and values, science and society, global interdependence, citizenship, or human dignity and freedom.
LAR 303 Impact Seminar: Connecting Knowledge to Choices and Actions – Junior or Senior Year This course will develop students’ teamwork and leadership skills to prepare for citizenship or work as they connect theory, practice and experience. Students, drawn from multiple fields of specialized study, will collaboratively analyze a complex real- world problem, develop an empathetic understanding of multiple perspectives needed to comprehend the issue, and propose possible solutions. Students will be engaged through experiential pedagogies such as service learning.
Key to Effective Scaffolding: Pedagogy is as important as content Active Learning Collaborative Learning Peer-to-peer education Role Playing Simulations – Reacting to the Past Inquiry-Based and Problem-Based Learning Service Learning Reflective Writing
Faculty Development Workshops: Feb 2012: Technology and Liberal Education – Bryan Alexander Aug 2012 Meeting Students Where They Are – Jennifer McCrickerd Aug 2012: Trends in General Education – John Burney Aug 2013: Developing Key Skills and Courses for the Core Curriculum – Engebretson, Kalbach, Marley and Burney Jan 2014: Student Reflection and Student Portfolios – John Zubizarreta May 2014: Rubric norming with LAR faculty – Marley and Vertin Aug 2014: Cooperative Learning as an Effective Pedagogy – Karl Smith Jan 2014: Active learning strategies, exploring initial assessment data – Marley and Vertin Faculty teams sent to conferences: AAC&U General Education Collaboration for the Advancement of College Learning HLC Assessment Reacting to the Past Annual Institute
Need for New Direct Assessments Had no direct evidence of student achievement in LAR 101 Now adding two more seminars Faculty buy-in for implementation would be essential Assessment data could provide evidence of seminar effectiveness New Assessment Team of faculty from across college – Charged to develop direct assessments for DCC – 9 faculty and dean Team attended a fall HLC assessment workshop, met weekly Scaffold of Liberal Arts Seminars Tremendous opportunity for DEVELOPMENTAL assessment of outcomes in seminar series
Assessment Team Identified Common Outcome Threads in Seminar Scaffold Critical Thinking Communication Reflection, Synthesis, Integration Collaboration Intellectual Skills Intellectual Engagement
Threads for New Direct Assessment: Intellectual Skills Critical Thinking Communication Intellectual Engagement Learner Self-Awareness Intellectual Curiosity Engagement with Diverse Cultures and Communities Connections Across Disciplines Connections Among Academic and Co-Curricular Experiences
Rubric building Two teams of 6 faculty were tasked with building and testing the rubrics Rubric criteria and descriptions of development written and revised Two rounds of using rubrics with student papers and revision Intellectual Engagement team also developed reflective writing prompts for each level of seminar Rubrics and norming process shared periodically with faculty January faculty workshop on reflective writing with John Zubizarreta May faculty workshop to scale the norming to all LAR and interested faculty Teams provided select sample papers for replication of scoring and discussion in small groups
Doane College Intellectual Skills Rubric CriteriaExceeding (4)Achieving (3)Progressing (2)Beginning (1) Does not meet min. req. (0) Engage in discovery: Student explores a main topic and articulates a position. Problem or issue The central problem or issue is defined clearly along with subsidary questions that explore the complexity of the problem or issue. The central problem or issue is clearly articulated along with indication of the complexity of the issue. The central problem or issue is stated although complexity of the issue may not be addressed. The problem or issue is identified in vague terms and/or distracting sub problems are identified. Problem or issue is not identified. Position - argument, claim, hypothesis, thesis Specific position is stated and sustained throughout. It is thoughtful, creative or unique, and takes into account the complexities of the issue and limitations. Position is stated and sustained throughout. Position is stated but may not be sustained consistently. Position is stated, but may be simplistic, vague and/or confusing. No position is stated. Gather and evaluate facts and assumptions: Student provides context and evidence in support of the position. Context Student provides an exceptional treatment of the context for the problem or issue. Relevant assumptions are presented along with potential biases. Student provides a thorough treatment of the context for problem or issue and is able to articulate relevant assumptions and potential biases. Student provides adequate context for the problem or issue, though may be somewhat lacking in information and/or include irrelevant information. Some assumptions are identified. Student provides some context for the problem or issue, though may lack critical information or irrelevant information may be included. Assumptions are not likely to be identified. No context is provided. Evidence Relevant, credible evidence is gathered, reported, and interpreted clearly, fairly, and accurately. In addition student utilizes evidence in a novel and/or esp. effective way. Relevant, credible evidence is gathered, reported, and interpreted clearly, fairly, and accurately. Relevant evidence is gathered and reported accurately. There may be limited interpretation of evidence or some evidence may not be from credible sources. A minimal amount of relevant evidence is gathered and reported but interpretation is probably not provided. Some irrelevant or distracting evidence may also be included, or evidence may not be from credible sources. No evidence is provided. Develops conclusion with relevant evidence: Student arrives at overall conclusion and articulates implications and consequences. Conclusion and Implications Student clearly and articulately develops overall conclusion from position and evidence. Provides nuanced exploration of implications and consequences. Student develops overall conclusion from position and evidence. Explores implications and consequences of conclusion. Overall conclusion stated. Some statement of implications and consequences may be included, but may be poorly explored. Overall conclusion stated. May not flow clearly from position and evidence, implications and consequences not explored. No conclusion is offered. Practices Effective Communication: Student considers purpose, structure, organization, and appropriate conventions to effectively convey meaning to an intended audience. Purpose and Audience Demonstrates an exceptional understanding of audience and purpose that focuses all elements of the work to effectively convey meaning. Demonstrates understanding of audience and purpose that conveys meaning. Demonstrates an occasionally inconsistent understanding of audience and purpose that occasionally obscures meaning. Demonstrates an inconsistent understanding of audience and purpose that frequently obscures meaning. Lacks understanding of audience and purpose. Structure and organization appropriate to genre, discipline, medium Demonstrates a sophisticated organization to achieve maximum coherence and momentum. Demonstrates coherent organization with consistent flow and continuity. Demonstrates organization but may lack overall coherence, flow, and/or continuity. Demonstrates inconsistent organization, lacking overall coherence, flow, and/or continuity. Lacks adequate organiz. or structure to convey meaning. Conventions approp. to genre, discipline, medium - such as in writing - grammar, sentence fluency, spelling Skillfully communicates meaning with particular clarity, fluency, and possibly creativity. It is virtually error free. Employs skillful communication that conveys meaning effectively. It may have errors, but they do not interfere with understanding. Uses straightforward communication that conveys meaning. It may include errors that occasionally obscure meaning. Uses communication that may include significant number of convention errors which frequently obscure meaning. Use of appropriate communication conventions is so poor as to obscure meaning
Doane College Intellectual Engagement Rubric CriteriaExceeding (4)Achieving (3)Progressing (2)Beginning (1) Does not meet minimum requirements (0) Learner self- awareness Demonstrates the skills of a mature learner, able to reflect in depth, yielding a nuanced or complex awareness of their learning process intrinsically motivating them to guide their own development as a learner. Demonstrates the ability to candidly reflect on own learning process yielding awareness and takes some responsibility to guide own development as a learner. Demonstrates the ability to reflect on own learning process but superficial awareness and action regarding own development as a learner. Demonstrates a willingness to reflect, may inadequately identify strengths and weaknesses, may not understand own role in own learning. No willingness to reflect on own learning. Intellectual curiosity Independently motivated to learn for the sake of increasing own knowledge and understanding, often tenacious, innovative or creative and tend to extend curiosity beyond the classroom and apply learning in other settings. Motivated to acquire knowledge beyond practical ends, appreciates understanding the significance and application of knowledge. Demonstrates interest in acquiring knowledge for practical ends such as improving own performance, shows interest in applying knowledge or understanding its significance. Demonstrates some interest in acquiring knowledge for practical ends such as exams or grades but not in its application or significance. Demonstrates little to no interest in or is apathetic toward the acquisition, application or significance of knowledge. Connections across disciplines Integrates and synthesizes examples, facts, or theories from more than one field of study or takes a multi-disciplinary approach to draw well-reasoned conclusions or creative insights. Connects examples, facts, or theories from more than one point of view or discipline to gain insight. Connects examples, facts, or theories from more than one point of view or discipline. May contain inaccurate or oversimplified statements. Provides examples, facts, or theories from one point of view or discipline. Does not present any facts, examples or theories. Connections among academic, co- curricular, professional and life experiences Meaningfully synthesizes connections among academic, co-curricular, professional and/or life experiences to enhance understanding, broaden own perspectives, and translate learning gains to new situations. Identifies connections among academic, co-curricular, professional and/or life experiences to enhance understanding and broaden own perspectives. Identifies connections among academic, co-curricular, professional and/or life experiences and can state the importance of those connections. Can identify significant academic, co-curriculuar, professional and/or life experiences but has difficulty identifying connections among them. Does not identify signficant academic, co- curricular, professional or life experiences. Engagement with diverse cultures and communities Actively engaged with diverse cultures and communities in order to develop an empathetic understanding of a mutlifaceted world and is eager to continually examine own perspectives in the face of new experiences or learning. Demonstrates an empathetic understanding of a multifaceted world and is willing to examine own perspectives in the face of new experiences or learning. Reflects on how own attitudes and perspectives may be different from those of other cultures and communities and is willing to consider what can be learned. Expresses attitudes and perspectives as an individual, respects differences between themselves and those of other cultures and communities. Expresses attitudes and perspectives as an individual and recognizes differences between themselves and those of other cultures and communities.
Evaluation of assessment and portfolio management systems Team of faculty explored multiple systems Identified critical parameters – ease of use, cost structure, reporting capabilities Scheduled open demonstrations for faculty and staff Collected survey evaluations and recommended LiveText Implementation of the system began with LiveText in spring 2014
Assessment for learning Intellectual Skills Faculty have students submit an assignment that is already part of their course plan targeting student development toward these outcomes Faculty assess in LiveText using Intellectual Skills Rubric Intellectual Engagement Faculty engage students in critical reflection throughout term to deepen student self-awareness, responsibility for own learning Faculty have students respond to a reflective prompt/portfolio that is common across all sections at each seminar level Faculty assess in LiveText using Intellectual Engagement Rubric
LAR 101 Reflection Prompt Critical Reflection assignment instructions: First, read the LAR 101 Doane Core learning outcomes… Second, read William Cronon’s essay “‘Only Connect…’ The Goals of a Liberal Education.” Finally, in an essay of at least three pages address the following: Identify two Doane Core learning outcomes that you think will be important in your own development and connect each to one or more Cronon characteristic. Reflect on the personal importance of these connections, supporting your reflection with specific evidence from your own life. Looking to the future, what will you do next to continue growing? How will you act, in choosing courses or activities or experiences, in ways that will help you to identify or explore your interests, clarify your values, and strengthen your intellectual skills?
LAR 202 Reflection Prompts DIVERSITY AND DEMOCRACY: What do you see as the critical issues that we need to face to create a well-functioning democracy…? Where in your range of academic or co-curricular experiences have democratic processes played a role? What have you learned… Describe an experience when you have engaged with people from diverse background that are different from your own. In this case, how are you defining diversity? What assumptions did you have going into the experience? How did the experience challenge your perspectives and assumptions? If it did not… EDUCATION: Reread your LAR 101 critical reflection on the essay by William Cronon “‘Only Connect…’ The Goals of a Liberal Education” and reflect on how your perspective(s) on liberal education may have changed based on the work you have done…
LAR 303 Showcase Portfolio The comprehensive project in this course will require leadership and collaboration. In order to make the collaboration as effective as possible it is important to know the strengths, skills, and values that each member brings to the team. To open your portfolio, write a cover letter in application for a team that summarizes your strengths, skills, and values. What are you most passionate about? What are your goals for the future? What can you bring to the team to help it construct an understanding of a critical issue… How will your engagement with the team over the critical issue being addressed in your section both further your own goals and help the team to succeed?
LAR 303 Portfolio Evidence Your portfolio should showcase examples of your work that provide evidence of your development in the areas described below. Each sample should be introduced with a short reflection addressing the prompts below and any other relevant ideas. Engage in intellectual inquiry – what is your best example from previous work of developing an argument supported by relevant evidence in order to analyze a critical issue? Communicate effectively – what is/are your most effective achievement(s) in writing, speaking, and/or creative work while in college? Build insights across disciplines – where have you connected knowledge from two or more courses in different disciplines…? Be a productive team member and work collaboratively to solve a problem – what specifically stands out in your contributions to past team efforts…? Pursue an understanding of a multifaceted world – where have you engaged with people from different cultures, beliefs, appearances, abilities, backgrounds, or perspectives, etc in order to challenge and expand your world view…?
LiveText training, implementation, analysis of first term of data Training LiveText staff came to campus during August Faculty week Faculty received accounts and got into system to explore student and faculty views Implementation Fall term 2014: LAR 101 students on our traditional campus and adult campuses successfully submitted assignments, instructors assessed them Analysis Jan 2015 Rubric Review: Team of faculty reviewed initial data, conducted inter and intra-rater reliability scoring Jan 2015 Workshop on Outcomes and Assessment: shared initial results and debrief of processes
Initial results for intellectual skills development in LAR 101 at traditional and adult campuses
Initial results for intellectual engagement in LAR 101 at traditional and adult campuses Trad. Only Adult Only
Next steps Revise and practice Revise rubrics, prompts Foster faculty practice and collaboration regarding rubrics Support development Support development of LAR 202 sophomore seminars and LAR 303 junior/senior seminars Support development of reflective writing and student-centered teaching practices Look ahead Set targets for student development Collect evidence of development across the sequence of seminars, analyze, respond
ASSESSMENT & E-PORTFOLIO SOLUTIONS A versatile and extensible learning tool from LiveText
THE FUTURE OF LEARNING 33 Extracurricular Experience Academic Experience Professional Experience Key Assessments Organizational Achievements Prior Studies Hallmark AssignmentsResume / Profile Personal Achievements Professional Membership Hobbies and Interests Learning Journals Instructor/Faculty Assessment Achievement of University Goals Journal / Life-wide Learning