Student Interpretation of Learning Outcomes What was learned “testing” the DQP at Saint Mary’s College

Lumina Foundation DQP http://degreeprofile.org/

What is the DQP? “A learning-centered framework for what college graduates should know and be able to do to earn the associate, bachelor’s or master’s degree.” … “proposes specific learning outcomes that benchmark the associate, bachelor’s and master’s degree levels…regardless of a student’s field of specialization.” Inclusive of co-curricular learning

DQP Five Learning Categories Specialized knowledge Broad and integrative knowledge Intellectual skills analytical inquiry use of information resources engagement with diverse perspectives ethical reasoning qualitative fluency communicative fluency Applied and collaborative learning Civic and global learning Institution specific**

How Did the DQP Projects Arise?

Who Developed the DQP? Lumina Foundation Funded Written by: Cliff Adelman – Institute for Higher Education Policy (IHEP) Peter Ewell – VP for the National Center for Higher Education Management Systems (NCHEMS) Paul Gaston – Trustees’ Professor at Kent State University and author of The Challenge of Bologna Carol Geary Schneider – President of Association of American Colleges and University (AAC&U)

Who is Working on this Nationally? Lumina Foundation Funded Projects Higher Learning Commission (HLC) Test the usefulness of framework for focusing student learning outcomes at degree level Western Association of Schools and Colleges (WASC) Redesign accreditation process for member schools with DQP as central reference Council of Independent College (CIC) Explore applicability and usefulness for independent colleges Association of American Colleges and Universities (AAC&U) Test the usefulness of the DQP to facilitate transfer between institutions Since 2011, projects from more than 400 colleges and universities, 4 of the 7 regional accrediting associations, CIC, AASCU, and AAC&U.

Accreditation: Pathways HLC New Accreditation Process Improvement Process Testing the Lumina DQP Assurance Process New Assurance System

Project Goals Provide feedback to the Lumina Foundation and DQP authors Contribute to the national conversation around a degree qualifications framework Reflect on the institution’s own general education program using a national framework for comparison

DQP Testing Projects Departmental College Wide Student Survey Departmental Gap Analysis with Specialized Knowledge Outcomes College Wide College-Wide Gap Analysis with DQP Responding to Gap Analysis Results Consider Potential Tools and Benchmarks for Sophia Program

Specialized Knowledge Student Survey

Specialized Knowledge Student Survey Specialized Knowledge: “…across all fields that we call “majors” lie common learning outcomes involving terminology, theory, methods, tools, literature, complex problems or applications, and cognizance of the limits of the field.” Junior and senior level major students were surveyed from four college departments Nursing Biology Modern Languages Psychology

Specialized Knowledge Student Survey Asked to explain in their own words (in their disciplinary context) the DQP learning outcomes to assess if their major focused on these outcomes, and the extent they believe they achieved them

Qualitative Results: Outcome Example 1 “Defining and properly using the principal specialized terms used in your field, both historical and contemporaneous.” “Principal specialized terms” was often mentioned by students as unclear Questioned how one could identify or agree upon the “principal specialized terms”

Qualitative Results: Outcome Example 2 “Defining and explaining the boundaries and major sub-fields, styles, and/or practices in your field.” “Boundaries” was interpreted in a variety of ways based on the student’s academic discipline. Nursing majors referenced professional boundaries in care provided Some Modern Languages majors interpreted the term “boundaries” to relate to global boundaries sometimes brought on by divides in language.

Qualitative Results: Outcome Example 3 “Demonstrating fluency in the use of tools, technologies, and methods common to your field.” This outcome was understood similarly by Biology, Nursing, and Psychology majors. Modern Languages majors varied in their understanding of the phrase “tools, technologies and methods common to your field” as it relates to their major. “Fluency” was unclear for Modern Languages students who noted the term has a particular meaning in their field.

Findings: Interpretation of Outcomes Able to identify similar general understanding & intention of the outcome Interpretation of some outcomes lack meaningful specificity Outcomes interpreted in the context of the major which sometimes lead to dissimilar understanding Specific words and phrases interpreted in disciplinary context alter outcome meanings

Implications and Follow-up Questions Should we expect students to be able to form a common understanding of student learning outcomes? If not, what does that mean for assessment? And how do students understand our own outcomes? Does disciplinary background impact the interpretation of other outcomes? What happens if students achieve higher level outcomes?

Reaction to DQP Based on Project Findings

Reaction to DQP Based on Findings In theory, it makes sense to have different outcomes for degree levels and the DQP uses a developmental model The DQP relegates mission outcomes to the Institution Specific category. This minimizes the importance of integration of mission across the curriculum. The DQP outcomes can be overly prescriptive or overly broad. (addressed in 2.0 revision) Interpretation of outcomes can be impacted by disciplinary background, leading to lack of common understanding. (Tuning)

Reaction to DQP Based on Findings Overly focused on and too prescriptive about the ideas of problem-solving, the use of multiple media and the use of a non-English language (addressed in 2.0 revision) Unclear about it’s usefulness with students because of lack of clarity in the DQP language Contains implied curriculum requirements that institutions may not currently have (addressed in 2.0 revision)

DQP 2.0

DQP Revision Increased clarity to the document Greater specificity of purpose and specify of assumptions Notes lack of intent toward perscriptiveness Institution-specific outcomes

Replication of this study Further Research Replication of this study Research Institutional benefit Faculty interpretation of learning outcomes Implications for assessment Tuning projects Advising as teaching model

Relevant for AICUP Institutions Broad reach Expansion – framework for certificates and doctoral degrees Relationship to mission Student interpretation of outcomes Expansion – Carnegie Classifications

Jessica Ickes – Director of Institutional Research ickes@lvc.edu Questions? Jessica Ickes – Director of Institutional Research ickes@lvc.edu