Re-Thinking Assessment in Student Affairs Marilee J. Bresciani, Ph.D. Associate Professor, Postsecondary Education and Co-Director of the Center for Educational Leadership, Innovation, and Policy San Diego State University 3590 Camino Del Rio North San Diego, California, U.S.A

Bresciani, M.J. Session Overview  Context of accountability – building bridges for comparability  Overview of a business lens on product development and quality  Application to student learning and development  Questions to stimulate further thought

Bresciani, M.J. Questions to Consider  How do you explain how well your students have learned what you expect them to learn to: Each other? To the people hiring them? To your institutional governing board members? To your legislators?

Bresciani, M.J. Drivers of Outcomes-Based Assessment ( Ewell, 2003)  To improve the underperforming student  Competency Movement in Business and Industry International Trade Agreements Affecting higher Education - Competition for new providers in Postsecondary or Tertiary Education Transnational Education or Globalism Massification of Education

Bresciani, M.J. Drivers, Cont.  Bologna Declaration of 1999  Government Conversation, 1985 The Higher Education Re-authorization Act Testimonies in USA, 2002 and 2006  Response to NCLB Legislation  Regional Accreditation – flexibility CRAC – 2003, 2004  Both documents focus on student learning

Bresciani, M.J. Drivers, Cont. National Commission on the Future of Higher Education Demand for Public Information about Performance Transparency of outcomes and results Comparable measures of quality Demonstration of value-added of the entire educational experience

Bresciani, M.J. Drivers, Cont.  Accountability requirements handed back to states Performance indicators  Discipline Standards could be designed by disciplines Comparisons still required  Increased focus on private education

Bresciani, M.J. Given the Aforementioned Information  Would you change the manner in which you explain how well your students have learned what you expect them to learn to: Each other? To the people hiring them? To your institutional governing board members? To your legislators?

Context is the Board Member/Legislator who through a Business Lens… is wanting to compare the quality and cost of your students’ learning and development across institutions

Bresciani, M.J. Framing Questions  How do those in the business world evaluate success and quality of their products of their companies?  How does that compare to the manner in which higher education evaluates success and quality?  Given our answers, how do we present success and quality of our students’ learning and development?

Bresciani, M.J. The Context for Business Planning  The purpose of the product is very clear  How the product fits in with the larger picture is also discussed

Bresciani, M.J. The Context, Cont.  Before the product is created, certain steps are taken market analysis  Is the product needed/desired?  Who are the consumers?  Is it a regional market or a broader market?  What price point can the consumers afford?

Bresciani, M.J. The Context, Cont. A development plan identifying the resources that will be needed and their costs is determined.  Who will be involved in development as well as production and how will they be involved?  Development and production costs are compared to potential revenue generated.  Determine what can be outsourced to save money without jeopardizing quality.

Bresciani, M.J. The Context, Cont. Net revenue projections are stated within a timeframe Investors secured; partnerships formed Detailed plans made Create plan to determine quality of the product and compare quality of the product with competitors’ similar products

Bresciani, M.J. The Context, Cont. Devise marketing plan  Refer back to market analysis Create instruction booklets or guidelines Identify warranty, guarantees, and insurance recommended for purchase, if applicable

How does this context apply to how we design, deliver, and evaluate student learning and development in student affairs? How could we explain quality of your students’ learning and development in a business context?

Bresciani, M.J. Questions to Consider  Is our institution clear on the purpose(s) of what student affairs offers? Knowledge Skills Dispositions A combination of the aforementioned Is SA purpose to enhance the specialty

Bresciani, M.J. Questions to Consider, Cont.  How widely shared is that purpose(s)?  How widely adopted is the commitment to practice that purpose(s) across the institution?

Bresciani, M.J. Questions to Consider, Cont.  How does your purpose fit into the larger focus of expected learning within the discipline?  The institution?  The region?  The world?

Bresciani, M.J. Questions to Consider, Cont. Do we want what we do to contribute to economic growth? Do we want what we do to contribute to intellectual development?

Bresciani, M.J. Questions to Consider, Cont.  Do we understand the purpose of what we do within the context of a market analysis? What knowledge/skills/dispositions are needed now and in the future? What kind is desired?  Now and in the future

Bresciani, M.J. Questions to Consider, Cont.  Who are the consumers of what we have to offer? The students? Their employers? Their graduate/professional schools? The faculty within the disciplines?

Bresciani, M.J. Questions to Consider, Cont.  Are your consumers local? Or international?  What price point can the consumer afford?  How can you make the learning and development experience more affordable?

Bresciani, M.J. Questions to Consider, Cont.  What is the plan to develop the student learning “curriculum”?  Who is involved in development?  With whom can we partner?  What are the intended end results?

Bresciani, M.J. Questions to Consider, Cont.  What resources are needed to design the “curriculum” and what is the cost?  What is needed to deliver the curriculum and what are the costs?  What is the cost to get the student to be able to receive and apply the curriculum?

Bresciani, M.J. Questions to Consider, Cont.  What could be out-sourced to make the learning opportunity more affordable without sacrificing quality?  Can quality be sacrificed to make learning more affordable? Or more accessible?

Bresciani, M.J. Questions to Consider, Cont.  What are the net learning and development gains projected and by when?  Who are the potential investors in the student learning and development?  When can they expect to see a return on their investment?

Bresciani, M.J. Questions to Consider, Cont.  How will we determine the quality of the student learning?  Situational within “subject”  Transferable to discipline  Comparison Within subject across discipline within institution Within subject across institutions Within subject across discipline across institutions

Bresciani, M.J. Questions to Consider, Cont. How will we compare that quality of learning with our competitors? How do we determine who the competitors are? How will we use consumer ratings in our comparison reporting?

Bresciani, M.J. Questions to Consider, Cont. How will we use experts’ ratings of our student learning in comparison reporting?  What internal and external influences on quality will we take into account? Do we stay within our regional market when comparing? Or is our market broader than our region?

Bresciani, M.J. Questions to Consider, Cont. What is the plan for marketing the student learning?  To whom are we marketing?  How and what will we market? Description of what student learning is What you have to do to participate What you will get from participating The overall benefits of having the learning

Some products are virtually worthless without their networks on which they run

Bresciani, M.J. Networks  What type of networks are needed to ensure the student learning and development keeps working? On-going facilitation of learning? On-going stimulation of learning? On-going support?

Bresciani, M.J. Product Warranty  How long is the learning guaranteed for?  Can they return for a full refund if their learning isn’t applicable? Or a “merchandise” certificate? Will there be varying return policies for various types of learning?

Bresciani, M.J. Returns, Cont. Or will the return policies be determined by institution and without regard to type of learning? What are the limitations on the return?  See “product disclaimers”

Bresciani, M.J. Upgrade Announcements  When are you warned that your learning will most likely be “out-dated” or in need of replacement?  Are you told how to get systematic upgrades of your learning?

What set of instructions and guidelines should go along with the application of the student learning?

Bresciani, M.J. Product Disclaimers Student inputs Student investment in learning/development Student learning/development challenges Transferability to a new “discipline” or region may be questionable

Bresciani, M.J. The Bottom Line  Provide clarity around purpose of and costs of delivering and evaluating student learning  Provide clarity about benefits of student learning  Allow consumers to determine the value of their net revenue/benefits generated

Bresciani, M.J. Tips from Good Practice Institutions  They clearly define their “product” through learning outcomes  They align the process for creating the product with the manner in which they evaluate the product  They advertise the variances in their processes  They advertise the “success” of their product

Bresciani, M.J. Tips, Cont.  They invite peer critiques, internal to their organization and external to their organization  They use testimonials  Some articulate expectations of “investments” to students and faculty  Some are mindful of their ”inputs”

Questions or Thoughts?

Bresciani, M.J. Citation for this Presentation  Bresciani, M.J. (Under Review). Uncovering General Learning: Presenting Findings of General Education within a Bottom-line Business World. NASPA Leadership Exchange

Bresciani, M.J. References, Cont.  Bresciani, M. J. (October 10, 2007). Accountability in higher education: Driven by business or social responsibility (Part I). Net Results: NASPA’s E-Zine for Student Affairs Professionals. m?id=1606  Bresciani, M. J. (October 24, 2007). Accountability in higher education: Driven by business or social responsibility (Part II). Net Results: NASPA’s E-Zine for Student Affairs Professionals. m?id=1609

Bresciani, M.J. References, Cont.  Bresciani, M. J. (Ed.). (2007). Good practice case studies for assessing student learning in general education. San Francisco, CA: Jossey Bass.  Higher Education Act Drafts and Amendments  Jaschik, S. (November 20, 2007). Another Call for Assessment. insidehighered.com