1. Advancing a Massachusetts Culture of Assessment
Standard 8: Educational Effectiveness
There really is no place to hide!
Advancing a Massachusetts Culture of Assessment
Quinsigamond Community College
November 30, 2018

2. NECHE?! What happened to CIHE of NEASC?
The Commission on Institutions of Higher Education of the New England Association of Schools and Colleges has become the New England Commission of Higher Education! We are a separate 501(c)(3) organization that maintains an historic connection and cordial relationship with NEASC.

3. NECHE: What’s changed? New name New acronym (pronounced neh-chee)
New logo
New letterhead
New addresses (first
New website (by year’s end)

4. NECHE: What hasn’t changed?
Our address
Our staff (except for two new staff members)
Commission members (except for usual rotation)
The Standards, policies & procedures
The decennial accreditation cycle
Your students’ access to federal financial aid
Our commitment to the two purposes of accreditation: assuring quality and fostering improvement

5. Dual Purposes of Accreditation
Assure quality
Foster improvement

6. NECHE: What’s in process?
Reprinting Standards, manuals
Updating policies with new header
Transition to new website
Annual Meeting: December Boston Marriott Copley Place

7. Revised Standards (effective 2016)
Mission and Purposes
Teaching, Learning and Scholarship
Planning and Evaluation
Institutional Resources
Organization and Governance
Educational Effectiveness
The Academic Program
Integrity, Transparency, and Public Disclosure
Students
Format:
Statement of the Standard – a summary in bold
Numbered paragraphs – to explicate the statement of the Standard
Subheadings – for organization and clarity

8. The current environment …
Senator Lamar Alexander (R, TN), Chairman of the Senate Committee overseeing higher education: “Lawmakers have a duty to make sure students spend their federal aid at a good college. We need to find a way to make accreditation work better.”
24 U.S. Senate Democrats: Accreditors have often failed to establish standards that ensure sufficient numbers of students are persisting in their course of study, completing their programs, and are able to find new or better jobs in their field when they graduate.
Judith Eaton, President of the Council for Higher Education Accreditation: “… credibility and confidence in an accreditor increasingly will rest on data about institutional performance on student achievement.”
Mount Ida College …

9. Three major changes in 2016 Less emphasis on inputs/resources:
Standards 7 (Library and Other Information Resources) 8 (Physical and Technological Resources) and 9 (Financial Resources) combined into a single new standard, Institutional Resources

10. Three Dimensions of Quality
Input Process Outcome
Are there enough books in the library?
Are students gaining skills of information literacy?
Are students using the books?
…………………………………………………………
Are the faculty well qualified?
Is the curriculum appropriate?
Is there good instructional practice?
Do students get practice and feedback?
Are students achieving the learning outcomes of the program and institution?

11. Three major changes in 2016
2. Combine Standards 10 (Public Disclosure) and 11 (Integrity) to form a new standard: Integrity, Transparency, and Public Disclosure
More emphasis on what is owed to the public
Especially For...
Prospective Students
Current Students
Faculty and Staff
Alumni and Donors
Parents and Visitors

12. Standard 9: What to disclose?
What Aunt Miriam wants to know …
Mission, character, size, location(s), president, board
Student body: Who goes there?
Programs: What’s on offer for study?
Resources: Faculty, library, labs, technology
Services: What help is available?
Policies: How can credit transfer?
Opportunities: What else besides class?
Cost: Tuition & fees, aid, debt, default, repayment
Results: What do graduates do? [How] are they successful?

13. More about disclosing “results”
9.16 The institution has readily available valid documentation [for any statements and promises regarding such matters as program excellence, learning outcomes, success in placement, and achievements of graduates or faculty.
9.24 The institution publishes statements of its goals for students’ education and success of students in achieving those goals. Information on student success includes rates of retention and graduation and other measures of student success appropriate to institutional mission. If applicable, recent information on passage rates for licensure examinations is also published.
9.25 The expected amount of student debt upon graduation and the institution’s cohort default and loan repayment rates are published to help students and prospective students make informed decisions.

14. Student Success on the Annual Report
URLs of the principal websites where the institution provides the public with information about student achievement and success
An interesting exercise:
Go to your institution’s website and search for:
Retention rates Graduation rates
Default rates Outcomes
How many clicks? Is there a “story”?
Do you understand what you see?

15. Three major changes in 2016 3. Heightened Emphases on Student Success
Existing Standards:
Parts of Standards 2 (Planning and Evaluation) and 4 (The Academic Program) focus on evaluating and improving student learning and success
New Standard: Educational Effectiveness*:
Combines parts of Standards 2 (Planning and Evaluation), 4 (The Academic Program), 6 (Students)
More attention to mission-appropriate measures of student success, including quantitative measures of success (e.g., rates of progression, retention, transfer, graduation; licensure passage rates; employment)
* aka: the “no place to hide” standard

16. Standard 2: Planning and Evaluation
The institution undertakes planning and evaluation to accomplish and improve the achievement of its mission and purposes. It identifies its planning and evaluation priorities and pursues them effectively. The institution demonstrates its success in strategic, academic, financial, and other resource planning and the evaluation of its educational effectiveness.

17. Standard 2: Measures of Student Success
2.7 The institution’s principal evaluation focus is the quality, integrity, and effectiveness of its academic programs. Evaluation endeavors and systematic assessment are demonstrably effective in the improvement of academic offerings, student learning, and the student experience. Systematic feedback from students, former students, and other relevant constituencies is a demonstrable factor in institutional improvement. 2.8 The institution has a demonstrable record of success in using the results of its evaluation activities to inform planning, changes in programs and services, and resource allocation.

18. Standard 4: The Academic Program
The institution’s academic programs are consistent with and serve to fulfill its mission and purposes. The institution works systematically and effectively to plan, provide, oversee, evaluate, improve, and assure the academic quality and integrity of its academic programs and the credits and degrees awarded. The institution sets a standard of student achievement appropriate to the degree or certificate awarded and develops the systematic means to understand how and what students are learning and to use the evidence obtained to improve the academic program.

19. Standard 4: Measures of Student Success
4.4 The institution offering multiple academic programs ensures that all programs meet or exceed the basic quality standards of the institution and that there is a reasonable consistency in quality among them. The institution provides sufficient resources to sustain and improve its academic programs. 4.6 The institution develops, approves, administers, and on a regular cycle reviews its academic programs under institutional policies that are implemented by designated bodies with established channels of communication and control. Review of academic programs includes evidence of student success and program effectiveness and incorporates an external perspective. Faculty have a substantive voice in these matters.

20. Standard 8 Educational Effectiveness
The institution demonstrates its effectiveness by ensuring satisfactory levels of student achievement on mission-appropriate student outcomes. Based on verifiable information, the institution understands what its students have gained as a result of their education and has useful evidence about the success of its recent graduates. This information is used for planning and improvement, resource allocation and to inform the public about the institution. Student achievement is at a level appropriate for the degree awarded. (Standard 8)

21. Standard 8: Measures of Student Success
8.6 The institution defines measures of student success and levels of achievement appropriate to its mission, modalities, and locations of instruction, including any specifically recruited populations. These measures include rates of progression, retention, transfer, and graduation; default and loan repayment rates; licensure passage rates; and employment.
8.7 The institution uses additional quantitative measures of success, such as further education, civic participation, religious formation, and others, as appropriate to its mission, to understand the success of its recent graduates. Information from students and former students is regularly considered.
8.1 The institution enrolling multiple student bodies, by degree level, location, modality, or other variable, develops and uses the data, evidence, and information below for each student body.

22. Student Achievement & Success Forms
Student Success: Making Assessment More Explicit (E Series)
Select and declare a basic approach to assessment and summarize the findings.
Data First Forms for Standard 8
Report data on retention rates, graduation rates, and other measures of student success appropriate to the institution’s mission.
Forms were revised in 2016 and are available on the Commission website.

23. E-Series: Making Assessment More Explicit
Choose one of the four options:
E.1 Inventory of educational effectiveness indicators and specialized and program accreditation
E.2. Voluntary System of Accountability plus program review
E.3 Institutional claims for student achievement with validating information
E.4 Measures of student success: Comparison with peers
Or, use a combination or design your own in
consultation with Commission staff.

24. Standard 8: Data First Forms
8.1 Undergraduate Retention and Graduation Rates (IPEDS – including Outcomes Measures – and others)
8.2 Student Success and Progress Rates and Other Measures of Student Success
8.3 Licensure Passage Rates; Job Placement Rates; Completion and Placement Rates for Short-term Vocational Programs
8.4 Graduate Programs, Distance Education, Off-campus Locations
Each form asks for information about the most recent year, one and two years prior, and goals for the future thereby promoting trend analysis.

25. Form 8.2: Other Measures

26. Mission-appropriate measures
Success in remedial courses
Graduation+transfer out+still enrolled
Number of students receiving Rhodes, Marshall, Fulbright, Truman & Goldwater Scholarships
Percentage of students enrolled in service-learning activities
Percentage of internship students offered permanent placements
Doctoral degrees earned by discipline by institution
Percentage of graduates working in the non-profit sector
Percentage of graduates starting their own business
Employment status and average starting salary
Number of graduates nominated for an Emmy

27. Using the Data Forms in accreditation reports
Start early so you can see what data you have and what else you want to collect.
Engage the campus in a discussion about student success.
What retention and graduation rates beyond IPEDS are helpful to us?
How do we define student success? How do we measure success?
What quantitative, qualitative, and anecdotal evidence about student success do we have?
What else do we need?

28. Using the Data Forms (continued)
Analyze the data – what do these data tell us about our institution? … about how well we fulfill the Standards? … about our capacity to collect, analyze and use important institutional data, especially data about student success and achievement? … about where and how we need to improve?
Incorporate the data into the narrative.
“You can see a lot just by looking.” Yogi Berra

29. Three commendations
Mission really does inform everything, including how institutions understand student success
Student-centeredness is alive and well in New England!
Institutions find many ways to get value from the accreditation process.

30. Three recommendations
Give as much attention to results and analysis as you do to process.
Tell your story of student success in all its nuance, richness, and complexity.
Put on your Aunt Miriam hat and explore your website … then sit down with the folks in charge of your site.
Process
Results
Analysis
Process
Results
Analysis

31. Time for your questions and insights Thank you for your participation!
Pat O’Brien SND Laura Gambino
Senior Vice President Vice President
New England Commission of Higher Education
3 Burlington Woods Drive, Suite 100
Burlington, MA
Thank you for your participation!