THE COLLEGE OF NEW JERSEY The Role of Data and IT in Accreditation and Assessment Paula Maas Debra Frank

Input Model Output Model Role of IT Role of Program Describe Faculty Describe Resources Describe Courses Provide Basic Demographic Information Role of IT Role of Program Analyze Data Input Assessment Data Develop Assessments Assist in Retrieving Data for Reporting Create Databases to Maintain Data Create Systems for Data Input Major Paradigm Shift in Accreditation

Middle States Accreditation Standards Major principle guiding standards revision: “greater emphasis on institutional assessment and the assessment of student learning”

Assessment of Institutional Effectiveness Implement Strategies Assess Achievement Use the Results Define Goals Middle States Standard 7

Assessment of Student Learning Offer Student Opportunities Assess Achievement Use the Results Develop Learning Goals Middle States Standard 14

Learning Outcomes Assessment 101 Learning Outcome: How do you know this? How do you know you are successful? At the completion of this (course, major, general education program, etc) students will know or will be able to do _______________. We will ask 3 questions on the final exam in course xxx 123 assessing this knowledge or skill. 90% of our students will score 80% or above on these three questions

Characteristics of Effective Assessments Useful Cost-effective Reasonably accurate and truthful Planned Organized, systematized, and sustained

Roles for IT in Effective Assessments Useful Cost-effective Reasonably accurate and truthful Planned Organized, systematized, and sustained

NCATE Accreditation Standard 2: the unit has an assessment system that collects and analyzes data

Data Collection Using built-in assessment functions in course management system Scored and un-scored rubrics Checklists Teamwork assessments

Data Storage ACCESS Database SA System Other

Increasing Data Volume

Linking Data to Other Sources Scored and un- scored rubrics Checklists Teamwork assessments Using built-in assessment functions in course management system Course grades GPA Connecting to student information PRAXIS II SAT scores Connecting to standardized test data

Reporting Data Communication between faculty/staff and IT professionals is key to managing the web of data Data

Analyzing Data Programs analyze data IT can provide technical support

Communication Issues What you may hear: “I have no idea how to do this, but this is what I need…” What they mean: “I’m looking to you to be the technology expert” “I need you to translate my needs into language that data administrators understand”

What data will you need from existing systems? Student record data Student bio- demographics? Courses completed with term? Course grades? Faculty data Courses taught with term, enrollment, grades? Degrees, certifications, experience? Course Management System Embedded assessment results?

Other Questions to Ask Once per semester? Once per year? More often? How often will the data need to be refreshed? What data security mechanisms are in place? Who will have access to the data? What format will the final output need to be in? When (how often) do you need the output?

More Questions to Ask Are there existing interfaces? Note – the assessment folks won’t know the answer to this! If it is an “off the shelf” product – what are the specs for data import, and can your existing systems produce what is needed?

Some Available Products WEAVEonline  Xitracs  Chalk & Wire  EduMetry  TaskStream  Tk20  Waypoint Outcomes  om/index.cfm om/index.cfm Listing of products within this presentation does not constitute endorsement.

A Word of Caution An “off the shelf” product may be very helpful for organizing assessment data Learning outcomes Performance criteria Relationship to academic program It will not perform the assessment! Communication of results Application to program changes It will not, in and of itself, “complete the loop” But it can be more cost effective alternative

Assessment 102 (Live) 1. The main point of this presentation is: A. IT can solve accreditation needs. B. As accrediting bodies move toward outcomes assessment, the role of IT is increasing C. Institutions need to decide whether to use home grown or off the shelf products for accreditation 2. What is meant by assessing institutional effectiveness?

Assessment 102 (Live) 3. What is meant by assessing student learning? 4. Where does IT fit into data management for accreditation? 5. What kinds of questions can IT ask to facilitate the assessment and accreditation process?

Contact Paula A.Y. MaasPaula A.Y. Maas, Executive Director, Center for Institutional Debra L. Frank, Coordinator of Assessment, School of Education, The College of New Jersey- L. Frank