1. THE COLLEGE OF NEW JERSEY The Role of Data and IT in Accreditation and Assessment Paula Maas Debra Frank

2. Input Model Output Model Role of IT Role of Program Describe Faculty Describe Resources Describe Courses Provide Basic Demographic Information Role of IT Role of Program Analyze Data Input Assessment Data Develop Assessments Assist in Retrieving Data for Reporting Create Databases to Maintain Data Create Systems for Data Input Major Paradigm Shift in Accreditation

3. Middle States Accreditation Standards Major principle guiding standards revision: “greater emphasis on institutional assessment and the assessment of student learning”

4. Assessment of Institutional Effectiveness Implement Strategies Assess Achievement Use the Results Define Goals Middle States Standard 7

5. Assessment of Student Learning Offer Student Opportunities Assess Achievement Use the Results Develop Learning Goals Middle States Standard 14

6. Learning Outcomes Assessment 101 Learning Outcome: How do you know this? How do you know you are successful? At the completion of this (course, major, general education program, etc) students will know or will be able to do _______________. We will ask 3 questions on the final exam in course xxx 123 assessing this knowledge or skill. 90% of our students will score 80% or above on these three questions

7. Characteristics of Effective Assessments Useful Cost-effective Reasonably accurate and truthful Planned Organized, systematized, and sustained

8. Roles for IT in Effective Assessments Useful Cost-effective Reasonably accurate and truthful Planned Organized, systematized, and sustained

9. NCATE Accreditation Standard 2: the unit has an assessment system that collects and analyzes data

10. Data Collection Using built-in assessment functions in course management system Scored and un-scored rubrics Checklists Teamwork assessments

11. Data Storage ACCESS Database SA System Other

12. Increasing Data Volume

13. Linking Data to Other Sources Scored and un- scored rubrics Checklists Teamwork assessments Using built-in assessment functions in course management system Course grades GPA Connecting to student information PRAXIS II SAT scores Connecting to standardized test data

14. Reporting Data Communication between faculty/staff and IT professionals is key to managing the web of data Data

15. Analyzing Data Programs analyze data IT can provide technical support

16. Communication Issues What you may hear: “I have no idea how to do this, but this is what I need…” What they mean: “I’m looking to you to be the technology expert” “I need you to translate my needs into language that data administrators understand”

17. What data will you need from existing systems? Student record data Student bio- demographics? Courses completed with term? Course grades? Faculty data Courses taught with term, enrollment, grades? Degrees, certifications, experience? Course Management System Embedded assessment results?

18. Other Questions to Ask Once per semester? Once per year? More often? How often will the data need to be refreshed? What data security mechanisms are in place? Who will have access to the data? What format will the final output need to be in? When (how often) do you need the output?

19. More Questions to Ask Are there existing interfaces? Note – the assessment folks won’t know the answer to this! If it is an “off the shelf” product – what are the specs for data import, and can your existing systems produce what is needed?

20. Some Available Products WEAVEonline  http://www.weaveonline.com/ http://www.weaveonline.com/ Xitracs  http://www.xitracs.com http://www.xitracs.com Chalk & Wire  http://www.chalkandwire.com/ http://www.chalkandwire.com/ EduMetry  http://www.edumetry.com/ http://www.edumetry.com/ TaskStream  https://www.taskstream.com https://www.taskstream.com Tk20  http://www.tk20.com/ http://www.tk20.com/ Waypoint Outcomes  http://www.subjectivemetrics.c om/index.cfm http://www.subjectivemetrics.c om/index.cfm Listing of products within this presentation does not constitute endorsement.

21. A Word of Caution An “off the shelf” product may be very helpful for organizing assessment data Learning outcomes Performance criteria Relationship to academic program It will not perform the assessment! Communication of results Application to program changes It will not, in and of itself, “complete the loop” But it can be more cost effective alternative

22. Assessment 102 (Live) 1. The main point of this presentation is: A. IT can solve accreditation needs. B. As accrediting bodies move toward outcomes assessment, the role of IT is increasing C. Institutions need to decide whether to use home grown or off the shelf products for accreditation 2. What is meant by assessing institutional effectiveness?

23. Assessment 102 (Live) 3. What is meant by assessing student learning? 4. Where does IT fit into data management for accreditation? 5. What kinds of questions can IT ask to facilitate the assessment and accreditation process?

24. Contact Paula A.Y. MaasPaula A.Y. Maas, Executive Director, Center for Institutional Effectiveness-maas @tcnj.edu Debra L. Frank, Coordinator of Assessment, School of Education, The College of New Jersey- frank@tcnj.eduDebra L. Frank