1. The Academic Computing Assessment Data Repository: A New (Free) Tool for Program Assessment Heather Stewart, Director, Institute for Technology Development, Seton Hall University Dr. Bert Wachsmuth, Chair, Dept. of Mathematics and Computer Science, Seton Hall University Dr. Eric Fountain, Research Associate, Institute for Technology Development, Seton Hall University Copyright Heather Stewart, Bert Wachsmuth, Eric Fountain 2001. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

2. Heather Stewart, Director, Institute for Technology Development, Seton Hall University Dr. Bert Wachsmuth, Chair, Dept. of Mathematics and Computer Science, Seton Hall University Dr. Eric Fountain, Research Associate, Institute for Technology Development, Seton Hall University The Academic Computing Assessment Data Repository: A New (Free) Tool for Program Assessment A New Tool for Assessment Heather Stewart: The Assessment of Large-Scale Technology Initiatives Eric Fountain: ITD and ACADR Bert Wachsmuth: A New Online Survey Design and Administration Tool

3. Charge of the Mobile Assessment Team Assess the impact of mobile computing on fall 1998 entering freshman This includes student satisfaction with the learning environment and student learning outcomes

4. Narrowing the Question Fantasy: a + b = c a = teaching, b = technology, c = learning Reality: a + b + c + d + e + f… discipline, teaching style, technology proficiency, fit of application to task, desired learning objectives...

5. When Assessment Occurs During Make Changes in Process Before Determine Needs After Evaluate Experience

6. Examples of Assessment Topics FacultyStudentsInstitution Technical Skills Instructional Goals Technical Skills Enrollment Decisions Community Engagement Resources Support Time Distribution Access General Satisfaction Bottlenecks Network Stability Training Needs Tool Fits Goal Info. Access Communication Technical Skills Delivery Change Evaluate Tech. Options D A B

7. Types of Assessment Focus Groups Interviews Journals Observations Portfolios Performances Standardized Tests Surveys Tracking Systems Longitudinal Records Document Reviews Skills Application Demonstrations

8. The Academic Computing Assessment Data Repository: A New (Free) Tool for Program Assessment ITD and ACADR

9. The Academic Computing Assessment Data Repository: A New (Free) Tool for Program Assessment ACADR Vision Resource for Assessment Qualitative and Quantitative Data Best Practice Information Research and Assessment Reports

10. Survey Tool – First Offering The Academic Computing Assessment Data Repository: A New (Free) Tool for Program Assessment What will it do? Conditions of Use Shared Resource Data – Across Institutions and Over Time

11. ACADR ACADR: TechoTalk Overview Taking a Survey Downloading/View Data Administrating Surveys The Inside View Participate or Download Eric Fountain & Bert Wachsmuth

12. ACADR Overview ACADR: Academic Computing Assessment Data Repository Collect, evaluate, and share technology assessment data General tool for survey administration and evaluation Three Web-based Modules Submission module for users to take a survey Download/View module for researchers to evaluate data Administration module to create and manage surveys Free Availability Use Seton Hall server for technology assessment Install on your own server for general use Eric Fountain & Bert Wachsmuth

13. ACADR Taking a Survey Pick a survey or jump via direct link Password-protected surveys Customized survey appearance Captures user and survey-specific information Recognizes returning users Planned Improvements: Authenticate against LDAP server Restrict survey to selected users Eric Fountain & Bert Wachsmuth

14. ACADR Download/View Data Available to ‘registered researchers’ only User names available to ‘trusted’ researchers only View online descriptive summary Combine data across surveys for cross-referencing Download data in Excel or SPSS format Planned Improvements: Expand cross-referencing capabilities Add additional descriptives to online summary Eric Fountain & Bert Wachsmuth

15. ACADR Administrating Surveys Available to ‘registered administrators’ only Support for multiple preconfigured ‘campuses’ Create surveys, factors, metrics, etc. online Define valid date range and password per survey Reuse and customize surveys via ‘instances’ An ‘instances’ customizes a survey for a campus A ‘survey’ consists of one or more groups A ‘group’ consists of factors and metric groups Tracks editing changes Eric Fountain & Bert Wachsmuth

16. ACADR The Inside View Uses secure, modern technologies Linux Operating System Apache Web Server IBM DB2 SQL server Java Servlets Uses Open Standards Technology ‘Frontend’ works with any Java Servlet 1.0 web server ‘Backend’ works with any JDBC 2.0 SQL server Planned Improvements Move to SSL-secured web site Eric Fountain & Bert Wachsmuth

17. ACADR Participate or Download For Technology Assessment in Higher Education Manage surveys through SHU server w/o setup or cost Benefit from shared surveys, factors, and metrics Enhance all IT assessment via cross-referencing For General Use Download and install software (GNU-like license) Use for institutional assessment (e.g. faculty evaluations) Support faculty research needs Available for IT Assessment Pilots now Available for (registered) download Jan. 15, 2002 Eric Fountain & Bert Wachsmuth

18. The Academic Computing Assessment Data Repository: A New (Free) Tool for Program Assessment An Invitation http://repository.itd.shu.edu fountaer@shu.edu Heather Stewart, Director, Institute for Technology Development, Seton Hall University Dr. Bert Wachsmuth, Chair, Dept. of Mathematics and Computer Science, Seton Hall University Dr. Eric Fountain, Research Associate, Institute for Technology Development, Seton Hall University