1. Assessing Distance Learning Programs – A Model League for Innovation in the Community College Conference on Instructional Technology 2006 Mary Beth Orrange Professor of Mathematics Jason L. Steinitz Professor of Social Science Erie Community College

2. Session description in program Participants develop a national model for distance learning program assessment, including identifying the variables and discussing issues related to interpreting results. The assessment plan and the results of the two-year study of Erie Community College’s distance learning program are presented and used as the starting point for the development of the assessment model. The goal is to have participants develop a list of variables that provide meaningful measures of student learning in distance learning programs.

3. Outline of this session Introduction Introduction Brief outline of the assessment of Erie Community College’s distance learning program presented Brief outline of the assessment of Erie Community College’s distance learning program presented Session participants will brainstorm a list of variables for distance learning assessment using the think-pair-share collaborative learning technique Session participants will brainstorm a list of variables for distance learning assessment using the think-pair-share collaborative learning technique Participants will then prioritize variables Participants will then prioritize variables Participants develop a national model for distance learning program assessment Participants develop a national model for distance learning program assessment Next year’s round table discussion on distance learning assessment will be planned. Next year’s round table discussion on distance learning assessment will be planned.

4. Erie Community College ECC A three campus college with an approximate FTE count of 10,000 A three campus college with an approximate FTE count of 10,000 Part of SUNY (State University of New York) but not part of SLN Part of SUNY (State University of New York) but not part of SLN A full complement of programs (64) and courses A full complement of programs (64) and courses Strong union environment Strong union environment

5. Distance Learning at Erie Community College Approximately 130 online courses each semester Approximately 130 online courses each semester Approximately 30 telecourses each semester Approximately 30 telecourses each semester Approved by the Commission on Higher Education, Middle States Region, to offer programs at a distance Approved by the Commission on Higher Education, Middle States Region, to offer programs at a distance Approved by the State Education Department of New York to offer programs at a distance Approved by the State Education Department of New York to offer programs at a distance

6. View ECC’s Assessment Project Results Visit the website http://south.ecc.edu/dl Visit the website http://south.ecc.edu/dlhttp://south.ecc.edu/dl Click on Assessment>Perkins Year 2 Click on Assessment>Perkins Year 2

7. Assessment items Passing rates of courses taught at a distance Passing rates of courses taught at a distance Passing rates of seated versions of courses taught at a distance Passing rates of seated versions of courses taught at a distance Determination of when the passing rates are significantly different between seated and distance courses using the chi-square test Determination of when the passing rates are significantly different between seated and distance courses using the chi-square test Pivot table in Excel for passing rates for all four semesters Pivot table in Excel for passing rates for all four semesters

8. Assessment items Demographics of students enrolled in distance courses Demographics of students enrolled in distance courses Student satisfaction Survey administered to students enrolled in distance courses Student satisfaction Survey administered to students enrolled in distance courses Self-assessment of skills improvement administered to students enrolled in online, telecourses, and seated courses Self-assessment of skills improvement administered to students enrolled in online, telecourses, and seated courses Characteristics of faculty members teaching at a distance Characteristics of faculty members teaching at a distance

9. Assessment items Reasons for withdrawal from courses at a distance Reasons for withdrawal from courses at a distance Study of when students stopped working in courses at a distance vs when they withdrew Study of when students stopped working in courses at a distance vs when they withdrew Comparison of passing and withdrawal rates for pairs of courses taught by the same instructor in the same semester. Comparison of passing and withdrawal rates for pairs of courses taught by the same instructor in the same semester.

10. Develop a Model The think part Use the next five minutes to make a list of items that you think should be included in the assessment model.

11. Develop a Model The pair part Turn to someone seated near you and spend the next five minutes discussing your two lists. Combine the two lists into one.

12. Develop a Model The share part We will develop a master list of all the variables listed by each group in the room.

13. Develop a Model We will prioritize the list using the quizdome product. For each item, select: “a” if you feel the item must be included “b” if you feel the item may be included “c” if the item does not belong in the model.

14. Implement the Model Return to your campus and collect data regarding one of the items we selected.

15. Refine the Model Come back next year and discuss your results at the roundtable session on the Assessment of Distance Learning.

16. Contact us: Jason L. Seinitz, PhD steinitz@ecc.edu Professor of Social Science (716) 851 1305 Mary Beth Orrange orrange@ecc.edu Professor of Mathematics (716) 851 1784 Erie Community College, South Campus 4041 Southwestern Blvd. Orchard Park, NY 14127 See you next year! This PowerPoint show is available at http://south.ecc.edu/orrange