“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online

Assessing Online Instruction at Carteret Community College Donald Staub, Title III Director Carteret Community College ? ? ? ? ? ?

The Lineup DL at CCC The Need to Assess How We’re Doing It Conclusions & Solutions Discussion

CCC DL is 10 years old 34 CCC ~1700 enrollment SP09 Fall total sections taught 270 seated sections 61 Internet 15 Hybrid

The Need to Assess SACS Common Sense it’s the right thing to do we need some answers

“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.” (CS 3.3.1) COC Distance Education Policy Statement education.pdf

CCC 270 seated sections 73% success / 22% failed or withdrew (15% W) 61 internet sections 65% success / 33% D, F, WD (16% W) 15 hybrid sections 81% success / 18% D, F, WD (9% W) FA07 internet: 59 sections; 34% D, F, W FA06 internet: 48 sections; 34% D, F, W

How We’re Doing It QAP Grade Distributions ILLOs & PLLOs Same instructor

The QAP (Quality Assessment Plan) Developed in-house (using models) Peer-reviewed by content specialist Stipends All current online courses have been reviewed All new online courses must be reviewed

The QAP Score Card

The QAP Score Card: Learning

Grade Distributions (Fall 2008)

ILLOs Communication Computer Literacy Critical Thinking Humanities & Fine Arts Information Literacy Personal Growth & Responsibility

Computer Literacy - CIS 110 Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers. Assessment: Final Exam in CIS 110. This exam consists of a hands-on skills assignment for Word, Excel, Access and PowerPoint and a 25 question Multiple choice assignment with questions regarding computer hardware, system software, internet browsers and other related computer terminology.

Computer Literacy 5 Sections: CIS WB; 2-IN Attrition:  IN01: 35%  IN02: 26%  WB03: 17%  WB04: 28%  WB05N: 18% Cross-section comparison (i.e. one course, different instructors)

Computer Literacy - CIS 110 Same instructor teaching one section of each

Personal Growth & Responsibility ACA 115 Outcome: Students demonstrate the ability to understand and manage self effectively.

Personal Growth & Responsibility ACA 115

Advisement Project In the fall of 2008, 79% of all ACA students (10 sections) scored 80 or above. For online sections (2 sections), 70% (7 out of 10) scored 80 or above.

Personal Growth & Responsibility PSY 150

PLLOs Same instructor teaching one section of each (take a look at a common assignment from each section) ACA 115 ACC 120 BUS 110 ECO 251 EDU 119

ACA Learning Styles Project Students identify and explain the strategy used to prepare for an exam.

ACC 120

BUS 110 ECO 250

EDU test test items related to: working/communicating with parents, parental rights, and regulations dealing with parental contact/information. n=20n=12

Conclusions & Solutions “The largest issue confronting online assessments is that of the dynamics that occur when the instructor and student do not share the same immediate space.” Milam, et al. (2004). In, Developing and Implementing Assessment of Student Learning Outcomes

Instructors who don’t treat online & seated equitably (e.g. additional support to seated sections) Using the observation instrument correctly. Some of this was a direct result of them using a document that was scanned and they had to print/copy to use. I believe some of the students had trouble with the process. (EDU 119) Math & the fax machine

“The seated students did however have the opportunity to submit their observation checks sheets for revision if needed.” “Students will no longer have to print the inventory results. Instead, they will write a narrative explaining their results.” “Students will no longer need to submit a curriculum check sheet (on-line students had difficulty accessing them from the website).”

Discussion “In many instances, the assessment of an online course is not in alignment with the type of instruction that may have occurred in the course.” Palloff & Pratt (2009) Assessing the Online Learner

Donald Staub Title III Director Carteret Community College