Presented by Jennifer Fager For University of Wisconsin-Superior Enhancement Day 1/19/2011

Advice for Assessing Student Learning in an Online Environment Multiple Choice and Short Answer Questions Not always the best technique in either online or ground formats Lend themselves to cheating more so in online Questions, regardless of ground or online, must be written well The “test” has been overused and relied upon unless the instructor is an expert in writing good test items

Advice continued Use multiple forms of assessment Has corroborating evidence of student achievement More personalized assessments reduce the likelihood of plagiarism “Term paper” vs. application paper using personal examples General research paper vs. select topics focus Use examples more “authentic” in format Consider problems of the field Engage in “conversation’s” online

Advice continued Security options Lockdown browser Student can’t go to another website during the test NOTE: Most students now have multiple devices so they can go to their IPhones, blackberries, laptops, IPads, etc. to research the “answers”

Advice Continued Set test time limits Even if the student gets help, time restricts the amount Can punish the slower readers/processors What do you want in terms of meaningful, thoughtful responses

Advice continued Assume any online test is, by definition, open book And perhaps open notes, and open neighbors Don’t try to make it otherwise

Advice continued Quiz/test software products are available Can generate multiple versions of tests Each student can receive a “different” test Have students explain their answers 25% have to explain ‘A’ answers, 25% ‘B’ answers, etc.

Advise continued Other security ideas “thumb” identification Webcams Face-to-face testing at a testing center

Online Assessment Resources Commercial Assessment Data Management Tools – E.g. TracDat, WeaveOnline, Chalk & Wire Curriculum Design Tools E.g. WIDS – link outcomes to Performance Assessment Tasks Survey Instruments Survey Monkey Zoomerang Portfolios Course Management Systems Quiz and Analysis Functions Blackboard and others

Resources Continued Google Docs Jing Reflective Instruments Leadership Personality Style Teamwork Merlot (has content template builder) Clickers AdobeConnect with polling features Lock-down Browsers for online testing

Fundamental Questions What evidence do you have that students achieve your stated learning outcomes? How have you articulated these outcomes to students and adjunct faculty? In what ways do you analyze and use evidence of student learning?

Assessment Process The same process is used for online programs and courses as for ground (traditional): Needs Assessment Needs of the profession Needs of the learners Articulate your expectations Measure achievement of expectations Collect and analyze data Use evidence to improve learning Assess effectiveness of improvement

Measure Online Learning Direct Methods Mostly the same as ground Can use any student work products that can be saved in an electronic form Some challenges exist with lab work, oral performances, etc. Indirect Methods Again, same as ground Survey response rates are notoriously low

Advantages of Online Assessment Artifact collection can be automated through the course management software Can work for ground as well Assignments, rubrics can be standardized and mapped to specific competencies/outcomes Again, can work for ground as well

New Opportunities with Online Assess quality of discussions, group work Particularly effective with asynchronous Assess individual learning in more depth Student advising can be more robust Use of standardized rubrics can result in greater consistency in grading Particularly effective when combined with periodic instructor norming and use of anchor samples

Discussion & Group Work Faculty can isolate and organize contributions to a threaded discussion and give feedback on how to improve Can archive individual contributions and group discussions Formative and summative Course, program, and institutional levels Create a record of instructor effectiveness Useful for annual review, tenure

Individual Student Learning Individual learning/achievements can be tracked across the educational experience E-format allows for wide dissemination of collected data Current software packages provide various levels of data collection & dissemination

Rubrics Clear articulation of expectations that are linked to specific course & program outcomes A means through which correspondence of online & ground rigor can be ensured (alignment) Increases consistency in grading across sections, courses, programs, even colleges Holistic vs. Analytic

Example of a Successful Online Assessment Strategy Develop course/program maps which articulate learning outcomes and link them to specific courses and course assignments Collect and archive artifacts through the course management software Post rubrics through course management software for student and faculty use

Ensure Validity, Reliability Tracking inter-rater reliability helps achieve consistency in grading Post rubrics, anchor samples for 24/7 faculty use Full-time/core faculty should formulate competencies, achievement standards & interpret results

Sample Assessments Review the Video Gaming Assignment included at the end of this packet. Determine how this assignment could be adapted for online learning

General Education Outcomes Example The ability and inclination to think and make connections across academic disciplines How would you assess this outcome in ground or online courses?

Websites and Resources courses.asp courses.asp eteaching.htm eteaching

Questions and Concerns What issues have you encountered when assessing online learning? What are your concerns regarding the use of online assessment techniques? What policies and/or procedures need to be in place to reduce or alleviate these concerns?

Questions Still Burning? What else do you need to know?

The Bottom Line? Assessment in online learning can be the same as assessment on ground—artifacts and tools can be the same even if the medium is different!