The Development of a Pre- and Post-Test for Assessing First-Year Library Instruction, Delivered Using Blackboard Randall Bowman Reference/Instruction Librarian Elon University

Introduction Problems with Assessment Formative assessments at Elon Printed feedback survey “One minute paper” Online survey form None of these approaches measured student learning

Formative vs. Summative Assessment Summative assessment measures actual student learning Librarians have tried: Workbooks Exercises Assessment tests

Challenges in testing Test use valuable class time How to deliver test? Pencil and paper Online tests Lack of expertise in test design

Development of tests at Elon 1 st draft not good; not linked to any class Sought examples on ILI-L Responses included samples of tests Literature review Began with library literature; also education literature Colborn and Cordell article Fagan article

Further Development Link test to FY composition course, ENG 110 Pre-test and Post-test scenario Measure what students know before instruction Pre-test given to ELITE student assistants for testing and feedback Tests take final form 12 questions, 11 multiple choice, 1 matching

Method of Delivery Three important questions to answer How to deliver the test? Web-based assessment Course Management System (Blackboard) Institutional-level test initiatives (SAILS, others) How to convince faculty to participate Timing of tests

Blackboard Create “Library Instruction Course Site” Test Manager Have class enrolled in course sitecourse site For additional classes, IDD can create additional course sites

How Blackboard works Export tests Place tests in assignment folder Make available and choose options Gradebook Download scores to Excel

Faculty participation Fall 2007: three agree, with two sections each Tests initially given during instruction session Limited to ENG 110 sessions that meet with librarian twice Results

Timing of tests Have tests taken on class time Give post-test after assignment turned in Measuring student’s retention of material Not right after instruction Did they use what they learned?

Spring 2008 Tested six more sections Pre- and post-test questions “aligned” for easier comparison Answers to test questions compared Results

Statistical Analysis Obtained help of professor Suggested “paired” test Hypothesis Derivation of P-value Results

Conclusion Test shows promise Currently being used this semester Questions?

Selected Bibliography Byerly, Gayla; Downey, Annie; Ramin, Lilly. “Footholds and Foundations: Setting Freshmen on the path to lifelong learning.” Reference Services Review 34.4 (2006): Carey, Lou M. Measuring and Evaluating School Learning, 2 nd edition. Needham Heights, Massachusetts: Allyn and Bacon (1994). Colborn, Nancy Wootton; Cordell, Rosanne M. “Moving from Subjective to Objective Assessments of Your Instruction Program.” Reference Services Review 26.3 (1998): Fagan, Jody Condit. “Selecting Test Items Types to Evaluate Library Skills.” Research Strategies 18.2(2001): Riddle, John S.; Hartman, Karen A. “But Are They Learning Anything?” Designing an Assessment of First-Year Library Instruction.” College & Undergraduate Libraries 7.2 (2000):