Culture of Assessment NCLB Outcomes-based assessment Accountability

Pressure to Assess At UNO--MyMapp Programmatic assessment

ACRL training Model for real-life library assessment Advantage of developing model before committing resources Feedback from ACRL instructors

Ask yourself a few questions Need to Determine: Scope Measurement instrument Population Timeframe Delivery Method Sustainability

What do you want to measure? Determine type of instrument Pre and Posttest to measure ENGL116 Composition II students information literacy skills Pool of questions Variety of questionsmultiple answer, drag and drop and fill in the blank

Who can help? Institutional assistance Faculty with expertise in assessment English faculty to assist in question development and review Graduate students to assist with data analysis Other librarians with assessment experience

Consultants Use consultants to: Define your scope Suggest testing instrument models Test your test Help navigate IRB process Review results and draw conclusions

Delivery Online vs. Paper Course management software Blackboard Campus IT support Familiar Convenient Accessible 24-7

Time Planning for assessmentour timeline Timeline for launching Timeline for measuring Longitudinal? One time measure? What can you sustain?

Money Do you need it? What it did for us Incentive Stipend for consultants Student employee wages Data analysis

Staff How to create buy-in with: Library staff Librarians English professors Adjuncts Campus stakeholders Highlight benefits to all stakeholders

Curriculum Build on what you have in place Keep it simple to start Be consistent with learning objectives Reinforce key concepts Share decision-making with English or other faculty stakeholders

Promotion English Department faculty meetings Library Instruction sessions Library Newsletterwebsite Continuous need for promotion every semester until it becomes routine

Benefits of assessment in library Working with faculty consultants campus-wide opened our eyes to the ways in which different units measure learning Engaging in larger assessment conversation developing on campus future committee work opportunities

ObservationHypothesisPrediction Theory Tests Consistent Not Consistent Modify Hypothesis

Measurements Pre and post questionnaires Student surveys Blackboard organization Controlled group of librarians, English instructors, and class sections Questions Collaboration with English department Prioritized outcomes with ACRL IL Standards Used challenging, performance-based questions

Background and Online Assessment Tools Why did we select Blackboard?

Saving time and money with little training Blackboard – The Foundation

Sample Size 15 sections of English 1160/ students completed both questionnaires Pre-questionnaire mean = 8.23 out of 15 Post-questionnaire mean = 9.89 out of 15 Statistically significant (p <.001) Findings from Initial Semester

What did you find most useful in the library instruction? A compleat waste of my time and tuition. Learned what some stuff did All of the engines that we could use to find things, and how to use them. The library instructor only showed us a couple, but after being shown how to use a few, I went through and figured out how to use others without being intimidated by all of the information. Everything..I woul not have found my sources if it wasnt for the library instruction

Changes implemented after initial semester Advice offered from faculty consultants, library instructors, and English faculty: Add Survey questions to pre- questionnaire Count every correct multiple answer score Allow 2 attempts rather than 3 Remove controls Offer assessment to all librarians, English instructors and students

147 completed the pre and post questionnaires 8 scores rejected as outliers 139 scores used Pre-questionnaire mean score = Post-questionnaire mean score = Results were NOT statistically significant Results from Second Semester

Second Semester – Survey Questions

Correlation between Score Difference and Number of Days Between Tests– Second Semester

Factors affecting success Voluntary (librarians, instructors, students) Fewer Controls Different Curriculum Different Teaching Styles Lack of Incentives Low Motivation

Rewards Focused and clarified the purposes of the information literacy sessions Prioritized and used the most relevant standards and outcomes that fit our program and institution Continued to build relationships with the English department faculty

Future assessment ideas Verbal demonstrations Keyboard analysis Surveysin depth Longitudinal study of information literacy skills Citation analysisMyMapp e-portfolio

Learn about outcomes-based assessment Keep it simple to start Seek guidance from university experts in assessment Use existing course management software Use campus IT support Remember that assessment is organic and on-going Practical Tips

Thank You Questions? Discussion