1. LOEX 2007 Cathy Palmer Head, Education and Outreach UC Irvine Libraries Applying Student Learning Data to Enhance Instruction An Anteater Initiative LOEX 2007

2. Outline  Background  Program Overview  Incentives/Challenges  Methodology  Results  Lessons Learned  What next?

3. Writing Program/Library Collaboration  Lower-division writing requirement  Fulfilled in part by successful completion of Writing 39C: Argument and Research  Course integrated library research assignment since 1986  Library research assignment evolved as course curriculum evolved

4. Library Incentives for Evaluation  Major investment of library resources Personnel Space  Availability of numeric data Number of students reached Number of sessions taught Number of librarian hours spent  Were our resources being put to the best use?

5. Yearly Statistics 1996/1997 1727 Students 76 Sessions 82 Hours 2005/2006 2504 Students 126 Sessions 150 Hours

6. Session Outline  Libraries Homepage (Lecture/Demo) Ask a Librarian Connect from off-campus  ANTPAC catalog (Lecture/Demo)  Expanded Academic (Lecture/Demo)  Hands-on exploration of assigned databases Sociological Abstracts L/N Congressional Universe United States Newspapers  Review of each database (Discussion)  Evaluations

7. Evaluation 1.0 Overall Value of Session to Student Very Useful59144% Useful62146% Somewhat Useful1159% Not Useful101% No Response00% Total1338100

8. Student rating of Librarian Presentation Understand Clearly61650% Understand52543% Understand Somewhat 20% Don't Understand00% No Response847

9. Understand Clearly52940% Understand70252% Understand Somewhat695% Don't Understand30% No Response353% Choose Databases—student self rating

10. Select Search Terms-student self rating Understand Clearly52339% Understand61846% Understand Somewhat695% Don't Understand20% No Response1269%

11. Evaluate Results of Search-self rating Understand Clearly59344% Understand53040% Understand Somewhat 796% Don't Understand60% No Response13010%

12. Interpret Citations-Find Material-self-rating Understand Clearly45734% Understand59144% Understand Somewhat 14511% Don't Understand111% No Response13410%

13. But... What were the students REALLY learning in the session?

14. The "Eureka!" moment Esther Grassian workshop Ask the students questions about the content of the session as part of the evaluation.

15. The BIG IDEA Explicit Objective: Students will understand that they have a variety of information resources available to them and that each resource has strengths and weaknesses. They will be challenged to answer the question "Why use this resource rather than that resource?"

16. The Invisible BIG IDEAs Implicit Objectives: Students will leave knowing that the librarians are there to help them. Involve students in their own learning by making the session as hands-on as possible.

17. Learning Outcomes At the end of the 50 minute session, students will:  Recognize the characteristics of generalized and specialized information resources and be able to differentiate between them.  Learn and apply basic search strategies that will help them locate information  Select, explore and use 3-5 library databases to find information on their topic.

18. Session Outline  Libraries Homepage (Lecture/Demo) Ask a Librarian Connect from off-campus  ANTPAC catalog (Lecture/Demo)  Expanded Academic (Lecture/Demo)  Hands-on exploration of assigned databases Sociological Abstracts L/N Congressional Universe United States Newspapers  Review of each database (Discussion)  Evaluations

19. Early Results

20. Read each statement and indicate the best resource to use by placing a check in the appropriate column. (Fall, 2003—383 responses) ANTPACExpanded Academic ASAP Sociological Abstracts UCelinksAsk a Librarian Find books Find scholarly articles Find scholarly and popular articles Get help Find electronic copy 62% 38% 53% 62% 77%

21. Let's use Sociological Abstracts as the demonstration database instead of Expanded Academic ASAP. Another Eureka Moment

22. Read each statement and indicate the best resource to use by placing a check in the appropriate column. (Spring 2004, 809 evaluations) ANTPACExpanded Academic ASAP Sociological Abstracts UC elinks Ask a Librarian Find books Find scholarly articles Find scholarly and popular articles Get help Find electronic copy 73.9% 48.3%52.7% 55.8% 74.7% 64.4%

23. Student Performance Comparison Fall, 2003Spring, 2004 Use ANTPAC to find books61.873.9 Use Soc Abs to find scholarly articles 3852.7 Use Expanded Academic to find popular and scholarly 5354.8 Ask a Librarian for help76.674.7 Use UCeLinks to find electronic copies 62.164.4

24. Bonuses Ability to look for themes in student comments What parts of the presentation helped you the most?  Following along step by step (71)  Visual aid/demonstration (73)  Group work (83)

25. Student comments cont. What helped you the least?  Group work (59)  Not enough time (38)  Nothing/everything helped (202) What could improve the session?  Longer session (184)  Slower and more detail about resources (58)  Speak louder/microphone

26. Still more student comments I still need to learn more about:  What each database is used for and when to use it for certain searches (78) The other databases we didn't get to explore (59) How to find the actual copies of books and articles in the library (34)

27. Problems with Paper ANTPACExpanded Academic ASAP Sociological Abstracts UCelinksAsk a Librarian Find booksXXXXX Find scholarly articles XXXXX Find scholarly and popular articles XXXXX Get helpXXXXX Find electronic copy XXXXX

28. Problems with Paper ANTPACExpanded Academic ASAP Sociological Abstracts UCelinksAsk a Librarian Find booksXXX Find scholarly articles XXX Find scholarly and popular articles XXXXX Get helpXXX Find electronic copy XXX

29. Problems with Paper ANTPACExpanded Academic ASAP Sociological Abstracts UCelinksAsk a Librarian Find booksXXX Find scholarly articles XXX Find scholarly and popular articles XX Get helpXXXXX Find electronic copy XXX

30. Benefits of Automating Evaluation Form  Could only select one response  More comments/longer comments  Results available immediately  Results could be sorted in a variety of ways

31. Numeric Results  Individual Librarian/Individual Session  Individual Librarian/All Sessions  Overall evaluations by quarter  Overall evaluation by academic year

32. Individual Performance Improvements Assessment/Feedback methodology  Use of Observation Rubric –Presentation style Use of projected image Can be heard –Session Content Outlines session Pacing –Classroom management Interaction with students Execution of learning techniques  Comparison between Observation and Evaluations

33. Results Clear, concise presentation + Hands-on + Review of experience Best performance by students on selection of best resource

34. Example

35. Another Example

36. Librarians: All Academic year: 2006 - 2007 Total evaluations: 1733 Q1. Session Content The Library Research Skills session was useful. Strongly agree:873(50%) Agree:778(45%) Neutral:64(4%) Disagree:9(1%) No opinion:1(0%) I learned new skills.Strongly agree:907(52%) Agree:713(41%) Neutral:78(5%) Disagree:18(1%) No opinion:3(0%)

37. Librarians: All Academic year: 2006 - 2007 Total evaluations: 1733 Q2. Instructors The librarian presented information in a way that I could understand. Strongly agree:970(56%) Agree:689(40%) Neutral:42(2%) Disagree:15(1%) No opinion:4(0%) The librarian encouraged and responded to questions. Strongly agree:960(55%) Agree:636(37%) Neutral:105(6%) No opinion:8(0%) Disagree:8(0%)

38. Q3. Choose the Best Resource A. I can use this resource to find out if the UCI Libraries own the book Reckoning with Homelessness. Antpac:1315(76%) ASAP:134(8%) Ask a Librarian:74(4%) Sociological Abstracts:84(5%) UC-eLinks:73(4%) D. I can get help with my topic by using this resource. Antpac:165(10%) ASAP:176(10%) Ask a Librarian:1160(67%) Sociological Abstracts:76(4%) UC-eLinks:99(6%) E. I can use this online resource to see if there is an electronic copy of an article available. Antpac:334(19%) ASAP:197(11%) Ask a Librarian:46(3%) Sociological Abstracts:50(3%) UC-eLinks:1042(60%)

39. Q3. Choose the Best Resource B. I can use this online resource to locate scholarly peer-reviewed journal articles on the topic of homelessness. Antpac:160(9%) ASAP:660(38%) Ask a Librarian:21(1%) Sociological Abstracts:779(45%) UC-eLinks:52(3%) C. This is a good resource to use when I am beginning my research because it covers both scholarly journals and popular magazines. Antpac:211(12%) ASAP:1078(62%) Ask a Librarian:25(1%) Sociological Abstracts:287(17%) UC-eLinks:77(4%)

40. Future Wishlist  Pre and post testing  Ability to share results of evaluation with students  More nuanced testing of ability to apply concepts learned  Consistent evaluation of learning outcomes from other library research skills sessions

41. Questions?

42. Cathy Palmer cpalmer@uci.edu 949-824-4972 Contact Me