1. EPIC Online Publishing Use and Costs Evaluation Program

2. Overview Overview of Evaluation Program Research update

3. Purpose of EPIC Evaluation Program To investigate how online electronic resources affect different aspects of the scholarly communication process. a) general perspective b) through lens of EPIC projects

4. How Does the Shift to Electronic Resources Affect the Following Groups? Publishers Administrators/Information Technology Librarians Faculty Students

5. Research Approach Interviews Focus Groups Surveys Weblog analysis Info gatheringSurvey development

6. Research Projects to Date Librarian Focus Group Librarian Online Survey Faculty Interviews Faculty Online Survey Student Interviews Student Online Survey http://epic.columbia.edu/eval/

7. Librarian Online Survey

8. Methodology a 15 minute online survey of librarians was conducted in Winter, 2002 e-mail invitations were sent to librarians at higher education institutions across the country All were librarians at institutions that are either CIAO or Earthscape subscribers, or have been identified as being potential subscribers 1007 respondents (response rate: 32%)

9. Objectives Learn what librarians perceive as the benefits and disadvantages of electronic resources. Gain insights into how electronic resources affect the role of the librarian. Gain insights into how electronic resources affect the role of print resources.

10. Conclusions from Librarian Survey Electronic Resources: Allow libraries to serve patrons more effectively Put a strain on some libraries Do not replace print resources

11. Faculty Survey

12. Methods a 25 minute online survey of faculty and graduate students was conducted in Winter 2003 e-mail invitations were sent to scholars at higher education institutions across the country scholars were involved in the disciplines of International Affairs or Earth Sciences 845 respondents (response rate 14%)

13. Objectives Gain insights into how academics are using electronic resources in their academic work Gain insights into the perceived benefits and disadvantages of electronic resources Learn how academics perceive electronic resources as affecting undergraduate students’ scholarship Examine differences based on stage of career, type of discipline, and type of institution

14. Respondent Profile

15. Gender & Age N=845

16. Position & Type of Institution N=845

17. Academic Responsibilities and Field of Study N=845

18. Electronic Resources and Academics’ Work Habits

19. Use of and Proficiency with Electronic Resources N=845

20. Electronic Resources’ Affect on Work Habits N=845

21. Work Habits and The Library N=845

22. Perceived Benefits and Disadvantages of Using Electronic Resources

23. Benefits of Electronic Resources--Convenience N=845 Slide 1 of 2

24. Benefits of Electronic Resources—Improved Availability N=845 Slide 2 of 2

25. Disadvantages of Electronic Resources N=845

26. Benefits and Disadvantages by Discipline N=845 All of the differences in means by discipline on this chart are statistically significant.

27. Benefits and Disadvantages by Seniority N=845 All of the differences in means by seniority on this chart are statistically significant.

28. Teaching with Electronic Resources

29. Use of Electronic Resources for Teaching N=419

30. Use of Electronic Resources for Teaching by Discipline N=419 All of the differences in means by discipline on this chart are statistically significant.

31. Electronic Resources & Students—Benefits N=419 Slide 1 of 2

32. Electronic Resources & Students—Problems N=419 Slide 2 of 2

33. Class Time Spent Instructing Students How to Evaluate Online Information N=419

34. Availability of Courses Instructing Students How to Evaluate Electronic Resources N=419

35. The Effect of Electronic Resources on Overall Quality of Students’ Learning Experience N=419 MEAN 4.00

36. Satisfaction with Electronic Resources

37. Importance of and Satisfaction with ER

39. Important and Satisfied Ability to download Ability to print Currency of info Speed of sites Reputation of providers Text in PDF or other standard format Full text available Search capabilities Ease of Navigation Availability of older/archived information Ability to tell if peer reviewed Important and Not Satisfied

40. Other suggestions Reduce price Standardize navigation and search procedures across databases Synthesize databases so that they are accessible through a single portal Faster turn-around time/More frequent updates Clearly identify or label sources

41. Conclusions Electronic resources have become the main tool for scholars’ information gathering. Electronic resources are seen as providing convenience more so than increased availability of information. The main problem with electronic resources is having to separate out reliable from unreliable information. Most faculty believe students have an enhanced learning experience because of the information available through electronic resources. Future efforts could focus on providing increased access to older and full text material, and improving search and navigation capabilities.

42. Christina Norman, Ph.D. Research Director, EPIC Columbia University 211 International Affairs Building 420 West 118th Street, Mail Code 3301 New York, NY 10027 cn2005@columbia.edu EPIC evaluation web site: http://epic.columbia.edu/eval/