1. Building a Journal Collection Tohoku University Library, Kanako Katsumoto Aiko Watanabe 2-Step Map Evaluation Approach

2. OUTLINE Tohoku University Questions on Research Evaluation Using Citation Metrics 2-Step Map Application to Library’s Collection Development Effectiveness and Issues

3. Tohoku University  Sendai City, Miyagi Prefecture  National university founded in 1907  Research-oriented university with 10 undergraduate departments and 15 graduate schools  Student Enrollment: 17,900; Number of Faculty and Staff: 5,000  “ Research First ” ; ” Open Door ” Sendai ● Tokyo

4. Sendai – “Mori no Miyako” nature vol.442, no.10, p714-716 (2006.8.10) vol.442, no.10, p714-716 (2006.8.10)

5. Library System  Main Library,4 Branch Libraries,13 Division Libraries  Collection: Approx. 3.7 million Volumes  No. of Staff: 130

6. Electronic Resources Environment  Site-wide license Web of Knowledge (Web of Science, JCR, ESI) OVID (MEDLINE, Current Contents, PsycINFO, ERIC, Biological Abstracts, CINAHL, EBMR) …  Resources maintained by faculty research fund SciFinder Scholar, CrossFire, INSPEC, …  Others Scopus, PubMed, …

7. Electronic Journal Environment Paper version + EJ Package EJ EJ Package EJ Paper version + Paper version + … ＋ EJ Which are “essential journals”?

8. What is the “fair cost burden”? Electronic Journal Environment

9. Citations and Research Evaluation  The need for research evaluation  Introduction of “ Web of Science ” in 2001  Utilization as a research evaluation tool The number of citations received by papers Impact factor of the journal publishing those papers Perhaps a little excessiv e?

10. Cautions for Use of Citation Metrics  Scope of journals included in the DB (Mostly European and American, Core journals only)  Field-dependent differences (Circumstances surrounding paper/citations)  The significance of citations (excessive self- citations, courtesy/critical citations)  Slippage loss of counting due to variants and journal title changes

11. Cautions for Use of Impact Factors  Average number of citations in the journal （ not each paper ）  Different citation trends by document type （ Review journals ）  Calculated for the past 2 years （ Basic v.s. applied fields ）  Differences between general and specialized journals

12. 2-Step Map  Draw an arrow to the 2 highest citing journals (exclude self-citation)  Avoid overlapping of the arrows  Present each journal proportionate to its publication size B 2.345 C 3.242 A 1.012 Receiving many arrows = Core

13. Advantages of 2-Step Map  Relationships between journals can be observed visually  Tendencies that are often missed with citation metrics can be recognized  Journals that receive many arrows ＝ Core journals ≠ journals with high impact factors Narin F, Carpenter MP, Berlt NC “Interrelationships of scientific journals” Journal of the American Society for Information Science vol.23, p323-331, 1972

14. Materials Science, Multidisciplinary (JCR 2004) 177 Journals More than 100 papers Top 50 journals by IF Review journals not included

15. Materials Science, Multidisciplinary (JCR 2004) Physical Review B Journal of Applied Physics Applied Physics Letters

16. １ ２ ３ ４ ５ ６ ７ ８ ９ 10 2.25 5 3.075 4.30 8 ４ 13 10 Materials Science, Multidisciplinary (JCR 2004)

17. １ ２ ３ ４ ５ ８ ９ 10 Materials Science, Multidisciplinary (JCR 2004) NANO CHEMISTRY

18. ６ ７ Materials Science, Multidisciplinary (JCR 2004) Acta Materialia Materials Science & Engineering A Physical Review B

19. １ ２ ３ ４ ５ ６ ７ ８ ９ 10 2.25 5 3.075 4.30 8 ４ 13 10 Materials Science, Multidisciplinary (JCR 2004)

20. Creating a 2-Step Map Review 177 Journals More than 100 papers Top 50 journals by IF

21. Journal Details Basic Information Categories Detailed Calculation

22. Journal Details founded in 2002?

23. Journal Details Review journal verification Source data for 2-Step Map!

24. Journal Details Record top 3 citing journals

25. Application to Collection Development Which are “essential journals”? User ObservationSurveysUsage StatisticsImpact Factor 2-Step Map?

26. Mapping Based on Subscriptions ２０９ titles JCROPAC

27. Journals subscribed at Agricultural Library (JCR 2004) Drawn by Aiko WATANABE Agricultural Library On campus Unpurchased

28. Drawn by Aiko WATANABE Journals subscribed at Agricultural Library (JCR 2004) The Journal of Biological Chemistry Nature Proceedings of the National Academy of Sciences, USA

29. Journals subscribed at Agricultural Library (JCR 2004) ANIMAL FOOD FISHERIES ANALYSIS MEDICINE AGRONOMY ECOLOGY PLANT CELL GEOPHYSICS BIOCHEMISTRY Drawn by Aiko WATANABE

30. Advantages of Maps Based on Subscriptions  Degree of availability of journals receiving many arrows  Gaining an understanding of the importance of a journal  Relationships between fields and balance of journals purchased

31. Assessment by Researchers  Gaps between the arrow concentrations and their professional perceptions  Are maps based on subscriptions limited?

32. Example: Fisheries Science Marine Ecology Progress Series Limnology and Oceanography

33. What is Causing This Gap?  Citation data for non-subscribed journals also needed?  Citation data for related fields also needed?  Comparison of this map to another map including all journals in the field

34. Drawn by Aiko WATANABE Fisheries + Marine & Freshwater Biology + Oceanography (JCR 2004) Limnology and Oceanography

35. 2-Step Map of the Overall Field  Map of all journals in the field + subscription information  Facilitates the holistic picture of the field  Helps to recognize essential journals that have not been subscribed

36. Effectiveness of 2-Step Map  Visually recognize the trends that are difficult to be drawn by citation metrics alone Journals with high IF values ≠ Journals that receive many arrows  Observe the relationships between journals and categories  Observe the research trends within fields  Utility in journal collection development by adding subscription information  Utility for journal evaluation supplementing IF values Researchers show high level of interest!

37. Issues Surrounding 2-Step Map  Limitations due to JCR For 1-year sets of data, journals cited over a long period of time are not well presented Journals included in the JCR are limited Journals after title change are treated as new journals

38. Issues Surrounding 2-Step Map  Difficulty in Extracting Appropriate Field Research fields becoming interdisciplinary and segmentalized JCR Category ≠ Research focus of your institution  Complex nature of data extraction and map creation

39. Difficulty in Extracting Appropriate Field  Graduate School of Agricultural Science (Plant sciences) Plant Sciences → × (Too broad) Agronomy → × (Content is irrelevant)  Essential to know the research focus before starting!

40. Field Analysis Based on Journal Usage Survey  Journals in Use (2005) Journals in which our faculty published their work Journals that were cited in their work  Analysis of the use frequency and the fields of these journals

41. Extraction of Journals in Use Journal that Published Paper Citation Information Publication Year ＋ Affiliation

42. １６３４ titles 43 categories １２３ titles EJ available: 100 Not available on campus: 10 Items not indexed in JCR: 7 Analysis of Extracted Data

43. Creating a Map of Extracted Fields REPRODUCTIVE BIOLOGY (JCR 2005) Used more than 10 times Used more than once Non-subscribed journals Human Reproduction

44. Drawn by Aiko WATANABE 2 3 2 1 1 23 6 4 3 6 4 2 4 4 1 24 10 55 10 15 24 15 Fisheries + Marine & Freshwater Biology + Oceanography (JCR 2004)

45. Effective Application to Library Collection Development  Determine the field(s) you wish to analyze  Create a 2-Step map with all journals in the field(s)  Add subscription information  Add usage frequency information

46. Multiple Indicators for Collection Development  Use a variety of indicators to make multi-faceted and comprehensive evaluation  Associate with the characteristics of the research focus  Also keep in mind the characteristics of each journal 2-Step Map Impact Factors (2, 5, 10 … years) Statistics on Electric Journals usage (COUNTER) Researcher surveys …

47. Conclusions  Overcoming excessive reliance in impact factors Providing researchers with accurate information Enhancement of the JCR – for utilization of multiple indicators  Facilitating easier data extraction  IF values longer than 2 years  Refining category schemes  Handling journal title changes

48. Thank you ！