1. Informetrics, bibliometrics, scientometrics, altmetrics: What is it all about? SIGMet-sponsored panel at the ASIS&T Annual Meeting 2014, Seattle Tuesday Nov. 4  -10:30 More information:

2. Experts on the panel Judit Bar-Ilan, Professor, Department of Information Science, Bar-Ilan University Gali Halevi, Senior analyst and program director, Elsevier Stefanie Haustein, Post-doctoral researcher, Canada Research Chair on the Transformations of Scholarly Communication, University of Montreal Andrea Scharnhorst, Head of Research & Innovation, Data Archiving and Networked Services (DANS) institution, the Netherlands Jevin West, Assistant Professor, iSchool, University of Washington Chair: Isabella Peters, Professor, ZBW Leibniz Information Centre for Economics, Germany

3. Use case A: You work in a tenure committee
Use case A: You work in a tenure committee. Two candidates have similar CVs but you need a decision by the end of the day. Only one candidate can get tenure. You want to choose the candidate with the most impact. How can you find out who is the one?

4. Experts’ statement to use case A: Hiring committees may use qualitative assessments (peer review) if they have enough time for that. If not they will rely on numbers. Even if there are qualitative assessments these are often backed-up or compared with numbers. What are these numbers? Today these are mainly based on citation and publication counts of journal articles and possible some conference papers. There are some derived measures like the h-index that has shown to be particular tempting as it pretends to reflect a researcher’s entire research output and impact in only one number. Unfortunately from the decision makers’ point of view, it is not as simple. Even if we focus on the research aspect and disregard teaching and management qualities, it is still not easy to determine the better researcher based on a few indicators. Books are problematic because citation databases do not cover them well. Also citation databases do not equally cover well all disciplines. Altmetrics as applied on platforms as ResearchGate, or SlideShares with downloads, views, likes and alike are often not comparable across candidates. Currently the major databases used for bibliometric analysis are Web of Science, Scopus and Google Scholar. Which one to use? Often there can be huge difference in these “numbers” when different sources are used. If a bibliometric analysis is to support your decision, it should be as comprehensive as possible providing a broad and detailed overview of the candidates’ publication patterns. It should include the number of publications both in full and fractional counts to account for differences in co-authorship patterns and distinguish between first and co-authored papers. Although a citation analysis is mostly limited to journal articles covered by the specific databases, the publication analysis should include all types of documents including edited books, book chapters and monographs, conference papers and journal articles. Information about the publication venues and document types can indicate where the authors publish and co-authorship patterns indicate how well they are connected within their scientific community. For those publications, where citation data is available, situations should be normalized in particular if candidates from different (sub-)fields are compared. On author level it is preferable to indicate the share of self-citations to make sure that high citation counts actually indicate influence on the scientific community. Particularly when used in research evaluation, it is essential to conduct bibliometric studies on the same database at the same point in time to gain comparable results. In the end, it is in the hands of the committee to decide which indicators are most important.
Some “no-gos” in author evaluation include:
Summing up impact factors of the journals for the papers they published in. The so-called cumulative impact factor is meaningless, because the papers of a journal determine a journal’s average citation rate, i.e. the impact factor, but not vice versa.
Ranking authors based on the h-index, particularly if they do not have the same academic age and are not from the exact same research area. The h-index indicates the number of h papers with at least h citations and disregards all papers and citations above and below h. Both values are affected by the discipline as well as how long someone has been publishing.
Making a difference between 5 and 6 publications or 10 and 13 citations. With a low number of papers, bibliometrics become unreliable and comparisons should be handled with care.

5. Use case B: You finished an exciting research project and gained some spectacular results. Unfortunately, with the end of the project you also lost your job at the university. What would be the best strategy to get the most out of the results so that you will quickly find a new and better paid job?

6. Experts’ statement to use case B: You know that nowadays bibliometrics is used, numbers are in favour, visuals go nicely and that in general you need to raise attention! One way to raise attention is to put visuals into your CV or some infographics (e.g., with Vivoweb). But be also careful! Visualization falls and raises with the data and designed used! If you think in terms of presenting your bibliometric or altmetric profile, you quite naturally want to make your numbers look as high as possible. This might influence the choice of the database you use. On the other side, which number you present might also depend to which system you have access. In any case, you should indicate how and when data was collected. Be transparent! The hiring committee will probably want to use the same data source for all applicants. You might think about indicating Web of Science, Scopus and Google Scholar citation counts in your list of publications favorably with a link to the records to facilitate committee members to conduct their own analyses. If you do not want or cannot provide a detailed bibliometric study as described above, it is advisable to use author ids such as ORCID, Researcher ID, Scopus Author ID and a Google Scholar profile to link to your publications and make sure that your work can be found and linked to you easily. Publishing open access and self-archiving on platforms such as arXiv, SSRN or institutional repositories, increase visibility and facilitates access to your publications.   Social media platforms such as Academia.edu and ResearchGate help you to create an online presence of you as a researcher and your achievements including publications but also other outputs such as slides and datasets on SlideShare, FigShare and Dryad or code on GitHub. These sources also help in showing the visibility/importance of one’ work by displaying how often it was used. This can be done through altmetrics – additional sources of measuring “impact”. These include Mendeley readership counts, downloads (also of applications/data), views of slideshare, figshare, ResearchGate, Twitter mentions, etc. Altmetrics can be especially useful for newer publications that did not have enough chance for receiving citations. Platforms such as Altmetric.com and ImpactStory help you to monitor and present altmetric information.

7. Use case C: You are new in a discipline and just started your PhD
Use case C: You are new in a discipline and just started your PhD. Your advisor asked you to summarize the most influential literature on a certain topic. Unfortunately, you do not really know where to start your search. You know that you can use the library but the summary is due in two weeks. What would help in determining what literature is a must-read and finding it?

8. Experts’ statement to use case C: One knows that almost all searches start in Google! But hits from there might be far too many, or unspecific. Next best option: go to your library OPAC (or use WorldCat) and talk to your librarian – or look up their search recommendations. For looking up terms and notions Wikipedia is not a bad start at all, in particular if you can compare Wikipedia entries in different languages. This already gives you a critical view on the sources. If you are new to a discipline, then it is also good to start with a textbook or a review article on the topic. But, if you want to do a more professional search and you cannot find either textbook nor review article, then choose appropriate keywords and look for a few highly cited or a highly “read” publications on the topic. Databases indexing peer-reviewed journal articles like the Web of Science and Scopus which also contain reference lists – citation indexing – are a first class source for academic information retrieval. Their features can help you to find review articles about your subject that are helpful to provide an overview of a field and introduction to a research area. Then you should do both forward and backward chaining. “Backward chaining” means to look at the reference lists of the initially chosen items, and “forward chaining” means to check who cited the initially chosen items or the useful items you found in the references. Forward chaining is facilitated by the citation databases. If the found documents are too old or not at all available, ranking relevant papers by the number of citations can help you to choose picking a few papers to start with, particularly from a large amount of documents. The focus should, however, be on the topical relevance which should be determined by at least reading paper title and abstract. A citation analysis in particular if visualized in network graphs, for example a map of co-cited title terms can help you to get a rough overview of the field in question. Tools like VOSviewer, CitNetExplorer are relatively easy to use and help to produce these maps from Web of Science and Scopus download files – so-called Science Maps. SCI2 is a tool which allows a more detailed network analysis and also offers a lot of visualizations which help you to gain an overview.

9. Use case D: Budgets are always decreasing
Use case D: Budgets are always decreasing. You know that some of the researchers of your institution favour particular journals and demand access to them. However, you are not sure if these journals are really the best buy. What could help you with that decision?

10. Experts’ statement to use case D: Journals selection must be based on its quality and its usefulness to the institutions’ faculty and students, therefore you should take into account local usage. If certain requests by researchers from your institutions are available, these are the journals you should choose. With the advent of e-publishing and the remote availability of scientific publications, most publishers now have usage tracking systems. These systems, such as COUNTER, track and capture page views; full text downloads and clicks through of online content and can help to find out which journals are used most frequently and determine cost per use ratios to make informed decisions about subscriptions and cancellations. Although publishers provide statistics, you should take into account also usage stemming from preprint servers and self-archiving, however for these it is more difficult to get data. And yes, for journals, one can use journal impact measures, like the journal impact factor, SNIP, SJR, Eigenfactor, article influence, but do not rely only on these create an institution-based analysis on journals usage. Download statistics can be complemented by a reference analysis to find out which journals researchers from your institutions most often cite. Combining an analysis of a journal’s citations rates with local usage rates can better inform the selection decision and help with building a customized collection that will better serve the library users. However, all these analyses should be normalized by discipline or done separately in order to account for smaller faculties and less research-oriented departments. Also note that the different measures can possibly lead to contradictory decisions. Another important factor is bundling by the publishers/aggregators. Often it is more expensive to subscribe to a few specific journals than to by a package.

11. Use case E: You consider yourself a researcher and you work in a scientific environment in large research groups. Mostly you develop software or generate large data sets. Oftentimes you engage with the public since your research has been reported in newspapers and blogs. Also, politicians seek your advice. Since all of that is very time-consuming your publication list is rather short. But there is that job offer, searching for engaged and influential researchers. How can you still satisfy?

12. Experts’ statement to use case E: In this case you should definitely try to show your influence using altmetrics. List these engagements and outputs as you would list your publications, as they are part of your work and equally important. If you provide open source code and software on platforms such as GitHub, you can report statistics on how often your code and tools have been used and further developed by others. Not sure that this will be convincing enough, though, since it is not yet proven whether or what kind of impact various social media metrics differ. But you can use them to demonstrate that your work has been shared, discussed, viewed or saved online.

13. Informetrics, bibliometrics, scientometrics, altmetrics: What is it all about? More information: Get in touch: