How Do Academic Librarians Use Research Impact Metrics? Guest post by Rachel Miles

Rachel Miles, Research Impact Librarian at Virginia Polytechnic Institute and State University, summaries the results from a recent National Survey, giving an insight on how Academic Librarians are using altmetrics in the US.

Altmetrics: What is going on?

“There is a growing interest in altmetrics.”

“Altmetrics have become the newest tool among academic librarians.”

“Scholarly communication is becoming more enhanced by the use of altmetrics.”

You may have heard one or more of these phrases in the past few years, especially if you are in the field of scholarly communication. However, such claims bring to mind advertisements of the same flavor: “The hottest and most popular gadget on the market!” Most of us are guilty of getting excited about the latest trend, tool, or gadget. In contrast, many of us are skeptical and critical of new tools and metrics.

I’m part of a research team that wanted to test whether these claims about librarians’ love for altmetrics were actually true. Along with Sarah Sutton (Emporia State University, Kansas, USA) and Stacy Konkiel (Altmetric, Digital Science, Minnesota, USA), I helped survey US librarians to determine the actual awareness and usage of altmetrics among academic librarians in the USA.  We also surveyed librarians about their awareness and use of other types of research impact indicators like citation counts, the Journal Impact Factor, and qualitative impact evidence. Our study (published recently in the Journal of Librarianship and Scholarly Communication) was the first large-scale, national study of its kind.

Librarians previously on the tenure track were much less likely to use altmetrics in their tenure and promotion dossiers, than academic librarians currently on the tenure track

Some of the most interesting results from this study include:

  • Academic librarians with regular scholarly communication duties are likelier to use research impact indicators, compared with other academic librarians;
  • There’s a growing interest among US academic librarians in using altmetrics as an indicator in promotion and tenure dossiers at institutions that offer tenure for librarians; and
  • Faculty tenure and promotion requirements tend to influence the likelihood of librarians addressing  JIF and citation counts during consultations

Let’s break this down in more detail.

Scholarly communication librarians are more “expert” in metrics than their colleagues

Not surprisingly, we found that academic librarians with regular scholarly communication duties (duties performed at least once a month) had more familiarity with and an overall higher usage of research impact indicators overall. Table 1 reflects this trend: much higher percentages of librarians with regularly scholarly communication duties responded “5 – I’m an expert” when asked to rank their knowledge of JIFs, citation counts, usage statistics, and altmetrics.

How do academic librarians use research impact metrics fig 1

American academic librarians are increasingly using altmetrics in promotion & tenure

Our analysis also found that academic librarians’ interest in using altmetrics for promotion and tenure is a relatively recent phenomenon. Librarians previously on the tenure track were much less likely to use altmetrics in their tenure and promotion dossiers, than academic librarians currently on the tenure track (Figure 1).

How do academic librarians use research impact metrics fig 2Figure 1. Comparison between librarians’ past versus intended future use of research impact indicators in tenure and promotion dossiers.

Of course, altmetrics have only been around for a few years, so it is only logical that tenured librarians would not have had much opportunity to use altmetrics in their dossiers. However, it is still exciting among altmetrics enthusiasts that this data shows a growing interest in the use of altmetrics among academic librarians on a more professional level.

Faculty want to learn about metrics for summative, not formative evaluation purposes

Though our research did not directly assess the use of metrics among faculty members, it did ask academic librarians about the likelihood of addressing research impact indicators during consultations with faculty. According to our statistical tests, the JIF, citation counts, h-index, and to some extent qualitative measures, are far more likely to be addressed during consultations with faculty concerning issues related to tenure, promotion, and grants than during consultations concerning publishing issues. So what does this say about faculty members’ use of research impact indicators?

How do academic librarians use research impact metrics fig 3Figure 2. Differences between the frequencies of addressing indicators of research impact during one-on-one consultations with faculty concerning publishing issues versus tenure, promotion, and grants.

First of all, we do not know for sure why these particular indicators were addressed during the consultations; faculty members could have initially asked about them, or the librarians could have mentioned them. We only know that the indicators were addressed by the librarians during consultations with faculty. At the very least, we can tell that assessing research impact may be more important for the sake of attaining tenure, promotion, and/or grants than it is for the determining how best to publish and disseminate research.

Other significant results are reported in our recent publication on this survey research, such as the reasons why librarians use the JIF and their varying levels of familiarity with certain research impact indicators. In addition, we emphasize the influence of tenure and promotion on researchers’ engagement with research impact metrics, and we call for more research on the influence of current research evaluation practices and career incentives on academic and scientific processes. Finally, and importantly, we call on the academic librarian community to take action in this important and emerging field to help influence and change current practices and to promote a healthier and more responsible approach to research evaluation.


rachel-miles-profile-picRachel Miles is the Research Impact Librarian at Virginia Polytechnic Institute and State University (more commonly known as Virginia Tech). She provides specialized research support to the Virginia Tech community in citation analysis, bibliometrics, altmetrics, the responsible use of research impact metrics, and emerging applications of impact data at individual, department, institutional, and other group levels. She also assists the community in developing and maintaining online researcher and professional profiles, developing outreach and communication strategies, instruction, research assistance, collection development, and partnerships in the library and across campus. https://orcid.org/0000-0002-8834-4304

 

Creative Commons Licence Unless it states other wise, the content of the 
Bibliomagician is licensed under a 
Creative Commons Attribution 4.0 International License.

3 Replies to “How Do Academic Librarians Use Research Impact Metrics? Guest post by Rachel Miles”

  1. You should declare the conflict of interest her: Stacy Konkiel, whom you say is ‘of Digital Science’ actually works for and promotes Altmetrics. So your study is not unbiassed, and it is disingenuous at the least, to claim it is.

    Like

    1. Hi Susannah,

      Our apologies if we weren’t clear in the blog post – Stacy does indeed work for Altmetric as well as Dimensions, which are both a part of Digital Science. For simplicity’s sake, we listed her affiliation as Digital Science and her Altmetric affiliation is described in the journal article itself, but you’re probably right in that her Altmetric affiliation could be clearer in this post. Sahar and the Bibliomagician team have updated the post to clarify things, we hope that helps!

      We also understand that Stacy’s affiliation might be cause for concern about bias for some, but hopefully this can help: the survey was vetted by consultants at Indiana University’s Center for Survey Research a few years before Stacy even joined Altmetric, Rachel and Sarah were pretty much fully responsible for the study’s statistical analyses, as well as the article’s results, discussion, and conclusion, and JLSC put us through a rigorous peer review prior to publication–so there were multiple checks in place to deter against bias.

      Sincerely,
      Rachel, Sarah, and Stacy

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.