Community responses to commercial metrics tools: the responsible use guides

Robyn Price and Andrew Gray introduce the InCites and SciVal responsible use guides, a new series from the LIS Bibliometric Committee to promote informed use of commercial metrics products.

InCites, owned by Clarivate Analytics, and SciVal, owned by Elsevier, are popular commercial analytics subscription tools used by institutions to report on research publication data.

Many of the pre-calculated metrics in InCites and SciVal are vital for institutions that want publication data and metrics but don’t have the resources of a statistically and theoretically trained bibliometrician able to compute raw data and draw meaningful conclusions from it. Even with this resource, the products are also essential data sources; for a lot of these types of data, open source alternatives do not always exist or are not yet of consistently good quality or interoperability. InCites and SciVal also perform mammoth tasks of ‘cleaning up’ data, such as disambiguating millions of author identities and constructing organisation hierarchies.

However, the data universe that these products create- comprising people, publications, publishers, institutional and geographic affiliations and more; are entities that in real life are not neat, discrete data units. Added to this are dozens of metrics purporting to measure values as cryptic as ‘impact’, ‘collaboration’ and more. The products present metrics accompanied by explanations, but the results are precalculated, sometimes with the calculation breakdown not visible or the underlying data not available or to be found in separate subscription data products.

The lack of transparency in data collection and metric calculations in commercial analytics tools is one of the problems identified by the Metric Tide Report as a barrier to practicing responsible metrics. This is especially pertinent when the data is then used to influence decision making, something that InCites’ own website alludes to with better data leads to better decisions…’. Many who use InCites or SciVal want to do so responsibly, but can feel put off by these issues or daunted by the number of options and settings available within the products.

Photo by Matthew Waring on Unsplash CC-BY

The series originated with Ian Rowlands documenting his knowledge of SciVal for his institution. Seeing a community need for editorially independent guidance for these tools, we then created an equivalent guide for InCites. The guides have been informed by members of the LIS Bibliometrics community and committee to share the experience of those who use the products in a real world, academic institutional setting. For all authors and contributors, understanding the potentials and limitations of quantitative metrics, especially in products that can compute an array of them in an instant, is key.

The guides include recommendations on which indicators, in our opinions, can be useful or informative, and which ones should be avoided, as well as a discussion of how the indicators are calculated. Throughout, there is a focus on responsible use, emphasizing reference to organisation responsible metrics policies or the DORA and Leiden principles.

The guides are not intended to be ‘how to’ user manuals. We encourage them to be used in complement with Clarivate Analytics and Elsevier’s own support documents and the advice of their product consultants.

Published under creative commons licenses, we hope other users will continue documenting and sharing best practice in this area.

Download: Using InCites responsibly: a guide to interpretation and good practice

Download: Using SciVal responsibly: a guide to interpretation and good practice


Robyn is the Bibliometrics and Indicators Manager at Imperial College London. In this role she is responsible for managing bibliometrics service with an emphasis on promoting responsible use of metrics. Previously, Robyn worked in the editorial teams of subscription and open access journals.

Andrew is the Bibliometric Support Officer for UCL Libraries, providing guidance and support on the use and interpretation of publication metrics to users across the university. He previously worked as the librarian for the British Antarctic Survey, where he first became interested in research metrics.

 
Unless it states other wise, the content of the 
Bibliomagician is licensed under a 
Creative Commons Attribution 4.0 International License.  

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.