Tracking citations in Law – Guest Post by Quentin Pope

Dr Quentin Pope, a Liaison Librarian and subject specialised librarian with specific responsibilities for the library and information requirements of QUT School of Law, outlines how you can track citations in Law.

The primary purpose of bibliographic reference research is to link references in citations of scholarly publications. For cross-linking citations to publications, an essential, but not sufficient or sole, condition is the extensive coverage of scientific literature by a publication provider/database, in which citation cross-referencing to be performed.

Legal publishers more than other scientific publishers, it appears, are protective of their content and, moreover, the coverage of the legal content by each provider/database is very limited (Table 1) [author data].

Table 1.  Approximate number of legal journals per provider and ranking list

Tracking Citations in Law post
*1findr data (Source: Dr Éric Archambault, President and CEO of Science-Metrix and 1science, (2017), Personal communication)

Successful automatic recognition of reference fields such as Author, Data, and Title fields is also necessary for cross-linking, and the success of the interlinking is highly dependent on how well structured the reference data are. Scientific articles with Reference/Bibliography lists have well defined and structured reference data, which are relatively easy to automatically cross-link. On the other hand, the majority of legal publications adhere to the convention of citing reference in footnotes without providing Reference/Bibliography lists at the end of the publications. Moreover, citations are often amalgamated within the explanatory text, which makes automatic cross-referencing even more challenging than that of the structurally uniformed Reference/Bibliography lists [1].

For a quick overview of the problems of the reference cross-linking of legal publications, Google Scholar was selected amongst other providers/databases of scientific literature as an example, as it is by far the largest provider of legal publications.

In summary, there are two major challenges for successfully collecting cites to legal publication:

  1. Even if the legal publishers’ content is available for indexing within Google Scholar, the full text indexing, which is essential for cross-linking references to publications, is not always available. This is most likely because of the particular arrangements that Google Scholar have in place with publishers as to what content and to what degree the content could be indexed by Google Scholar.
  2. Even if legal content is available and indexing of full text is permitted by the publishers, there are technical challenges in automation of cross-linking of references from footnotes.

As a consequence of these challenges it is essential to manually search citations across a large number of legal content providers/databases and then collate the data.

While this analysis is only a snapshot, it does provide a good overview of what factors play role in ability of Google Scholar, and other services, to cross-link legal references

A binomial logistic regression was performed on a small set of data – 15 publications with 167 citations, and only secondary materials such as journal articles, book chapters and books were considered for this analysis – and collected in a large number of legal databases and Google Scholar following procedure described under Searching for evidence of impact and / or engagement of legal research below). The purpose of the regression was to examine the effects of the existence of Bibliography list, provider/database and year of publication on the likelihood that cross-linking of citations is chanced by Google Scholar. Google Scholar was selected because coverage of the citing documents (journal articles, book chapters and books) in Google Scholar was, for the 167 cited references and 15 cited references, 100%, even though the cross-linking/interception of citations, which was of interest varied between 50-70% depending on the source of the particular original citation.

The logistic regression model was statistically significant: -2 Log Likelihood ratio= 39.602, p < .0005. The model explained 39% (Nagelkerke R2) of the variance in citations being cross-linked by Google Scholar and correctly classified 74% of cases. The existence of the Bibliography lists were 9 times more likely to result in Google Scholar capturing publications in the ‘cited by’ list than when the Bibliography lists are absent. Increased age of the publication was associated with a decreased likelihood of appearing in the ‘cited by’, but provider/database did not add significantly to the model (the number of providers/databases in this small datasets had only 4 categories (HeionOnline, Westlaw AU, Taylor and Francis and ‘other’).

While this analysis is only a snapshot, it does provide a good overview of what factors play role in ability of Google Scholar, and other services, to cross-link legal references. (Similar results have been observed for HeinOnline [data not shown]). The ‘capture effect’ is increasingly complex as Google Scholar captures data not only from publishers but from many open repositories and aggregators, including institutional repositories which often provide full text to otherwise protected materials as a result of the various mandates of funding bodies.  This can lead to ‘a diluted’ effect of provider/database.

The process of manually collecting citations to legal publications is very time consuming, and is not really plausible on a large scale, however, this process does yield good results and present a more or less complete picture of attention and impact of a particular researcher (via citations to his/her publications). Below is a proposed list of providers/databases and other resources, which could be full-text searched in order to collect information on how often a specific research publication has been cited by others.

The data collated as a result of the process lent itself to measuring seamlessly h-index and other author/publication level citation metrics. For benchmarking, please see an excellent discussion at LSE Impact Blog [2]

Tracking Citations in Law post_methodology
Searching for evidence of impact and/or engagement of legal research: Databases (legal and ‘other’-subject specific) of scholarly publications, and other resources.

 

 


Dr Quentin Pope | Law Librarian
QUT | Gardens Point | 2 George Street Brisbane QLD 4000|P: +61 7 3138 5047 | E: quentin.pope@qut.edu.au | www.qut.edu.au|W: https://www.library.qut.edu.au/about/contact/staff/q_pope.jsp

References

  1. Kim, Y.-M., Bellot, P., Faath, E., and Dacos, M. (2012). Automatic annotation of incomplete and scattered bibliographical references in Digital Humanities papers. In Proceedings of Conference en Recherche d’Information, Applications (CORIA). Bordeaux, France, pp. 329–40.
  2. The London School of Economics Impact of Social Science blog.  (n.d.) Impact of Social Sciences – 3: Key Measures of Academic Influence. Retrieved fromhttp://blogs.lse.ac.uk/impactofsocialsciences/the-handbook/chapter-3-key-measures-of-academic-influence/

Special thank you from the Author

[1] I am in debt to the support of my colleague and friend, Janet Baker, QUT Library ERA Officer, whose guidance and encouragement were essential to see this work progressing.


Creative Commons LicenceThis work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.