Katie Evans reflects on how their principles of research assessment and management are being applied at the University of Bath – one year on.
What difference does having a responsible metrics statement make? And how do you tell if a statement is doing what you need it to do?
We published the University of Bath’s Principles of research assessment and management in March 2017. Our intention was to ‘encapsulate current good practice and act as a guide for future activities’.
We didn’t have any major concerns about how the University was using metrics in research assessment, so we weren’t setting out to dramatically change behaviour. Nevertheless, I’d say that having the statement has made a positive difference: it makes our commitment to good practice explicit, helping to cultivate a healthy institutional culture.
To give just two examples:
I met with an early career researcher to advise them on using publication and citation indicators to evidence the strengths of their research track record for a grant application. Their initial reaction was delight at the data and analysis tools available to them, but then their face fell: ‘is this how my work will be assessed in the future?’ they asked. I was able to point to our statement of principles to reassure them that, within the University at least, assessment of their work would always be centred on expert judgement, with metrics playing a supporting role only if appropriate, and in only context.
My second example will probably be familiar to many Bibliomagician readers: a senior manager asking a professional services colleague if they could provide a simple traffic light rating of researchers to help them identify research stars. The motivation behind the request was innocent enough, but the multi-faceted complexity of research quality means that any attempt at such a score system would be dangerously inadequate. Having the University’s statement of principles gave us firm ground to stand on when saying we couldn’t provide this.
The results of the survey were mixed: looking on the bright side, over half our respondents were aware of the statement of principles. On the down side, it was only just over half!
So I know that the statement is helping me and my professional services colleagues who provide metrics. But what about everyone else?
Reviewing our statement one year on, we carried out an anonymous survey of Heads of Department, Deans and Associate Deans (Research) – all key targets of our communications plan due to their role in academic promotions and recruitment. The results of the survey were mixed: looking on the bright side, over half our respondents were aware of the statement of principles. On the down side, it was only just over half! Asked in what circumstances they would refer to the statement of principles, responses varied from ‘Never’ through to ‘Wide range of circumstances, from promotion review to setting REF strategy’. Clearly there’s still a need for internal advocacy. We’re addressing this through a drip feed at point of need approach, integrating awareness of the principles into working practices.
One important change we have made as a result of reviewing whether the statement of principles is doing its job is to add a contact name so that any member of the University who has concerns about practices failing to conform to the principles has a clear route for raising their concerns. Again, this reinforces the message of our commitment to good practice.
In short: 18 months on from publication, it’s clear a statement of principles is a way marker rather than the end of the journey of using metrics responsibly.
Katie Evans is the Research Analytics Librarian at the University of Bath, responsible for supporting the use of research publishing, citation and collaboration data across the University. Previously, Katie worked on open access at the University of Bath.
Unless it states otherwise, the content of The Bibliomagician is licensed under a Creative Commons Attribution 4.0 International License.
The Bibliomagician 