Linda Nasr interviewed for our series “Going abroad” Sabine Benoit, who moved from Germany to the UK. This is the second part of the interview with her.
So, what is special about working in academia in the UK?
I will first mention research before I elaborate on teaching. In research, the one thing that is omnipresent, is what they call REF. It is a research assessment that all Universities go through every 6-7 years. The first ever REF (back then called RAE) was in 2008, the second one was in 2014, the one upcoming one will be in 2021. The REF is an important driver of institutional reputation and an instrument for the government to allocate funds to the best performing ones. Therefore, the REF or “being REF’able” has had an enormous impact on hiring decisions and the job market, in particular shortly before cut off date.
University of Surrey
Can you tell us more about this being “REF’able” who decides that?
Coming to the assessment, how does the REF approach the task to validly and reliably evaluate the research performance of an entity and how does that relate to working in the UK? The answer is that research active staff members in Departments/Schools submit a certain number of publications (1-5 for REF 2021) published within the assessment time period. Some readers might think the most reliable at the same time manageable way to evaluate these publications is to agree on a common ranking and create a Department/ School score. The evaluation would then rest on the shoulders of Editors, journal reviewers, impact factors which are based by others using the research, thus a wide array of different actors with content expertise. The situation in the UK seems favourable for such an evaluation, since the ABS, the (British) Association of Business Schools issues a Journal Ranking that is updated every couple of years. However, this is not how it’s done: The last REF regulations explicitly forbid that rankings or Journal impact factors are used to evaluate research outputs. I had lively discussions about rankings in general and the ABS ranking in particular. Without the intention to repeat these arguments here we are probably all aware of the flaws of rankings, the relevant question is not how many flaws rankings have, but what method to evaluate research outputs is less flawless.
Service Frontiers, Summer 2017
Tell us how they evaluate the research outputs?
In REF the evaluation is done through panels which in Business and Management consisted of 26 members mostly full professors working at UK institutions. These panel members split up the publications of the entire UK Business discipline and then each publication is read by two panel members making judgment on how many stars each and every paper is worth (going from 1 up to 4). In 2014 all 101 Business Schools submitted in total 12,204 outputs, which are 469 x2 = 938 outputs per panel member. This process has two components: quantity and quality. First, 469 publications per panel member is a tiring amount of work and here we only mention the outputs, institutions submitted way more than only outputs to be evaluated, e.g. describing their research environment and research strategy and so-called impact case studies (see below). The whole process takes more than a year, for the last REF the cut of date was October 2013, the results were communicated in December 2014. One anonymous panel member wrote in the popular press that s/he had only about 20 minutes per paper.
The sheer volume does not allow in depths analysis. From a quantity perspective making a valid and reliable judgement on 938 papers that is funding relevant and can be a game-changer for some institutions is really remarkable. Second, if the Business discipline would be subdivided into 13 sub-fields and there are sure arguments for way more than that, this would mean two panel members decide upon the outputs of the entire Marketing discipline in the country. This is remarkable from a quality perspective. What this person is not in expert in all Marketing topics and/or methods (who is?), what if the Innovation panel member is way more generous in his evaluation? I have learned that the panel members go through an rigorous process of aligning judgment, but I’m still not sure this entirely convinced me. What if this person is very well connected in the country, which is usually true for people that get important jobs like this, won’t they have a bias towards the publications from their academic friends or former colleagues or from “good” institutions? Personally, I cannot imagine any scenario where such an evaluation method that is based on an individual opinion of two academics, who are very likely not familiar with the breadth of topics and methods submitted can be in any way more effective and fairer in evaluating research performance than using rankings. I totally don’t get it, to me that has so much unpredictability, variability and randomness! But as said I’m the newcomer to the system, I’m the odd one out, so its ok, I probably don’t have to understand it! Every system has its path dependency and I joined that path at a later stage.
Thought Leadership Conference at UQ in Brisbane, November 2017
This was all rather critical, does the REF has anything positive?
Oh yes absolutely, with all the above mentioned the REF has some great implications for the relevance of research in particular compared to the academic system in Germany. Because Universities scoring high on research get funds, they have a very tangible incentive structure to give individuals with 3-star or 4-star publication potential time to do research. For me personally this is great, my teaching load is very manageable, it is about ¼ of the teaching load I had in Germany. To be fair in Germany full professors usually have research assistants that support teaching and a personal secretary supporting admin work. Thus, if you think about getting a job in the UK and you are a “fairly good” researcher, meaning you publish in at least 3* Journals, no need to hit for the big 4, you’ll be treated well and you’ll probably like it.
Is there anything else worth mentioning about the REF?
Yes, indeed. The second remarkable aspect about REF is what they call “impact”, which means Schools submit a couple of so “impact called case studies”, in which they describe how research and it has to be published research rather than consultancey has had an impact on the non-academic world, e.g. policy, an industry or company. For example, if your research changes how a big retailer runs its loyalty programme or how government policy regulates food advertising, then you will be likely one of the selected few with an impact case study. The perspective on impact sometimes makes people “shine” that are not the usual 4-star-hitters, but that do solid and relevant work with government bodies or companies. Encouraging academics to think about how their research can make a difference outside the academic world is a brilliant idea, in my view we should all adapt to this kind of thinking. We are not doing this research just for the academic community. It really changed my perspective on the dissemination of my research, suddenly submitting the final version and looking through the proofs is the end of the publication process, but it becomes the start of the dissemination process of this publication. Personally, I have started developing infographics and videos for my publications to enhance the access to my results for non-academic audiences: see my youtube channel or sabinebenoit.com for some examples.
Linda: I heard that some people here in the US have part-time jobs in the UK, how’s that?
That’s has all got to do with the REF. The 2014 REF rules specified that every academic working at a UK institution on the 31st of October 2013 with an at least 20% fractional was REF’able, meaning that some institutions hired part time staff members to be able to submit more and better publications and then get a higher reputation and more funds. Even though these institutions played by the rules, the rules have changed now to make harder to submit “remote” superstar “staff members” that the students will never see in their lifetime.
Sabine Benoit is Professor of Marketing,
Director for External and Business Engagement for SBS
at the University of Surrey
see her youtube channel or sabinebenoit.com for more insights.
