{"id":6338,"date":"2018-04-18T18:11:11","date_gmt":"2018-04-18T22:11:11","guid":{"rendered":"http:\/\/www.servsig.org\/wordpress\/?p=6338"},"modified":"2018-04-26T10:52:02","modified_gmt":"2018-04-26T14:52:02","slug":"going-abroad-explaining-the-ref-system-in-the-uk","status":"publish","type":"post","link":"https:\/\/www.servsig.org\/wordpress\/2018\/04\/going-abroad-explaining-the-ref-system-in-the-uk\/","title":{"rendered":"Going abroad: Explaining the REF System in the UK"},"content":{"rendered":"

\"\"<\/a><\/p>\n

Linda Nasr interviewed for our series \u201cGoing abroad\u201d Sabine Benoit, who moved from\u00a0Germany to the UK<\/em>. This is the second part of the interview with her.<\/span><\/p>\n

So, what is special about working in academia in the UK?<\/strong> <\/em><\/p>\n

I will first mention research before I elaborate on teaching. In research, the one thing that is omnipresent<\/em>, is what they call REF. It is a research assessment that all Universities go through every 6-7 years. The first ever REF (back then called RAE) was in 2008, the second one was in 2014, the one upcoming one will be in 2021. The REF is an important driver of institutional reputation and an instrument for the government to allocate funds to the best performing ones. Therefore, the REF or \u201cbeing REF\u2019able\u201d has had an enormous impact on hiring decisions and the job market, in particular shortly before cut off date.<\/p>\n

\"\"

University of Surrey<\/p><\/div>\n

Can you tell us more about this being \u201cREF\u2019able\u201d who decides that? \u00a0<\/em><\/strong><\/p>\n

Coming to the assessment, how does the REF approach the task to validly and reliably evaluate the research performance of an entity and how does that relate to working in the UK? The answer is that research active staff members in Departments\/Schools submit a certain number of publications (1-5 for REF 2021) published within the assessment time period. Some readers might think the most reliable at the same time manageable way to evaluate these publications is to agree on a common ranking and create a Department\/ School score. The evaluation would then rest on the shoulders of Editors, journal reviewers, impact factors which are based by others using the research, thus a wide array of different actors with content expertise. The situation in the UK seems favourable for such an evaluation, since the ABS, the (British) Association of Business Schools issues a Journal Ranking that is updated every couple of years. However, this is not how it\u2019s done: The last REF regulations explicitly forbid that rankings or Journal impact factors are used to evaluate research outputs. I had lively discussions about rankings in general and the ABS ranking in particular. Without the intention to repeat these arguments here we are probably all aware of the flaws of rankings, the relevant question is not how many flaws rankings have, but what method to evaluate research outputs is less<\/em> flawless.<\/p>\n

\"\"

Service Frontiers, Summer 2017<\/p><\/div>\n

Tell us how they evaluate the research outputs? \u00a0<\/em><\/strong><\/p>\n

In REF the evaluation is done through panels which in Business and Management consisted of 26 members mostly full professors working at UK institutions. These panel members split up the publications of the entire UK Business discipline and then each publication is read by\u00a0two panel members making\u00a0judgment on how many stars each and every paper is worth (going from 1 up to 4). In 2014 all 101 Business Schools submitted in total 12,204 outputs, which are 469 x2 = 938 outputs per panel member. This process has two components: quantity and quality. First, 469 publications per panel member is a tiring amount of work and here we only mention the outputs, institutions submitted way more than only outputs to be evaluated, e.g. describing their research environment and research strategy and so-called impact case studies (see below). The whole process takes more than a year, for the last REF the cut of date was October 2013, the results were communicated in December 2014. One anonymous panel member wrote in the popular press that s\/he had only about 20 minutes per paper.<\/p>\n

The sheer volume does not allow in depths analysis. From a quantity perspective making a valid and reliable judgement on 938 papers that is funding relevant and can be a game-changer for some institutions is really remarkable. Second, if the Business discipline would be subdivided into 13 sub-fields and there are sure arguments for way more than that, this would mean two panel members decide upon the outputs of the entire Marketing discipline in the country. This is remarkable from a quality perspective. What this person is not in expert in all Marketing topics and\/or methods (who is?), what if the Innovation panel member is way more generous in his evaluation? I have learned that the panel members go through an rigorous process of aligning judgment, but I\u2019m still not sure this entirely convinced me. What if this person is very well connected in the country, which is usually true for people that get important jobs like this, won\u2019t they have a bias towards the publications from their academic friends or former colleagues or from \u201cgood\u201d institutions? Personally, I cannot imagine\u00a0any<\/em>\u00a0scenario where such an evaluation method that is based on an individual opinion of two academics, who are very likely not familiar with the breadth of topics and methods submitted can be in any way more effective and fairer in evaluating research performance than using rankings. I totally don\u2019t get it, to me that has so much unpredictability, variability and randomness! But as said I\u2019m the newcomer to the system, I\u2019m the odd one out, so its ok, I probably don\u2019t have to understand it! Every system has its path dependency and I joined that path at a later stage.<\/p>\n

\"\"

Thought Leadership Conference at UQ in Brisbane, November 2017<\/p><\/div>\n

This was all rather critical, does the REF has anything positive? \u00a0<\/em><\/strong><\/p>\n

 <\/p>\n

Oh yes absolutely, with all the above mentioned the REF has some great implications for the relevance of research in particular compared to the academic system in Germany. Because Universities scoring high on research get funds, they have a very tangible incentive structure to give individuals with 3-star or 4-star publication potential time to do research. For me personally this is great, my teaching load is very manageable, it is about \u00bc of the teaching load I had in Germany. To be fair in Germany full professors usually have research assistants that support teaching and a personal secretary supporting admin work. Thus, if you think about getting a job in the UK and you are a \u201cfairly good\u201d researcher, meaning you publish in at least 3* Journals, no need to hit for the big 4, you\u2019ll be treated well and you\u2019ll probably like it.<\/p>\n

Is there anything else worth mentioning about the REF? \u00a0<\/em><\/strong><\/p>\n

Yes, indeed. The second remarkable aspect about REF is what they call \u201cimpact\u201d, which means Schools submit a couple of so \u201cimpact called case studies\u201d, in which they describe how research and it has to be published research rather than consultancey has had an impact on the non-academic world, e.g. policy, an industry or company. For example, if your research changes how a big retailer runs its loyalty programme or how government policy regulates food advertising, then you will be likely one of the selected few with an impact case study. The perspective on impact sometimes makes people \u201cshine\u201d that are not the usual 4-star-hitters, but that do solid and relevant work with government bodies or companies. Encouraging academics to think about how their research can make a difference outside the academic world is a brilliant idea, in my view we should all adapt to this kind of thinking. We are not doing this research just for the academic community. It really changed my perspective on the dissemination of my research, suddenly submitting the final version and looking through the proofs is the end of the publication process, but it becomes the start of the dissemination process of this publication. Personally, I have started developing infographics and videos for my publications to enhance the access to my results for non-academic audiences: see my youtube channel<\/a> or sabinebenoit.com<\/a> for some examples.<\/p>\n

Linda: I heard that some people here in the US have part-time jobs in the UK, how\u2019s that? <\/em><\/strong><\/p>\n

That\u2019s has all got to do with the REF. The 2014 REF rules specified that every academic working at a UK institution on the 31st<\/sup> of October 2013 with an at least 20% fractional was REF\u2019able, meaning that some institutions hired part time staff members to be able to submit more and better publications and then get a higher reputation and more funds. Even though these institutions played by the rules, the rules have changed now to make harder to submit \u201cremote\u201d superstar \u201cstaff members\u201d that the students will never see in their lifetime.<\/p>\n

\"\"<\/a>Sabine Benoit is\u00a0Professor of Marketing,<\/em>
\nDirector for External and Business Engagement for SBS<\/em>
\nat the University of Surrey<\/em><\/p>\n

see\u00a0her\u00a0youtube channel<\/a>\u00a0or\u00a0sabinebenoit.com<\/a>\u00a0for more insights.<\/em><\/p>\n

 <\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Linda Nasr interviewed for our series \u201cGoing abroad\u201d Sabine Benoit, who moved from\u00a0Germany to the UK. This is the second part of the interview with her. So, what is special about working in academia in the UK? I will first mention research before I elaborate on teaching. In research, the one thing that is omnipresent, […]<\/p>\n","protected":false},"author":1,"featured_media":6339,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17,10],"tags":[],"_links":{"self":[{"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/posts\/6338"}],"collection":[{"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/comments?post=6338"}],"version-history":[{"count":4,"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/posts\/6338\/revisions"}],"predecessor-version":[{"id":6551,"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/posts\/6338\/revisions\/6551"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/media\/6339"}],"wp:attachment":[{"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/media?parent=6338"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/categories?post=6338"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.servsig.org\/wordpress\/wp-json\/wp\/v2\/tags?post=6338"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}