Guest article by Jonas Holmqvist, winner of the best Reviewer Award 2016 of the Journal of Service Theory and Practice (JSTP)
Like football referees, reviewers are often blamed when things go badly, seldom thanked when things go well, but absolutely crucial for the outcome. In this short piece, I first talk briefly about my own views on how a reviewer should act, and then provide some insights on the review process that I hope will help authors, especially young researchers.
Before starting, a word on terminology: in this paper, I’ll sometimes talk about a good review or a bad review. When I started submitting, my friends and I equated “good” with “positive”. That is a rookie mistake; a good review provides helpful insights, and there is little or no correlation between good and positive reviews. I’ve had great, outstanding reviews that recommend reject and I believe some of my own rejects as reviewer have also been my most helpful reviews. With this in mind, I move on to some thoughts about how to be a good reviewer.
Don’t make it “your” paper!
The biggest trap for a reviewer is to start steering the paper towards how we would have done it ourselves. This is a mistake, as our task is to evaluate the paper, not make it ours. I know researchers who are passionate about either qualitative or quantitative research, and that’s fine – as long as it’s in our own papers. As reviewers, we must never hold it against authors that they approach a topic from another direction, as long as they do it well. This is not limited to methods; some researchers prefer service-dominant logic, others service logic; some focus on customers, others on service employees, etc. So you may prefer qualitative service-dominant logic research from the customer perspective and receive a paper on quantitative service logic research on service employees, or vice versa. As reviewers, we should evaluate the papers for what they are, not for what we would have wished them to be.
Better no review that a bad review
Writing a review is accepting a responsibility, accepting to spend time with the manuscript to provide detailed feedback to help the authors. As author, it has happened a few times (very few times), that I’ve received reviews that were just a few sentences. In one extreme case, my co-authors and I received a review recommending “major revisions” – and the reviewer had not written even one word! Such cases are rare, but it is not entirely uncommon to receive a review consisting of a paragraph or two. Unless it’s for a (conditional) accept, no review of just a few lines is a good review. No matter if we like the paper or if we reject it, we should always explain our reasoning to the authors. This is especially true for ‘major revisions’, where the authors need our guidance. However, even if we find a paper truly lacking and an obvious reject, I believe we should still be able to write at least a page to explain why the paper is not acceptable and what the authors need to improve.
“How can I get my paper accepted?”
Moving on to my suggestions for authors, the main question young researchers ask is how to make sure the paper is accepted. If I had the answer to that question, I would never be rejected and unfortunately that’s not the case. However, there are some important aspects to keep in mind
The reviewer is not always right – but always a judge on your paper!
As author, you may think the reviewer is wrong. This happens, reviewers can be wrong! I remember a reviewer writing that Finnish and Swedish are so similar that being served in one or the other would not matter. In reality, Finnish and Swedish are less similar than English and Russian; we politely explained this to the reviewer, who had no problem accepting our argument. I also remember doing a review for a major marketing journal, and one of my fellow reviewers was insulted in three different replies by the author(s) who felt my fellow reviewer was wrong. Not only did that reviewer react badly, both the third reviewer and I also reacted negatively to the author(s) using insults instead of factual arguments. This is the key: even if you think the reviewer is wrong, you still need to explain it factually and politely.
Minor revisions are not an accept
The major revisions have been excellent, the paper has gone on to minor revisions – and suddenly the authors ignore half of what the reviewers said and expect an accept to follow. This is a mistake; minor revisions do not mean that the suggestions are less important. I have never seen a paper go from minor revisions to reject, but it is not unheard of for a paper to go from minor back to major. As author, my most extensive reply ever was for a paper that already was conditionally accepted – just one detailed remained, but we really needed to get that one detail right and it took a lot of effort. Of course “minor revisions” are better news than “major”, but never think it’s a done deal.
Revision notes really really matter!
I cannot emphasize this point enough. The first time I had the pleasure of being guest editor, my main surprise was the enormous difference in the revision notes. Some authors made excellent, detailed notes to each comment while others barely bothered and just wrote a few lines about what they had done in general. Needless to say, the first group fared much better with the reviewers in the next round. If you’ve received revisions, either major or minor, do remember that you need to be able to address every comment the reviewers made; it’s not just about addressing it in the paper, you also need to explain it in the revision notes. This is particularly true if you disagree with a comment, in which case you need to provide a detailed and strong argument.
Why are papers rejected?
If I look back at the reviews I’ve done, it’s easy to see that most rejects come down to a handful very common errors. The reason these errors lead to a reject is that they are almost impossible to fix without writing a completely new paper.
No real contribution
This is by far the most common reason for a reject. I’ve lost count of how many times I’ve read impressive studies, even multiple studies in the same paper, just to find something completely trivial. A weak study leading to a strong contribution is unlikely to go far, but could conceivably lead to major revisions with new data collection. A paper no contribution will not get even major no matter how good the study.
Obvious hypotheses
Connected to the lack of contribution, weak hypotheses are a major problem. Several times a year, I read hypotheses suggesting that good service quality leads to higher satisfaction or something else that seem both obvious and long since established. Point a. and b. combined almost always lead to a reject. While many errors can be fixed in a revision, a missing contribution will almost always lead to a reject.
Poor methodology
This is the second most common reason for a reject. I see, on a regular basis, how researchers pick a methodology that makes it impossible to use the data. I remember a very interesting manuscript that used Hofstede’s cultural dimensions and had me hooked already in the introduction, and only grew stronger in the literature review. In the methodology I discovered that the author(s), rather than collecting data in two different cultures, had decided to ask half the respondents to answer in their native language and the other half in English, and then used these respondents answering in English as their British(!) sample. While it is true that the language we use will influence us, a Chinese, Korean, Brazilian or Russian respondent does not become “British” just by answering a questionnaire in English. This example stands out as a particularly unfortunate error, but even small errors in the methodology can be crucial as they undermine the reliability of the data. I really cannot emphasize enough the need to carefully think through the methodology.
Not knowing the field
Serious omissions in the literature are problematic but fortunately a poor literature review is something that a good revision can fix if the rest of the paper is good; personally, I’ve never rejected a paper just because the literature review had omissions. However, not knowing the literature can also have more serious consequences if it turns out that previous research the authors did not know already did what the paper intended to do. This is not exceedingly common, but it has happened on some occasions that I’ve had to reject as I found that the contribution the authors thought they made had in fact been completely covered in a few previous articles the authors did not know.
“Shotgun submissions”
Every editor and reviewer know them – manuscripts obviously written for another journal, rejected and now submitted to our journal without any real changes (hence “shotgun”, lack of precision). If I review for a service journal and receive a manuscript that does not mention services or has forced the word service into the manuscript in several places but without contributing to the service literature, then a reject is likely. Do remember that there is nothing wrong with submitting to a new journal if you were rejected – but make sure you have adapted the manuscript for the new journal. This is not about just making superficial changes; it should be a new manuscript that takes into account the positioning of the journal.
Rejected – what now?
After having worked for months on a manuscript, it is a tremendous disappointment to receive that exciting mail from the editor just to discover that your manuscript is rejected. Do remember that this happens to us all! Being rejected is something to which every researcher, no matter how successful, has had to get used. What really matters is what you do next. Embarrassing as it is to admit, it took me some time to learn this. During my early years of research, I could think that the reviewers were wrong, and didn’t understand my manuscript. This, I can assure you, was not the case; going back to those early manuscripts now, I can easily see that – apart from one or two exceptions – the reviewers were right and made very valid points, and rejecting those manuscripts I sent was the right decision.
If we refuse to acknowledge that the reviewers’ reject may be right, we are much more likely to just make a few small changes before resubmitting in the hope of getting “good” reviewers. This way of thinking is a serious trap, and will likely lead to successive rejects. No matter what the reviewers say, they probably have a reason for saying it! If we feel that the reviewers missed our point, well, we may be right – but then we should ask ourselves why they missed it? Perhaps it was clear in our head when we wrote but we failed to communicate it clearly enough? So while many authors do spend time on revising after receiving “major revisions”, good authors also spend time revising after rejects, making sure they correct the shortcomings that led to the reject before resubmitting.
Jonas Holmqvist is a Assistant Professor at Kedge Business School and the winner of Best Reviewer Award for the Journal of Service Theory & Practice 2016
