Stages in the Initial Review of a Future Masterpiece
Guest article by Mahesh Subramony, best reviewer 2017 of JSR.
It is “that letter” – you know the one I am talking about . . . the one that fills you with excitement, then dread, followed by disappointment (or mild ecstasy), and then frustration (or resolve). The first decision from the Action Editor could be a rejection or a revise and resubmit, but either way, authors are typically astounded by the inability of anonymous reviewers to truly recognize the lucidity of the theoretical arguments or rigor of the methodology.
What is reviewer 2 really thinking? What gives reviewer 2 the competence to say what he/she just said? Who is reviewer 2 anyway? For the purpose of this article, I am reviewer 2, and let me reveal to you what goes on in my mind during various stages of the review process.
Stage 1. Agree to review the submission. My first reaction to an invitation to review is curiosity. After briefly scanning the title, I decide whether this submission is within my area of competence, and then scroll down to the abstract to confirm my intuition of fit/misfit. Abstracts that clearly convey the objective, methodology, and results of the study, without *any* grammatical errors are ready for my review if and only if, they also have something new to say about a particular topic. Submissions that simply state the obvious (e.g., when customers are happy, they are also loyal; employees who are busy have less time to spend on customers; bullied employees are unhappy employees) and test these propositions using cross-sectional survey data, don’t make my cut. I am not saying that these are not worthwhile topics, just they are not of enough interest to the reviewer who will invest a significant part of his/her day reading and commenting on the manuscript.
Stage 2. Reading the manuscript. The first three pages of the manuscript need to fully state the case for the study. Upon reading these pages, reviewer 2 should be able to answer the following questions: What is this study about? What theoretical questions does this study purport to answer? Do the authors have mastery over the literature and highlight the main gaps/opportunities? Is the methodology rigorous (i.e., are the sample size and characteristics appropriate to investigate the problem?)? I am fairly confident that most reviewers can accurately predict whether the submission stands a chance of a revise and resubmit (R/R) after a thorough reading of the first three pages. Is that fair? Is it not possible that all the gems are hidden in the middle of the manuscript? I would urge you to think of the most recent fiction or non-fiction book that you returned to the library unread. What led you to borrow it in the first place, what made you turn the first few pages and then decide to not read it anymore? Could it be that it was not interesting or relevant, or that it was not well-written, or inappropriately biased in favor of or against a certain viewpoint? Reviewers are readers too.
Stage 3. Looking for rigor and consistency. If the manuscript explores an important topic using appropriate methods, I start looking for an internally consistent story that is built upon the foundations of rigorous theory and research. I like hypotheses that emerge from theory, not ones that seem to have been force-fitted into the manuscript. Research is an uncertain enterprise and a lot of times our data does not yield to the theorized relationships (e.g., employee customer-orientation does not predict customer loyalty). It is absolutely fine to find the absence of a hypothesized relationship. If the authors have conducted the study with rigor, typically with data from multiple sources, with time lags, included adequate controls, and carefully considered the context, an absence of a relationship can mean several things including the presence of suppressors or moderators and the limitations of the theory itself. Once again, that is perfectly cool. What is *not cool* is when the hypotheses are twisted post-facto to suggest a counter-intuitive finding, or when variables are dropped from the study to allow the emergence of significant results. Really? Hermione was in love with Draco all along?
Stage 4. Writing the review. I have faith in the transformative power of developmental reviews. A review that is straightforward (not irritable or condescending) in its tone, rich in content (with relevant references and leads for the author), and open to challenge, can turn adequate submissions into impactful articles. I attempt to write reviews that assure authors who are asking interesting questions that their work needs a certain limited number (usually 4-5) changes to contribute to the body of literature; and try to guide others toward more fruitful explorations. Maybe manuscript X requires an additional study or a different methodology, perhaps the research question needs to be recrafted, the literature needs to be comprehensively reviewed . . . I like to write such reviews because, as a researcher, I have always benefited from reading ones that embody these characteristics. Writing can be often be a lonely process and the blind review process should produce clear and objective external-feedback from friendly critics with tough standards.
I hope to write about the revise and resubmit process, and the action-editors’ perspective on manuscripts in a future article. Until then, I wish us all an engaging and developmental review process.
Mahesh Subramony is Associate Professor of Management at Northern Illinois University and Director at the Centerof Human Capital Management.
