8

Suppose there is a paper with 3 promised reviews, 2 of which have been submitted. In this scenario there will usually be no decision while the editor waits for the third review. Will the author want to see the submitted reviews before a formal decision is made?

Intuitively I'd guess "yes", especially if the decision turns out to be "revise", but I am not so sure if the decision turns out to be accept/reject. It could be jarring for the authors to see reviews with minor comments, and then receive a scathing 3rd review & a reject decision, for example.

Another possible issue is if the 3rd promised review never arrives, which from the editor's perspective will inevitably happen for some papers, but this seems less likely to aggravate the authors.

Allure
  • 127,528
  • 50
  • 325
  • 493
  • 2
    The only scenario that I could think of where this would not be beneficial (or at least not causing any negative consequences) is if the third reviewer raises issues that directly conflict with the other reviews, as this might lead to work having to be redone. – Sursula Jan 12 '23 at 08:13
  • @Snijderfrey The time would be spent better by actually looking at the manuscripts From my perspective, that would be yet another reason to have full-time journal staff. Also, I'm not sure how other editors feel about it, but I like to read reviews as they come in (especially since sometimes the review is very poor and we might need to invite more reviewers). From there, one could configure the Editorial Management System to send reviews to authors in automated fashion. – Allure Jan 12 '23 at 09:01
  • @Allure, I converted my comment into an answer, therefore deleted the comment. – Snijderfrey Jan 12 '23 at 12:50
  • Makes no difference whether authors would prefer it or not, as it isn't likely to happen. Among other reasons, it presents authors with too much opportunity to interfere in the review process. – Scott Seidman Jan 13 '23 at 21:37
  • @ScottSeidman how would the authors interfere in the review process if they are shown the currently-received reviews? – Allure Jan 14 '23 at 03:58
  • @Allure. They could try to figure out who the reviews they see came from, and pressure them to change it, they could then try to figure out who the reviewer who hasn't responded yet is (if the field is small enough), and pressure them. They could also start whining to the editor about the quality of the reviews before a decision in made, which is something editors don't need. – Scott Seidman Jan 14 '23 at 13:47

5 Answers5

11

There is little disadvantage in having information earlier rather than later, so I doubt many authors would object to getting some reviews earlier (though I don't think editors usually give this option). Certainly some academics might want to hold off starting any revisions until they have all the reviews, just in case they end up changing something based on incomplete information and making their paper worse! (E.g., in the case where two referees ask for inconsistent things.) I don't think it matters too much if the last review is a rejection, since you will usually want to incorporate the suggested revisions anyway before resubmitting elsewhere.

In regard to the possibility of a jarring rejection in the presence of other positive reviews, I think most experienced academics are aware that referee reports are pot-luck and they are prepared for wild inconsistency in opinions on their paper. Most academics who have submitted a lot of papers have probably encountered the situation you describe, where one referee loves your paper and another rejects it entirely (sometimes the latter is just a scope decision even if the quality of the paper is good, so that is also reasonably common). As to the possibility that the final review never arrives, I guess eventually you have to prod the editor for a decision.

Ben
  • 68,453
  • 9
  • 142
  • 263
  • 1
    also, if the third never hands in their report, the editor will either decide that 2 reports are sufficient, or find a replacement reviewer, none of which really changes the situation. – Sursula Jan 12 '23 at 07:20
  • 4
    I don't think editors usually give this option That's actually kind of why I asked the question - maybe editors should start doing it? For third review never arriving, I was thinking of e.g. if the two reviews received are negative, the editor sends them and informs the authors that there is a third review due in X days, and then the third review never arrives. Maybe the authors will say there is a third [positive] review that is being withheld from them, that the journal isn't being transparent, etc. – Allure Jan 12 '23 at 07:32
  • Yes, I think it could be reasonable to provide staggered reviews (especially in cases where one review is unreasonably delayed), so I'm not averse to editors changing their process to do this. I imagine that many editors/authors might decide it's too much hassle, but personally I wouldn't mind. – Ben Jan 12 '23 at 07:38
6

The way you ask the question, the answer inevitably is: Some authors will prefer early sharing of referee reports, some won't. I personally would not care too much about it. However, I am not sure if early sharing would improve the overall process, and therefore would indeed be in the authors' interest.

Generally, it is in the authors' interest to have a smoothly working publication process. That includes a transparent review process. You may argue that by early sharing you would increase transparency, and would enable the authors to start revising early. In my view, this would be a marginal improvement at best because also currently authors eventually see the referee reports.

However, of similar importance in the publication process is a responsive editorial office. Every message you send to the authors (automated or not) has a certain probability to generate a response, and I predict that you would end up with a lot of additional (unnecessary?) communication between editor and authors. Such responses may be, among others:

  • Asking when you will come to a decision, given that one referee report is already received.
  • Asking to use the two received referee reports to come to a decision due to some external pressure to publish rapidly (thesis defense, project reports etc.)
  • Already submitting a revised version taking into account what the first referees said.

Therefore, early sharing would have to go hand in hand with a strategy how to deal with such responses (I don't mean chat bots...) without imposing too much extra workload on the editors (who may even work on a voluntary basis). This is to ensure that the responsiveness of the editorial office for really relevant issues is as good as it should be. Editors also should be able to spend some time to actually look at the manuscripts.

Snijderfrey
  • 8,961
  • 2
  • 29
  • 48
6

Getting one review per week instead of all 3 after 3 week wait (yeah, we can hope), would only make me, as an author, start fretting about response to the reviewer, wondering whether the paper is getting accepted or not, or starting to fix the paper according to wishes and whims of the first reviewer.

Assuming I did start acting on the first review, then, another week fixing stuff for the second reviewer, then yet another week to try accommodating the third reviewer. In the final week I also have to make sure I didn't now overwrite the first reviewer's opinions (it would be mightily unfair to have his opinion count the least because he submitted it first), but also that I didn't slight the third one which only got a week of my attention instead of 3 the first reviewer did.

And, even if I didn't bother fixing anything, I also don't gain much by getting those 2 reviews sooner. In the best case I would probably only think more about the just submitted paper, instead of forgetting it for a while and starting to work on the fresh new thing - so, in effect I would waste 3-4 weeks on review instead of 1-2, no matter how I approach this.

But, IF I am given the option to read the reviews as they come, would I say thanks but no? Hell no. Even knowing it will be quite possibly counter-productive waste of time, I would still love to read those reviews as soon as possible. Because, on the bright side, they might even help shape future research if they found some flaw in the current paper.

Zizy Archer
  • 513
  • 2
  • 7
1

As an author, I would appreciate this. One reason is simply that it would give me some sort of progress report. It's usual for me to have several papers under consideration by various journals and to have no idea how any of them are progressing until I get a decision. It would be nice to know "we have received two of three promised reports, based on which we want to wait for the third", even without getting to see the first two. (As well as simply "nice" it may be useful, if I can see early on that a particular journal is behind schedule and should be gently reminded.)

The additional advantage of actually seeing the first two would typically come if major revisions are required, particularly if a reviewer has identified some gap that needs fixing. It may be that it will be more convenient to think about how to solve the issue now, compared to after the third report comes in, either because one of the authors has more time available or because it is possible for authors to meet in person.

Especially Lime
  • 9,290
  • 1
  • 22
  • 38
1

I’m sure some authors will want to but most editors will not. The reason is simple enough: the editor can choose to ignore some of the reviews.

Editors don’t necessarily keep track in real time of the referee reports and any report transmitted to the author should be scanned so it does not contain objectionable comments that would jeopardize the refereeing process. An obvious example would be a report that contains insults but some reports might need to be invalidated for a number of reasons: they could be unclear, reveal information about the referee, be clearly biased (positively or negatively), *etc.

Ultimately, the referee reports are only recommendation given to the editor by (presumed) experts in the field, so the editor is well within their right to edit or not to transmit reports that they judge inappropriate.

ZeroTheHero
  • 27,311
  • 2
  • 48
  • 111
  • Presumably, the editors would need to vet the reports & remove egregiously bad comments before showing them to the authors. I think it's common editors already do this, come final decision time. – Allure Jan 14 '23 at 03:59
  • @Allure precisely! But this is rarely done in real time, i.e. I don’t know of an editor who reads referee reports one at the time, as they come in. Never mind the frequent interruption this would be, it would also generate redundant tasks since most editors read all reports at the same time before taking a decision. Moreover, what if the editor decides to ask another opinion because the current reports are split in their recommendation? – ZeroTheHero Jan 14 '23 at 05:42