13

I know it is fairly common to review the same paper twice. However, I want to ask whether this can be unfair to the authors.

I recently had a paper rejected in two very prestigious applied math journals. Both times I received one very good review and both times I received the same very negative review (with minor adjustments to accomodate for the improved manuscript).

The problem is that the negative review is blatantly wrong and off the mark, criticizing everything and not suggesting anything specific. It seems to be a very peculiar opinion about my paper, written by someone that can only appreciate his own views of the subject.

It seems unfair to me that one single person was able to ruin my chances in both journals, with a possibly odd opinion. When I was at the other end of this situation, I denied to review the paper for a second time, as I'd want the authors to have a fresh judgement of their work.

So, can it be that our common policy of giving multiple reviews of the same paper, especially when we don't propose any improvement to be done, is unfair to the authors as we can be the ones wrong?

And can I, as an author, do something about this situation?

Shake Baby
  • 4,072
  • 2
  • 23
  • 50
  • 5
    In some subfields, the total worldwide number of qualified reviewers could be very small. So if you have a paper of moderate quality and you submit unsuccessfully to a few top-quality journals, you might "use up" all qualified reviewers and make your paper effectively unpublishable, even though those same reviewers might agree it is perfectly suitable for a medium-quality journal. Is that good? – Nate Eldredge Jun 27 '17 at 00:46
  • 4
    Much of this question is a rant I want to advocate is not generally a good question on SE. You might want to edit it down. – StrongBad Jun 27 '17 at 02:15
  • https://academia.stackexchange.com/questions/7200/asked-again-to-review-a-paper-when-the-authors-dont-wish-to-modify-it – StrongBad Jun 27 '17 at 02:15
  • @StrongBad Yes. I'm sorry. It is kind of a rant. But I think in the end I managed to come up with a question. – Shake Baby Jun 27 '17 at 02:40
  • @NateEldredge my problem is in the other direction: making the paper unpublished without necessarily using up all qualified reviewers. – Shake Baby Jun 27 '17 at 02:42
  • I think the edited title was much better, but don't want to start an edit war. – StrongBad Jun 27 '17 at 03:19
  • 5
    One should never blame the referee: he/she is representative of the community, at least in this case by the standards of two allegedly prestigious journals. If the referee doesn't like the first sentence, change the first sentence or the entire first paragraph. If you resubmit the duplicate of rejected manuscript, you should not be surprised if the outcome is a duplicate of your original submission. –  Jun 27 '17 at 04:36
  • @ZeroTheHero nobody is perfect. The referee may be simply wrong, that's why he shouldn't review the same paper twice. – Shake Baby Jun 27 '17 at 15:01
  • @ShakeBaby As you state the issue, you are not only suggesting the referee is wrong, but also that two journals were wrong in selecting this allegedly incompetent referee, and that two editors were wrong in considering comments from this referee. Keep in mind that editors can choose to ignore reports they feel are inappropriate, yet two different editors chose to accept the comments of this referee as valid. However frustrating, and unless you can argue clearly that this second referee has a bias, it's unlikely the editors will do much because you disagree with referee comments. –  Jun 27 '17 at 15:43
  • 3
    @ZeroTheHero: "One should never blame the referee: he/she is representative of the community," That doesn't make any sense to me. It's like saying "One should never blame the president; he/she is representative of the United States." Clearly this is not followed, and it shouldn't be. First, the referee is a representative of the community, not the only representative. Second: the community is not automatically right either. There are notorious examples of academic communities discriminating against individuals or groups of researchers. – Pete L. Clark Jun 27 '17 at 16:19
  • one single person was able to ruin my chances in both journals — One single person did not ruin your chances. Referees do not accept or reject papers; they only make recommendations. The editors decided to reject your paper. If you believe that the referee's interpretation of your results is wildly off the mark, appeal to the editor. – JeffE Jun 27 '17 at 16:22
  • @JeffE I feel this is not true in my area. I often see that editors, while very prestigious in their field, deal with submissions from a wider range of expertises, and thus, will reject a paper that receives a single negative report (probably due to the volume of submissions). – Shake Baby Jun 27 '17 at 16:42
  • @ShakeBaby Even under the circumstances you describe, the editor is making the decision. (My area is not that far from yours.) – JeffE Jun 27 '17 at 16:46

3 Answers3

16

If you have concerns that the reviewer isn't being fair or just, you can always contact the editor and ask for a new reviewer. It is not entirely uncommon, I know of researchers that have a "blacklist" of people that they do not allow to review their works.

But consider that the reviewer might have a point. I know it is not your case, but I get really disappointed when I get the same paper twice and it is exactly the same text. Regardless if is the same process or not, a proper review would most likely improve the paper, so to see that not even the typos got fixed (in my case) is hard.

More to the point, how did you change your paper to make sure nobody else reading it would reach the same conclusions as this reviewer?

Fábio Dias
  • 8,732
  • 1
  • 31
  • 55
  • 4
    So why not just turn down the review instead of forcing your opinion on the authors again. If you are right, then the other reviewers should catch the issue. – StrongBad Jun 27 '17 at 02:26
  • Thanks for answering. I would definitely blacklist this guy if I knew who he was.. As for your suggestion, I haven't had any good outcome in replying to rejected outcomes. I think the editor would only see me as a complainer independently of what the reviewer has said. – Shake Baby Jun 27 '17 at 02:45
  • If the reviewer makes suggestions, I'd definitely try my best to incorporate all of them on my revised manuscript. However, if he blatantly misreads my paper and disagrees with my first sentence, I don't see how to rewrite anything. – Shake Baby Jun 27 '17 at 02:48
  • @ShakeBaby Well, maybe your first sentence is wrong... – David Richerby Jun 27 '17 at 07:44
  • 2
    I rarely see the same paper for multiple journals, but I frequently see the same paper in multiple conferences. As a reviewer I want to provide feedback when I've already read it. Sometimes I can see it's been improved, sometimes I feel they've ignored my comments, and sometimes the paper is literally submitted unmodified. When completely unmodified, they are frequently rejected without review when this is pointed out. // Reviewing is an iterative process. Authors are expected to try to address comments. And reviewers are assumed to be competent to evaluate. – Fred Douglis Jun 27 '17 at 13:16
  • @DavidRicherby what if it is not? – Shake Baby Jun 27 '17 at 14:59
  • 3
    @ShakeBaby Then maybe you wrote it in a way that's so hard to understand that an expert in your field misunderstood it. If an expert in your field cannot understand your paper, you need to consider the possibility that you didn't write it very well. – David Richerby Jun 27 '17 at 15:06
  • @DavidRicherby the point is not the first sentence, but the fact that he disagrees with the whole research. I can't fix the whole research. It's not a matter of presentation. – Shake Baby Jun 27 '17 at 15:12
  • 2
    When you say that the person disagrees with the whole research, is that because there is a well-known theoretical or methodological rift in your field? If not, then the person is probably misunderstanding. I have multiple examples of someone thinking my work is worthless until I identify the source of their misunderstanding. Then I add a paragraph or diagram and poof -- they now think it is amazing work. If I get a reviewer who totally misunderstands, I know I have to find and add the poof. – Dawn Jun 27 '17 at 15:50
  • Continued - Sometimes the paragraph takes the form of "The problem at hand is somewhat similar to the established problem of X. However, it is distinct in several ways... Because of these differences between the two problems, the approach used in problem X is not suitable here. ..." – Dawn Jun 27 '17 at 15:56
  • 1
    @StrongBad Nobody is forcing anything on anybody. The editor is asking for an opinion. – JeffE Jun 27 '17 at 16:25
  • @Dawn. I think the problem in my field is more dramatic. Simplifying, the issue at hand is that a more theoretical inclined review will only judge my paper based on my theory, disregarding all other aspects of my contribution. – Shake Baby Jun 27 '17 at 16:47
  • @Dawn Exactly what I was trying to convey: If the reviewer misunderstood, your text let that happen. Fix the text, fix the reviewer :) – Fábio Dias Jun 28 '17 at 19:56
9

So, can it be that our common policy of giving multiple reviews of the same paper, especially when we don't propose any improvement to be done, is unfair to the authors as we can be the ones wrong?

I don't think it's unfair. Moreover, I have a feeling that if it was the positive review of your paper that you got twice from the same reviewer, you wouldn't come here to complain that this was unfair. This reminds me of the joke about a professor who had a student who failed the final exam in his course and had to retake it. On the second attempt, the professor gave the exact same final exam, and the student failed it again. He then went to the professor and said "professor, I think it's unfair that you failed me a second time." Incredulous, the professor says "but I gave you the same final exam and you still failed it!", to which the student replies "exactly - I already failed that exam so it was unfair of you to give the same one; you should have given me a different exam!"

Jokes aside, what is potentially unfair is to get a negative review from someone who clearly hasn't understood your paper or even bothered to try. Even then there is an argument to be made (that has been made here by various people in connection with past questions) that that means your paper is at fault for not making it easy enough for the referee to understand your results. In any case, whether it's unfair or not, the unfairness is the same whether that lazy/ignorant/mean-spirited referee reviewed your paper once, twice, or any other number of times.

And can I, as an author, do something about this situation?

Yes, you can. You can write the editor a polite email in which you give your best attempt at explaining why the reviewer's report doesn't make sense and should be ignored. You should have done that already after the first submission, but you can equally well do it now after receiving the same negative report the second time. You can also point out to the editor that this is the second time you're getting the same referee report, but personally I don't think that's a valid argument that should sway the editor's decision. My advice to you is to focus on the substantive issue of why the report is not a useful one, and not the number of submissions it was used in.

Dan Romik
  • 189,176
  • 42
  • 427
  • 636
  • Thanks. Great answer. I was already convinced and I've written to the editor after @JeffE's comments. – Shake Baby Jun 27 '17 at 18:55
  • The joke is great (I knew it, but it is still great!) - and the response, of course. "focus on the substantive issue of why the report is not a useful one, and not the number of submissions it was used in" +1 – Captain Emacs Jun 27 '17 at 19:32
3

Yes, like almost everything in life, it's possible that it's unfair, e.g. for the reasons that you describe. Often, though, it's an advantage:

  • It puts less pressure on the system in general, since fewer reviewers are required per paper. It's much quicker to get up to speed on a revised paper if you've read a previous version before.
  • It's almost always possible to find something wrong with a paper. If it's sent to a new reviewer each time, they won't see the improvements being made and will simply identify new things that need changing. You'll get caught in an endless cycle of revisions.

And consider the alternative: if papers were sent to a different set of reviewers each time, authors would be more able to play the odds, and simply keep submitting until they happen to get a favourable set of reviewers. As the most suitable reviewers are used up, it gets sent to people with less expertise until the paper's (possible) flaws get overlooked. This is still unfair, just in a different way.

Finally, bear in mind that a reviewer cannot accept or reject a paper, that job lies with the editor. Therefore, the editor provides a check against possible rogue reviewers: if the reasons for suggesting rejection are unreasonable, the editor is at liberty to ignore them.

user2390246
  • 12,317
  • 6
  • 35
  • 56