45

I recently received a referee report on a math paper in a mid-tier journal.

In this paper I proved (to give an example avoiding technicalities) that all dogs are cute. But the referee claimed in his report that I claimed that all animals are cute (that is, he claimed that I proved something much stronger than what I really proved).

There is no way to understand the more general statement from my paper, and my guess is that the referee simply did not read the paper carefuly.

Now, the referee recommended acceptance, and the editor accepted his recommendation.

Should I do something about it? should I explain this to the editor?

Gabriel
  • 2,676
  • 2
  • 20
  • 30
Confused
  • 431
  • 4
  • 4
  • 14
    I would not do something about it, but just leave it. Congratulations on the acceptance. – gerrit Jan 11 '17 at 14:26
  • How many referees commented on your paper in total? If it was only one, the issue would be greater than with two or three. – lighthouse keeper Jan 11 '17 at 16:06
  • 2
    @lighthouse: In mathematics, it is quite common to have only one referee. – Pete L. Clark Jan 11 '17 at 16:12
  • 9
    Science is not about getting published no matter what. You should most definitely clarify the misunderstanding of the referee. – Gabriel Jan 11 '17 at 17:23
  • 21
    That's what you get for reading the comments on an acceptance. Don't do that. – mikeazo Jan 11 '17 at 19:50
  • 16
    It is entirely possible that the editor understands the distinction between "dogs" and "animals", noticed and understood the referee's error, and then accepted the paper anyway based on their own expert judgement. (Of course, the only way to know whether this happened is to ask the editor.) – JeffE Jan 11 '17 at 20:54
  • 1
    Is the title of your paper something like "All dogs are cute: A proof"? If so, I would assume that the editor knows what your paper is about, and read right past the reviewer's error. – 1006a Jan 11 '17 at 22:01
  • 2
    After mulling this over for some hours today, I really think that more context would be helpful. If there's just one wrong word "animal" instead of "dog," and of course if the paper itself is clear and correct about the distinction, I think it's really likely that this was just a misspeaking on the referee's part. Depending upon the situation, I might choose to assume the best and let it drop without much worry. But if the report calls attention to the stronger result, and especially if there is any evidence that the weaker result would yield a different outcome: that's something else. – Pete L. Clark Jan 11 '17 at 22:05
  • 1
    @Pete: in particular, the "something else" is similar to "suppose you submitted a paper and were offered the choice whether it's reviewed or just published in the same venue without peer review at all". Because a review that's gravely misunderstood the paper is in a way worse than no review: (a) you don't have the benefits of review; (b) you know for a fact that your paper is capable of being completely misunderstood. Scary stuff ;-) It's not even "just" an ethical matter: it's a real risk your paper is bad. – Steve Jessop Jan 12 '17 at 17:35
  • 1
    Adding to @JeffE, from the information we have it could also be that the referee thought "some animals" and the fingers typed "all animals" out of a momentary distraction, although the referee actually did understand the paper. I wouldn't jump to conclusions without talking with said referee (which might be impossible). – Nemo Jan 12 '17 at 18:12
  • 10
    @Steve: I'm not sure how much experience you have with refereeing of mathematics papers. I have a lot at this point, and I'm sorry to tell you that the quality control is just not there: while some referee reports are heroically terrific and more than half are fine, a sizable chunk are either manifestly bad or might be the result of a referee who put essentially no time or effort in and there is no way of knowing. Upshot: if you're worried that your math paper is bad, you'd better do something about it yourself rather than relying on the referee. – Pete L. Clark Jan 12 '17 at 18:32
  • Actually, @Pete makes a good point. My experience with writing proofs (of unimportant things) is that no one reads them. This includes the coauthors, supervisor/boss, and the reviewers. I guess, this may (or may not, as OP's case suggests) be different when a proof is the whole point of a paper. – Alexey B. Jan 13 '17 at 00:36
  • 2
    @PeteL.Clark: out of interest, then, is there much momentum in the mathematical community to formally do away with peer review entirely? Or, if it amounts to the same thing, to formally stop reading journals and rely solely on arXiv, each others' web pages, and recommendations what to read from people who currently are journal editors but perhaps more productively could just curate recommended reading lists? – Steve Jessop Jan 14 '17 at 19:02
  • 3
    @Steve: I don't think there is much interest in doing away with the journal publication process. It still plays the role of determining how much value the community places on each particular work (quite imperfectly, of course, but a lot better than nothing). It doesn't play much of a role in certifying correctness, and so far as I know most mathematicians are willing to rely on results from arxiv preprints as published papers. – Pete L. Clark Jan 15 '17 at 00:45
  • 3
    This is not so great, because the journal refereeing process definitely plays some role in vetting papers for accuracy and improving them. As a referee, I have pointed out small mistakes in papers many times, and major mistakes a few times. As an author I've often received referee feedback improving the paper. I think it comes down to this: gross errors are easy to catch, and often the experts are not fooled for a second. Subtle errors are caught only by those who put time and effort into finding them. The referee may not do so, and someone who is not the referee may do so. – Pete L. Clark Jan 15 '17 at 00:56

7 Answers7

36

I would probably mention it, but not make a big deal out of it. The referee here was lazy (by not reading carefully), the editor was lazy (by not looking at the main result of the paper), so my plan would be to be lazy too. I'd write an email back with the final revisions and mention casually in the text of the email "just to correct the record the referee wrote animals, but they must have meant dogs which is what's actually in the paper." Odds are the editor won't read your email carefully and won't even notice your comment, but you'll feel better.

Peer review is such a random crapshoot anyway, and you obviously thought this journal was appropriate, so there's very little actual harm here. You can be sure that at some point the same thing will happen in the opposite direction and you'll have a paper unfairly rejected by a referee who didn't bother to read the main result and the editor almost certainly won't do anything to correct the error.

Noah Snyder
  • 32,772
  • 7
  • 71
  • 128
  • 4
    This answer is the one I agree with most. Mention it, but don't yourself delay the publication process or indicate that something needs to be done to delay the process. – Pete L. Clark Jan 11 '17 at 22:00
  • 1
    Mention to whom? If two persons nearby you fail to see or pretend not to see someone is in trouble, perhaps because that would imply them to do something about it, would you also pretend not to have seen it? What if, in the end, the author is the person in trouble? – CPHPython Jan 12 '17 at 11:16
  • 1
    I'd mention it to the editor in my reply to the editor. As I explained in the second paragraph, there's very little harm here, and nothing analogous to an injured person. – Noah Snyder Jan 12 '17 at 13:03
  • 1
    @NoahSnyder even if the editor does not read the email carefully but they realize the mistake, won't it be likely that the referee will end up knowing it as well? Since it was his fault in the first place, won't they try to push that responsibility? I made an analogy with physical injury, but in this case someone's reputation may be injured by a carefree (in)action. I understand that there are unfair situations in the review process, but is that an excuse to ignore responsibility and twist this situation in someone's favor? – CPHPython Jan 12 '17 at 16:20
24

If you care about being a person of integrity (not everybody does), then yes, you should explain this to the editor. Your paper was accepted based on a misunderstanding of its contribution; do you really want to have this "achievement" on your record, and forever have to feel a pang of guilt every time you discuss the paper with someone and they ask you where it was published and then nod with approval when you tell them the name of the journal?

The ethical thing to do is to clear up the misunderstanding. Hopefully the result that all dogs are cute (far from an obvious fact if you ask my opinion ;-)) is impressive enough that your paper will still be accepted, and then you can have the satisfaction of knowing that this happened for the paper's actual merits. And you will score major points with the editor and referee for your honesty, which may offer its own practical benefits someday.

Dan Romik
  • 189,176
  • 42
  • 427
  • 636
  • 20
    I actually think, unfortunately, that this advice may be a smidge overly-ethical. I don't have time to talk about it in detail now, but the gist of it is: the whole system is not under the submitter's control or even transparent to the submitter, but it often happens that a faulty negative opinion leads to a rejection, and then most of the time the editor is not very interested in revisiting the matter. I think it is too much to ask for those in positions of lesser authority to be more ethical than those in positions of greater authority. – Pete L. Clark Jan 11 '17 at 15:48
  • 20
    ...Also: "forever have to feel a pang of guilt every time you discuss the paper with someone..." I think this takes too seriously the meaning of any one published paper in any one journal. We all know that papers get published or not published for many contingent reasons. Just because the referee (apparently) didn't understand the paper doesn't mean that it's not as good as other papers published in the journal, and I'm not convinced it increases the likelihood of that. Ultimately an author has a feeling about whether her work is good or not; that's what counts. – Pete L. Clark Jan 11 '17 at 15:53
  • 8
    @Pete whether it is a smidge over-ethical, under-ethical, or just-right ethical, is really a personal matter of conscience (but I am glad we agree up to a smidge). You do make a good point, and I think for precisely the reason you pointed out -- the asymmetry between faulty negative opinions and faulty positive opinions from referees -- the editor is unlikely to wish to revisit the matter, as a way of correcting the asymmetry. But you are suggesting for OP to take it into their own hands to correct the asymmetry through somewhat dishonest (in)action. I still say let the editor handle it. – Dan Romik Jan 11 '17 at 15:58
  • 6
    @Pete as for your second comment, the point is OP knows that the referee misunderstood the strength of their result. Of course there were many contingent reasons, but if you are aware that one of them was a pure accident - say, the editor clicking "accept" instead of "reject" in an online publication system - it is unethical to carry on pretending that never happened; maybe only a "smidge" unethical, but still unethical. And I'm not taking anything too seriously - obviously OP can decide however he/she wishes and the world will keep turning either way. Just offering my opinion FWIW. – Dan Romik Jan 11 '17 at 16:03
  • @DanRomik Your answer says that an author in this situation boldface-should talk to the editor; your comment says that it's just a matter of personal conscience. Which is it? – David Richerby Jan 11 '17 at 17:56
  • 2
    Again, sorry to only have the time to quickly comment, but: based on the OP's description it is not completely clear (to me) how serious was the refereee's misunderstanding. "Animal" might have been a typo for "dog." In my experience exactly how much the referee understands is one of the things that is not always made sufficiently clear. If it is clear that both the referee and the editor thought it was "animal" and also that it being "animal" and not "dog" played a role in the decision, then for me the ethical issue becomes more serious. – Pete L. Clark Jan 11 '17 at 18:25
  • 2
    @DavidRicherby Which is it? Both. OP should talk to the editor if he cares about being a person of integrity (as I wrote in the answer) and if he agrees with my analysis according to which being a person of integrity requires clearing up the misunderstanding. Ergo, this is a matter of personal conscience as I said in the comment (for example, Pete Clark apparently disagrees with me somewhat, and he is certainly a reasonable person). I'm not claiming that my answer is some sort of universal truth. – Dan Romik Jan 11 '17 at 18:39
  • 1
    @PeteL.Clark OP wrote: "he claimed that I proved something much stronger than what I really proved". Seems pretty clear cut to me that the strength of the result would play a role in the decision to accept the paper. Would you consider revising your verdict based on this? – Dan Romik Jan 11 '17 at 18:41
  • 1
    @PeteL.Clark it occurred to me that your "many contingent reasons" argument could be used to justify all kinds of unethical behaviors. For example, would it be acceptable for someone to say "Stock investments yield a good or poor return based on many contingent reasons. I think you are taking too seriously the meaning of any one investment." in the context of an accusation that they engaged in insider trading of stocks? – Dan Romik Jan 11 '17 at 18:43
  • @Dan: "he claimed that I proved something much stronger than what I really proved" Right, I find that a bit ambiguous. Also, I don't find your stock analogy a very good one. Still without time to fully respond but: if one only thinks ethically it is possible to get carried away. I agree that it is a matter of personal conscience, but I don't think it is only a matter of personal conscience. For instance, what if someone misses out on an academic career because of the delay in re-refereeing the paper to come to the same conclusion? – Pete L. Clark Jan 11 '17 at 20:12
  • 2
    I guess I would say the following is the principle at work here: if you participate in a lengthy, multi-step process that is opaque to you, but you have some reason to believe that a mistake was made at some stage, do you have an ethical obligation to start it over no matter what, or only when you think the wrong outcome was reached? If you apply for a visa and you think they thought you lived in place X rather than place Y at a certain time, do you have to ask them about it even if you think you deserve a visa? I don't think one has unlimited obligation to correct other's mistakes. – Pete L. Clark Jan 11 '17 at 20:16
  • @PeteL.Clark That argument reads as "the end justify the means", and could potentially be used to justify virtually any academic misconduct. "I don't think one has unlimited obligation to correct other's mistakes". That's a misleading exaggeration of the situation. – Gabriel Jan 11 '17 at 20:20
  • 6
    @Gabriel: The proposed action of letting it go is not illegal (unlike insider trading) and it is at the least not clear that it is academic misconduct. But yes, in real life one does sometimes not take the maximally ethical or academically proper action because of the possible consequences. If you've spent X hours doing a literature search, it would be more academically proper to spend X+1 hours: you can't be sure you won't find something else. One does have to take practical considerations into account at some point. – Pete L. Clark Jan 11 '17 at 20:32
  • Except that by spending X hours reading instead of X+1 I'm not exploiting any weakness in the scientific process of research+publish. You most definitely are if you agree to be published knowing the referee accepted your manuscript under a wrong impression. – Gabriel Jan 11 '17 at 20:41
  • @PeteL.Clark you make some good and valid points. In particular your principle that one is not obligated to always take the maximally ethical action regardless of practical considerations is very reasonable and I've applied it myself in other answers. Anyway, to the extent that we disagree here, it's a very minor disagreement. Thank you for a fun debate. :-) – Dan Romik Jan 11 '17 at 22:29
23

No, you don't have to do anything.

However, if you have the chance to do a minor revision, try to see where the misunderstanding of the referee came from and rewrite these parts so that it is clearer, that you are really only proving things about dogs and not all animals. In addition, I would not suggest to put such a clarification in the title or abstract, but only in the introduction (or wherever in the text this is appropriate) - I think an abstract that stresses that you do not treat a case and that you never intended to, may read strange in the end.

The case would be different, if there is an actual serious error in your paper that the referee did not catch. If that would be the case, e.g. if the proof of your main result has a gap, or the result does actually not hold, you should stop the publication of the paper by withdrawing the paper.

Dirk
  • 38,078
  • 9
  • 96
  • 154
  • 4
    I'm sorry but this is just some terrible advice. They know the referee made a mistake when reviewing their paper. Saying nothing is highly unethical. – Gabriel Jan 11 '17 at 17:26
  • 15
    @Gabriel I don't find it obviously unethical, and clearly the OP and this poster didn't either. Maybe make an answer explaining why you think so, rather than a brief comment. – Jeff Jan 11 '17 at 18:05
  • 2
    @Gabriel: When the mistake is in the other direction, we're the ones who gets screwed over, and editors ignore the matter. Somehow that's OK, yet it's unethical when submitters do it? – user541686 Jan 12 '17 at 07:50
  • 3
    @Mehrdad: carelessly giving a wrong review is incompetent. Perhaps to the point of unethical negligence, but perhaps only incompetent. Unethical is the name for doing something bad when you know it's bad, so the ethical issue for the questioner is not, "are other people sometimes frustratingly incompetent to my detriment", it is "would it be bad to allow a paper to be published without competent peer review, and am I knowingly making that bad thing happen for my benefit?" – Steve Jessop Jan 12 '17 at 17:41
  • 3
    ... or in other words, it is unethical to just decide to keep a mobile phone that you found on the table in a bar, even if your phone was nicked yesterday and the thief wasn't prosecuted. Of course, one might think it's good, not bad, to publish a good paper that wasn't properly reviewed. But the fact of papers being wrongly rejected isn't what makes it good. – Steve Jessop Jan 12 '17 at 17:57
  • 1
    @SteveJessop: I wasn't talking about the reviewer being unethical, I was talking about the editor to whom you subsequently complain and who ends up completely ignoring you about 99.9% of the time being unethical. They don't even care enough to reply, let alone address your complaint. So if that's OK, then I don't see why submitters should go out of their way in the reverse direction. – user541686 Jan 12 '17 at 20:01
  • "I don't see why submitters should go out of their way in the reverse direction", because it's unethical not to do so. – Gabriel Jan 12 '17 at 21:06
  • @Mehrdad: so you see no point in taking the ethical path here, OK. But that doesn't make the thing you're resigned to lower yourself to, in order to be successful, suddenly not unethical after all. That said, it might not be very unethical: opinion is quite divided as to whether it's the author's responsibility to seek peer review at all. I don't think it's totally unreasonable to address it as an adversarial system in which you have no responsibility other than to advocate for your paper without actively lying, but a lot of people don't think publishing should be that way. – Steve Jessop Jan 14 '17 at 19:18
  • @SteveJessop: Well, I don't think it should be that way either. In my ideal world I would definitely bring this up to the editor's attention, and the editor would also definitely not ignore me when I was bringing the opposite to their attention. In my ideal world a lot of things would work out nicely and cards wouldn't be heavily stacked against one side. I just wish I lived in that world. – user541686 Jan 14 '17 at 22:46
20

Yes, you should explain this to the editor.

Science publishing should not rely upon taking advantage of a referee making a mistake. Your ultimate goal should be the advancement of science, not how many papers you can push through the refereeing process at any cost.

If the referee approved your article based on an improper understanding of it, it invalidates the entire purpose of refereeing in the first place.

A referee is not an obstacle to conquer at any cost. It's an allied, a colleague, pursuing the same goal you are (or at least should be) pursuing.

Gabriel
  • 2,676
  • 2
  • 20
  • 30
  • 5
    Yes, this, very much. The referee understood that the paper was proving a much bigger result than actually proved, and still failed to spot the fact that the proof does not support the stronger statement. It follows that the referee would be unable to detect any actual errors in the weaker statement, should they be present. The referee is failing the author at the one thing they're meant to be doing. – E.P. Jan 11 '17 at 19:32
  • 1
    I agree with the principle that referees should be your allies, but I find it very common that a paper will be accepted based on reviews which are clearly not sufficiently deep to ensure that the reviewers would have spotted any serious problems if there had been some. In such cases, by your logic, should one refuse to publish and ask the editor for better reviews? I think reviewer feedback should be taken seriously, but I don't think it is the authors' responsibility to obtain deeper reviews if their paper was accepted based on superficial or even erroneous reviews. – a3nm Jan 15 '17 at 21:43
1

Are you sure that the reviewer made a mistake? As a reviewer I usually say something like "The author proved that all animals are cute, which is interesting." or "The author proved that all dogs are cute, which is boring because cats are much more interesting." The editors cannot be expected to understand the difference, in particular if you submit to a general journal, so in the first case it is not necessary to give too much detail. The author on the other hand wants to know why his article got rejected, so in the second case I have to be much more precise.

So if the short description is followed by a list of small remarks, then it might be that the reviewer was deliberately vague. If there are no detailed remarks, the reviewer either didn't read the article or your style of writing is outstanding.

Jan-Christoph Schlage-Puchta
  • 3,819
  • 13
  • 19
0

Before you decide, get a second opinion or two.

Select one or two people knowledgeable in the area, and ask them to evaluate your paper. If one of them comes up with the same inflated wrong view of your claim and proof, then you have a problem and need to think about bringing it up with the editor, rewriting, etc.

If the second opinion is that the paper proves what you think it does, and that the paper is worth publishing in the target journal, then you can rest easy and look forward to seeing the project in print soon.

aparente001
  • 38,999
  • 8
  • 65
  • 153
0

Would you have explained to the editor when the referee made a mistake that would lead them to reject the paper? If yes, then I think you should do so here as well. Then it's up to the editor to leave this be if he/she would have done the same in case of a reject.

Vincent
  • 1,789
  • 1
  • 10
  • 13