47

I recently received a request to referee a manuscript for a journal and, while doing so, I pointed out a crucial mistake undermining most of the calculations. After two unsuccessful answers on the part of the authors (where they addressed other concerns of mine, but basically ignored the mistake I was pointing out), I recommended the journal reject the paper and they did so.

Very recently, and within a couple of weeks after this rejection, the authors have openly posted their article on arXiv, without correcting their mistake. While I understand that manuscripts from arXiv should be taken with a grain of salt, as it is not peer reviewed, making a clearly incorrect result public seems wrong to me, both from an ethical standpoint and from a practical one as it might mislead researchers.

I am unsure how to proceed. Should I ignore it or take some measures such as publishing a reply/correction? If the latter, is there any way to do it for arXiv submissions? Is it possible to do it anonymously?

CGCampbell
  • 258
  • 7
  • 12
D34
  • 471
  • 4
  • 3

6 Answers6

31

I think it is commendable of you to care, but it would be best for you to keep anonymity with respect to the authors.

A good middle ground would be writing someone who knows your identity already, and is known by the authors: for example the editor in charge of the paper for the journal you reviewed for. Typically this editor is known to the authors, and they could pass on your comments without disclosing your identity.

In any case, the editor should be able to at least give you a qualified opinion on how to proceed.

hertzsprung
  • 135
  • 4
Pronte
  • 1,163
  • 8
  • 9
  • 5
    This has nothing to do with the editor anymore, so I don't see any reason why the editor should even care. –  Oct 22 '19 at 10:25
  • 58
    @corey979 Because they are a scientist? Of course, they could just ignore it, but science is a collective effort, and some people are nice. – Captain Emacs Oct 22 '19 at 11:20
  • Why do you think the OP should hide their identity? IMO there's no reason reviews should be anonymous in the first place. – spacetyper Oct 23 '19 at 04:26
  • 3
    In theory, everyone should be nice and accept other people's judgement. In practice, some people are bad and may retaliate against OP for the negative judgement, e.g., by being adversarial next time they review a paper by OP. – Pronte Oct 23 '19 at 12:04
  • 5
    @spacetyper There are plenty of reasons for reviews to be anonymous. If you want to know what they are, post a question. – David Richerby Oct 23 '19 at 18:17
  • @DavidRicherby I know what they are, but I think the bad parts of anonymous reviews far outweigh the good. Anonymous reviewers are much more likely to make ridiculous requests or to be uncivil in their reviews. They can tell you to cite their own papers and you wouldn't know if there was bias in that decision. Readers don't know what bias there was when a paper was accepted using a sample size of two (or if you're lucky three) when they don't know the reviewers. – spacetyper Oct 24 '19 at 04:19
  • 2
    @spacetyper If a reviewer asks you to do something ridiculous or to cite an irrelevant paper, then don't do it. Explain why you didn't do it in your response to the editor. – David Richerby Oct 24 '19 at 06:59
28

Maybe the authors wanted to secure their authorship of the idea, are working on the correction, or uploaded it so that they can send it to another expert in the field to help them with the corrections. There are many plausible reasons, so I wouldn't jump to conclusions.

Contact the authors first. Since they made the preprint public, obviously anyone could react to it. Just send them an email pointing out the error and asking for their thoughts on it. Be careful not to use the exact same phrases from your review!

If the authors won't answer, or will try to avoid the problem in another way, then you can write a short note and post it to arXiv as well. What the authors and the community will do next is up to them.

  • 38
    I think if you contact them, pointing out the same mistake that was mentioned in the referee report, then they will have a very strong suspicion that you are the referee - even if you use different phrasing. True, it isn't 100% proof, but they don't need 100% proof to be nasty to you in the future. – Nate Eldredge Oct 22 '19 at 13:44
  • 4
    I would even suggest to weaken the complaint to a question perhaps, or ask a friend to have alook into the problematic section and if daid friend finds the mistake on their own, apploud and encourage them to contact the authors. This may sound awfully tricky, but might at least help anonymity – Hagen von Eitzen Oct 22 '19 at 15:57
  • 1
    I think contacting the authors could be dagerous, for the reason Nate Eldredge mentions. – Terry Loring Oct 22 '19 at 22:14
  • 2
    @NateEldredge I often have a very stron suspicion about the identity of the referees of my papers – so what? –  Oct 23 '19 at 09:42
  • 1
    @corey979: So, if you disagreed with their recommendation on your paper, and you weren't a very nice person, you could retaliate in various ways: don't invite them to speak, reject their papers for no good reason, advise people not to hire them, etc. In this case it seems clear that the authors disagree with OP's opinion on the paper, and we don't know whether they are nice people. If they're not, then the more clues OP gives about their identity, the more likely they are to experience retaliation. – Nate Eldredge Oct 23 '19 at 13:31
  • 3
    That the authors are establishing priority is an interesting idea. – David Richerby Oct 23 '19 at 18:23
6

You're completely correct that what they've done is wrong and you're right to be upset about it, but that doesn't mean there's anything you can or should do about it now. You already did your duty as a referee by pointing out the error. If you ever have a role evaluating these people (as an editor, on a hiring committee, as a tenure letter writer, etc.) you can remember that they don't care about the accuracy of their papers or about their reputation as a scientist. But currently you don't have any power here and it's not your responsibility. If this were a major result in the field, then perhaps it would be appropriate, after further consultation with experts and with the authors, to post something explaining why the result is incorrect. But assuming this is a minor result that people aren't really paying attention to, posting a rebuttal would be an overreaction. Hopefully future referees will also be careful and reject the paper, and if not you can consider contacting the journal where it's published down the line to try to get it retracted (though this can be difficult). I don't think the arxiv has the time or the resources to investigate technical errors in papers like this, so I wouldn't be inclined to try to get it taken down from the arxiv.

Noah Snyder
  • 32,772
  • 7
  • 71
  • 128
  • I do not believe OP is trying to get this removed from arXiv, just making sure that his work finding this issue is not wasted: even if it is just an arXiv, people may take the result for correct as-is and use it in their research, propagating the error. So it would be suitable for the authors to at least acknowledge the presence of "possible issues" in the concerned part of the work – Pronte Oct 23 '19 at 12:07
  • 5
    There’s lots of things that the authors should do. But if they wouldn’t listen to a referee they’re not going to do anything. The question is whether there’s anything OP should do, and unfortunately there’s nothing reasonable to do. – Noah Snyder Oct 23 '19 at 12:54
4

Unless the paper could do real harm (e.g., public health policy), I wouldn't encourage you to moralize over the the ethics of someone posting a paper to the arXiv that wasn't accepted by you for publication. Spend your time on something constructive.

If you are certain that the paper is flawed and certain that explaining the fault would shed light on important issues in the field, it could be worthwhile to write a comment on the topic.

If, on the other hand, highlighting the fault in the paper doesn't lead to any greater conceptual understanding of the issue, reverse any widely-held approaches or attitudes to the issue, or prevent an erroneous result becoming important in the field, I see no real benefit of any further action. In fact, most colleagues won't have time to examine the details, and may just be left with the impression that you are argumentative, and like telling other people that they're wrong and you're right. So it's probably not a good move.

Lastly, for completion, let me address your questions about the arXiv. I don't think there is an 'official way to submit comments on the other papers to the arXiv - they are just ordinary submissions. Anonymous submissions are not allowed on the arXiv - see https://arxiv.org/help/registerhelp and this question. The relevant part of the arXiv registration policy is that:

It is a violation of our policies to misrepresent your identity or organizational affiliation. Claimed affiliation should be current in the conventional sense: e.g., physical presence, funding, e-mail address, etc. Misrepresentation of identity or affiliation, for any reason, is possible grounds for immediate and permanent suspension.

innisfree
  • 911
  • 5
  • 15
2

My understanding is that if you put something on arxiv and never publish it, it has minimal impact on the scientific community (with some well-known exceptions), especially if it is wrong. I would wait to see if it gets published somewhere else. If it does, then submit a comment on the paper. If it doesn't, it is hard to see what else you can do.

stochastic
  • 472
  • 3
  • 10
  • 6
    I don't think this is really true. My understanding is that, for example, in theoretical physics, ArXiv publications are very frequently cited, and formal publication can lag years behind. In a field like that, an incorrect ArXiv paper could lead to multiple publications before anybody realises "Hey, how come that one hasn't been published, yet?" Also, not every field has mechanisms for publishing comments on papers, though one could always write a short paper in that case. – David Richerby Oct 23 '19 at 18:21
  • @DavidRicherby Thanks for the comment, I removed the controversial statement – stochastic Oct 24 '19 at 20:13
  • @DavidRicherby years would be uncommon, and suggest that problems were uncovered during review. 3 - 6 months is normal. arXiv preprints of papers can indeed be well-cited before publication – innisfree Oct 25 '19 at 01:31
0

When something gets published in a journal and you have some disagreement over the contents, it's common to publish a rebuttal. Why not put a rebuttal on the arXiv if the authors won't respond to your contact with them?

sintax
  • 539
  • 2
  • 6