What should you do if you spotted a non-trivial error in a highly cited paper?

Question

I have seen question on here regarding a citation error in a paper here: What should I do if I found a citation error in a published paper?, but the nature of this problem is very different.

Basically, I came across an error in a paper (call this error 2).

I have previously contacted the author before regarding a different problem (call this error 1, say) in this paper, and I got no response. (That was what I still believe to be an unproven proposition, and is actually also related to this problem.)

The reason I decided to ask here this time is the following:

1. I am highly confident about error 2.

2. Its correctness has a direct impact on my current research.

3. The mistake cannot be easily fixed. (At least it appears to me, for the time being). I have spent months on a related problem before revisiting this paper and realised that I have tried this same technique in problem, which did not work in my problem, which led me to check why it worked in his paper - and it turned out, it did not.

4. The result was probably the first/the only result of its kind in the literature and the paper is fairly important with 150+ citations.

What should I do here? I have previously contacted the author and I have no reason to believe he would respond this time. I want it to be correct/easily patched because of reason 2. This paper was published in 1997 so it has been a while....

EDIT: Precisely for reason 2, I would much prefer some way of getting the author to respond.

EDIT: I had a discussion with my supervisor. He immediately agreed with me it was a mistake. The thing is when we tried to use a similar technique for a different problem, we fell down the same trap about 10 times. At the moment, we have no fix and the mistake is serious enough that it takes away a lot of credit from the paper.

Answer 1

The main answer to your question is easy: talk to your advisor.

I find it a little curious that you have not done this already. As a graduate student, you read a very famous paper (in mathematics, more than 150 citations is a very large number) that was written a while ago, and at the first sign of trouble you contacted the author of the paper. That is already a little strange: although both are possible, the probability that you, a relatively young graduate student, have misunderstood something in the paper (or are applying standards that are different from those of the field you will be working in, or some other similar issue related to the fact that you probably do not yet have expert level knowledge and experience in this field) is higher than that the author has made a serious mistake. Writing to someone that you don't know at all and who is much more senior than yourself and pointing out a mistake is not without risk: the risk is that you will be wrong and they will dismiss you in the future as a less than serious person.

Your advisor is there exactly for such things: she is the person who is helping you transition from a neophyte to a journeyman to an expert, and she needs to see your mistakes and flawed reasoning in order to do this properly. Some outside expert really does not: they can wait to see the finished product that you become. Moreover, you are presumably reading the paper because your advisor wants you to, perhaps even because she directly told you to read it. Therefore a mistake in the paper is your advisor's problem as well as yours. Why are you holding that information back from her? You shouldn't.

Having said that, I feel like the OP is getting a fair amount of advice that would be more appropriate in other fields than mathematics. Especially:

In mathematics it is very difficult to write a paper whose sole or even primary purpose is to point out a mistake (even if it includes a correction) in someone else's work.

This is a cultural reality of the field; it is certainly not entirely positive. In many other academic fields, one can "score points" by pointing out others' mistakes, and in some fields this is one of the best ways to score points. Mathematics is not like this: if you can get such a paper published it will "take points away" from the author and give you a certain amount of notoriety, but if this is for instance your first published paper then many people will look at you strangely, almost as if they expect you to make further trouble. (Again, I'm not endorsing this cultural practice; I'm just describing it.)

Similarly, I would say that contacting the editor of the journal in question is maybe step 10 of a procedure that is mostly designed to terminate after one of the first 9 steps. You should do this only after you have exhausted every other possible avenue.

In mathematics -- very much unlike in some other fields -- it is prohibitively unlikely that an editor will publish any "commentary/correction/erratum" by you unless she believes that you are mathematically correct in the point you're making, and convincing someone with standing in the field like a journal editor that you are mathematically correct is largely what you're trying to do anyway.

Finally, while I barely know what a white paper is and to the extent that I do I'm not sure it's the appropriate terminology here -- in content I do agree with @eykanal's suggestion: it will certainly help your advisor if along with communicating the surprising news that Famous Paper X is wrong, you can supply a written version of your arguments. This can be hard to do: explaining why a difficult argument is subtly wrong is one of the sternest expository challenges in mathematics that I can think of. It takes a lot to go from "I don't understand the argument and find it rather unclear" to "I am convinced that it is wrong." By the way, in writing to an author it is a good strategy to err on the side of the former quotation.

Answer 2

I would do the following:

1. Write up a rigorous white paper demonstrating the issue. This should be your first step, as putting the idea to paper will help you think of aspects you may not have thought of yet. If you can successfully convince yourself in a written paper that this issue is valid, then...

2. Present the white paper to your advisor. This has numerous benefits:

   • Your advisor will critique your writing, which is always a good thing; you will become a better scientific writer.
   • Your advisor can provide specific feedback on specific aspects of your argument by simply referencing what you wrote and providing counterexamples. This will make the process of convincing him that there is a problem much easier.

   If you can sucessfully convince your advisor that the problem exists, then it's time for..

3. Work with your advisor to reach out to the author. Working with your advisor will significantly increase your chances of success. You may also be able to progress to
4. Write up your findings in a letter or article of your own, depending on the nature of the issue.

Do note that you state that this paper has been reviewed and cited many times. While it is possible that you have found something everyone else has missed, your ideas will need to be iron-clad before anyone will even listen to you. I would spend the most time right now making sure you haven't made a mistake before reaching out to the author.

Answer 3

Ask someone else - your age listed in your profile suggests that you're a grad student. Have you talked to your supervisor about this? Even better, do you know someone in your own department that has collaborated with the paper's author?

Put yourself in the author's shoes. He's probably bombarded with emails all the time from grad students he's never heard of, so unless you've collaborated before then he may not even read your email. An email from your supervisor or one of the author's collaborators is much more likely to get serious attention.

Answer 4

I agree with Moriarty entirely. The first step is to get independent confirmation that there is an error.

The second is to judge its severity and how it impacts on the conclusions of the paper.

The third is to contact the authors directly and perhaps ask them to list or fix the errata on their homepage and on any pre-prints.

Mainly my answer is to introduce the fourth option. Many journals allow for publishing reviews, comments or corrections on papers that have already been published. Here's an example of this in a reputable computer science journal (TKDE). Particularly if the paper is highly cited and the error underlines grave concerns about its contributions, this option is recommendable.

If the journal does not allow corrections, you may consider writing your own independent "correction" paper instead (as others have suggested), but this way, you may be expected to present new results, not just pointing out errors.

---

But again, like Moriarty, I would highly recommend that you get the advice of a more experienced researcher, in particular before engaging in the fourth or fifth option.

Answer 5

Let me share my personal experience in a vaguely similar situation: Back in 2007 and within a couple of months of publication I spotted a huge mistake in a paper published in a journal that used to be A class but is more like B class these days. I wrote a proof of the error, showed how to correct it, and sent it to the journal. It took over one year of back and forth between the editor and I to get him to admit he had published a profoundly incorrect paper and to publish my "Comment": but I'll say that I probably got lucky that he admitted his error, got lucky that he cared to correct it, got lucky that he read my correspondence, etc.. If my experience is any indication, it will be very difficult to get across to the editor(s). At some point, about 6 months into the exchange, when it was clear that the editor was assuming I was in the wrong (he never read the math bits), I wrote to another editor and asked him if he thought I could damage my reputation by insisting so much, and he told me not to worry and go ahead on the quest for truth (but there was nothing he could do to help). The bottom line is you will hurt several people's pride and could make some enemies. I personally couldn't care less, but I'm atypical. And if you looked at my career, you'd probably think twice about upsetting the editors of an influential journal. Whatever you do, all the best!

Answer 6

The answer may be field-dependent, but in theoretical physics it is: write your paper. If the error is non-trivial, showing why something not works (or an counterexample) may be valuable. The same thing if the error is relatively simple, but most people are not aware of it.

Sometimes a critical flaw in one paper gives raise to a new field - see How the No-Cloning Theorem Got its Name by Asher Peres (or for a longer story, How the Hippies Saved Physics by David Kaiser).

An erroneous paper (with an error being non-trivial at this time) resulted in the non-cloning theorem, and later - quantum cryptography, quantum information etc. The error (assumption that a machine could copy any quantum state) could be addressed by a few lines of an elementary proof there no such machine could exist.

Answer 7

If you can give all the relevant details in a reasonably concise way, then it may be appropriate to ask a question on mathoverflow.net. I would strongly recommend that:

• You prepare your question very carefully, and review and revise it several times before posting to ensure that it is as clear and complete as possible and correct in all details.
• You phrase your question as "I do not see how this can be correct, please explain what I am missing" (even if you feel very sure that you are not missing anything).

If your problem is in an area that is well covered by mathoverflow, then this should be effective. Either someone will explain what you are missing, or the relevant experts will become aware that you have uncovered a genuine problem.

Answer 8

My advice is pretty much the same as everyone else's: talk to your advisor, go over the suspected problem point with him; if there is no resolution, reach out to others in the department (your advisor should do this).

Having said this, I'd like to relate a few incidents from my own experience. As a graduate student, I independently discovered a result in number theory that did not appear in the literature. I was unaware of the significance of the result, but it came up in discussion with my advisor. He informed me that Professor X had found this result, but had never published it, and that I should not publish it, thereby "scooping" the (world-famous) expert.

While working with my second advisor, in a different area, I read a paper by a major player in that area who had developed a widely used technique. I mantioned to my advisor that there was a point in the proof of that technique that I didn't understand and he told me that no one understood the proof, but all assumed it worked.

Many years later, I now work in a different area. But while I was getting started in it I ran across a theorem in a book by a well-known expert in the field. What it said contradicted other things I had read. Not fully trusting myself, I reached out to the author who acknowledged that what he had written was not correct. He was very grateful for the correction and saved it against the advent of a second edition of his book.

Finally, I wrote a paper which contained an error, an error I should not have made because I was familiar with a counterexample to the statement I made. No referee caught the mistake, and only one person ever wrote to me about it, and I am very grateful to that person. Everyone reacts to these things differently, but I did not take offense at the correction. Granted this situation is a bit different than yours, because the correction came from a well-known source, and not from a graduate student. I would like to think I would have been equally accepting if the correction came from a student. I was embarrassed to be corrected by an expert, whereas I might have been peeved to be corrected by a student. However, were it you who wrote to me, you would never have known my reaction. I would have thanked you for pointing out the error, and perhaps have apologized for making it. But that's just me, and I'm far from well-known in the field. So, I have no idea of what my more famous colleagues have to deal with on a daily basis. The paper in question has never been corrected to my knowledge.

As I said, everyone reacts differently in these cases. Work with your advisor. The most important thing for you is to determine if there is indeed an error in the paper, and, if not, to understand why the paper is correct so that you can move forward with a correct grasp of what the paper established.