1

Quite often authors make some claims with some "to the best of our knowledge" disclaimer.

Example:

"To the best of our knowledge this is the first time this approach has been used to address this problem".

Is there any research/study/survey that tried to look at how often such claims are false? (e.g. some counterexample can be found)

I am mostly interested in the field of machine learning / data mining in English-speaking venues, however any reference on occurrence of such false assertions would be of value.

recursion.ninja
  • 338
  • 4
  • 12
Franck Dernoncourt
  • 33,669
  • 27
  • 144
  • 313
  • 3
    Quite often, I would hope. Appending or prepending "AFAIK" to a phrase is really only appropriate when you have a pretty good idea - but just aren't 100% certain - that what you're saying is true. – Moriarty Nov 30 '14 at 18:31
  • 8
    The example you give is of a statement that can be shown to be false, but can't be shown to be true (which is why the authors qualify it in the first place). So I don't know what it means to ask "how often such claims are true." – ff524 Nov 30 '14 at 18:32
  • @ff524 I was trying to be positive :) In my mind true == failure to show it is false but yes you're right, the direct way is to come up with a counterexample. I edited the question to reflect this. (apparently I messed up my votes, I can't upvote your comment) – Franck Dernoncourt Nov 30 '14 at 18:37
  • 2
    A meta-comment: @FranckDernoncourt, you frequently ask questions about researches or studies on a variety of topics which, really, do not seem to attract much research, at least by judging the answers you get, which, AFAIK, are mostly based on personal experience. If there is a thread connecting this series of questions, which do not seem to be driven only by curiosity, it would be nice to mention it explicitly, maybe you'd get more (meaningful/useful?) answers. Moreover, it would be nice if you add to each question which publications or databases you've already searched to find an answer. – Massimo Ortolano Nov 30 '14 at 23:08
  • @MassimoOrtolano Thread connecting this series of questions: often personal reflections on the organization of research, and I sometime wonder to what extent ideas stemming from open science could help research to be more effective. Regarding your second meta-comment, I generally Google + Google Scholar a bunch of words I put in the question. It sounds a bit tedious to list all queries I have made, and it might be counterproductive as results for a given queries depend on the location, time, etc., I only look at the few first pages of results, and I may miss some interesting results. – Franck Dernoncourt Nov 30 '14 at 23:18
  • 2
    Like @MassimoOrtolano says, in most SE communities, when someone requests surveys or solutions to his particular problem he must also provide a) evidence of his efforts so far or b) share the information he has acquired so far to the community. You seem to do neither of those two things, which to my eyes is a bit rude. – Alexandros Dec 01 '14 at 16:54
  • 1
    @Alexandros a) I generally Google + Google Scholar a bunch of words I put in the question. It sounds a bit tedious to list all queries I have made, and it might be counterproductive as results for a given queries depend on the location, time, etc., I only look at the few first pages of results, and I may miss some interesting results. b) I do share any information I have found so far, e.g. http://academia.stackexchange.com/a/27497/452 but as you may have noticed through the number of answers my questions asking for references (as Massimo did) it's unfortunately often tough to find. – Franck Dernoncourt Dec 01 '14 at 17:00
  • @FranckDernoncourt It is hard to believe that in more than 10-20 similar questions you have posted, you have not found a single reference or survey. It seems more probable that you actually found something that you did not want to share. – Alexandros Dec 01 '14 at 17:06
  • 5
    @Alexandros Yes of course I keep all the information to myself, it's very valuable and one day I'll become rich thanks to it. – Franck Dernoncourt Dec 01 '14 at 17:13
  • 1
    not about academia. at best, about one field thereof. – keshlam Aug 05 '15 at 17:54
  • 2
    @keshlam I have no idea why you believe this isn't about academia. – jakebeal Aug 05 '15 at 18:16
  • 2
    It's no more about academia than any other data-mining, linguistics or philosophy question would be. And it's a "please do a literature search for me" question, which is generally considered bad form unless the OP explains what efforts they've already made to answer the question themselves. I'm willing to be outvoted, but if so I don't understand which rules are actually in effect. – keshlam Aug 05 '15 at 19:38
  • @keshlam Well it's about academic research. – Franck Dernoncourt Aug 05 '15 at 20:03
  • To the best of my knowledge, this isn't about academia as defined for this area. Ymmv. – keshlam Aug 05 '15 at 20:08
  • -1 because this question pertains to, "The content of your research, rather than the process of doing research". In particular, since you don't intend to share what you find except for some sort of personal gain, I don't believe that this can be construed as a contribution to the forum. Everyone has vague, half-baked ideas. The correct way to pursue them is by talking to colleagues/mentors and/or getting a research assistant. – webelo Aug 05 '15 at 21:59
  • @Moriarty, I disagree on hoping it happens often. It would mean sloppy work of the authors. On the other hand, it would be nice if such claims get really followed up, if only for the sake of historical accuracy. – vonbrand Aug 05 '15 at 22:01
  • Isn't this what reviewers are supposed to check? Even if the author uses "to the best of my knowledge" as a euphemism for "I can't be bothered to spend more than five minutes at finding out", the referees are supposed to judge (and be able to judge!) the originality of the work. Hence in reviewed papers I would assume a high probability that the approach truly is original. – user3697176 Aug 05 '15 at 22:10

1 Answers1

2

To the best of my knowledge, it is unlikely anyone has done the gruelling work of trawling through the literature, looking for such statements, and then doing the detailed work of finding out if the claim was true. On the other hand, if I state such, you'd have to prove I did have prior knowledge to refute my statement. In itself, that could come close to an accusation of plagiarism in the context you cite. Or at least of sloppy checking of previous work.

vonbrand
  • 9,982
  • 1
  • 25
  • 46