6

I'm not an academic, but I notice a trend in the top journals where replication studies aren't taken with the same glamour as original studies. Considering mountains of recent research show the majority of studies in many fields may in fact be irreproducible 1 2 3 4 etc, how can journals as a whole be taken seriously?

As a scientist, surely the scientific method must be respected, and one tenet of that method is reproducibility. This fundamental part of science seems to have taken a backdoor seat, and is not taken too seriously it seems. Be it due to funding, lack of prestige, or whatever. The end results are tainted/irreproducible studies that are cited many hundreds of times, distorting the truth of the world.

I respect there are individual initiatives recently to replicate large numbers of studies, but replication should be the norm, not the exception.

My question is, why are prestigious journals relied upon in terms of status, knowledge and information exchange, when in fact the replication rates of their studies are quite low, and there is no guarantee of any study you publish, cite or rely upon actually being accurate unless it's been replicated?

user4779
  • 176
  • 3
  • 2
    Stack Exchange is not for rants, no matter their possible truth or value. – Zev Chonoles May 16 '15 at 01:46
  • Zev, feel free to edit the question. But I'm honestly attempting to gain an insight into this phenomena. – user4779 May 16 '15 at 01:52
  • 1
    I am voting to put this on hold as too broad for the moment. You are asking three very different questions: 1) How about this reproducibility problem? 2) Are publications meaningful when stripped of technical jargon? 3) What should we do instead of journals? Please edit to focus on one of these. – jakebeal May 16 '15 at 02:28
  • I've amended the question to focus on point 1) The reproducibility problem, and the reliance of knowledge and directions of research based on studies that haven't been independently verified. – user4779 May 16 '15 at 05:38
  • I retracted my close vote after you updated your question (I won't be able to vote to close again this time around). – Nobody May 16 '15 at 05:53
  • When a journal accepts a research paper containing experiments, it is the duty of its editors and reviewers to ensure that the paper contains enough details about the experiments that allow others to reproduce the results without making unreasonable assumptions. So reputed journals should contain papers that allow its readers to reproduce the results. If a reader complains that something is not reproducible, the journal might verify the claims and if necessary withdraw the paper. – Arani May 16 '15 at 06:02
  • @Arani That's a good thing, but before citing a study, in proper scientific spirit, shouldn't the presumption be that the study is in fact wrong until it's been replicated and accepted by a number of independent sources, before it can then be referenced and relied upon? I realize this isn't what happens most of the time, but when it doesn't happen the consequences can be severe (eg:medicine). Even at an undergrad level, students must cite studies that haven't been replicated, and base conclusions off that. But isn't that poor science? Should the practice not be to only cite reproduced studies? – user4779 May 16 '15 at 06:09
  • "How can journals as a whole be taken seriously?". Journals need not to be taken seriously by the general public or specifically you. They only need to be taken seriously by the related scientific community and people who publish there. Second, as recent events showed (Obokata case) even if some fraud paper manages to fool peer-review it is discovered very soon and the related people pay the consequences. So, problem solved. Critics of peer-review (like you) must acknowledge that peer-review in its current form actually works as it is evidenced from the recent technological advancements. – Alexandros May 16 '15 at 07:10
  • 1
    @Alexandros,There is no doubt that a vast quantity of published studies have revolutionized and continue to revolutionize our world, but the evidence seems to suggest that more fraud and irreplicable studies are never detected than those few cases that are found and made public. If which studies are correct or not is esoteric and available only to insiders in that community, how are politicians, entrepreneurs or other people who base decisions off science supposed to make informed decisions as to whether the study they're reading is even correct? Why is replication not held as a gold standard? – user4779 May 16 '15 at 07:38
  • @Alexandros I thought the OP is questioning why the replication rates of studies are quite low, not critisizing the peer review process. Am I mistaken? – Nobody May 16 '15 at 08:32
  • @scaaahu I think the OP "due to the low replication rates of studies" doubts the entire peer-review regime. I might be wrong though. – Alexandros May 16 '15 at 08:36
  • I think the peer review process and journals are necessary, but the common practice of basing new research or findings off prior findings that haven't been replicated, or citing such studies, or industry/government/academia making decisions based on faulty studies, can and does have catastrophic consequences. I feel all this could be avoided by only citing or basing further research off studies that have been replicated. I appreciate the resources involved in replication are immense, but I think the consequences of not doing so are greater, and I'm curious why this isn't standard practice. – user4779 May 16 '15 at 08:56

0 Answers0