2

Academic papers often make claims about observational data and results from computations. The papers only refer to names of data sources (such as an observatory) and of mathematical methods without any specifics, other than some of it illustrated in a few graphs and tables. References lead almost only to other papers of similar format.

In order to replicate (or even peer review) a paper, obviously one must have access to specifics such as the actual calculations, the raw data, the software, the blue prints of the measurement equipment and logs of how it has been operated.

What data do peer reviewers and replicating scientists (and the public) have access to beyond a paper making some claims?

Two examples: The results of the infamous cold fusion press conference of 1990 could never be satisfactorly replicated. But were the attempts based only on the paper they then published, or did they publish on the side also a wealth of data about their procedures? And the more recent "neutrinos travel faster than light" media debacle was, as far as I know, all about the details of equipment, was all relevant information about it made public (for the science community)?

called2voyage
  • 6,312
  • 1
  • 32
  • 62
LocalFluff
  • 11,383
  • 4
  • 31
  • 73
  • I hope my question isn't perceived as a suspicion of bad science. I think astronomy would be a very bad choice of subject for cheating scientists. Beside my curiosity about the practicalities of the scientific method, I rather think about how I could access source code and other yummies from the very edge of the science community. :-P – LocalFluff Mar 21 '14 at 15:25
  • Unless you're talking about review papers (ones which talk generally about a specific field of astronomy/astrophysics), it is simply not realistic for scientists to report this level of detail in published papers. Usually they will have a methods or samples (or both) section to their paper highlighting the source of their data as well as the analysis/reduction procedure which was done on it. Most likely, they do not make the specific chunk of data they use available simply because it comes from some publicly available database. Providing the coordinates for the region should be .. – astromax Mar 23 '14 at 20:45
  • sufficient to be able to reproduce the result. All of that being said, some papers do actually provide the exact data they use to publish their paper with. I'd imagine that paper within the fields of research which are under the highest levels of scrutiny (climatoloty, medical research, high-profile astronomy projects like Kepler) probably make their own datasets available more often than ones in fields which are not. – astromax Mar 23 '14 at 20:49

3 Answers3

5

Most catalogs have a public release somewhere on the web. Here is the best listing of astronomical catalogs that I know of: http://vizier.u-strasbg.fr/viz-bin/VizieR

There is also a NASA Extragalactic Database (NED) that can be helpful: http://ned.ipac.caltech.edu/

In general, astronomical observations are available to the public to be analyzed. Different telescopes/space missions maintain their own archives. Some give an observer a proprietary period (6-7 months is the time frame I have heard given for most observatories) to give the person who requested the observations time to analyze the data before it is released to the general public.

In publications the author(s) will list they catalogs the used to come to their conclusions so replicating scientist can use the same dataset.

moonboy13
  • 76
  • 4
2

If you read scientific papers (at least in astronomy, as far as I am concerned), you will always read of a section which is called Observations and data, or similar.

There you (as author) have to explain which kind of data you used and the detailed method for data reduction.

Most of the time, this section goes along with a table in which all used data are described in detail: the observation log. The guide principle in this section is that, any reader can reproduce your results.

In the same section, information about instrument calibration is also reported in detail. Also, error sources and error determinations. Basically, everything that is included in your published data.

Of course, mistakes can happen, and sometimes we do not consider variables which are important. I do not know the details of the two examples that you mention, but they are not the first and will not be the last.

Paradoxically, what you report is exactly the reason for disproving those experiments. Both major and minor results are always tested twice or thrice or more, by different groups, different facilities/observatories, different software versions. There is no way you can guarantee for a breakgrounding result without being "differently" tested. In the end, the method works well in this way! If I can't reproduce your results by following your description, either you are a bad writer, or your experiment is going to be confuted.

All the rest is explained well in the answer from OP @moonboy13, with the only exception that, for my experience, when data are private, they are released after 1 year.

Py-ser
  • 1,965
  • 11
  • 12
0

All data from the Hubble Space Telescope that is used in scientific research is released into the public domain (usually within one year, sometimes less) and available from the Mikulski Archive for Space Telescopes.

Donald.McLean
  • 1,024
  • 1
  • 10
  • 22