7

This excellent, thorough and well-sourced answer to Has a gravitational microlensing event ever been predicted? If so, has it been observed? includes four links to papers on adsabs.harvard.edu pages, and all of these have arXiv preprints.

In fact a large number of posts link to the arXiv preprint version of astronomical papers and I suspect most astronomy papers have arXiv preprints.

To my understanding, this allows other astronomers earlier access to and awareness of ongoing research results which may allow the field to progress faster, and also allows everyone access to the paper without penetrating a paywall. This may have implications for the folks who collect the money which is paid at the wall.

Question: How did astronomers manage generally agree to do this? Was there much pushback by astronomical journals and/or by some authors in the astronomical community? If so, how was that resolved?

note: This may also be on-topic in Academia SE, but I think that both the pools of potential answerers and of interested readers of this bit of history in the field of Astronomy here is substantial.


Obligatory XKCD, hat tip to @JamesK

Shhh, you'll Jinx it. Obligatory XKCD

uhoh
  • 31,151
  • 9
  • 89
  • 293
  • 1
    Coincidentally, this topic came up a couple of days ago in The h Bar – PM 2Ring Apr 03 '21 at 04:56
  • @PM2Ring I came to write this question after making an edit on an old question in SE which contained "arXiv" and "astronomy". I was so happy that I could instantly read all of the sources sited in the answer to my microlensing question, and so tired of not being able to read physics papers until I get to the university or to break down and get a VPN, and simply wondered why astronomers were "so much smarter and nicer" than physicists ;-) – uhoh Apr 03 '21 at 05:04
  • 2
  • 1
    @JamesK updated :-) – uhoh Apr 03 '21 at 06:59
  • 1
    Preprints have existed since before the internet. – ProfRob Apr 03 '21 at 08:30
  • @ProfRob I've already distinguished how a publicly accessible internet website like arXiv differs from a paper preprint sitting on some table in some department libraries. Paper preprints were self-limiting due to printing and postal costs; there is no analogy whatsoever between how arXiv is used today internationally especially by the more tech-savvy folks, and how one might have browsed though what came in the mail before the internet. – uhoh Apr 03 '21 at 08:39
  • 3
    But no difference in terms of a culture of "posting" your research and passing it on to others before publication and with the full knowledge of the publishers. – ProfRob Apr 03 '21 at 08:53
  • @ProfRob yes that's certainly right. – uhoh Apr 03 '21 at 09:00
  • @uhoh There's plenty of physics preprints at Inspirehep.net, going back decades. – Dr Chuck Apr 03 '21 at 15:34
  • 3
    @uhoh The crucial bit is that the transition to sharing preprints online through the arxiv happened long before the commercial arm of the publisher realized how big the internet was going to be. – TimRias Apr 03 '21 at 23:01
  • @mmeent yes I think that could indeed explain a lack of "pushback by astronomical journals" early on, back when publishers had "commercial arms". I would expect that commercial considerations are more pervasive in all aspects of publishing in order to survive. – uhoh Apr 03 '21 at 23:08
  • 1
    @ProRob In the late 1960's you received preprints if you belong to a research "bubble". This allowed for pre-publication vetting by colleagues and gave an all-important "early preview". Now arXiv makes this available to all researchers. This was particularly useful if the paper was a conference paper since the published version would only be accessible in (ridiculously) expensive books or journals that were not available in all libraries. The journal publishing industry is worth 5-10 billion dollars a year - authors derive little benefit from that and have to give up their copyright. – JonesTheAstronomer Apr 07 '21 at 09:39
  • 1
    @JonesTheAstronomer you are instructing me on something I know very well. Preprints were still distributed well into the 90s I'm not sure what your point is, you seem to be arguing that publishers are being daft to allow this to continue? Authors still have to give up their copyright if they want their paper in a journal or conference proceedings. – ProfRob Jun 30 '21 at 18:50
  • 1
    @ProfRob: sorry to give the wrong impression - I am vehemently anti those rapacious publishers like Elsevier who profit from our effort in doing the research, writing it up, and then typesetting it for them. Peter Coles (aka Telescoper) has arranged a system whereby an arXiv e-Print can be refereed and given a DOI number. A very good start if the community supports him. The problem then shifts to getting this approval of a paper recognized by those who evaluate us for future jobs; a paper published in Nature is worth more brownie points than one published in Swaziland Science. – JonesTheAstronomer Jul 14 '21 at 15:32

4 Answers4

9

This is an interesting question, and I think it can productively be split into a couple of distinct parts, separating the publisher side from the author side:

  1. Was/is there any pushback from journals about posting pre-publication versions of papers on the web?

  2. Why do research article authors in astronomy post papers on arxiv at a relatively high rate?

Part 1 is easier to answer, since there are fewer individual actors to consider. As far as I can tell, the answer to this is a flat "no". As @ProfRob notes in a comment, there was a long tradition of dissemination of preprints on paper before the web existed, so it likely would have been difficult for publishers to forbid that practice once electronic dissemination became a possibility. Many journal publishers are directly connected with professional societies, and even for-profit journals need to show support for wide dissemination of ideas. I don't know of any journal that publishes astronomy research that doesn't allow posting of preprints.

Contrary to what one often hears, the higher-profile journals like Science and Nature are on board with this idea as well; both of them are very explicit in stating policies that allow both pre-publication posting on sites like arxiv, and free discussion of results at meetings. (current Nature policy, current Science policy). What they don't allow is authors discussing results with the press before publication. Nature in particular seems to emphasize the importance of preprints; they published their first editorial on it in 1997, and they seem to publish articles every few years that say, in effect, "no really, it's ok."

Part 2 is harder to answer, as is any question about why a practice evolves in some particular setting that involves individual decisions by many different people to adhere to some unstated norm.

First I thought I'd check to see whether it's really true that astronomy is different from other physical sciences in its use of preprints. The article linked in the answer from @Thomas suggests that astro is on the high side (at least as of 2011) in use of arxiv. I did my own spot check with a few ADS queries. To keep it simple, I just picked a few journals. In the years 2015-2019, this ADS query shows that there were a combined 24,634 refereed articles published in The Astrophysical Journal and Astronomy & Astrophysics (to pick a major American journal and a major European journal). Of these, 21,568 have preprints available on arxiv. (You get this by adding the qualification "property:eprint_openaccess" to the query.) So that's almost 88% of papers, which seems high indeed. To compare to another sub-field, I chose the journal Physical Review B (which publishes papers on condensed matter physics, since I know there's a cond-mat section of arxiv), yielding 25,952 articles, of which 16,936 have arxiv preprints. So that's 65%, still pretty high, but a fair bit lower than astronomy.

So, why might this be? My guess is that part of the answer might lie in the creation of ADS, and in particular the ADS project to scan all of the previous astronomical research literature and make it freely available online. That made it possible to sit at your desk and do all of your bibliographic research, rather than going to the library. Once people got used to doing that, it was natural to want to do it for the newer literature, too. Of course those electronically published articles were likely available to many researchers directly from the publisher, through their home institutions, but that early digital access was pretty heterogeneous by journal and sometimes not so smooth. That might have contributed to people wanting to put papers on arxiv to make them more available.

It's certainly more complicated than this, but I do think the existence of ADS (including its indexing of arxiv) may have made a difference in earlier / more complete adoption of electronic dissemination in astronomy compared to some other fields.

A related thing that I think is interesting is that the use of arxiv in astronomy has now become so ubiquitous that some researchers consider posting on arxiv to be important not just for early visibility, but for visibility at all. By that I mean that the daily arxiv postings have become, for many, the way that they learn about new literature, supplanting the role of journal table-of-contents services. I don't know how widespread the practice is (and I'm sure there are some generational differences), but I have certainly heard people say that arxiv is the only resource they use for regularly keeping up on new papers that are published. In other words, there's a sense in which simply publishing your paper in a refereed journal is not enough to make it fully visible to the community, at least not in the short term (though people will likely find it in literature searches later).

To check this idea, I went back to the ADS astronomy searches referenced above and looked at citation numbers. The 21,568 papers with arxiv preprints have 532,493 citations total, while the 24,634 papers overall have 561,738 citations. Subtracting, that means that the 3,066 papers without arxiv preprints have 29,245 citations. Looking at average citations per paper, that means that the arxiv preprint papers have 24.6 citations per paper on average, while the non-arxiv-preprint papers have 9.5 citations per paper on average. That is such a huge difference that it actually makes me wonder if there is some confounding factor I'm not considering - I encourage others to check the numbers and/or to try other searches. But if that's true, it's a huge effect, and bears out the idea that arxiv plays an important role in the community in research visibility overall, not just in early access.

Finally, I'll note that the importance of arxiv is now so complete that you even see an open letter from the AAS journal editors to arxiv, asking them to include the Research Notes of the AAS (a new publication of short, unrefereed papers) in the arxiv. There has been an ongoing discussion of this, since arxiv moderators started (mostly) rejecting arxiv postings of RNAAS papers.

Eric Jensen
  • 4,864
  • 9
  • 17
  • 1
    I got the impression that many/most Nature papers lack an arXiv entry. Am I confused? – ProfRob Apr 05 '21 at 08:54
  • If they don’t have arxiv preprints, it’s because of the authors’ choice, or possibly a misunderstanding of the policy. I tried to test it via ADS queries, but it’s complicated by ADS’ indexing of Nature news articles etc. I partially worked around it by requiring an abstract with at least one of a long list of astronomy words (most news pieces don’t have abstracts) and at least 2 citations. With those caveats I find 203 with arxiv preprints and 150 without. So it’s more than half, but not as high as for other sources. – Eric Jensen Apr 05 '21 at 13:19
  • I seem to recall there was a point in the late 1990s/early 2000s when Science (but not Nature was a bit negative about posting on the arXiv... – Peter Erwin Apr 06 '21 at 16:41
  • @PeterErwin Sounds like a research project for the Wayback Machine! – Eric Jensen Apr 06 '21 at 19:08
3

It is the usual policy of publishing companies (as per their terms and conditions) that you can publish preprints of the submitted manuscript on non-commercial websites, not only on arXiv but also university websites and personal websites etc. And you can keep it there after the paper has been published, although you should then provide reference to the journal publication and add a copyright notice (because normally the publishing company would then have the copyright).

Publishing your preprint on commercial websites like Researchgate and Academia would usually be in violation of their copyright agreement though.

This holds not only for astronomy papers but in other areas as well.

This has of course also the welcome benefit that you can have your paper 'published' even when it is rejected by the journal (which many manuscripts on arXiv appear to be).

So whilst there is nothing conspirational or even illegal about sharing your journal article on arXiv (as it is in line with the policies of the publishing houses) it seems to be true that Astronomy has one of the highest arXiv publishing rates (see the graphics in this arXiv article (published elsewhere as well)). One can only speculate about the reason for this. It is interesting in this context that Astronomy has also one of the shortest publication delays (as shown in the same link). Maybe these facts are related somehow. It seems the traditional rule 'Publish or Perish' holds particularly strongly in Astronomy, possibly because everybody wants to stake out their claim of being the first in case of some theoretical or observational discovery (for which a publication on arXiv should be enough).

Thomas
  • 3,061
  • 5
  • 12
  • 2
    Your latter point is a bad thing about arXiv for me and I don't agree with putting stuff on arXiv before it has gone thru the refereeing process. I generally don't cite work that is "submitted". Not least because something might change and then that makes you look like you've made a mistake because invariably your reference to an arXiv submission gets updated upon publication to point to the actual accepted version of a paper, which may be different. Nature and Science still have embargo rules don't they? – ProfRob Apr 03 '21 at 12:29
  • 1
    Thanks but this does not at all addresses the question that I've asked. This is just general academic background but I've explained why I asked here specifically. My experience is confirmed by the existence of an XKCD; there's something unusual about Astronomy compared to other fields. Somehow Astronomers are more in agreement that one should share a preprint to arXiv than other physical sciences. If you feel that there was never push or agreement to do this, and instead it "just happens to be this way for no reason" then that would be an answer to the question as asked. Thanks! – uhoh Apr 03 '21 at 13:37
  • @uhoh Please see my edited answer. – Thomas Apr 03 '21 at 15:50
  • 1
    @ProfRob You can still find useful articles an arXiv, even if they were never published (you have to consider that the acceptance rate of a journal is often fixed and sometimes as low as 25% or so, so the editors have to make up some rejection reasons in order to fulfil their quota). Citing an arXiv article that has not been published elsewhere (yet) can of course be dangerous and you should always think twice before doing it. Usually the information is not so original though that you could not find an alternative citation instead. – Thomas Apr 03 '21 at 16:16
  • I see you've made an edit. In Stack Exchange we try not to just say "see this 25+ page article", we include a block-quote of the specific section that supports our assertion within our answer, and mention the section or page number where it is found in the link. Otherwise we run the risk of having a link-only answer where there facts are elsewhere. I see that as discussed you've added "One can only speculate about the reason..." and while that's usually discouraged it at least begins to address the question as asked. – uhoh Apr 03 '21 at 16:20
  • @uhoh I have been thinking about inserting one or two graphics from this publication in my answer, but I did not want to cause any possible copyright issues for Stack Exchange (as these have been published in some journal). The same holds for copying whole paragraphs of text from other publications. That's why I left it with a link here. – Thomas Apr 03 '21 at 16:31
  • None of the major journals in Astronomy (apart from perhaps Nature and Science - which in any case do not tolerate arXiv distribution before publication) have acceptance rates as low as 25% (I've never been involved with a paper that has been rejected). Papers should not be "published" until they have been peer-reviewed; it's as simple as that. Conference publications etc. - fair enough. – ProfRob Apr 03 '21 at 16:37
  • @ProfRob I received the following response recently regarding a submission: "Given the current competition to publish in Physics Letters A (with an 75% rejection rate) and the burden that the present volume of submissions places on referees, we now have no choice but to make difficult decisions at any early stage in the manuscript evaluation process. We do not evaluate the validity of the paper's content or estimate its value within certain scientific communities" – Thomas Apr 03 '21 at 16:55
  • So get it accepted in another peer-reviewed journal and then put it on arXiv. – ProfRob Apr 03 '21 at 17:28
  • 3
    Whether to put on arXiv at the time of submission or acceptance is a matter of taste, I think. I understand why some people prefer to wait, but I've always put them on both simultaneously. If it's good enough to submit to a journal, it's good enough for the arXiv. More importantly, it gives other people a chance to read it and give comments, which might be useful – pela Apr 03 '21 at 22:17
  • 2
    @ProfRob You must be accustomed to quick review times. As a pure mathematician, my threshold for "this review seems to be taking a while to complete, maybe I should e-mail the editor" is around six months. It's not uncommon for a lot of papers these days to appear on the arxiv a year or more before they are formally published. Nor were such delays uncommon before arxiv; I've read quite a lot of pre-internet papers where the "submitted" date was a year or two prior to the "accepted" date. Arxiv's been wonderful for us because it is a MASSIVE speedup in dissemination time. – zibadawa timmy Apr 03 '21 at 22:51
  • 1
    Yes, a few weeks is typical for astronomy journals. @zibadawatimmy – ProfRob Apr 03 '21 at 23:01
  • @Thomas The main astronomy journals have acceptance rates of 75 to 85%. Yes, this is quite different from many other fields, including much or most of physics. For whatever reason, we just don't have the "prestige journals must reject most submissions" mentality, which may be a good thing. – Peter Erwin Apr 06 '21 at 16:58
  • @zibadawatimmy From an analysis of submissions in 2006 to The Astrophysical Journal: "88% are eventually accepted for publication.... The mean time for the first review is 31 days." – Peter Erwin Apr 06 '21 at 17:01
  • @ProfRob Nature explicitly allows you to post submitted versions of papers. E.g., from a 2005 editorial: "As first stated in an editorial in 1997, and since then in our Guide to Authors, if scientists wish to display drafts of their research papers on an established preprint server before or during submission to Nature or any Nature journal, that's fine by us.") – Peter Erwin Apr 06 '21 at 17:18
  • The reason for the short "publication delay" for astronomical papers is almost certainly the deliberate and intense effort made by astronomical journals to streamline the publication process, beginning in the mid-1990s if not before. The editorial process may be more efficient as well (e.g., usually only one referee report to deal with, less editorial overhead -- e.g., more freedom for scientific editors to accept papers directly -- etc.). – Peter Erwin Apr 06 '21 at 17:42
  • @PeterErwin Nature not only allows but encourages posting preprints elsewhere. From their policies page: Nature Portfolio journals encourage posting of preprints of primary research manuscripts on preprint servers, authors’ or institutional websites, and open communications between researchers whether on community preprint servers or preprint commenting platforms – Thomas Apr 06 '21 at 17:47
  • @Thomas So based on what did they reject that paper, if not on the validity or possible impact of the paper? The number of English mistakes? – Federico Poloni May 08 '21 at 10:08
  • @FedericoPoloni They said it would not be suitable for the journal as its scope had changed somewhat in the last few years (my paper was a direct comment on an article that appeared in the journal some years ago).. Looking at recent publications in the journal, that may be true to some extent, but since I feel that journals should be accountable for the papers they are publishing, they should at least have given my comment a proper review. – Thomas May 08 '21 at 13:57
3

Just thinking about the push back from journals (and thus is just speculation). The number of "main" astronomy journals is small, ApJ ( ran by the American astronomical society), MNRAS ( ran by the royal astronomical society), and A&A ( a pan European organisation). There are of course other journals (nature, science, Phys rev etc) but most articles i expect are published in one of these three groupings (I'm bundling the various letters and supplemental journals together).

Two of these journals are ran by astronomy societies, and the third was a merger of various European publishing efforts. The arixiv is a great way to get astronomy research out there and help the societies fulfill their mission in promoting astronomy. I expect also being ran by scientific societies means there's less pressure to generate large profits with the journals. Thus the arxiv would be seen less of an issue/competitor in "stealing" potential revenue from a journal.

Rob
  • 2,035
  • 1
  • 11
  • 14
1

Anecdotally, the following is from Scientific American's December 10, 2015 Astronomers Skeptical Over "Planet X" Claims:

That is, perhaps, until now. On December 8 researchers from Sweden and Mexico quietly submitted two papers to the prestigious journal Astronomy & Astrophysics, announcing their discovery of not one but two possible Planet X candidates. The quiet did not last for long. Even though neither paper has yet been accepted for peer-review and publication, the researchers uploaded both to the arXiv, a public online repository for preprint papers, where they appeared last night. Today, as claims of newfound planets in our solar system reverberate around the world in news stories and blog posts, other astronomers are reviewing the papers and reacting mostly with skepticism. The ensuing discussions between experts in public forums like Twitter and Facebook offer a rare, real-time glimpse of the sometimes messy scientific process as it unfolds.

“Normally I prefer to only upload accepted papers,” says Wouter Vlemmings, an astronomer at Chalmers University of Technology in Sweden and co-author on both studies. “This time, however, we had exhausted our ideas... With the arXiv upload we specifically wanted to reach the community that could tell us if we overlooked something, in which case we fully intend to withdraw the papers... What I personally did not count on was the impact it has had outside the astronomy community.”

I ran across this from What about the U planet? and then Wikipedia's (Object) U (TNO) which links to it.

uhoh
  • 31,151
  • 9
  • 89
  • 293