3

How are journals evaluated?

A researcher is evaluated on the basis of his/her publications. If a researcher has good publications in his/her PhD he/she is offered a Post-Doc position and eventually a faculty position.

My question is:

How is a journal judged?How does a faculty search committee evaluate a person's publications based on the journals where one has published?Is it done on the basis of impact factors of journals?

Learnmore
  • 2,772
  • 3
  • 18
  • 32
  • I hope the h index of the scholar and the journals are much more important...at least better measure to me which impact a scholar has on the peers in his field, while impact factor is more an interdisciplinary measure, but imo needs many publications to judge his interdisciplinary impact or if he is a "one hit wonder" – user48953094 Mar 22 '19 at 10:25
  • Seen this question earlier today - this will depend on what you need and what I want may well be different... – Solar Mike Mar 22 '19 at 10:45
  • 3
    You seem to be seeking an algorithmic way to evaluate a concept that relies heavily on human social perception. This is doomed to failure. – Jon Custer Mar 22 '19 at 12:58
  • How is a journal judged for what purpose? If you're really asking "How do faculty search committees evaluate applicants' publications?", then just ask that. (The answer is more complicated than "see how good the journals are"!) – JeffE Mar 22 '19 at 13:41
  • 1
    There was an editorial in Physical Rreview Letters in 2009 (https://journals.aps.org/prl/edannounce/PhysRevLett.102.060001). To quote the intro: "Impact factors are a bit like television’s Nielsen ratings. You scrutinize them and take credit if you are a beneficiary, but they are a tad unsavory!" It went on to explain why PRL has no intention of trying to increase its impact factor... – Jon Custer Mar 22 '19 at 15:15
  • 1
    Possible duplicate of How do you judge the quality of a journal? – enthu Mar 22 '19 at 18:58

2 Answers2

2

Let me start by saying that publications are usually evaluated not only according to their journal, but also according to the times they have been cited and to the impact they have had, which is a bit harder to gauge and a less objective metric. There are some cases where (at least where I work) the committee requires candidates to select a small-ish number of publications that they deem as the most important (5-12), and these are then read and carefully evaluated individually.

Regarding journals, the answer, as often happens, pretty much depends on the field. In many communities, the impact factor (IF) is surely important, but there are also other (often intangible) factors that come into play.

If many fields, the journals at the very top are few and well known (often Science, Nature and a few others). Papers published in these journals are usually considered to be the top of the cream. For all the other journals, the way they are considered by single scientists or committees is a combination of IF and of these people's own background.

For instance, I work in a pretty multidisciplinary field (in between physics, chemistry and materials science). Some journals are more chemistry- and materials-science-related, and these tend to have higher impact factors than their physics counterparts. As a result, I often find myself publishing in journals that, according to their IF, would be very good from a physicist perspective (>10). However, in the physics community these are regarded way lower than other physics-only journals that have a long history of publishing exceptionally important papers. A notable example is Physical Review Letters, which has published many famous papers like the original papers on the Higgs' Boson or the first detection of gravitational waves and has an IF of ~ 8.8 or PNAS, which is widely known and has an IF of ~ 9.5. I can tell you that, for many physicists, a paper published in PRL is valued higher than a paper published in a materials science journal with an IF of, say, 15 (everything else, such as the number of citations, being the same). I believe that in other communities the attitude is probably very similar: journals that are well known and have a long history will be ranked high, regardless of their IF.

lr1985
  • 238
  • 3
  • 8
  • 1
    fun fact: many chemistry/material science journals have higher IF than the best phyiscs journals, still nobel prize in chemistry often goes to a physicist...I also had to think of gravitational waves published in PRL. Sorting by "h index", PRL is also the best – user48953094 Mar 22 '19 at 12:39
  • 2
    In some fields, at least in some departments, publications are also evaluated by people actually reading them. – JeffE Mar 22 '19 at 13:42
  • Given that IF a) depends on the number of people in the field, and b) only takes into account citations in the first two years, it's kind of pointless to try to use it (and only it) to infer anything about the relative merits of two journals from two different (albeit connected) fields. – Anyon Mar 22 '19 at 23:05
  • 1
    @JeffE thanks for the comments, I have updated the answer. – lr1985 Mar 25 '19 at 10:11
0

I feel it may differ from country to country, or even from institution to institution. It may also differ across disciplines. Where I am working, there is what is called journal reputability assessment guideline, which outlines several criteria in assessing journals in which a faculty has published articles. These include the following.

  1. Journal impact factor (JIF) - journals with higher JIF are given higher weights.
  2. The current volume of the journal - as in JIF, journals with higher volumes currently are assigned higher weights.
  3. Indexing - journals indexed in known indexing services are considered more trustworthy and hence are given more weight than journals not indexed in known indexing services.
  4. Publisher - though assessment of this criterion is more subjective, journals considered as being published by popular publishers get more weight while those considered as being published by less known publishers are given lesser weights.
  5. DOI number - journals assigning digital object identifier (DOI) to their articles are given more weights.
  6. ISSN number - if the journal has an ISSN number (though it is less likely that a journal will not have one), it is rated more positively (i.e., it gets higher weight).
  7. The composition of the editorial team and editorial policy - this is also subjective; journals considered to have a high profile editorial team/board and standard editorial policy are assigned higher weights than those otherwise.
  8. Regularity and continuity of publications - Journals that publish regularly and without interruption between volumes or issues are assessed more positively (i.e., get higher weights) than those with irregularity and discontinuity in publishing.

So, the bottom line is, the criteria may differ and the way the evaluation is accomplished could also be different, but universities have such set of criteria for assessing journals.

Ayalew A.
  • 1,485
  • 1
  • 10
  • 11
  • 1
    No mention of the quality of the research that the journal publishes? Really? – JeffE Mar 23 '19 at 07:53
  • @JeffE: I don't mean these are the only set of criteria. In other institutions, there may be other criteria, and what I pointed out here may not be considered useful. The quality of research the journal publishes could also be considered if there is some measure of quality. The message I wanted to convey in my answer is that, at least from my experience, there are criteria that institutions set in an attempt to evaluate journals. You can add to the list. – Ayalew A. Mar 23 '19 at 11:46