How can I judge whether an alternative "impact factor" is from an accepted and genuine source?

Question

So I was wondering whether the SJIF (Scientific Journal Impact Factor) calculation was authentic and accepted.

How can someone identify whether an alternative impact factor is generally accepted and from a reputable source? What kind of things should one look for in evaluating an impact factor metric?

Since we don't generally answer questions about a specific company here (see "Can I ask about my specific situation" in the [help/on-topic]), I've modified the post to ask more generally how someone can determine whether an "impact factor" comes from a reputable source. — ff524, Nov 19 '15 at 04:59
related: Which quantitative metrics are there of the quality and prestige of journals, and how do they compare? — fileunderwater, Nov 19 '15 at 08:45
They don't even tell you how are evaluated, just a vague list of things that they look at. — Davidmh, Nov 19 '15 at 08:46

score 18 · Answer 1 · edited Apr 13 '17 at 12:49

18

One way to judge a journal metric is to look at a few of the journals that are ranked highly by the metric, and judge for yourself whether they seem like important, reputable journals. (See How do you judge the quality of a journal.)

According to the SJIF metric, this is the publisher of the US-based journal with the highest impact factor. Visit their website (if you dare - it may give you a headache). I think it's pretty clear that this journal is not an important or reputable journal. The other journals ranked by this metric are similarly unprofessional.

Also: here's Jeffrey Beall's list of criteria that he uses for identifying a "fake" impact factor, and a list of metrics that he believes meet those criteria.

edited Apr 13 '17 at 12:49

Community

1

answered Nov 19 '15 at 05:09

ff524

108,934
49
421
474

6

That is hideous! – Davidmh Nov 19 '15 at 08:40
9

That website looks like it was designed by someone whose only exposure to web design was Geocities sites from the 90s. – JAB Nov 19 '15 at 13:41
1

Well, that website has certainly made an impact on me! – David Richerby Nov 19 '15 at 18:27

score 6 · Answer 2 · answered Nov 19 '15 at 05:34

Mirroring ff524's advice, go to journals that should be ranked highly, and see if they are. Either if their rank is appropriate, or if they're there at all.

For example, the American Journal of Public Health or the American Journal of Epidemiology are decent, society level journals that should be well-represented by any meaningful citation metric that's in the public health or epidemiology sphere. Neither even appears in SJIF's list. This indicates two things:

The journal isn't of the mind that being listed by that impact factor index is meaningful or worthwhile. That's a pretty strong signaling mechanism.
The service isn't attempting coverage of journals that actually represent where scholarship is being done. Which raises the question of what they are making their decisions based on - likely processing fees.

These types of services may (and hopefully should...) disagree on their metrics somewhat, but if the service is reputable, the major journals of your field should appear there in about the order you'd expect them to be in.

I like your first point. Just please note that there are journals which oppose the idea of bibliometrics and are not listed in any of them. But it's an exceptional thing, and while they may be important for the community, the big journals don't adopt this policy. — yo', Nov 19 '15 at 09:11
@yo' That's possible, and while I think I might let "They're missing one or two..." go, but if its a systematic pattern, it's a bad sign. — Fomite, Nov 19 '15 at 10:19

fileunderwater · Answer 3 · 2015-11-19T15:11:12.053

The first thing to check is what the index is claiming to measure, and how it is trying to measure that. Due to it's name, the Scientific Journal Impact Factor appears to be similar to the traditional Journal Impact Factor (JIF) from Thomson-Reuters. From its name you would therefore assume that the SJIF is measuring some sort of average citation rate of papers published in the journal. However, the list of evaluation critera of SJIF doesn't include any information about citation rates. It also doesn't mention a database of citations, which is needed if you want to evaluate journals based on citation performance. In contrast, the JIF from Thomson-Reuters is using citation information from the Web of Science database while the SCImago Journal Rank (SJR indicator) and the Source normalized impact per paper (SNIP) are using the Scopus citation database.

In fact, the evaluation critera that SJIF claim to use includes just about everything except information about the citation performance of papers. The index is therefore very misleading, since most people probably read it in comparison to the normal Thomson-Reuters JIF. Instead, they claim to evaluate such things as the Editorial Board, print quality, Internationalization, indexation, review quality etc. It is however completey opaque how all these components are weighted to produce their index. The minimum evaluation critera of SJIF also includes that; "At least one issue [and] At least 3 articles must have been published". To me, this shows quite clearly what kind of information their evaluation is based on.

Having said this, I also agree with the other answers that an easy check is to look at the score of well-known journals in your field (if they are ranked at all), to see how they are ranked in comparison to other journals.

How can I judge whether an alternative "impact factor" is from an accepted and genuine source?

3 Answers3