0

There have been some discussions here on how long to spend on reviewing a paper (e.g., here).

This question is a bit different in that I'm wondering what are your thoughts about reviewing a paper that is not going to be accepted.

I recently got a request to review a journal paper for a leading IEEE journal. From reading the abstract, introduction, and related work, it is clear that the authors are not aware of the progress that was done in recent years. The paper cites, evaluates, and compares against, 15-20 years old approaches that are far behind the current state of the art.

The problem now is that solution is quite involved and would take some significant time to understand and give feedback on. The only way for the work to be publishable is for the authors to compare against modern techniques and show merit, so the current submission cannot possibly be accepted.

  • Would you still spend the time in reviewing the proposed algorithm?
  • Would a review that reads "please compare your solution with XX and YY before consider it" be reasonable?

I have mentioned that this is a significant journal since I know that in some journals showing yet-another-solution for an important problem may be published if the solution is reasonable (e.g., if it is simpler than the state of the art but not as effective).

Jan
  • 1
  • This sounds like a question for the editor. – Buffy Nov 24 '19 at 15:58
  • I sometimes received this type of papers for review. Usually, the authors are hoping to get lucky. They know the work is not publishable because it is old. So it is perfectly fine to say the work is dated and is of no interest. – Prof. Santa Claus Nov 24 '19 at 18:28
  • I agree with Prof Santa Claus but I want to add that in principle a paper can discuss something old in a very new fashion and not necessarily compares with state of the art benchmark values or practices... – Alchimista Nov 26 '19 at 07:16

1 Answers1

6

In my experience, it is perfectly valid to reject such a paper on the basis of an inadequate review of the literature and/or limited relevance compared to the state-of-the-art. However, you may (understandably) have some reservations about rejecting the paper outright. In this case, I suggest considering adding something to your review along the lines of:

The authors do not clearly establish the relevance of their approach given the state of the art (e.g., Article1, Article2). Therefore, at this time, this reviewer recommends resubmission only after addressing (if not directly comparing against) the state of the art.

At the end of the day, your review is a recommendation for the editor. They can choose to take your suggestion or not. But it sounds to me that your assessment (that the paper as currently written is ignoring a large chunk of recent & relevant work) would be quite helpful to the editor.

Regarding how long you should spend on evaluating the actual method - I think you should at least do your due diligence to make sure you've understood the method and the overall novelty/impact of the paper. (It sounds to me that you may have done this already.) As part of the review, you should include a summary of what you've understood about the paper - this helps both the editor to grasp the significance of the work and also conveys to the authors that you've shown them the respect of honestly trying to understand their contribution, and yet you still feel that something is lacking.

A helpful guide here discusses "Rejection After the First Reading", among other things.

J. Tylka
  • 521
  • 3
  • 8