25

There's been a case of alleged data manipulation in a recent Nano Letters paper (initial report here; now-retracted paper):

    enter image description here

You can surely make your own opinion on whether the images have been digitally modified, given the above snapshots (look for gray rectangles around the rods); I was careful to write “alleged” because the paper’s PI threatened legal action against the blogger who reported the issue.

Now, this case lead to a large number of comments here and there about whose responsibility it was to catch this issue (reviewers or editor). I tend to agree that the reviewer should probably have caught it, but this lead me to wonder:

As editor of a scholarly journal, what measures can I implement to prevent image/photo manipulation? And what about data manipulation? Organic Letters has made the news two months ago when they hired a in-house data analyst. Is that the way to go? Are there other measures one could take to reduce the threat of image and data manipulation?

F'x
  • 73,069
  • 23
  • 262
  • 392
  • 7
    I guess the answer is "None whose benefits will outweigh the costs on the long run". The photoshop gets better every day and the phrase "you can surely make your own opinion on whether the images have been digitally modified" certainly does not apply to me. I would just be cautious of authors with no or bad established reputation and stop there. It is the same issue as with plagiarism, etc. We may run a couple of obvious checks but it is just not our job to ascertain everything beyond any doubt or to defend our statements in the court. – fedja Aug 16 '13 at 21:21
  • 3
    Related for statistical data manipulation, What should raise red flags to detect fabricated data. – Andy W Aug 17 '13 at 01:59
  • 7
    If you are not already doing this, I would insist that all images be submitted and be made easily available online at an extremely high resolution. The size and resolution of images in print will usually not be sufficient for readers to find the manipulations. – Bitwise Aug 19 '13 at 15:21
  • 2
    Well, I was going to comment that the aberrations could simply be artifacts of the image compression algorithm, most likely jpeg. This particular compression algorithm is notorious for leaving artifacts after the compression.

    However, after looking at the images, I have to agree with the conclusion that they appear edited. And poorly so, I might add.

     – Jonathan E. Landrum Aug 20 '13 at 15:31
  • @JonathanLandrum: I've never seen jpeg rectangular artifacts that were rotated against the pixel x or y directions (unless of course the image was rotated and subsequently stored without further loss). And in the pictures above I'd have expected similar rectangular artifacts along those curved edges. – cbeleites unhappy with SX Aug 21 '13 at 20:36
  • @cbeleites, that is why I wrote at the end of my comment, "However, after looking at the images, I have to agree with the conclusion that they appear edited. And poorly so, I might add." I suppose I might not have been clear enough. The artifacts I saw in the images do not appear to be the result of a compression algorithm of any kind. They appear to be the result of sloppy editing. – Jonathan E. Landrum Aug 22 '13 at 15:07

4 Answers4

10

I think a search term for this type of manipulation detection is image forensics. Matthias Kirchner: Notes on Digital Image Forensics and Counter-Forensics may be a starting point.

What can you do?

I think a first step is to communicate clearly what image manipulations* are acceptable and which are not:

  • is it acceptable to adjust brightness and contrast?
  • correct unequal illumination?
  • whitelight correction based not on a whitelight measurement but on parts of the image?
  • digital sharpening or other enhancement filters?
  • where's the border between an inset picture and a fraudulent manipulation?

and so on.

Who should detect this?

I agree that the reviewer should have commented and asked about the pictures above, however my experience as reviewer is that I often get pictures in ridiculously low resolution (I comment on that). I'm not sure, but I think that the publishers may ask for high resolution pictures when they spot such problems in the production process. However, that would mean that the reviewers may not have seen the actual picture that is used for printing.

* manipulation in the data analysis sense: calculations that change the information content (e.g. enhance contrast), and cannot be easily undone (as opposed to a transformation where the back transformation is easily possible, e.g. rotation), not in the sense of fraud.

cbeleites unhappy with SX
  • 23,007
  • 1
  • 44
  • 91
3

There exist softwares dedicated to this. Journal of Cell Biology seems to have been a precursor in this in 2002.

A first simple check suggested in this paper is to open images in a image manipulation software and change "controls" (I guess: contrast, luminance, etc.) The problematic regions can then appear clearly.

I would say that, like plagiarism, this is the publisher to check or provide software to the editorial board, as this is technical and not scientific assessment.

F'x
  • 73,069
  • 23
  • 262
  • 392
Benoît Kloeckner
  • 14,620
  • 39
  • 77
1

It seems to me the only possibility is to subject all or particularly important images to an analysis detecting variation in noise levels (see examples here; service unfortunately closed). I personally do not know if such an analysis could be automated in an electronic submission system or be used as a tool by the editors. How much effort should be spent looking for fraud will obviously depend on assessments of, for example, how critical images are for the publication(s), the likelihood for fraud (random tests?) and the cost in terms of time and money for doing the testing.

It seems to me there would be much use to have a tool to do such an analysis for all submissions. The problem is of course that the noise level detection also identifies all kinds of manipulation and so it would seem reasonable to ask authors to provide a very detailed account of what has been done to each image so that the analysis can be set in a perspective.

F'x
  • 73,069
  • 23
  • 262
  • 392
Peter Jansson
  • 73,217
  • 12
  • 206
  • 344
-1

"Kill the chicken, and make the monkey watch"

Institute an editorial policy by which all convicted offenders are banned for life from the publication in question. Convince other publications to share such info, and act in the same way after definitive proof is presented. No second chances.

Publicize these events far and wide.

That will probably take care of 60-70% of the problem, at least the blatant cases such as the one above.

Unfortunately few publishers have the conviction to do that.

EDIT: To answer your question: "Who do you ban among N coauthors?" To first approximation, all of them as co-authorship implies shared responsibility. However, that can be more accurately determined on a per-case basis as the result of detailed investigation.

Detecting instances fraud is trivial if crowdsourced. Blatant image manipulation as the one shown above would eventually have been noticed by a reader of that paper. Same applies to other similar kinds of fraud. Relying on a single overburdened editor and a couple of bored referees for that task makes it much more difficult. Hiring staff to essentially redo part of the research reported in manuscripts submitted to the journal is just laughable.

The point is to demand ethical standards as a publisher, and raise the stakes so high that the penalty of getting caught, guaranteed loss of professional reputation and possibly employment, offsets any gain from publishing a single or a series of papers.

Relying on automatic detection schemes is inherently unreliable. Smart people will always find creative ways to cheat more effectively. To illustrate my point, consider the unending arms race between virus writers and antivirus software companies.

To conclude, and since you seem to be quite green, I suggest you give this a thorough reading: https://en.wikipedia.org/wiki/Schon_scandal

It won't take many Schönen to drive the point home...

None
  • 1
  • 5
    Life is complicated, shades of gray rather than black and white. There are questions with any black-and-white policy such as the one your propose, such as: “You have a paper with N coauthors, who do you ban?” I'm not convinced anything this would be enforceable, let alone give positive results. – F'x Aug 21 '13 at 19:19
  • Well, you did ask how you can PREVENT image/photo manipulation, rather than asking about how to detect it and confront authors about it. :) Emphasizing that manipulating photos like this is WRONG is one way to draw attention to the subject. – Irwin Aug 21 '13 at 22:34