2

I'd like to explore possibilities of applying deep learning on image noise reduction problem, more on photographic camera noise. What's a good NN architecture to solve problems like this?

EDIT 25,Nov,2017: I have a small dataset of clean/noisy reference (~15K 4Kres images) acquired from digital camera. The target is to denoise other images from this camera type but without a reference photo.

Wangsta
  • 23
  • 3

1 Answers1

0

You might want to look into autoencoders. What they do is learn the relationship between the input and a simplified version of it, and then try to reconstruct the original version of the input.

The good news about this architecture is that it is not a supervised learning technique, so you can feed it a lot of data without having to construct both input and output.

Hopefully, if you train it right, it will learn that the parts of the image that are not important in the result are the noisy parts that you intend to remove.

Note that autoencoders had already been used to reduce noise in audio and image with great success.

Update: Here's a quick not-so-technical explanation on why they work.

In the general architecture that auto-encoder networks have, you'll usually find an input layer, a certain amount of hidden layers which usually have less nodes than the input layer, and then a certain amount of hidden layers that gradually grow to have the same amount of input layers in the output layer.

Autoencoder network structure example

Why they learn to reproduce the input in the output layer, the data representation needs to necessarily go through the narrow middle layer -- at that point, you can consider that the input has been "compressed", usually with a lossly approach to have a representation of the data that can later on use to reproduce the input.

(Side note: this compression feature has been tested in replacement of compression algorithms but they are not so good at performing compression as particular algorithms designed for it.)

When reducing the error between input and output, the network then starts forgetting the pieces that don't add as much information to the input as other bits of data in it. For instance, if you always train these images with figures in white backgrounds, it is likely that the encoded representation of the image will not have any information about the white background itself, since it is not required to be recreated.

As such, the most common pieces will start falling apart, and with the right tuning, a general amount of noise will go away too. Uniform noise is easier to remove (for the very same reason as the white background example), but variable noise can be worked around too with the right training.

Alpha
  • 458
  • 3
  • 12
  • Thanks for your reply. I have a small dataset that could be used for supervised learning, Any good idea to fit them into auto-encoder framework? I could think of "detaching" the output and input and fine-tune the network with my dataset. – Wangsta Nov 25 '17 at 18:01
  • @Wangsta Yup! Autoencoders don't need labels (correct outputs) so it will be easier for you to gather lots of images for training. Also, check out techniques for data augmentation, which basically means altering your data to generate more examples for training. It's particularly easy for images. – Alpha Nov 25 '17 at 18:07
  • Recently I read a bit about GANs. It seems like VAEs are sometimes used as the generator for GANs and the noisy images are often used as the input prior for the generator. – Wangsta Nov 28 '17 at 21:23
  • @Wangsta That could work too — NNs are universal approximators, so with the right input, output, and hyper parameters, any architecture should work. It all depends on what information you have and how much computing power / training time you can afford. – Alpha Nov 28 '17 at 22:03