4

I have a CNN model using cifar -10 dataset. The model was built using Keras (Tensorflow).

Now based on this model, I have to generate an image embedding (vector). That means - an input image comes and I have to output the embedding vector of that image.

I am not sure how to do that. This is not a straight forward prediction/classification output. Rather I have to output the embedding of the input image (which is off couse the predicted embedding but an embedding vector nonetheless).

Any suggestion?

nad
  • 163
  • 1
  • 1
  • 5

1 Answers1

7

You should use something like an autoencoder. Basically. you pass your images through a CNN (the encoder) with decreasing layer size. The last layer of this network is the one that produce the embeddings (that is, a lower dimensional representation of your input), and the number of neurons you use here is the length of your vector embedding for the input images.

Now, your embeddings are useful only if they actual encode the data in your images. To achieve this, you need another network (the decoder) that takes as input the image embedding and outputs an image with the same dimension of the input. Here, you try to minimize a loss function that tells you the distance between the image you generate from the embedding and the initial image (it might be Euclidean distance between pixel values).

Finally, if you need to output the embedding of an image, you just need to pass the image through the encoder network and collect the output.

Here's a simple tutorial.

dpstart
  • 211
  • 1
  • 1
  • 1
    Thanks. I looked into your link. So suppose I have the encoder network only in my autoencoder model (no decoder as I only care about the image embedding). So in the autoencoder function, I do return conv3. After training I save the model. Now when a new test image comes, how do I collect the output? I understand I have to do pred = autoencoder.predict(test_data) but whats next? – nad Feb 09 '19 at 23:34
  • 1
    You just have to pass the image through the encoder, and the output is the embedding. Of course, for the embedding to make sense, you have to pair it with a decoder when training. At the end, to get the embedding, you can simply do something like pred = encoder.predict(test_data) – dpstart Feb 14 '19 at 13:52