Keras - no prediction probability for multiple output models?

Question

I have built the following model:

def create_model(conv_kernels = 32, dense_nodes = 512):

    model_input=Input(shape=(img_channels, img_rows, img_cols))
    x = Convolution2D(conv_kernels, (3, 3), padding ='same', kernel_initializer='he_normal')(model_input)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(2, 2))(x)
    x = Convolution2D(conv_kernels, (3, 3), kernel_initializer='he_normal')(x)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(2, 2))(x)
    x = Dropout(0.25)(x)
    x = Flatten()(x)

    conv_out = (Dense(dense_nodes, activation='relu', kernel_constraint=maxnorm(3)))(x)

    x1 = Dense(nb_classes, activation='softmax')(conv_out)
    x2 = Dense(nb_classes, activation='softmax')(conv_out)
    x3 = Dense(nb_classes, activation='softmax')(conv_out)
    x4 = Dense(nb_classes, activation='softmax')(conv_out)

    lst = [x1, x2, x3, x4]

    model = Model(inputs=model_input, outputs=lst)
    sgd = SGD(lr=lrate, momentum=0.9, decay=lrate/nb_epoch, nesterov=False)
    model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

    return model

When I do a prediction:

model.predict(X_test)

it works properly. However, when I want to get prediction probability like this:

model.predict_proba(X_test)

my model has no predict_proba function. Why not? Is it because of the multiple-output nature of the model?

Green Falcon · Answer 1 · 2017-12-20T15:41:59.047

6

As you can see here Keras models contain predict method but they do not have the method predict_proba() you have specified and they actually do not need it. The reason is that predict method itself returns the probability of membership of the input to each class. If the last layer is softmax then the probability which is used would be mutually exclusive membership. If all of the neurons in the last layer are sigmoid, it means that the results may have different labels, e.g. existence of dog and cat in an image. For more information refer here.

For more information as stated here in the recent version of keras, predict and predict_proba are the same i.e. both give probabilities. To get the class labels use predict_classes. The documentation is not updated. (adapted from Avijit Dasgupta's comment)

edited Dec 20 '17 at 15:41

answered Dec 20 '17 at 15:08

Green Falcon

14,058
9
57
98

2

Actually keras does have a predict_proba method, it's in the source code. The issue is that it's now outdated. Predict used to return classes , but now predict_classes returns labels and predict returns probabilities. predict_proba simply calls predict. – Tophat Dec 20 '17 at 15:15
2

predict method returns exactly the probability of each class. Although the first link that I've provided has referred to that point, I add here an example that I just tried: import numpy as np model.predict(X_train[0:1]) and the output is: array([[ 0.24853359, 0.24976347, 0.25145116, 0.25025183]], dtype=float32). Moreover, about the predict_proba, I tried to call it but there was not such method apparently. In the first link there was not discussion about that. for documentation you have to refer to the original docs. The reason is that the github version may still be unstable. – Green Falcon Dec 20 '17 at 15:30

score 2 · Answer 2 · answered Dec 20 '17 at 15:18

As you can see here, keras predict_proba is basically the same as predict. In fact, predict_proba, simply calls predict. As for why it is not working, I have no idea, it seems like it should work. However, it's mostly irrelevant since predict will give you the probabilities and predict_class will give you the labels.

Keras - no prediction probability for multiple output models?

2 Answers2

Linked

Related