KerasRegressor Coefficient of Determination R^2 Score

Question

I’m building a small neural net in Keras meant for a regression task, and I want to use the same accuracy metric as the scikit-learn RandomForestRegressor:

The coefficient R^2 is defined as (1 - u/v), where u is the regression sum of squares ((y_true - y_pred) ** 2).sum() and v is the residual sum of squares ((y_true - y_true.mean()) ** 2).sum().

It’s a handy metric because it shows values up to 1.0 (similar to percent accuracy in classification). Is my usage of Keras backend correct for the accuracy metric I want?

def create_model():
    model = Sequential()
    model.add(Dense(10, input_dim=X.shape[1], activation="relu"))
    model.add(Dense(10, activation="relu"))
    model.add(Dense(1))

    # Compile model
    model.compile(loss="mean_squared_error", optimizer="adam", metrics=[det_coeff])
    return model

# Is this computing the right thing?
def det_coeff(y_true, y_pred):
    u = K.sum(K.square(y_true - y_pred))
    v = K.sum(K.square(y_true - K.mean(y_true)))
    return K.ones_like(v) - (u / v)

This appears to work in that nothing errors and the metric is increasing towards 1 over time, but I want to be sure I implemented the metric correctly.

Asked By: Nick

||

Source

Answer 1

you can check this post out. I tested the following code and it works ok for your purpose.

from keras import backend as K

def coeff_determination(y_true, y_pred):
    SS_res =  K.sum(K.square( y_true-y_pred ))
    SS_tot = K.sum(K.square( y_true - K.mean(y_true) ) )
    return ( 1 - SS_res/(SS_tot + K.epsilon()) )

Answered By: Mingfei Sun

Answer 2

Tensorflow has add-ons as a separate module named "tensorflow-addons" which you can install using pip install tensorflow_addons. It includes some common metrics such as R2-score. To use R2-score as an evaluation metric, you can simply import it, instantiate it and pass it as a metric:

from tensorflow_addons.metrics import RSquare
model.compile(loss='mse', optimizer='rmsprop', metrics=[RSquare()])

Another option is to directly use sklearn.metrics.r2_score. However, because keras uses tensors and scikit-learn uses numpy arrays, you must set run_eagerly=True to avoid a NotImplementedError. It must be noted that this considerably slows down the code, so unless you have a small NN, it’s probably better to use the other option.

from sklearn.metrics import r2_score
model.compile(loss='mse', optimizer='rmsprop', metrics=[r2_score], run_eagerly=True)

Answered By: cottontail

KerasRegressor Coefficient of Determination R^2 Score

Question:

Answers: