Custom callback after each epoch to log certain information

Question

I know how to save the model after every epoch:

savemodel = ModelCheckpoint(filepath='models/model_{epoch:02d}-{loss:.2f}.h5')
model.fit(X, Y, batch_size=4, epochs=32, verbose=1, callbacks=[savemodel])

How to have a custom callback function to log certain informations:

def write_metrics(): 
    with open('log.txt', 'a') as f:  # append to the log file
        f.write('{epoch:02d}: loss = {loss:.1f}')

model.fit(X, Y, batch_size=4, epochs=32, verbose=1, callbacks=[savemodel, write_metrics])

?

With this code it won’t work because {loss} and {epoch} are not defined in f.write('{epoch:02d}: loss = {loss:.1f}').

Asked By: Basj

||

Source

Answer 1

Actually, you don’t need to define a function (or even use an explicit callback) for this, as such info is automatically returned by the fit method; from the docs:

History

keras.callbacks.History()

Callback that records events into a History object.

This callback is automatically applied to every Keras model. The
History object gets returned by the fit method of models.

You don’t even need to explicitly import anything; what you need is just:

hist = model.fit(X, Y, batch_size=4, epochs=32, verbose=1, callbacks=[savemodel]) # no other specific callback

and hist.history will contain the loss and any other metrics of interest you may have defined (e.g. accuracy), for the training and validation (if exists) sets, at the end of each epoch, which you can subsequently save into a file.

See here for detailed examples.

Answered By: desertnaut

Answer 2

Here is the solution, by subclassing Callback:

from keras.callbacks import Callback

class MyLogger(Callback):
    def on_epoch_end(self, epoch, logs=None):
        with open('log.txt', 'a+') as f:
            f.write('%02d %.3fn' % (epoch, logs['loss']))

then

mylogger = MyLogger()
model.fit(X, Y, batch_size=32, epochs=32, verbose=1, callbacks=[mylogger])

or even

model.fit(X, Y, batch_size=32, epochs=32, verbose=1, callbacks=[MyLogger()])

Answered By: Basj

Answer 3

A more straightforward approach to getting logs through Callbacks is using a CSVlogger callback.

CSVLogger class

tf.keras.callbacks.CSVLogger(filename, separator=",", append=False)

Callback that streams epoch results to a CSV file.

Supports all values represented as a string, including 1D iterables such as np.ndarray.

Example

csv_logger = CSVLogger('training.log')
model.fit(X_train, Y_train, callbacks=[csv_logger])

Arguments

filename: Filename of the CSV file, e.g. ‘run/log.csv’.
separator: String used to separate elements in the CSV file.
append: Boolean. True: append if file exists (useful for continuing training). False: overwrite the existing file.

Answered By: Tikendra Kumar Sahu

Custom callback after each epoch to log certain information

Question:

Answers:

CSVLogger class