Make a deep copy of a keras model in python

Question

I would like to make a deep copy of a keras model (called model1) of mine in order to be able to use it in a for a loop and then re-initialize for each for-loop iteration and perform fit with one additional sample to the model. I would like to be able to initialize the model after each iteration since after performing the fit (my model is modified however, I want it keep it as it is when i am loading from the path using the load_weights).

My code looks like:

model1= create_Model()
model1.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model1.load_weights('my_weights')

model_copy= create_Model()
model_copy.compile(optimizer='rmsprop', loss='categorical_crossentropy')

model_copy= keras.models.clone_model(model1)
for j in range(0, image_size):
      model_copy.fit(sample[j], sample_lbl[j])
      prediction= model_copy.predict(sample[j])

Also, it is not really efficient for me to load the model each time in the for-loop since that is time-consuming. How can I do properly the deep copy in my case? The code I posted give the following error that concerns the function .fit and my reference model model_copy:

RuntimeError: You must compile a model before training/testing. Use model.compile(optimizer, loss).

Asked By: Jose Ramon

||

Source

Answer 1

The issue is that model_copy is probably not compiled after cloning. There are in fact a few issues:

Apparently cloning doesn’t copy over the loss function, optimizer info, etc.
Before compiling you need to also build the model.
Moreover, cloning doesn’t copy weight over

So you need a couple extra lines after cloning. For example, for 10 input variables:

model_copy= keras.models.clone_model(model1)
model_copy.build((None, 10)) # replace 10 with number of variables in input layer
model_copy.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model_copy.set_weights(model.get_weights())

Easier Method 1: Loading weights from file

If I understand your question correctly, there is an easier way to do this. You don’t need to clone the model, just need to save the old_weights and set the weights at beginning of the loop. You can simply load weights from file as you are doing.

for _ in range(10):
    model1= create_Model()
    model1.compile(optimizer='rmsprop', loss='categorical_crossentropy')
    model1.load_weights('my_weights')

    for j in range(0, image_size):
          model1.fit(sample[j], sample_lbl[j])
          prediction= model1.predict(sample[j])

Easier Method 2: Loading weights from previous get_weights()

Or if you prefer not to load from file:

model1= create_Model()
model1.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model1.load_weights('my_weights')
old_weights = model1.get_weights()

for _ in range(10):
    model1.set_weights(old_weights)
    for j in range(0, image_size):
          model1.fit(sample[j], sample_lbl[j])
          prediction= model1.predict(sample[j])

Answered By: Tim

Answer 2

These days it’s trivial:

model2 = tf.keras.models.clone_model(model1)

This will give you a new model, new layers, and new weights. If for some reason that doesn’t work (I haven’t tested it) this older solution will:

model1 = Model(...)
model1.compile(...)
model1.save(savepath) # saves compiled state
model2 = keras.models.load_model(savepath)

Answered By: markemus

Answer 3

A very general method to get deep copies in python is deepcopy from the copy package:

import copy
model2=copy.deepcopy(model)

Are there any disadvantages in using this for keras models?

Edit: As pointed out in the comment by GRASBOCK, this solution (copy.deepcopy) does not work reliably for tf-models: https://stackoverflow.com/a/64427748/5044463.

Answered By: Jakob

Make a deep copy of a keras model in python

Question:

Answers:

Easier Method 1: Loading weights from file

Easier Method 2: Loading weights from previous get_weights()