Saving and Loading Tensorflow Model Results in Keras Error

Question

For a project that I’m working on, I have created a simple model in TensorFlow that consists of a dense features layer followed by three dense layers.

def build_model(arguments):
    model = tf.keras.Sequential([
        tf.keras.layers.DenseFeatures(arguments),
        tf.keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dense(5, activation='sigmoid')
    ])
    
    return model

I am unable to go into more detail about the parameter arguments, but the above model function works perfectly fine and can train and save a .h5 file perfectly fine using the code below.

# Create a path for the saving location of the model
model_dir = log_dir + "model.h5"

# Save the model
model.save(model_dir)

However, when I try to load the model back from the .h5 file,

model = tf.keras.models.load_model(model_path)

I get the following error message.

  File "sampleModel.py", line 342, in <module>
    model = tf.keras.models.load_model(model_path)
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkerassavingsave.py", line 1
82, in load_model
    return hdf5_format.load_model_from_hdf5(filepath, custom_objects, c
ompile)
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkerassavinghdf5_format.py",
 line 178, in load_model_from_hdf5
    custom_objects=custom_objects)
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkerassavingmodel_config.py"
, line 55, in model_from_config
    return deserialize(config, custom_objects=custom_objects)
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkeraslayersserialization.py
", line 175, in deserialize
    printable_module_name='layer')
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkerasutilsgeneric_utils.py"
, line 358, in deserialize_keras_object
    list(custom_objects.items())))
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkerasenginesequential.py",
line 487, in from_config
    custom_objects=custom_objects)
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkeraslayersserialization.py
", line 175, in deserialize
    printable_module_name='layer')
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkerasutilsgeneric_utils.py"
, line 358, in deserialize_keras_object
    list(custom_objects.items())))
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonkerasfeature_columnbase_fea
ture_layer.py", line 141, in from_config
    config['feature_columns'], custom_objects=custom_objects)
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonfeature_columnserialization.
py", line 186, in deserialize_feature_columns
    for c in configs
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonfeature_columnserialization.
py", line 186, in <listcomp>
    for c in configs
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonfeature_columnserialization.
py", line 138, in deserialize_feature_column
    columns_by_name=columns_by_name)
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonfeature_columnfeature_column
_v2.py", line 2622, in from_config
    config['normalizer_fn'], custom_objects=custom_objects)
  File "C:WINDOWSsystem32configsystemprofileAppDataRoamingPython
Python37site-packagestensorflowpythonfeature_columnserialization.
py", line 273, in _deserialize_keras_object
    obj = module_objects.get(object_name)
AttributeError: 'NoneType' object has no attribute 'get'

Looking around, I suspect it has something to do with the custom_objects tag for the load_model function, but I am not 100% sure of how to implement it.

The only custom objects that I could be using are my loss, which I declare below, and the accuracy metric that I use.

loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

Asked By: Aaron Jones

||

Source

Answer 1

Assuming your "arguments" is a list of feature_columns which you have created.
Assuming you have not created any custom training objects.
Use the TF format for saving the model.
This is tested for TF 2.3

model.save("./my_model/",save_format="tf")

For Loading model:

model2 = tf.keras.models.load_model('/content/my_model')

Answered By: Aniket Bote

Answer 2

After looking around some more and digging through some Github issues, I believe I’ve figured out a way around the issue. For my specific situation, I didn’t need to save the entire model, rather just the weights. For my configuration I’m using Tensorflow 2.3.0 and Keras 2.4.3.

SHORT ANSWER:

Fit your model for at least one epoch, then load in the weights.

LONG ANSWER:

To save the weights, I use the following function appended with my model file path above it.

# Create a path for the saving location of the model
model_dir = dir_path + '/model.h5'

# Save the model
model.save_weights(model_dir)

I first build my model from my question above and store it in a model object

model = build_model(arguments)

I add my loss function and optimizer, then compile my model to make sure it has all relevant features before loading in the weights.

loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
    
#Declare and set the parametors of the optimizer
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001, decay=0.001)
    
#Compile the model
model.compile(loss=loss_object, optimizer=optimizer, metrics=['accuracy'])

I found my answer from this line here, but at the very bottom it says to fit the model for 1 epoch before loading in the weights.

history = model.fit(test_data, batch_size=1, epochs=1)

Afterwards, you should be able to load in the weights just fine using the function below.

model.load_weights(model_path)

Answered By: Aaron Jones

Saving and Loading Tensorflow Model Results in Keras Error

Question:

Answers: