WARNING:tensorflow:sample_weight modes were coerced from … to ['…']
Question:
Training an image classifier using .fit_generator()
or .fit()
and passing a dictionary to class_weight=
as an argument.
I never got errors in TF1.x but in 2.1 I get the following output when starting training:
WARNING:tensorflow:sample_weight modes were coerced from
...
to
['...']
What does it mean to coerce something from ...
to ['...']
?
The source for this warning on tensorflow
‘s repo is here, comments placed are:
Attempt to coerce sample_weight_modes to the target structure. This implicitly depends on the fact that Model flattens outputs for its internal representation.
Answers:
I have taken your Gist and installed Tensorflow 2.0, instead of TFA and it worked without any such Warning.
Here is the Gist of the complete code. Code for installing the Tensorflow is shown below:
!pip install tensorflow==2.0
Screenshot of the successful execution is shown below:
Update: This bug is fixed in Tensorflow Version 2.2.
This seems like a bogus message. I get the same warning message after upgrading to TensorFlow 2.1, but I do not use any class weights or sample weights at all. I do use a generator that returns a tuple like this:
return inputs, targets
And now I just changed it to the following to make the warning go away:
return inputs, targets, [None]
I don’t know if this is relevant, but my model uses 3 inputs, so my inputs
variable is actually a list of 3 numpy arrays. targets
is just a single numpy array.
In any case, it’s just a warning. The training works fine either way.
Edit for TensorFlow 2.2:
This bug seems to have been fixed in TensorFlow 2.2, which is great. However the fix above will fail in TF 2.2, because it will try to get the shape of the sample weights, which will obviously fail with AttributeError: 'NoneType' object has no attribute 'shape'
. So undo the above fix when upgrading to 2.2.
I believe this is a bug with tensorflow that will happen when you call model.compile()
with default parameter sample_weight_mode=None
and then call model.fit()
with specified sample_weight
or class_weight
.
From the tensorflow repos:
fit()
eventually calls _process_training_inputs()
_process_training_inputs()
sets sample_weight_modes = [None]
based on model.sample_weight_mode = None
and then creates a DataAdapter
with sample_weight_modes = [None]
- the
DataAdapter
calls broadcast_sample_weight_modes()
with sample_weight_modes = [None]
during initialization
broadcast_sample_weight_modes()
seems to expect sample_weight_modes = None
but receives [None]
- it asserts that
[None]
is a different structure from sample_weight
/ class_weight
, overwrites it back to None
by fitting to the structure of sample_weight
/ class_weight
and outputs a warning
Warning aside this has no effect on fit()
as sample_weight_modes
in the DataAdapter
is set back to None
.
Note that tensorflow documentation states that sample_weight
must be a numpy-array. If you call fit()
with sample_weight.tolist()
instead, you will not get a warning but sample_weight
is silently overwritten to None
when _process_numpy_inputs()
is called in preprocessing and receives an input of length greater than one.
instead of providing a dictionary
weights = {'0': 42.0, '1': 1.0}
i tried a list
weights = [42.0, 1.0]
and the warning disappeared.
Training an image classifier using .fit_generator()
or .fit()
and passing a dictionary to class_weight=
as an argument.
I never got errors in TF1.x but in 2.1 I get the following output when starting training:
WARNING:tensorflow:sample_weight modes were coerced from
...
to
['...']
What does it mean to coerce something from ...
to ['...']
?
The source for this warning on tensorflow
‘s repo is here, comments placed are:
Attempt to coerce sample_weight_modes to the target structure. This implicitly depends on the fact that Model flattens outputs for its internal representation.
I have taken your Gist and installed Tensorflow 2.0, instead of TFA and it worked without any such Warning.
Here is the Gist of the complete code. Code for installing the Tensorflow is shown below:
!pip install tensorflow==2.0
Screenshot of the successful execution is shown below:
Update: This bug is fixed in Tensorflow Version 2.2.
This seems like a bogus message. I get the same warning message after upgrading to TensorFlow 2.1, but I do not use any class weights or sample weights at all. I do use a generator that returns a tuple like this:
return inputs, targets
And now I just changed it to the following to make the warning go away:
return inputs, targets, [None]
I don’t know if this is relevant, but my model uses 3 inputs, so my inputs
variable is actually a list of 3 numpy arrays. targets
is just a single numpy array.
In any case, it’s just a warning. The training works fine either way.
Edit for TensorFlow 2.2:
This bug seems to have been fixed in TensorFlow 2.2, which is great. However the fix above will fail in TF 2.2, because it will try to get the shape of the sample weights, which will obviously fail with AttributeError: 'NoneType' object has no attribute 'shape'
. So undo the above fix when upgrading to 2.2.
I believe this is a bug with tensorflow that will happen when you call model.compile()
with default parameter sample_weight_mode=None
and then call model.fit()
with specified sample_weight
or class_weight
.
From the tensorflow repos:
fit()
eventually calls_process_training_inputs()
_process_training_inputs()
setssample_weight_modes = [None]
based onmodel.sample_weight_mode = None
and then creates aDataAdapter
withsample_weight_modes = [None]
- the
DataAdapter
callsbroadcast_sample_weight_modes()
withsample_weight_modes = [None]
during initialization broadcast_sample_weight_modes()
seems to expectsample_weight_modes = None
but receives[None]
- it asserts that
[None]
is a different structure fromsample_weight
/class_weight
, overwrites it back toNone
by fitting to the structure ofsample_weight
/class_weight
and outputs a warning
Warning aside this has no effect on fit()
as sample_weight_modes
in the DataAdapter
is set back to None
.
Note that tensorflow documentation states that sample_weight
must be a numpy-array. If you call fit()
with sample_weight.tolist()
instead, you will not get a warning but sample_weight
is silently overwritten to None
when _process_numpy_inputs()
is called in preprocessing and receives an input of length greater than one.
instead of providing a dictionary
weights = {'0': 42.0, '1': 1.0}
i tried a list
weights = [42.0, 1.0]
and the warning disappeared.