tf.image.decode_jpeg – contents must be scalar, got shape [1]
Question:
I have build a server/client demo for image classification by tensorflow serving, following this tutorial
https://github.com/tmlabonte/tendies/blob/master/minimum_working_example/tendies-basic-tutorial.ipynb
The Client
It accepts an image as input, convert it to Base64, pass it to the server using JSON
input_image = open(image, "rb").read()
print("Raw bitstring: " + str(input_image[:10]) + " ... " + str(input_image[-10:]))
# Encode image in b64
encoded_input_string = base64.b64encode(input_image)
input_string = encoded_input_string.decode("utf-8")
print("Base64 encoded string: " + input_string[:10] + " ... " + input_string[-10:])
# Wrap bitstring in JSON
instance = [{"images": input_string}]
data = json.dumps({"instances": instance})
print(data[:30] + " ... " + data[-10:])
r = requests.post('http://localhost:9000/v1/models/cnn:predict', data=data)
#json.loads(r.content)
print(r.text)
The Server
Once loaded the model as .h5 the server must be saved as SavedModel.
the image must pass from the client to the server as a Base64 encoded string.
model=tf.keras.models.load_model('./model.h5')
input_bytes = tf.placeholder(tf.string, shape=[], name="input_bytes")
# input_bytes = tf.reshape(input_bytes, [])
# Transform bitstring to uint8 tensor
input_tensor = tf.image.decode_jpeg(input_bytes, channels=3)
# Convert to float32 tensor
input_tensor = tf.image.convert_image_dtype(input_tensor, dtype=tf.float32)
input_tensor = input_tensor / 127.5 - 1.0
# Ensure tensor has correct shape
input_tensor = tf.reshape(input_tensor, [64, 64, 3])
# CycleGAN's inference function accepts a batch of images
# So expand the single tensor into a batch of 1
input_tensor = tf.expand_dims(input_tensor, 0)
# x = model.input
y = model(input_tensor)
then the input_bytes become the input for the predition_signature in the SavedModel
tensor_info_x = tf.saved_model.utils.build_tensor_info(input_bytes)
At the end the server result like:
ยง saved_model_cli show --dir ./ --all
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['images'] tensor_info:
dtype: DT_STRING
shape: ()
name: input_bytes:0
The given SavedModel SignatureDef contains the following output(s):
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (1, 4)
name: sequential_1/dense_2/Softmax:0
Method name is: tensorflow/serving/predict
Sending Image
When I send the image base64 I received a run-time error from the server about the shape of the input that seems not scalar:
Using TensorFlow backend.
Raw bitstring: b'xffxd8xffxe0x00x10JFIF' ... b'0;sxcfJ(xa0hxffxd9'
Base64 encoded string: /9j/4AAQSk ... 9KKKBo/9k=
{"instances": [{"images": "/9j ... Bo/9k="}]}
{ "error": "contents must be scalar, got shape [1]nt [[{{node DecodeJpeg}} = DecodeJpeg[_output_shapes=[[?,?,3]], acceptable_fraction=1, channels=3, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_bytes_0_0)]]" }
As you see from the server the input_bytes is scalar as the shape=[], I have also tried to reshape it with tf.reshape(input_bytes, []) but no way, I got always the same error.
I did not find any solution in internet and here in Stackoverflow about this error. Can you please suggest how to fix it?
Thanks!
Answers:
I solved the issue and I would like to comment how so you can benefit of the solution!
When you send a json like this:
{"instances": [{"images": "/9j ... Bo/9k="}]}
actually you are sending an array of size 1 as you put the []
in case you would like to send 2 images you should write like that
{"instances": [{"images": "/9j ... Bo/9k="}, {"images": "/9j ... Bo/9k="}]}
here the size is 2 (shape = [2])
so the solution is to state in the placeholder to accept any type of size with shape=[None]
input_bytes = tf.placeholder(tf.string, shape=[None], name="input_bytes")
then if you are sending only 1 image the vector 1 can be converted to a scalar by:
input_scalar = tf.reshape(input_bytes, [])
Also there were another error in my code, I did not consider that in tensorflow/serving there is a feature to decode the base64 by explicitly stating ‘b64’ in the json please refere to RESTful API Encoding binary values, so if you send
{"instances": [{"images": {"b64": "/9j ... Bo/9k="}}]}
the server will automatically decode the base64 input and the correct bit-stream will reach the tf.image.decode_jpeg
input_bytes = tf.placeholder(tf.string, shape=[], name="input_bytes")
input_tensor = tf.image.decode_jpeg(input_bytes, channels=3)
the "tf.image.decode_jpeg" can only take scalar
I have build a server/client demo for image classification by tensorflow serving, following this tutorial
https://github.com/tmlabonte/tendies/blob/master/minimum_working_example/tendies-basic-tutorial.ipynb
The Client
It accepts an image as input, convert it to Base64, pass it to the server using JSON
input_image = open(image, "rb").read()
print("Raw bitstring: " + str(input_image[:10]) + " ... " + str(input_image[-10:]))
# Encode image in b64
encoded_input_string = base64.b64encode(input_image)
input_string = encoded_input_string.decode("utf-8")
print("Base64 encoded string: " + input_string[:10] + " ... " + input_string[-10:])
# Wrap bitstring in JSON
instance = [{"images": input_string}]
data = json.dumps({"instances": instance})
print(data[:30] + " ... " + data[-10:])
r = requests.post('http://localhost:9000/v1/models/cnn:predict', data=data)
#json.loads(r.content)
print(r.text)
The Server
Once loaded the model as .h5 the server must be saved as SavedModel.
the image must pass from the client to the server as a Base64 encoded string.
model=tf.keras.models.load_model('./model.h5')
input_bytes = tf.placeholder(tf.string, shape=[], name="input_bytes")
# input_bytes = tf.reshape(input_bytes, [])
# Transform bitstring to uint8 tensor
input_tensor = tf.image.decode_jpeg(input_bytes, channels=3)
# Convert to float32 tensor
input_tensor = tf.image.convert_image_dtype(input_tensor, dtype=tf.float32)
input_tensor = input_tensor / 127.5 - 1.0
# Ensure tensor has correct shape
input_tensor = tf.reshape(input_tensor, [64, 64, 3])
# CycleGAN's inference function accepts a batch of images
# So expand the single tensor into a batch of 1
input_tensor = tf.expand_dims(input_tensor, 0)
# x = model.input
y = model(input_tensor)
then the input_bytes become the input for the predition_signature in the SavedModel
tensor_info_x = tf.saved_model.utils.build_tensor_info(input_bytes)
At the end the server result like:
ยง saved_model_cli show --dir ./ --all
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['images'] tensor_info:
dtype: DT_STRING
shape: ()
name: input_bytes:0
The given SavedModel SignatureDef contains the following output(s):
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (1, 4)
name: sequential_1/dense_2/Softmax:0
Method name is: tensorflow/serving/predict
Sending Image
When I send the image base64 I received a run-time error from the server about the shape of the input that seems not scalar:
Using TensorFlow backend.
Raw bitstring: b'xffxd8xffxe0x00x10JFIF' ... b'0;sxcfJ(xa0hxffxd9'
Base64 encoded string: /9j/4AAQSk ... 9KKKBo/9k=
{"instances": [{"images": "/9j ... Bo/9k="}]}
{ "error": "contents must be scalar, got shape [1]nt [[{{node DecodeJpeg}} = DecodeJpeg[_output_shapes=[[?,?,3]], acceptable_fraction=1, channels=3, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_bytes_0_0)]]" }
As you see from the server the input_bytes is scalar as the shape=[], I have also tried to reshape it with tf.reshape(input_bytes, []) but no way, I got always the same error.
I did not find any solution in internet and here in Stackoverflow about this error. Can you please suggest how to fix it?
Thanks!
I solved the issue and I would like to comment how so you can benefit of the solution!
When you send a json like this:
{"instances": [{"images": "/9j ... Bo/9k="}]}
actually you are sending an array of size 1 as you put the []
in case you would like to send 2 images you should write like that
{"instances": [{"images": "/9j ... Bo/9k="}, {"images": "/9j ... Bo/9k="}]}
here the size is 2 (shape = [2])
so the solution is to state in the placeholder to accept any type of size with shape=[None]
input_bytes = tf.placeholder(tf.string, shape=[None], name="input_bytes")
then if you are sending only 1 image the vector 1 can be converted to a scalar by:
input_scalar = tf.reshape(input_bytes, [])
Also there were another error in my code, I did not consider that in tensorflow/serving there is a feature to decode the base64 by explicitly stating ‘b64’ in the json please refere to RESTful API Encoding binary values, so if you send
{"instances": [{"images": {"b64": "/9j ... Bo/9k="}}]}
the server will automatically decode the base64 input and the correct bit-stream will reach the tf.image.decode_jpeg
input_bytes = tf.placeholder(tf.string, shape=[], name="input_bytes")
input_tensor = tf.image.decode_jpeg(input_bytes, channels=3)
the "tf.image.decode_jpeg" can only take scalar