tf.data.Dataset does not fetch images from file path using function map

Tensorflow throws an error if I use tf.data.Dataset and .map to retrieve images from a file path.

Reproducible example on Google colab below. I have also attached notebook if anyone wants (you’d have to copy to your drive)

import tensorflow as tf
import PIL.Image
import numpy as np

data = np.random.randint(0,255,(28, 28, 3), dtype=np.uint8)
img = PIL.Image.fromarray(data, 'RGB')
img.save('abc.png')
img.save('pqr.png')

Now once I have these images, I am able to normal operations using a tensorflow dataset, but if I try to use this path to fetch an image, it throws a TypeError: expected str, bytes or os.PathLike object, not Tensor error

For instance, if I try to replace some values in the string (here I am just replacing "abc" with "xyz"), it works fine

def fn(x1,x2):
  if tf.strings.regex_full_match(x1[0],'.*abc.*'):
    return (tf.strings.regex_replace(x1[0], "abc", "xyz"),x1[1]),x2

  return x1,x2

aa = ['/content/abc.png','/content/abc.png','/content/pqr.png','/content/pqr.png']
bb = [1,2,3,4]
cc = [1,2,3,4]

xx = tf.data.Dataset.from_tensor_slices(((aa,bb),cc))

for x in xx.take(-1):
  print(x)

print('#--------')

xx = xx.map(fn)

for x in xx.take(-1):
  print(x)

>>>
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/abc.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=1>), <tf.Tensor: shape=(), dtype=int32, numpy=1>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/abc.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=2>), <tf.Tensor: shape=(), dtype=int32, numpy=2>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/pqr.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=3>), <tf.Tensor: shape=(), dtype=int32, numpy=3>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/pqr.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=4>), <tf.Tensor: shape=(), dtype=int32, numpy=4>)
#--------
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/xyz.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=1>), <tf.Tensor: shape=(), dtype=int32, numpy=1>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/xyz.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=2>), <tf.Tensor: shape=(), dtype=int32, numpy=2>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/pqr.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=3>), <tf.Tensor: shape=(), dtype=int32, numpy=3>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/pqr.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=4>), <tf.Tensor: shape=(), dtype=int32, numpy=4>)

But if I try to use these paths to fetch the image, the following happens

def fn(x1,x2):  
  if tf.strings.regex_full_match(x1[0],'.*png'):
    img = tf.keras.preprocessing.image.load_img(x1[0])
    img  = tf.keras.preprocessing.image.img_to_array(img)
    img = tf.cast(img,dtype=tf.float32)
    img = img / 255.
  return (img,x1[1]),x2


aa = ['/content/abc.png','/content/abc.png','/content/pqr.png','/content/pqr.png']
bb = [1,2,3,4]
cc = [1,2,3,4]

xx = tf.data.Dataset.from_tensor_slices(((aa,bb),cc))

for x in xx.take(-1):
  print(x)

print('#--------')

xx = xx.map(fn)
for x in xx.take(-1):
  print(x)
>>>((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/abc.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=1>), <tf.Tensor: shape=(), dtype=int32, numpy=1>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/abc.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=2>), <tf.Tensor: shape=(), dtype=int32, numpy=2>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/pqr.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=3>), <tf.Tensor: shape=(), dtype=int32, numpy=3>)
((<tf.Tensor: shape=(), dtype=string, numpy=b'/content/pqr.png'>, <tf.Tensor: shape=(), dtype=int32, numpy=4>), <tf.Tensor: shape=(), dtype=int32, numpy=4>)
#--------
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-6e29d6482ac7> in <module>
     19 print('#--------')
     20 
---> 21 xx = xx.map(fn)
     22 for x in xx.take(-1):
     23   print(x)

10 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py in wrapper(*args, **kwargs)
    690       except Exception as e:  # pylint_disable=broad-except
    691         if hasattr(e, 'ag_error_metadata'):
--> 692           raise e.ag_error_metadata.to_exception(e)
    693         else:
    694           raise

TypeError: in user code:

    File "<ipython-input-6-6e29d6482ac7>", line 3, in fn  *
        img = tf.keras.preprocessing.image.load_img(x1[0])
    File "/usr/local/lib/python3.7/dist-packages/keras/preprocessing/image.py", line 314, in load_img
        target_size=target_size, interpolation=interpolation)
    File "/usr/local/lib/python3.7/dist-packages/keras_preprocessing/image/utils.py", line 113, in load_img
        with open(path, 'rb') as f:

    TypeError: expected str, bytes or os.PathLike object, not Tensor

This seems to because if you print the file path, you get Tensor("args_0:0", shape=(), dtype=string). I understand that the map function is not executed eagerly so it doesn’t print specific values. But I’m not able to figure out why I’m not able to use the string to do anything.

I tried using img = tf.keras.preprocessing.image.load_img(x1[0].numpy()), but this just throws an error saying AttributeError: 'Tensor' object has no attribute 'numpy'

PS: I understand I can use functions like flow_from_directory, but I need to combine images with text and other numerical outputs, and using file paths and tf.data.Dataset is the easiest thing.