How can I efficiently concatenate images in a Tensorflow dataset?

Question

I currently have sixteen images (A,B,C,D,E,F,G,…) which must be concatenated into one as part of a Tensorflow Dataset workflow. Each image is 128 x 128 and has the shape of (128, 128, 3). The final output should be a 512 x 512 image of shape (512,512,3). All of the images come from an image sequence, known as img_seq. This img_seq has the shape of (None, 128, 128, 3)

Right now, this is accomplished through the following code:

@tf.function
def glue_to_one(imgs_seq):
    first_row= tf.concat((imgs_seq[0], imgs_seq[1],imgs_seq[2],imgs_seq[3]), 0)
    second_row = tf.concat((imgs_seq[4], imgs_seq[5], imgs_seq[6], imgs_seq[7]), 0)
    third_row = tf.concat((imgs_seq[8], imgs_seq[9], imgs_seq[10], imgs_seq[11]), 0)
    fourth_row = tf.concat((imgs_seq[12], imgs_seq[13], imgs_seq[14], imgs_seq[15]), 0)

    img_glue = tf.stack((first_row, second_row, third_row, fourth_row), axis=1)
    img_glue = tf.reshape(img_glue, [512,512,3])

    return img_glue

It is suspected that this method is inefficient and is learning to a bottleneck.
A different approach would be to allocate a 512 x 512 tensor and then replace the elements. Would this be more efficient? How would it be done? Can you please recommend a better approach?

Asked By: Tom Lin

||

Source

Answer 1

You can improve it about 3 times using something like this:

def glue_answer(imgs_seq):
    image = tf.reshape(imgs_seq, (4, 4, 128, 128, 3))
    image = tf.concat(image, axis=1)
    image = tf.concat(image, axis=1)
    
    return image

I tested the performance as follows:

def glue_to_one(imgs_seq):
    first_row= tf.concat((imgs_seq[0], imgs_seq[1],imgs_seq[2],imgs_seq[3]), 0)
    second_row = tf.concat((imgs_seq[4], imgs_seq[5], imgs_seq[6], imgs_seq[7]), 0)
    third_row = tf.concat((imgs_seq[8], imgs_seq[9], imgs_seq[10], imgs_seq[11]), 0)
    fourth_row = tf.concat((imgs_seq[12], imgs_seq[13], imgs_seq[14], imgs_seq[15]), 0)

    img_glue = tf.stack((first_row, second_row, third_row, fourth_row), axis=1)
    img_glue = tf.reshape(img_glue, [512,512,3])

    return img_glue

def glue_answer(imgs_seq):
    image = tf.reshape(imgs_seq, (4, 4, 128, 128, 3))
    image = tf.concat(image, axis=1)
    image = tf.concat(image, axis=1)
    
    return image

print("Method in question:")
%timeit -n 1000 -r 10 glue_to_one(imgs_seq)
print("Method in answe:")
%timeit -n 1000 -r 10 glue_answer(imgs_seq)

Output:

Method in question:
1.7 ms ± 212 µs per loop (mean ± std. dev. of 10 runs, 1,000 loops each)
Method in answe:
540 µs ± 28.8 µs per loop (mean ± std. dev. of 10 runs, 1,000 loops each)

Answered By: D.Manasreh

Answer 2

Simply use tf.split method instead of writing that much code…,

**Your Inputs seems to be a list of inputs**

def stack_and_concat(x):
    t = tf.split(x , 16 , axis=0)

    t = tf.reshape(tf.stack([tf.concat(t[(i*4): 4 * (i+1)] , axis=1) for i in range(4)],axis=2) , (512,512,3))
    
    return t

concat_inputs(x).shape

TensorShape([512, 512, 3])

for thousand iterations my method took 3.28 secs but your’s took 10.35 secs

Answered By: Mohammad Ahmed

Answer 3

tf.squeeze(tf.concat(tf.split(tf.concat(imgs_seq, axis=0),4),axis=1))

Testing:

imgs_seq = [ tf.random.normal(shape=(128,128,3)) for _ in range(16)]
out1 = glue_to_one(imgs_seq)
out2 = tf.squeeze(tf.concat(tf.split(tf.concat(imgs_seq, axis=0),4),axis=1))

#check whether both outputs are same.
np.testing.assert_allclose(out1.numpy(), out2.numpy())

%timeit glue_to_one(imgs_seq)
1.52 ms ± 123 µs per loop 
   
%timeit tf.squeeze(tf.concat(tf.split(tf.concat(imgs_seq,axis=0),4),axis=1))   
308 µs ± 13.6 µs per loop

Answered By: V.M

How can I efficiently concatenate images in a Tensorflow dataset?

Question:

Answers: