np.concatenate dimension problem of 1 & 3 channel image input

Question:

I’m trying to train a future 7days broccoli Unet model

I try to concatenate images, because my input have 8 channel(51channel 13channel[index 5])

appear this error message

        img_batch = np.concatenate(images, axis=3)________line172
        mask_batch = np.concatenate(masks, axis=2)________line173
        y_int = np.argmax(mask_batch, axis=2)  
        y_binary = to_categorical(y_int)  
        yield (img_batch, y_binary ,mask_batch )

error message

發生例外狀況: ValueError
all the input array dimensions for the concatenation axis must match exactly, but along dimension 3, the array at index 0 has size 1 and the array at index 5 has size 3
  File "C:Labbbunet_mao0data.py", line 173, in trainGenerator
    mask_batch = np.concatenate(masks, axis=2)
  File "C:Labbbunet_mao0main.py", line 88, in <module>
    train_history=model.fit_generator(myGene,steps_per_epoch=200,epochs=20,callbacks=[model_checkpoint])
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 3, the array at index 0 has size 1 and the array at index 5 has size 3

I think because my index5-image is 3 channel lead to error message happened,but I’m sure every mask image is grayscale, how to fix it?

before img, mask = adjustData,I print mask’s channels

Mask 1 channels: 1
Mask 2 channels: 1
Mask 3 channels: 1
Mask 4 channels: 1
Mask 5 channels: 1
Mask 6 channels: 3
 for i in range(len(image_folder)):
            img, mask = data[i], data[i + len(image_folder)]
            print("Mask", i+1, "channels:", mask.shape[-1])_______________print channels
            img, mask = adjustData(img, mask, flag_multi_class, num_class)
            images.append(img)

            masks.append(mask)
        img_batch = np.concatenate(images, axis=3)
        mask_batch = np.concatenate(masks, axis=2)

at full code, I set image_color_mode="rgb", mask_color_mode="grayscale" in traingenerator

Full code

def trainGenerator(batch_size, train_paths, image_folder, mask_folder, aug_dict,image_color_mode="rgb", mask_color_mode="grayscale", 
                   image_save_prefix="image", mask_save_prefix="mask", 
                   flag_multi_class=False, num_class=6, save_to_dir="data_train/test/", target_size=(512, 512), seed=1):

    image_datagen = ImageDataGenerator(**aug_dict)
    mask_datagen = ImageDataGenerator(**aug_dict)
    image_generators = []
    mask_generators = []
    for train_path in train_paths:
        
        image_color_mode = "rgb" if os.path.basename(train_path) == "clahe" else "grayscale"
        
        image_generator = image_datagen.flow_from_directory(
            train_path,
            classes=image_folder,
            class_mode=None,
            color_mode=image_color_mode,
            target_size=target_size,
            batch_size=batch_size,
            save_to_dir=save_to_dir,
            save_prefix=image_save_prefix,
            seed=seed
        )
        mask_generator = mask_datagen.flow_from_directory(
            train_path,
            classes=mask_folder,
            class_mode=None,
            color_mode=mask_color_mode,
            target_size=target_size,
            batch_size=batch_size,
            save_to_dir=save_to_dir,
            save_prefix=mask_save_prefix,
            seed=seed
        )
        image_generators.append(image_generator)
        mask_generators.append(mask_generator)

    train_generators = zip(*image_generators, *mask_generators)
    for data in train_generators:
        images = []
        masks = []
        ###for i in range(len(train_paths)):
        ###    img, mask = data[i], data[i + len(train_paths)]
        for i in range(len(image_folder)):
            img, mask = data[i], data[i + len(image_folder)]
            img, mask = adjustData(img, mask, flag_multi_class, num_class)
            images.append(img)
            
            print("Mask", i+1, "channels:", mask.shape[-1])
            
            masks.append(mask)
        img_batch = np.concatenate(images, axis=3)
        mask_batch = np.concatenate(masks, axis=2)
        y_int = np.argmax(mask_batch, axis=2)  # Assuming mask_batch has shape (batch_size, height, width, num_classes)
        y_binary = to_categorical(y_int)  # Convert integer labels to binary vectors
        yield (img_batch, y_binary ,mask_batch )
        #yield (img_batch, y_binary)
Asked By: Syuuuu

||

Answers:

I think the problem is that there are masks of different sizes, but np.concatenate requires np.arays in the same dimension.

For example:
This case will work well

np.concatenate([np.random.randint(0,10, size=(2,3,4))] 
               + [np.random.randint(0,10, size=(2,3,4))], axis=2)

But this one will show the same error that you have:

np.concatenate([np.random.randint(0,10, size=(2,3,4))] 
               + [np.random.randint(0,10, size=(1,3,4))], axis=2)

I would recommend to print

print("Mask", i+1, "channels:", mask.shape)

and deep dive into function adjustData for those samples where mask have different shape

Answered By: MaryRa