How to get a correct output predictions from unet_learner (fastai)?
Question:
Please, I’m working on an image segmentation project and I used the fastai library (specifically the unet_learner). I’ve trained my model and very thing is fine here is my code (in the training phase):
#codes = np.loadtxt('codes.txt', dtype=str)
codes = np.array(['bg', 'edge'], dtype='<U4')# bg= background
get_y_fn = lambda x: path_lbl/f'{x.stem}{x.suffix}'
# fastai codes
data = (SegmentationItemList.from_folder(path_img)
.split_by_rand_pct()
.label_from_func(get_y_fn, classes=codes)
#.add_test_folder()
#.transform(get_transforms(), tfm_y=True, size=384)
.databunch(bs=2,path=dataset) # bs = mimi-patch size
.normalize(imagenet_stats))
learn = unet_learner(data, models.resnet34, wd=1e-2)
learn.lr_find() # find learning rate
learn.recorder.plot() # plot learning rate graph
lr = 1e-02 # pick a lr
learn.fit_one_cycle(3, slice(lr), pct_start=0.3) # train model ---- epochs=3
learn.unfreeze() # unfreeze all layers
# find and plot lr again
learn.lr_find()
learn.recorder.plot()
learn.fit_one_cycle(10, slice(lr/400, lr/4), pct_start=0.3)
learn.save('model-stage-1') # save model
learn.load('model-stage-1');
learn.export()
My problem is when I try to make a prediction using the trained model, the output is always a black image. Below is the code In the prediction phase:
img = open_image('/content/generated_samples_masks/545.png')
prediction = learn.predict(img)
prediction[0].show(figsize=(8,8))
Please, any ideas on how to fix this issue? Thanks
Answers:
I think the prediction is ok. Are you expecting something like this?
This result is based on your posted prediction image.
To check how things are going, try this:
interp = SegmentationInterpretation.from_learner(learn)
mean_cm, single_img_cm = interp._generate_confusion()
df = interp._plot_intersect_cm(mean_cm, "Mean of Ratio of Intersection given
True Label")
i = 0 #Some image index
df = interp._plot_intersect_cm(single_img_cm[i], f"Ratio of Intersection given True Label, Image:{i}")
interp.show_xyz(i)
About your prediction result, it’s an image based in your classes values. If you take the (r,g,b) values from this image, you have (r, g, b) == 0
for your background and (r, g, b) == 1
for edges. If you have more classes, the next one will be as (r, g, b) == 2
and so on.
So you can just colorize your prediction result. I did it using OpenCV, something like this:
frame = cv2.imread("yourPredictionHere.png",1)
frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
for x in range(384): #width based on the size of your image.
for y in range(384): #height based on the size of your image.
b, g, r = frame[x, y]
if (b, g, r) == (0,0,0): #background
frame[x, y] = (0,0,0)
elif (b, g, r) == (1,1,1): #edges
frame[x, y] = (85,85,255)
cv2.imwrite("result.png",frame)
Best regards!
To view the prediction of unet_learner
overlayed over original image, you can do:
img = open_image("your_test_image.png")
prediction = learn.predict(img)
img.show(y=prediction[0])
Here is an example in fastai docs: https://docs.fast.ai/tutorial.inference.html#A-segmentation-example.
Please, I’m working on an image segmentation project and I used the fastai library (specifically the unet_learner). I’ve trained my model and very thing is fine here is my code (in the training phase):
#codes = np.loadtxt('codes.txt', dtype=str)
codes = np.array(['bg', 'edge'], dtype='<U4')# bg= background
get_y_fn = lambda x: path_lbl/f'{x.stem}{x.suffix}'
# fastai codes
data = (SegmentationItemList.from_folder(path_img)
.split_by_rand_pct()
.label_from_func(get_y_fn, classes=codes)
#.add_test_folder()
#.transform(get_transforms(), tfm_y=True, size=384)
.databunch(bs=2,path=dataset) # bs = mimi-patch size
.normalize(imagenet_stats))
learn = unet_learner(data, models.resnet34, wd=1e-2)
learn.lr_find() # find learning rate
learn.recorder.plot() # plot learning rate graph
lr = 1e-02 # pick a lr
learn.fit_one_cycle(3, slice(lr), pct_start=0.3) # train model ---- epochs=3
learn.unfreeze() # unfreeze all layers
# find and plot lr again
learn.lr_find()
learn.recorder.plot()
learn.fit_one_cycle(10, slice(lr/400, lr/4), pct_start=0.3)
learn.save('model-stage-1') # save model
learn.load('model-stage-1');
learn.export()
My problem is when I try to make a prediction using the trained model, the output is always a black image. Below is the code In the prediction phase:
img = open_image('/content/generated_samples_masks/545.png')
prediction = learn.predict(img)
prediction[0].show(figsize=(8,8))
Please, any ideas on how to fix this issue? Thanks
I think the prediction is ok. Are you expecting something like this?
This result is based on your posted prediction image.
To check how things are going, try this:
interp = SegmentationInterpretation.from_learner(learn)
mean_cm, single_img_cm = interp._generate_confusion()
df = interp._plot_intersect_cm(mean_cm, "Mean of Ratio of Intersection given
True Label")
i = 0 #Some image index
df = interp._plot_intersect_cm(single_img_cm[i], f"Ratio of Intersection given True Label, Image:{i}")
interp.show_xyz(i)
About your prediction result, it’s an image based in your classes values. If you take the (r,g,b) values from this image, you have (r, g, b) == 0
for your background and (r, g, b) == 1
for edges. If you have more classes, the next one will be as (r, g, b) == 2
and so on.
So you can just colorize your prediction result. I did it using OpenCV, something like this:
frame = cv2.imread("yourPredictionHere.png",1)
frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
for x in range(384): #width based on the size of your image.
for y in range(384): #height based on the size of your image.
b, g, r = frame[x, y]
if (b, g, r) == (0,0,0): #background
frame[x, y] = (0,0,0)
elif (b, g, r) == (1,1,1): #edges
frame[x, y] = (85,85,255)
cv2.imwrite("result.png",frame)
Best regards!
To view the prediction of unet_learner
overlayed over original image, you can do:
img = open_image("your_test_image.png")
prediction = learn.predict(img)
img.show(y=prediction[0])
Here is an example in fastai docs: https://docs.fast.ai/tutorial.inference.html#A-segmentation-example.