How to get region properties from image that is already labeled in OpenCV?
Question:
I am labeling images using the watershed algorithm in OpenCV (similar to this tutorial: https://docs.opencv.org/3.4/d3/db4/tutorial_py_watershed.html) such that at the end I obtain an array of labels where each region has an integer value corresponding to its label. Now, I want to obtain the coordinates of the bounding boxes and areas of each region.
I know this is easily done with skimage.measure.regionprops() but for considerations of speed of execution I would like to achieve this without importing skimage, ideally directly with OpenCV.
I have tried using cv2.connectedComponentsWithStats() but it only seems to work if the image is binary not if the labels are already defined.
I have tried to binarize the labeled image and then relabel it with connectedComponentsWithStats() as follows (note that the background has a label of 1 in this case and I want to remove it):
segmented = cv2.watershed(image.astype('uint8'), markers)
segmented_bin = segmented.copy()
segmented_bin[segmented < 2] = 0
segmented_bin[segmented > 1] = 255
num_labels, label_image, stats, centroids = cv2.connectedComponentsWithStats(segmented_bin.astype('uint8'), 4, cv2.CV_32S)
But this approach merges regions that are not separated by background which is not the desired effect.
Essentially I would like to know if there is a function similar to connectedComponentsWithStats() that deals with already labeled images?
Answers:
Since (if I’m not mistaken) each label represents a continuous area, we can just iterate over all the non-background labels.
for i in range(2, marker_count + 1):
For each label, we can create a corresponding binary mask (pixels with that label become 255, everything else becomes a 0) using numpy.where.
mask = np.where(segmented==i, np.uint8(255), np.uint8(0))
Since cv2.boundingRect can process single-channel images as well, we can use it to determine the bounding box directly from the mask.
x,y,w,h = cv2.boundingRect(mask)
The area of the label is simply the count of the pixels with given label (i.e. all non-zero pixels in the mask). We can simply use cv2.countNonZero for that. Since we already know the bounding box, we can save some work by processing only the corresponding ROI.
area = cv2.countNonZero(mask[y:y+h,x:x+w])
And we’re done.
print "Label %d at (%d, %d) size (%d x %d) area %d pixels" % (i,x,y,w,h,area)
Console output
Label 2 at (41, 14) size (47 x 49) area 1747 pixels
Label 3 at (111, 30) size (48 x 47) area 1719 pixels
Label 4 at (71, 51) size (56 x 48) area 1716 pixels
Label 5 at (152, 61) size (48 x 47) area 1676 pixels
Label 6 at (25, 75) size (47 x 48) area 1719 pixels
Label 7 at (109, 76) size (49 x 49) area 1748 pixels
Label 8 at (192, 82) size (49 x 48) area 1774 pixels
Label 9 at (64, 97) size (48 x 49) area 1695 pixels
Label 10 at (1, 114) size (47 x 48) area 1720 pixels
Label 11 at (139, 114) size (52 x 48) area 1727 pixels
Label 12 at (97, 132) size (48 x 48) area 1745 pixels
Label 13 at (181, 133) size (48 x 47) area 1667 pixels
Label 14 at (41, 140) size (47 x 48) area 1733 pixels
Label 15 at (129, 167) size (45 x 47) area 1666 pixels
Label 16 at (5, 169) size (50 x 48) area 1713 pixels
Label 17 at (72, 176) size (46 x 48) area 1745 pixels
Label 18 at (171, 177) size (50 x 49) area 1772 pixels
Label 19 at (35, 205) size (46 x 47) area 1702 pixels
Label 20 at (106, 207) size (55 x 49) area 1909 pixels
Label 21 at (155, 219) size (43 x 47) area 1537 pixels
Label 22 at (65, 237) size (51 x 48) area 1713 pixels
Label 23 at (25, 251) size (50 x 49) area 1818 pixels
Label 24 at (108, 264) size (48 x 47) area 1730 pixels
Label 25 at (155, 264) size (46 x 47) area 1711 pixels
Images
- Input
- Colored labels
- Labeled bounding boxes
The complete script
import numpy as np
import cv2
# START of original watershed example
# from https://docs.opencv.org/3.4/d3/db4/tutorial_py_watershed.html
img = cv2.imread('water_coins.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
# noise removal
kernel = np.ones((3,3),np.uint8)
opening = cv2.morphologyEx(thresh,cv2.MORPH_OPEN,kernel, iterations = 2)
# sure background area
sure_bg = cv2.dilate(opening,kernel,iterations=3)
# Finding sure foreground area
dist_transform = cv2.distanceTransform(opening,cv2.DIST_L2,5)
ret, sure_fg = cv2.threshold(dist_transform,0.7*dist_transform.max(),255,0)
# Finding unknown region
sure_fg = np.uint8(sure_fg)
unknown = cv2.subtract(sure_bg,sure_fg)
# Marker labelling
marker_count, markers = cv2.connectedComponents(sure_fg)
# Add one to all labels so that sure background is not 0, but 1
markers = markers+1
# Now, mark the region of unknown with zero
markers[unknown==255] = 0
segmented = cv2.watershed(img,markers)
# END of original watershed example
output = np.zeros_like(img)
output2 = img.copy()
# Iterate over all non-background labels
for i in range(2, marker_count + 1):
mask = np.where(segmented==i, np.uint8(255), np.uint8(0))
x,y,w,h = cv2.boundingRect(mask)
area = cv2.countNonZero(mask[y:y+h,x:x+w])
print "Label %d at (%d, %d) size (%d x %d) area %d pixels" % (i,x,y,w,h,area)
# Visualize
color = np.uint8(np.random.random_integers(0, 255, 3)).tolist()
output[mask!=0] = color
cv2.rectangle(output2, (x,y), (x+w,y+h), color, 1)
cv2.putText(output2,'%d'%i,(x+w/4, y+h/2), cv2.FONT_HERSHEY_SIMPLEX, 0.4, color, 1, cv2.LINE_AA)
cv2.imwrite('wshseg_colors.png', output)
cv2.imwrite('wshseg_boxes.png', output2)
If anybody else is interested, I ended up reverting to skimage.measure.regionprops() since I could not get cv2.connectedComponentsWithStats(). The time overhead is only in the tens of millisecond per image.
Like the object we had in this image is of coin but what if the we had irregular shaped object so the boundigrect() would not work well in that case?
I am labeling images using the watershed algorithm in OpenCV (similar to this tutorial: https://docs.opencv.org/3.4/d3/db4/tutorial_py_watershed.html) such that at the end I obtain an array of labels where each region has an integer value corresponding to its label. Now, I want to obtain the coordinates of the bounding boxes and areas of each region.
I know this is easily done with skimage.measure.regionprops() but for considerations of speed of execution I would like to achieve this without importing skimage, ideally directly with OpenCV.
I have tried using cv2.connectedComponentsWithStats() but it only seems to work if the image is binary not if the labels are already defined.
I have tried to binarize the labeled image and then relabel it with connectedComponentsWithStats() as follows (note that the background has a label of 1 in this case and I want to remove it):
segmented = cv2.watershed(image.astype('uint8'), markers)
segmented_bin = segmented.copy()
segmented_bin[segmented < 2] = 0
segmented_bin[segmented > 1] = 255
num_labels, label_image, stats, centroids = cv2.connectedComponentsWithStats(segmented_bin.astype('uint8'), 4, cv2.CV_32S)
But this approach merges regions that are not separated by background which is not the desired effect.
Essentially I would like to know if there is a function similar to connectedComponentsWithStats() that deals with already labeled images?
Since (if I’m not mistaken) each label represents a continuous area, we can just iterate over all the non-background labels.
for i in range(2, marker_count + 1):
For each label, we can create a corresponding binary mask (pixels with that label become 255, everything else becomes a 0) using numpy.where.
mask = np.where(segmented==i, np.uint8(255), np.uint8(0))
Since cv2.boundingRect can process single-channel images as well, we can use it to determine the bounding box directly from the mask.
x,y,w,h = cv2.boundingRect(mask)
The area of the label is simply the count of the pixels with given label (i.e. all non-zero pixels in the mask). We can simply use cv2.countNonZero for that. Since we already know the bounding box, we can save some work by processing only the corresponding ROI.
area = cv2.countNonZero(mask[y:y+h,x:x+w])
And we’re done.
print "Label %d at (%d, %d) size (%d x %d) area %d pixels" % (i,x,y,w,h,area)
Console output
Label 2 at (41, 14) size (47 x 49) area 1747 pixels
Label 3 at (111, 30) size (48 x 47) area 1719 pixels
Label 4 at (71, 51) size (56 x 48) area 1716 pixels
Label 5 at (152, 61) size (48 x 47) area 1676 pixels
Label 6 at (25, 75) size (47 x 48) area 1719 pixels
Label 7 at (109, 76) size (49 x 49) area 1748 pixels
Label 8 at (192, 82) size (49 x 48) area 1774 pixels
Label 9 at (64, 97) size (48 x 49) area 1695 pixels
Label 10 at (1, 114) size (47 x 48) area 1720 pixels
Label 11 at (139, 114) size (52 x 48) area 1727 pixels
Label 12 at (97, 132) size (48 x 48) area 1745 pixels
Label 13 at (181, 133) size (48 x 47) area 1667 pixels
Label 14 at (41, 140) size (47 x 48) area 1733 pixels
Label 15 at (129, 167) size (45 x 47) area 1666 pixels
Label 16 at (5, 169) size (50 x 48) area 1713 pixels
Label 17 at (72, 176) size (46 x 48) area 1745 pixels
Label 18 at (171, 177) size (50 x 49) area 1772 pixels
Label 19 at (35, 205) size (46 x 47) area 1702 pixels
Label 20 at (106, 207) size (55 x 49) area 1909 pixels
Label 21 at (155, 219) size (43 x 47) area 1537 pixels
Label 22 at (65, 237) size (51 x 48) area 1713 pixels
Label 23 at (25, 251) size (50 x 49) area 1818 pixels
Label 24 at (108, 264) size (48 x 47) area 1730 pixels
Label 25 at (155, 264) size (46 x 47) area 1711 pixels
Images
- Input
- Colored labels
- Labeled bounding boxes
The complete script
import numpy as np
import cv2
# START of original watershed example
# from https://docs.opencv.org/3.4/d3/db4/tutorial_py_watershed.html
img = cv2.imread('water_coins.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
# noise removal
kernel = np.ones((3,3),np.uint8)
opening = cv2.morphologyEx(thresh,cv2.MORPH_OPEN,kernel, iterations = 2)
# sure background area
sure_bg = cv2.dilate(opening,kernel,iterations=3)
# Finding sure foreground area
dist_transform = cv2.distanceTransform(opening,cv2.DIST_L2,5)
ret, sure_fg = cv2.threshold(dist_transform,0.7*dist_transform.max(),255,0)
# Finding unknown region
sure_fg = np.uint8(sure_fg)
unknown = cv2.subtract(sure_bg,sure_fg)
# Marker labelling
marker_count, markers = cv2.connectedComponents(sure_fg)
# Add one to all labels so that sure background is not 0, but 1
markers = markers+1
# Now, mark the region of unknown with zero
markers[unknown==255] = 0
segmented = cv2.watershed(img,markers)
# END of original watershed example
output = np.zeros_like(img)
output2 = img.copy()
# Iterate over all non-background labels
for i in range(2, marker_count + 1):
mask = np.where(segmented==i, np.uint8(255), np.uint8(0))
x,y,w,h = cv2.boundingRect(mask)
area = cv2.countNonZero(mask[y:y+h,x:x+w])
print "Label %d at (%d, %d) size (%d x %d) area %d pixels" % (i,x,y,w,h,area)
# Visualize
color = np.uint8(np.random.random_integers(0, 255, 3)).tolist()
output[mask!=0] = color
cv2.rectangle(output2, (x,y), (x+w,y+h), color, 1)
cv2.putText(output2,'%d'%i,(x+w/4, y+h/2), cv2.FONT_HERSHEY_SIMPLEX, 0.4, color, 1, cv2.LINE_AA)
cv2.imwrite('wshseg_colors.png', output)
cv2.imwrite('wshseg_boxes.png', output2)
If anybody else is interested, I ended up reverting to skimage.measure.regionprops() since I could not get cv2.connectedComponentsWithStats(). The time overhead is only in the tens of millisecond per image.
Like the object we had in this image is of coin but what if the we had irregular shaped object so the boundigrect() would not work well in that case?