How to obtain (x,y) coordinates of a shape found using mask in OpenCV, Python?
Question:
I am trying to locate certain items on an image. The image in simplified form looks like this:
I would like to obtain the (x,y) coordinates of the bold black text on top of the second rectangle, as well as of the three colored rectangles.
I have the masks ready, except for the mask of the black text that I wasn’t able to figure out. However, the text is always on top of the rectangle so if I’d be able to figure out the position of the bottom large rectangle, I would have the position of the text too.
These are the outputs I got:
I tried using the ConnectedComponents function based on this comment but apart from coloring and grouping the various objects, I didn’t manage to move forward, so I didn’t include that snippet below to make things as clear as possible.
Here is my code so far:
import cv2
import numpy as np
import imutils
PATH = "stackoverflow.png"
img = cv2.imread(PATH)
imgHSV = cv2.cvtColor(img,cv2.COLOR_BGR2HSV)
mask_border = cv2.inRange(imgHSV,np.array([0,0,170]),np.array([0,0,175]))
mask_green = cv2.inRange(imgHSV,np.array([76,221,167]),np.array([76,221,167]))
mask_pink = cv2.inRange(imgHSV,np.array([168,41,245]),np.array([172,41,252]))
mask_red = cv2.inRange(imgHSV,np.array([4,207,251]),np.array([4,207,251]))
#mask_black = ???
all_masks = cv2.bitwise_or(mask_border, mask_green)
all_masks = cv2.bitwise_or(all_masks, mask_pink)
all_masks = cv2.bitwise_or(all_masks, mask_red)
cv2.imshow("Masks", all_masks)
imgResult = cv2.bitwise_and(img,img,mask=all_masks)
cv2.imshow("Output", imgResult)
cv2.waitKey(0)
Answers:
Try out this example: simple blob detector. I would run it on your actual mask, rather than the rgb image since it will work pretty well on a binarized image like your mask. Then pulling the pt and size attributes from the keypoints returned will give you the center and radius of your blobs. (tbh not sure if it will be average or max radius without testing it). I probably wouldn’t even use the radius for your use case though, From the center point of each blob, I would search iteratively in the -x and -y axis (separately), out from that point. At each step I would check if it was a black or white pixel and I would wait until I saw some adjustable variable number of black pixels in a row (basically to avoid triggering on text, but trying to trigger on exit of box and entrance to background). Those coords for x and y will be the upper left hand corner of your box. If you want you can do it in the positive x and y direction too in order to get the full bounding box.
You can binarise image and then apply some morphological operations to get the proper connected components. Here is an approach. You can fine-tune this to get proper output.
import numpy as np
import cv2
import os
image=cv2.imread('path/to/image.jpg')
###binarising
gray=cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
ret2,th2 = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
###applying morphological operations to dilate the image
kernel=np.ones((3,3),np.uint8)
dilated=cv2.dilate(th2,kernel,iterations=3)
### finding contours, can use connectedcomponents aswell
_,contours,_ = cv2.findContours(dilated, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
### converting to bounding boxes from polygon
contours=[cv2.boundingRect(cnt) for cnt in contours]
### drawing rectangle for each contour for visualising
for cnt in contours:
x,y,w,h=cnt
cv2.rectangle(image,(x,y),(x+w,y+h),(0,255,0),2)
Normal binary image
Dilated image
output with detected bounding boxes
I think the easiest method is to apply cv2.ConnectedComponentsWithStats to your mask images. Included in the results are the Centroids, being the center coordinates of each "cluster".
See here: https://stackoverflow.com/a/35854198/221166
You might want to apply Dilation beforehand, to connect inner islands (e.g. in b, d, g characters) with the surrounding rectangle.
(_,contours,_) = cv2.findContours(dilated, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) does not generate the x,y cordinates of every pixel
I am trying to locate certain items on an image. The image in simplified form looks like this:
I would like to obtain the (x,y) coordinates of the bold black text on top of the second rectangle, as well as of the three colored rectangles.
I have the masks ready, except for the mask of the black text that I wasn’t able to figure out. However, the text is always on top of the rectangle so if I’d be able to figure out the position of the bottom large rectangle, I would have the position of the text too.
These are the outputs I got:
I tried using the ConnectedComponents function based on this comment but apart from coloring and grouping the various objects, I didn’t manage to move forward, so I didn’t include that snippet below to make things as clear as possible.
Here is my code so far:
import cv2
import numpy as np
import imutils
PATH = "stackoverflow.png"
img = cv2.imread(PATH)
imgHSV = cv2.cvtColor(img,cv2.COLOR_BGR2HSV)
mask_border = cv2.inRange(imgHSV,np.array([0,0,170]),np.array([0,0,175]))
mask_green = cv2.inRange(imgHSV,np.array([76,221,167]),np.array([76,221,167]))
mask_pink = cv2.inRange(imgHSV,np.array([168,41,245]),np.array([172,41,252]))
mask_red = cv2.inRange(imgHSV,np.array([4,207,251]),np.array([4,207,251]))
#mask_black = ???
all_masks = cv2.bitwise_or(mask_border, mask_green)
all_masks = cv2.bitwise_or(all_masks, mask_pink)
all_masks = cv2.bitwise_or(all_masks, mask_red)
cv2.imshow("Masks", all_masks)
imgResult = cv2.bitwise_and(img,img,mask=all_masks)
cv2.imshow("Output", imgResult)
cv2.waitKey(0)
Try out this example: simple blob detector. I would run it on your actual mask, rather than the rgb image since it will work pretty well on a binarized image like your mask. Then pulling the pt and size attributes from the keypoints returned will give you the center and radius of your blobs. (tbh not sure if it will be average or max radius without testing it). I probably wouldn’t even use the radius for your use case though, From the center point of each blob, I would search iteratively in the -x and -y axis (separately), out from that point. At each step I would check if it was a black or white pixel and I would wait until I saw some adjustable variable number of black pixels in a row (basically to avoid triggering on text, but trying to trigger on exit of box and entrance to background). Those coords for x and y will be the upper left hand corner of your box. If you want you can do it in the positive x and y direction too in order to get the full bounding box.
You can binarise image and then apply some morphological operations to get the proper connected components. Here is an approach. You can fine-tune this to get proper output.
import numpy as np
import cv2
import os
image=cv2.imread('path/to/image.jpg')
###binarising
gray=cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
ret2,th2 = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
###applying morphological operations to dilate the image
kernel=np.ones((3,3),np.uint8)
dilated=cv2.dilate(th2,kernel,iterations=3)
### finding contours, can use connectedcomponents aswell
_,contours,_ = cv2.findContours(dilated, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
### converting to bounding boxes from polygon
contours=[cv2.boundingRect(cnt) for cnt in contours]
### drawing rectangle for each contour for visualising
for cnt in contours:
x,y,w,h=cnt
cv2.rectangle(image,(x,y),(x+w,y+h),(0,255,0),2)
Normal binary image
Dilated image
output with detected bounding boxes
I think the easiest method is to apply cv2.ConnectedComponentsWithStats to your mask images. Included in the results are the Centroids, being the center coordinates of each "cluster".
See here: https://stackoverflow.com/a/35854198/221166
You might want to apply Dilation beforehand, to connect inner islands (e.g. in b, d, g characters) with the surrounding rectangle.
(_,contours,_) = cv2.findContours(dilated, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) does not generate the x,y cordinates of every pixel