yolo v8: does segment contain point?

Question

I’m using yolo v8 to detect subjects in pictures. It’s working well, and can create quite precise masks over subjects.

from ultralytics import YOLO

model = YOLO('yolov8x-seg.pt')

for output in model('image.jpg', return_outputs=True):
    for segment in output['segment']:
        print(segment)

The code above works, and generates a series of "segments", which are a list of points that define the shape of subjects on my image. That shape is not convex (for example horses).

I need to figure out if a random coordinate on the image falls within these segments, and I’m not sure how to do it.

My first approach was to build an image mask using PIL. That roughly worked, but it doesn’t always work, depending on the shape of the segments. I also thought about using shapely, but it has restrictions on the Polygon classes, which I think will be a problem in some cases.

In any case, this really feels like a problem that could easily be solved with the tools I’m already using (yolo, pytorch, numpy…), but to be honest I’m too new to all this to figure out how to do it properly.

Any suggestion is appreciated 🙂

Asked By: aspyct

||

Source

Answer 1

You should be able to get a segmentation mask from your model: imagine a binary image where black (zeros) represents the background and white (or other non zero values) represent an instance of a segmentation class.

Once you have the binary image you can use opencv’s findContours function to get a the largest outer path.

Once you have that path you can use pointPolygonTest() to check if a point is inside that contour or not.

Answered By: George Profenza

yolo v8: does segment contain point?

Question:

Answers: