Replace instance of a class with new instance

Question

I am working on detectron2 object detection. I am facing a problem in filtering the objects detected.

Here is the detectron2 predicted output:

Instances(num_instances=9, image_height=547, image_width=820, fields=[pred_boxes: Boxes(tensor([[3.1173e+01, 3.8368e+01, 5.3751e+02, 5.4078e+02],
    [5.9945e+02, 2.6412e+02, 6.8196e+02, 5.1333e+02],
    [4.4486e+02, 1.7210e+02, 4.9981e+02, 2.5596e+02],
    [1.1566e-01, 2.3533e+02, 8.5483e+01, 3.6838e+02],
    [3.0897e+02, 2.4964e+02, 3.5739e+02, 4.8948e+02],
    [7.6962e-03, 2.3240e+02, 8.5447e+01, 3.7128e+02],
    [2.7454e+02, 2.6212e+02, 3.3122e+02, 4.5928e+02],
    [6.4399e+02, 3.0057e+02, 6.6374e+02, 3.8033e+02],
    [3.1025e+02, 2.5372e+02, 3.3572e+02, 3.5059e+02]])), scores: tensor([0.9998, 0.9994, 0.9941, 0.8815, 0.8447, 0.3559, 0.1484, 0.1304, 0.0928]), pred_classes: tensor([ 0,  0, 67,  2, 27,  7, 27, 27, 27]), pred_masks: tensor([[[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    ...,

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]]])])

I did the filtering and created a new list(dict) with predicted object classes, scores and boxes. I wanted to plot and visualize this on image:

Filtering code:

idxofClass = [i for i, x in enumerate(list(outputs['instances'].pred_classes)) if (x == 0)]
outputs_new = [{'pred_classes': o.pred_classes[idxofClass], 'scores':o.scores[idxofClass], 'pred_boxes':o.pred_boxes[idxofClass] }]

Now, I able to get the filtered values as below:

[{'pred_classes': tensor([ 0,  0, 67]), 'scores': tensor([0.9998, 0.9994, 0.9941]), 'pred_boxes': Boxes(tensor([[ 31.1728,  38.3685, 537.5092, 540.7788],
    [599.4498, 264.1228, 681.9622, 513.3326],
    [444.8603, 172.1017, 499.8055, 255.9632]]))}]

While passing this value to Visualizer, getting the below error:

Traceback (most recent call last):
  File "apimodel.py", line 96, in <module>
    out = v.draw_instance_predictions(outputs_new)
  File "/root/anaconda3/envs/ml-engine/lib/python3.8/site-packages/detectron2/utils/visualizer.py", line 366, in draw_instance_predictions
    boxes = predictions.pred_boxes if predictions.has("pred_boxes") else None
AttributeError: 'list' object has no attribute 'has'

The data type of original output is a class instance:

o = outputs["instances"]
print("data type:", type(o))
<class 'detectron2.structures.instances.Instances'>

The output of newly created filtered output is a list(dict):

<class 'list'>

My objective is to plot the bounding box based on filtered score. I have been trying to replace original values of output, but not successful. Please assist on this.

Asked By: Rathish Kumar B

||

Source

Answer 1

After two days of searching, I found a way to achieve my objective. I am writing answer, as detectron2 class, so that, if anyone looking for similar approach will get benefit.

Filter Index of classes:

idxofClass = [i for i, x in enumerate(list(outputs['instances'].pred_classes)) if x == 0]

Create new class, boxes, scores & masks:

classes = o.pred_classes[idxofClass]
scores = o.scores[idxofClass]
boxes = o.pred_boxes[idxofClass]
masks = o.pred_masks[idxofClass]

Define new instance and set the new values to new instance. Note: detectron2 module provides this method set.

obj = detectron2.structures.Instances(image_size=(480, 640))

obj.set('pred_classes', classes)
obj.set('scores', scores)
obj.set('pred_boxes', boxes)
obj.set('pred_masks', masks)

Now you can use this new instance obj for other processing and visualization:

out = v.draw_instance_predictions(obj.to("cpu"))

Answered By: Rathish Kumar B

Answer 2

Here is an efficient way to create a new Instances object after applying a transformation on the previous Instances object’s data. In this case, I’m applying a score threshold to filter out boxes with low prediction scores below 0.5.

I’m using torch transformations to create indices used for filtering. Also, the new Instances class can be initialized with all fields set.

instances = predictor(img)["instances"]
score_threshold = 0.5

filter_mask = instances.scores > score_threshold
indices = torch.nonzero(filter_mask).flatten().tolist()
filtered_instances = Instances(
    image_size=instances.image_size,
    pred_classes=instances.pred_classes[indices],
    scores=instances.scores[indices],
    pred_boxes=instances.pred_boxes[indices],
    pred_masks=instances.pred_masks[indices],
)

Answered By: BushMinusZero

Replace instance of a class with new instance

Question:

Answers: