Image Processing: how to imwarp with simple mask on destination?

Question

Following my own question from 4 years ago, this time in Python only-

I am looking for a way to perform texture mapping into a small region in a destination image, defined by 4 corners given as (x, y) pixel coordinates. This region is not necessarily rectangular. It is a perspective projection of some rectangle onto the image plane.

I would like to map some (rectangular) texture into the mask defined by those corners.

Mapping directly by forward-mapping the texture will not work properly, as source pixels will be mapped to non-integer locations in the destination.

This problem is usually solved by inverse-warping from the destination to the source, then coloring according to some interpolation.

Opencv’s warpPerspective doesn’t work here, as it can’t take a mask in.

Inverse-warping the entire destination and then mask is not acceptable because the majority of the computation is redundant.

Is there a built-in opencv (or other) function that accomplishes above requirements?
If not, what is a good way to get a list of pixels from my ROI defined by corners, in favor of passing that to projectPoints?

Example background image:

I want to fill the area outlined by the red lines (defined by its corners) with some other texture, say this one

Mapping between them can be obtained by mapping the texture’s corners to the ROI corners with cv2.getPerspectiveTransform

Asked By: Gulzar

||

Source

Answer 1

For future generations, here is how to only back and forward warp pixels within the bbox of the warped corner points, as @Micka suggested.

here banner is the grass image, and banner_coords_2d are the corners of the red region on image, which is meme-man.

def transform_banner(banner_coords_2d, banner, image):
    # show_points_on_image("banner corners", image, banner_coords_2d)

    banner_height, banner_width, _ = banner.shape
    src_banner_points = np.float32([
        [0, 0],
        [banner_width - 1, 0],
        [0, banner_height - 1],
        [banner_width - 1, banner_height - 1],
    ])

    # only warp to size of bbox of warped corners, not all of the image
    warped_left = np.round(np.min(banner_coords_2d[:, 0])).astype(int)
    warped_right = np.round(np.max(banner_coords_2d[:, 0])).astype(int)
    warped_top = np.round(np.min(banner_coords_2d[:, 1])).astype(int)
    warped_bottom = np.round(np.max(banner_coords_2d[:, 1])).astype(int)
    warped_width = int(warped_right - warped_left)
    warped_height = int(warped_bottom - warped_top)

    dst_banner_points = banner_coords_2d.astype(np.float32)
    dst_banner_points[:, 0] -= warped_left
    dst_banner_points[:, 1] -= warped_top

    tform = cv2.getPerspectiveTransform(src_banner_points, dst_banner_points)

    warped_banner = cv2.warpPerspective(banner, tform, (warped_width, warped_height))
    # cv2.imshow("warped_banner", warped_banner)
    image_with_banner = image.copy()
    image_with_banner[warped_top: warped_bottom, warped_left: warped_right][warped_banner != 0] = warped_banner[
        warped_banner != 0]
    # cv2.imshow("image_with_banner", image_with_banner)

    return image_with_banner

Likely, this can be done more neatly, I am open to edits.

Answered By: Gulzar

Image Processing: how to imwarp with simple mask on destination?

Question:

Answers: