How to replace cropped rectangle in opencv?

Question

I have managed to cropped a bounding box with text, e.g. given this image:

I’m able to exact the following box:

with this code:

import re
import shutil

from IPython.display import Image

import requests
import pytesseract, cv2

"""https://www.geeksforgeeks.org/text-detection-and-extraction-using-opencv-and-ocr/"""
# Preprocessing the image starts
# Convert the image to gray scale
img = cv2.imread('img.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Performing OTSU threshold
ret, thresh1 = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU | cv2.THRESH_BINARY_INV)

# Specify structure shape and kernel size.
# Kernel size increases or decreases the area
# of the rectangle to be detected.
# A smaller value like (10, 10) will detect
# each word instead of a sentence.
rect_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (18, 18))

# Applying dilation on the threshold image
dilation = cv2.dilate(thresh1, rect_kernel, iterations = 1)

# Finding contours
contours, hierarchy = cv2.findContours(dilation, cv2.RETR_EXTERNAL,
                                                 cv2.CHAIN_APPROX_NONE)

# Creating a copy of image
im2 = img.copy()


for cnt in contours:
    x, y, w, h = cv2.boundingRect(cnt)
    # Drawing a rectangle on copied image
    rect = cv2.rectangle(im2, (x, y), (x + w, y + h), (0, 255, 0), 2)
    # Cropping the text block for giving input to OCR
    cropped = im2[y:y + h, x:x + w]
    
cv2.imwrite('image-notxt.png', cropped)
Image(filename='image-notxt.png',  width=200)

Part 1: How do I replace the cropped box and put back a clear text box? e.g. to get something like:

I’ve tried:

    for cnt in contours:
        x, y, w, h = cv2.boundingRect(cnt)
        # Drawing a rectangle on copied image
        rect = cv2.rectangle(im2, (x, y), (x + w, y + h), (0, 255, 0), 2)
        # Cropping the text block for giving input to OCR
        cropped = im2[y:y + h, x:x + w]
        text = pytesseract.image_to_string(cropped).strip('x0c').strip()
        text = re.sub(' +', ' ', text.replace('n', ' ')).strip()
        if text:
            # White out the cropped box.
            cropped.fill(255)
            # Create the image with the translation.
            cv2.putText(img=cropped, text="foobar", org=(12, 15), fontFace=cv2.FONT_HERSHEY_TRIPLEX, fontScale=0.3, color=(0, 0, 0),thickness=1)
            cv2.imwrite('image-notxt.png', cropped)
            Image(filename='image-notxt.png',  width=200)

That managed to white out the cropped box and insert the text like this:

Part 2: How to create an opencv textbox rectangle with the same size as the cropped box? e.g. given a string foobar, how to get the final image like this:

Asked By: alvas

||

Source

Answer 1

In Python/OpenCV/Numpy, use Numpy to write a color to the area in the format:

img[y:y+h, x:x+w] = color tuple

For example:

# Part 1 --- Blank Area

# read image
photo = cv2.imread('Wurst.png')

# define region arguments
x,y,w,h = 40,40,150,45

# blank area
result1 = photo.copy()
result1[40:40+45, 40:40+150] = (255,255,255)

# save results
cv2.imwrite("Wurst_blank_result.png", result1)

# show the results
cv2.imshow("blanked result", result1)
cv2.waitKey(0)

# Part 2 --- Write text

# define text arguments
text = "FOOBAR"
thickness = 2
scale = .8
pad = 2

# determine size for text image
(wd, ht), baseLine = cv2.getTextSize(text, cv2.FONT_HERSHEY_SIMPLEX, scale, thickness)
print (wd, ht, baseLine)

# add black text to white background image padded all around
th = ht + 2 * pad
tw = wd + 2 * pad
text_img = np.full((th,tw,3), (255,255,255), dtype=np.uint8)
text_img = cv2.putText(text_img, text, (pad,ht+pad), cv2.FONT_HERSHEY_SIMPLEX, scale, (0,0,0), thickness)
print(text_img.shape)

# insert text image into region of photo
if w>=tw and h>=th:
    # compute offset
    offx = int((w-tw)/2)
    offy = int((h-th)/2)
    # combine the text with the image
    result2 = result1.copy()
    result2[y+offy:y+offy+th, x+offx:x+offx+tw] = text_img
    # save results
    cv2.imwrite("Wurst_text_result.png", result2)
    # show the results
    cv2.imshow("text image", text_img)
    cv2.imshow("text result", result2)
    cv2.waitKey(0)
else:
    print("Text Is Too Large")

See cv2.getTextSize() at https://docs.opencv.org/4.1.1/d6/d6e/group__imgproc__draw.html#ga3d2abfcb995fd2db908c8288199dba82 and cv2.putText() at https://docs.opencv.org/4.1.1/d6/d6e/group__imgproc__draw.html#ga5126f47f883d730f633d74f07456c576

Answered By: fmw42

Answer 2

Part 1: How do I replace the cropped box and put back a clear text box?

After cropping out the box, fill it up with:

cropped.fill(255)

That will produce

Part 2: How to create an opencv textbox rectangle with the same size as the cropped box?

Putting in the text, it’s a little more nuance, but the steps are first:

Create the image with the text in it using cv2.putText()
But there are multiple things that
- length and font of the text that you want to put in and if they fit in the box
- location/position to put the text in the box

TL;DR

for i, chunk in enumerate(textwrap.wrap(translation, width=20)):
    cv2.putText(img=cropped, text=chunk, org=(12, 15+i*10), 
         fontFace=cv2.FONT_HERSHEY_TRIPLEX, fontScale=0.3, 
         color=(0, 0, 0),thickness=1)
    im2[y:y + h, x:x + w] = cropped

To handle the length of the text, I’ve to use the Python textwrap library to break the string into multiple substrings

Then iterating through the substrings, I putText each of the substring into the cropped image.

Finally, replace the portion of the original image with the edited cropped image with the text putted into it like im2[y:y + h, x:x + w] = cropped

A working example can be found on https://www.kaggle.com/code/alvations/image-translate

Answered By: alvas

How to replace cropped rectangle in opencv?

Question:

Answers:

Part 1: How do I replace the cropped box and put back a clear text box?

Part 2: How to create an opencv textbox rectangle with the same size as the cropped box?

TL;DR