Python Image Processing on Captcha how to remove noise

Question

I am so new on Image Processing and what I’m trying to do is clearing the noise from captchas;

For captchas, I have different types of them:

For the first one what I did is :

Firstly, I converted every pixel that is not black to the black. Then, I found a pattern that is a noise from the image and deleted it. For the first captcha, it was easy to clear it and I found the text with tesseract.

But I am looking for a solution for the second and the third.

How this must go like? I mean what are the possible methods to clear it?

This is how I delete patterns:

def delete(searcher,h2,w2):
    h = h2
    w = w2
    search = searcher
    search = search.convert("RGBA")
    herear = np.asarray(search)
    bigar  = np.asarray(imgCropped)

    hereary, herearx = herear.shape[:2]
    bigary,  bigarx  = bigar.shape[:2]

    stopx = bigarx - herearx + 1
    stopy = bigary - hereary + 1

    pix = imgCropped.load()

    for x in range(0, stopx):
        for y in range(0, stopy):
            x2 = x + herearx
            y2 = y + hereary
            pic = bigar[y:y2, x:x2]
            test = (pic == herear)
            if test.all():
                for q in range(h):
                    for k in range(w):
                        pix[x+k,y+q] = (255,255,255,255)

Sorry for the variable names, I was just testing function.

Thanks..

Asked By: Ahmet Aziz Beşli

||

Source

Answer 1

This is as far as I can get:

You probably know about medianBlur function which finds the median value in every kernel and substitute that value to kernel’s center. We can do something similar to that but instead of the median, use the max value then the min value. With a median bluring too, I got some results. I know they are not perfect but I hope it gives you some ideas ( you can play with the sizes of the input image and the kernels, it may make the results a little better).

I don’t have python installed right now, so I share the exact C++ code that I have used:

Mat im1 = imread("E:/1/3.jpg", 0);
Mat im2, im3;

im2 = Mat::zeros(im1.size(), CV_8U);
for (size_t i = 1; i < im1.rows-1; i++)
{
    for (size_t j = 1; j < im1.cols-1; j++)
    {
        double minVal, maxVal = 0;
        minMaxIdx(im1(Rect(j - 1, i - 1, 3, 3)), &minVal, &maxVal);
        im2.at<uchar>(i, j) = maxVal;
    }
}

imshow("(1) max bluring", im2);

medianBlur(im2, im2, 3);

imshow("(2) median bluring", im2);

im2.copyTo(im1);

im2 = Mat::zeros(im1.size(), CV_8U);
for (size_t i = 1; i < im1.rows - 1; i++)
{
    for (size_t j = 1; j < im1.cols - 1; j++)
    {
        double minVal, maxVal = 0;
        minMaxIdx(im1(Rect(j - 1, i - 1, 3, 3)), &minVal, &maxVal);
        im2.at<uchar>(i, j) = minVal;
    }
}

imshow("(3) min bluring", im2);

Mat tmp;
double st = threshold(im2, tmp, 10, 255, THRESH_OTSU);
threshold(im2, im2, st + 14, 255, THRESH_BINARY_INV);
//dilate(im2, im2, Mat::ones(3, 3, CV_8U));

imshow("(4) final", im2);

waitKey(0);

By the way in such cases, deep Learning methods like YOLO and RCNN are the best methods. Try them too.

Answered By: MeiH

Answer 2

You can use opencv library for image processing. Very usefull could be this opencv documentation page. Then try to extract your number through findCountour method like:

import cv2 
import numpy as np 

image = cv2.imread('C:\E0snN.png')
cv2.waitKey(0) 

# Grayscale 
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) 
gray = cv2.bitwise_not(gray)

# Threshold
ret,thresh = cv2.threshold(gray,150,255,1)
# Get countours
contours,h = cv2.findContours(thresh,1,2)
# Draw
cv2.drawContours(image, contours, -1, (0, 255, 0), 3) 
cv2.imshow('Contours', image) 
cv2.waitKey(0)

cv2.destroyAllWindows()

After that there is following result:

It’s far from perfect but if you try with different threshold values e.g. :

ret,thresh = cv2.threshold(gray,127,255,1)

you can get better results.

Answered By: ElConrado

Answer 3

Here is my solution,

Firstly I got the background pattern(Edited on paint by hand). From:

After that, I created a blank image to fill it with differences between the pattern and image.

img = Image.open("x.png").convert("RGBA")
pattern = Image.open("y.png").convert("RGBA")

pixels = img.load()
pixelsPattern = pattern.load()

new = Image.new("RGBA", (150, 50))
pixelNew = new.load()

for i in range(img.size[0]):
    for j in range(img.size[1]):
         if(pixels[i,j] != pixelsPattern[i,j]):
             pixelNew[i,j] = pixels[i,j]

new.save("differences.png")

Here are the differences..

and finally, I added blur and cleared the bits which are not black.

Result :

With pytesseract result is 2041, it is wrong for this image but the general rate is around %60.

Answered By: Ahmet Aziz Beşli

Python Image Processing on Captcha how to remove noise

Question:

Answers: