What features do glitched images have that I could detect?

Question

I’m trying to build a footage filter that only sends only "good" frames to the database.

Here is my current rating function:

def rateImg(img):
    try:
        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    except:
        gray = img
    edges = cv2.Canny(gray, 0, 255)
    countours, _ = cv2.findContours(
        edges, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
    num_of_countours = len(countours)
    lap = cv2.Laplacian(gray, cv2.CV_64F).var()
    lap = round(lap, 2)
    return [lap, num_of_countours]

First off, I use variance of Laplacian to calculate the sharpness of an image from a particular time window.

It should technically provide me a "good" frame, but that’s not always the case.

The camera I have to use isn’t great and sometimes glitches out like this and frames like this have the highest variance of Laplacian.

So, my current solution is to calculate the number of countours in an image and if an image crosses a particular threshold I classify it as "glitched". But with this approach the algorithm rates images with a lot of objects as "glitched".

Also, I have tried detecting squares and rectangles, but that proved to be much less effective than the countour approach.

Is there any way to detect obvious glitches in an image?
I feel like there should be, because as a human I can easily classify glitched and normal images at a glance. I just can’t seem to pin-point what exactly makes them different.

Asked By: Jonas

||

Source

Answer 1

Is there any way to detect obvious glitches in an image?

Yes, but probably not for complex random glitches, have a look in this similar question

In that case, you can detect if there is a large area of the image containing the same color. Photo taken from the camera would never contain the same RGB value although they look similar. However, this would be perfectly normal if the images are arts drawn on a digital devices.

As a human I can easily classify glitched and normal images at a
glance… What exactly makes them (me and a program) different. Is there any way to detect obvious glitches in an image

In fact, you can’t identify a glitched image. You try to recognize the objects in it. When you see something "weird" that you don’t recognize, you consider it as a glitched image. The machine can neither achieve this. You can train an AI that report images with unrecognizable "parts" as glitched but it will never be 100% accurate

Answered By: imamangoo

Answer 2

Converting your image to HSV and runnign the Brightness Channel through an edge filter on ImageJ gives me this:

As you can see, the "glitched" region appears pretty uniformly brighter then the rest of the image, and should be detectable in some form. How often do you get a picture from you camera ? Depending on how much change occurs between two pictures, you might get away with subtracting the current one from the one before it to just look at changes.

You have not shown what an image with

a lot of objects

looks like, so you’d have to try if this works for those cases.

OpenCV functions needed for this workflow would be:

cvtColor() with COLOR_BGR2GRAY for color conversion (there might be faster ways to get a good greyvalue then HSV)
one of the edge detectors. Canny() or Sobel() would be the first i’d try
some greyvalue statistics. threshold() and CountNonZero() for a simple approach, which you could refine for sectors on the image or such

PS:

I feel like there should be, because as a human I can easily classify
glitched and normal images at a glance.

That’s a common fallacy: Us humans (the sight-centric beings that we are) are fantastic at pattern recognition and interpolation and are rarely aware how much of that (including a lot of error correction) is happening every microsecond. A puny 2D camera can not hope to live up to that. (obligatory XKCD: https://xkcd.com/1425/)

Answered By: nick

What features do glitched images have that I could detect?

Question:

Answers: