Fast way to convert string to numpy ndarray

Question

I am currently doing a python program to convert from image to hex string and the other way around. I need two functions, one that takes an image and returns a hex string that corresponds to the RGB values of each pixel, and another function that takes a hex string, two ints, and generates a visible image of that size corresponding to that hex string.

I currently use imageio to get an RGB matrix from the image and then convert that to hex. That one is fast, around 2.5 seconds for a 442KB image of 918 x 575 pixels.

In order to get the image from the string, I convert that to a matrix of hex values and then convert that to RGB to use imageio to create an image. This one is where the problem arises since it takes 36 seconds to do the process on the string corresponding to the same 918 x 575 image.

How could I make it quicker?

Here’s the code:

def rgb2hex(rgb):

    """
    convert a list or tuple of RGB values
    to a string in hex
    """

    r,g,b = rgb
    return '{:02x}{:02x}{:02x}'.format(r, g, b)


def arrayToString(array):
    """
    convert an array to a string
    """

    string = ""
    for element in array:
        string += str(element)

    return string


def sliceStr(string,sliceLenght):
    """
    slice a string in chunks of sliceLenght lenght
    """

    string = str(string)
    array = np.array([string[i:i+sliceLenght] for i in range(0,len(string),sliceLenght)])
    return array



def hexToRGB(hexadecimal):
    """
    convert a hex string to an array of RGB values
    """
    h = hexadecimal.lstrip('#')
    if len(h)!=6:
        return
    return [int(h[i:i+2], 16) for i in (0, 2, 4)]

def ImageToBytes(image):
    """
    Image to convert from image to bytes
    """
    dataToEncrypt =imageio.imread(image)

    if dataToEncrypt.shape[2] ==4:
        dataToEncrypt = np.delete(dataToEncrypt,3,2)

    originalRows, originalColumns,_ = dataToEncrypt.shape


    #converting rgb to hex
    hexVal = np.apply_along_axis(rgb2hex, 2, dataToEncrypt)
    hexVal = np.apply_along_axis(arrayToString, 1, hexVal)
    hexVal = str(np.apply_along_axis(arrayToString, 0, hexVal))

    byteImage = bytes.fromhex(hexVal)

    return (byteImage, [originalRows,originalColumns])

def BytesToImage(byteToConvert,originalRows,originalColumns,name):

    """
    Convert from Bytes to Image
    """
    Data = byteToConvert.hex()


    stepOne = sliceStr(Data,originalColumns*6)

    stepTwo = []

    for i in stepOne:
        step = sliceStr(i,6)

        #Add lost pixels
        while len(step) != originalColumns:
            step = np.append(step,"ffffff")
        stepTwo.append(step)


    stepThree = []
    for i in stepTwo:
        d = []
        for j in i:
            d.append(hexToRGB(j))

        if len(stepThree) < originalRows:
            stepThree.append(d)

        Img = np.asarray(stepThree)


    imageio.imwrite(name,Img)

Asked By: Rararat

||

Source

Answer 1

remove incorrect indentation on this line:

Img = np.asarray(stepThree)

It is inside the for loop and it should not

Also the code is doing innecesary conversion from byte to string to int instead of doing it directly byte to int, consider changing to the following which is shorter and faster

def BytesToImage2(byteToConvert,originalRows,originalColumns,name):

    """
    Convert from Bytes to Image
    """
    ia=[]
    for i in range(0,len(byteToConvert),originalColumns*3):
        row=[[y for y in x] for x in [byteToConvert[j:j+3] for j in range(i,i+originalColumns*3,3)]]
        ia.append(row)
             
    Img = np.asarray(ia)
    imageio.imwrite(name,Img)

Answered By: wrbp

Answer 2

If your bytes length is compliant, the following code should be the fastest method, it is 500k times faster than list comprehension on a 918 * 575 image (ignore the time of imwrite):

def BytesToImage(byteToConvert, originalRows, originalColumns, name):
    img = np.frombuffer(byteToConvert, np.uint8).reshape(originalRows, originalColumns, 3)
    imageio.imwrite(name, img)

Answered By: Mechanic Pig

Fast way to convert string to numpy ndarray

Question:

Answers: