Python – What is the most efficient way to generate padding?

Question

Here’s the problem: I’m reading binary files in fairly large blocks (512 KiB) and wish to pad the last block with zeros whenever it is shorter than the block size.

Currently, I’m doing something like this:

bytes = f.read(self.chunksize)
if len(bytes) > 0:
    len_diff = self.chunksize - len(bytes)
    if len_diff > 0:
        bytes += reduce(lambda x,y: x+y, ["" for i in range(0, len_diff)])

Obviously this is terribly inefficient since that reduce will make a lot of string concatenations. I’m wondering though, how can I achieve this with Python? In C, I’d simply calloc and be done with it.

If it isn’t achievable with Python, I’m willing to convert this code into a C module and/or abandon Python entirely for this project, since it’s still on the early stages.

Cheers!

EDIT: I’m feeling terrible for not remembering to use the * operator. 🙂

This solution worked perfectly for me:

bytes += "" * len_diff

EDIT #2: Using ljust() instead simplified my code a bit, so the correct answer goes to Jeff.

Asked By: João Neves

||

Source

Answer 1

bytes += ""*len_diff

should help

Answered By: John La Rooy

Answer 2

try this.

bytes = "" * self.chunksize
rbytes = f.read(self.chunksize)
bytes[:len(rbytes)] = rbytes

or

bytes = f.read(self.chunksize)
bytes += "" * (self.chunksize - len(bytes))

Answered By: Pavan Yalamanchili

Answer 3

How about:

bytes += ""*len_diff

Answered By: tom10

Answer 4

Since you’re working with strings, you can use ljust() to do the padding.

readBytes = f.read(self.chunksize)
if readBytes:
    readBytes = readBytes.ljust(self.chunksize, b'')

Answered By: Jeff Mercado

Answer 5

I needed to pad something encrypted. Here’s how to do it.

from Crypto.Util.Padding import pad

...
_bytes = pad(_bytes, block_size)

Answered By: Elijah

Python – What is the most efficient way to generate padding?

Question:

Answers: