How to calculate CRC32 with Python to match online results?

Question

I’m trying to calculate/generate the CRC32 hash of some random strings using Python but they do not match the values I generate from online sources. Here is what I’m doing on my PC,

>>> import binascii
>>> binascii.crc32('hello-world')
-1311505829

Another approach,

>>> import zlib
>>> zlib.crc32('hello-world')
-1311505829

The fact that the above results are identical tells me that I’m calling the function correctly. But, if I go to the following online sources,

http://www.lammertbies.nl/comm/info/crc-calculation.html
http://crc32-checksum.waraxe.us/
http://www.md5calc.com/ (select CRC32B from drop-down)

For the string “hello-world” they all give the same value = b1d4025b

Does anyone know what I need to do, to get matching results?

As I was typing this question it occurred to me that I might need to convert my Python result to hex,

>>> hex(zlib.crc32('hello-world'))
'-0x4e2bfda5'

Unfortunately, that hasn’t helped either. 🙁

Asked By: chronodekar

||

Source

Answer 1

Python 2 (unlike py3) is doing a signed 32-bit CRC.

Those sites are doing an unsigned 32-bit CRC.

The values are the same otherwise, as you can see from this:

>>> 0x100000000 - 0xb1d4025b == 0x4e2bfda5
True

One quick way to convert from 32-bit signed to 32-bit unsigned is:^*

>>> -1311505829 % (1<<32)
2983461467

Or, in hex:

>>> hex(-1311505829 % (1<<32))
'0xb1d4025b'

& 0xFFFFFFFF or % 0x100000000 or & (2**32-1) or % (2**32) and so on are all equivalent ways to do the same bit-twiddling; it just comes down to which one you find most readable.

_{* This only works in languages that do floored integer division, like Python (-3 // 2 == -2); in languages that do truncated integer division, like Java (-3 / 2 == -1), you’ll still end up with a negative number. And in languages that don’t even require that division and mod go together properly, like C, all bets are off—but in C, you’d just cast the bytes to the type you want…}

Answered By: abarnert

Answer 2

zlib.crc32 documentation suggests using the following approach “to generate the same numeric value across all Python versions and platforms”.

import zlib
hex(zlib.crc32(b'hello-world') & 0xffffffff)

The result is 0xb1d4025b as expected.

Answered By: Alexey

Answer 3

It seems that python is returning an signed integer (hence the negative number), whereas the others are returning an unsigned integer.

I have tried using a modulus with 2^32, and it gave the same value as these sites.

>>> hex(zlib.crc32(b'hello-world')% 2**32)
'0xb1d4025b'

Answered By: chw21

How to calculate CRC32 with Python to match online results?

Question:

Answers: