Python's pycrypto library for random number generation vs os.urandom
Question:
I was trying to understand and figure out if I should use os.urandom() or Crypto.Random.new() for cryptographically secure pseudo-random numbers.
The following website seems to suggest to use os.urandom():
https://github.com/mozilla/PyHawk/pull/13
but I don’t really see why and the other websites I found online don’t really explain which one to use.
If anyone knows which one is secure for cryptographic use, I would appreciate the advice!
One of the things that specifically worry me is that, I plan to use it to generate random numbers but I am worried that one day I generate some random numbers and the next day the state of the generator is the same and it starts to generate the same “random” numbers.
Answers:
urandom does return cryptographically secure numbers, with a caveat. If it runs out of entropy (real random data generated by mouse movement, CPU temperature fluctuations on Ivy Bridge+ chips, etc), it will start producing less secure pseudo-random numbers. These are generated by a strong generator, but depending on your application they may not be random enough. On Unix systems, /dev/random will block while waiting for more entropy, while /dev/urandom will start generating pseudo-random numbers.
I’m not knowledgeable on PyCrypto, however on reading the source, it seems the Crypto/Random/OSRNG/posix.py simply reads from /dev/urandom – so I wouldn’t worry too much about the choice.
In the link you gave, the only reason given to prefer urandom() is that it pulled less code in (the OS implements “most of it”, and os.urandom() is built in to Python).
If you’re going to distribute a Python package, you can simplify users’ lives by minimizing external dependencies. That’s the entire point of the link you found.
In terms of quality, either way should work fine. I prefer urandom() because I understand what it does; I never dug into the guts of PyCrypto. But urandom() has been criticized for use in some environments. Click this and scroll down to the part that starts
Gutterman, Pinkas, & Reinman in March 2006 published a detailed
cryptographic analysis of the Linux random number generator …
I go for os.urandom. On all (recent) Python implementations I checked, it does the correct thing by simply opening an unbuffered connection to /dev/urandom or the equivalent device on other non-Linux platforms.
On the other hand, PyCrypto’s Crypto.Random is a very complex wrapper based on Fortuna. Such complex construction was probably done in the hope of mitigating some flaws of the underlying OS. Unfortunately:
- It still pulls entropy from
/dev/urandom (on Linux), so if the OS is broken, PyCrypto’s Crypto.Random will be broken too (defeating its purpose)
- That choice has backfired, since it is very hard to deal with forking and cases where the same entropy gets reused by different processes (see CVE-2013-1445).
- The Fortuna code is not covered at all by unit tests and Fortuna algorithm itself does not come with test vectors. That alone forces you to make a (big) leap of faith.
From Python 3.6, the secrets module, which is based on os.urandom and random.SystemRandom, is the suggested secured random number generator.
Warning: Note that the pseudo-random generators in the random module should NOT be used for security purposes. Use secrets on Python 3.6+ and os.urandom() on Python 3.5 and earlier.
PEP 506 has a detailed discussion on this.
I was trying to understand and figure out if I should use os.urandom() or Crypto.Random.new() for cryptographically secure pseudo-random numbers.
The following website seems to suggest to use os.urandom():
https://github.com/mozilla/PyHawk/pull/13
but I don’t really see why and the other websites I found online don’t really explain which one to use.
If anyone knows which one is secure for cryptographic use, I would appreciate the advice!
One of the things that specifically worry me is that, I plan to use it to generate random numbers but I am worried that one day I generate some random numbers and the next day the state of the generator is the same and it starts to generate the same “random” numbers.
urandom does return cryptographically secure numbers, with a caveat. If it runs out of entropy (real random data generated by mouse movement, CPU temperature fluctuations on Ivy Bridge+ chips, etc), it will start producing less secure pseudo-random numbers. These are generated by a strong generator, but depending on your application they may not be random enough. On Unix systems, /dev/random will block while waiting for more entropy, while /dev/urandom will start generating pseudo-random numbers.
I’m not knowledgeable on PyCrypto, however on reading the source, it seems the Crypto/Random/OSRNG/posix.py simply reads from /dev/urandom – so I wouldn’t worry too much about the choice.
In the link you gave, the only reason given to prefer urandom() is that it pulled less code in (the OS implements “most of it”, and os.urandom() is built in to Python).
If you’re going to distribute a Python package, you can simplify users’ lives by minimizing external dependencies. That’s the entire point of the link you found.
In terms of quality, either way should work fine. I prefer urandom() because I understand what it does; I never dug into the guts of PyCrypto. But urandom() has been criticized for use in some environments. Click this and scroll down to the part that starts
Gutterman, Pinkas, & Reinman in March 2006 published a detailed
cryptographic analysis of the Linux random number generator …
I go for os.urandom. On all (recent) Python implementations I checked, it does the correct thing by simply opening an unbuffered connection to /dev/urandom or the equivalent device on other non-Linux platforms.
On the other hand, PyCrypto’s Crypto.Random is a very complex wrapper based on Fortuna. Such complex construction was probably done in the hope of mitigating some flaws of the underlying OS. Unfortunately:
- It still pulls entropy from
/dev/urandom(on Linux), so if the OS is broken, PyCrypto’sCrypto.Randomwill be broken too (defeating its purpose) - That choice has backfired, since it is very hard to deal with forking and cases where the same entropy gets reused by different processes (see CVE-2013-1445).
- The Fortuna code is not covered at all by unit tests and Fortuna algorithm itself does not come with test vectors. That alone forces you to make a (big) leap of faith.
From Python 3.6, the secrets module, which is based on os.urandom and random.SystemRandom, is the suggested secured random number generator.
Warning: Note that the pseudo-random generators in the random module should NOT be used for security purposes. Use secrets on Python 3.6+ and os.urandom() on Python 3.5 and earlier.
PEP 506 has a detailed discussion on this.