In Python, what is `sys.maxsize`?
Question:
I assumed that this number ( 2^63 - 1 ) was the maximum value python could handle, or store as a variable. But these commands seem to be working fine:
>>> sys.maxsize
9223372036854775807
>>> a=sys.maxsize + 1
>>> a
9223372036854775808
So is there any significance at all? Can Python handle arbitrarily large numbers, if computation resoruces permitt?
Note, here’s the print-out of my version is:
>>> sys.version
3.5.2 |Anaconda custom (64-bit)| (default, Jul 5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)]'
Answers:
Python can handle arbitrarily large integers in computation. Any integer too big to fit in 64 bits (or whatever the underlying hardware limit is) is handled in software. For that reason, Python 3 doesn’t have a sys.maxint constant.
The value sys.maxsize, on the other hand, reports the platform’s pointer size, and that limits the size of Python’s data structures such as strings and lists.
Documentation for sys.maxsize:
An integer giving the maximum value a variable of type Py_ssize_t can take. It’s usually 2**31 – 1 on a 32-bit platform and 2**63 – 1 on a 64-bit platform. python3
The largest positive integer supported by the platform’s Py_ssize_t type, and thus the maximum size lists, strings, dicts, and many other containers can have. python2
What is Py_ssize_t?
It is an index type (number type for indexing things, like lists). It is the signed version of size_t (from the C language).
- We don’t use a normal number/ int, because this is unbounded in Python.
- In Python, we don’t use
size_t because we want to support negative indexing, in Python we can do my_list[-4:]. So Py_ssize_t provides negative and positive numbers between a range.
- The
_t stands for type, to inform developers that size_t is a type name, not a variable. Just a convention.
So what is the effect of having a limit on Py_ssize_t? Why does this limit list, strings, dict size?
- There is no way to index a list with an element larger than this. The list cannot get bigger than this, because it won’t accept a non
Py_ssize_t.
- In the dictionary case,
Py_ssize_t is used as the hash. Python doesn’t use linked lists in its dictionary implementation, it uses Open Addressing/ probing, where if a collision is found, we a systematic way of getting another place to find the key and put the value. So you can’t have more than Py_ssize_t in a dictionary in Python.
In all practical cases (64 bit machines aka. probably you), you will run out of memory before you max out Py_ssize_t. Trying dict.fromkeys(range(sys.maxsize + 5)) never got there, it just slowed my computer down.
I assumed that this number ( 2^63 - 1 ) was the maximum value python could handle, or store as a variable. But these commands seem to be working fine:
>>> sys.maxsize
9223372036854775807
>>> a=sys.maxsize + 1
>>> a
9223372036854775808
So is there any significance at all? Can Python handle arbitrarily large numbers, if computation resoruces permitt?
Note, here’s the print-out of my version is:
>>> sys.version
3.5.2 |Anaconda custom (64-bit)| (default, Jul 5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)]'
Python can handle arbitrarily large integers in computation. Any integer too big to fit in 64 bits (or whatever the underlying hardware limit is) is handled in software. For that reason, Python 3 doesn’t have a sys.maxint constant.
The value sys.maxsize, on the other hand, reports the platform’s pointer size, and that limits the size of Python’s data structures such as strings and lists.
Documentation for sys.maxsize:
An integer giving the maximum value a variable of type Py_ssize_t can take. It’s usually 2**31 – 1 on a 32-bit platform and 2**63 – 1 on a 64-bit platform. python3
The largest positive integer supported by the platform’s Py_ssize_t type, and thus the maximum size lists, strings, dicts, and many other containers can have. python2
What is Py_ssize_t?
It is an index type (number type for indexing things, like lists). It is the signed version of size_t (from the C language).
- We don’t use a normal number/ int, because this is unbounded in Python.
- In Python, we don’t use
size_tbecause we want to support negative indexing, in Python we can domy_list[-4:]. SoPy_ssize_tprovides negative and positive numbers between a range. - The
_tstands for type, to inform developers thatsize_tis a type name, not a variable. Just a convention.
So what is the effect of having a limit on Py_ssize_t? Why does this limit list, strings, dict size?
- There is no way to index a list with an element larger than this. The list cannot get bigger than this, because it won’t accept a non
Py_ssize_t. - In the dictionary case,
Py_ssize_tis used as the hash. Python doesn’t use linked lists in its dictionary implementation, it uses Open Addressing/ probing, where if a collision is found, we a systematic way of getting another place to find the key and put the value. So you can’t have more thanPy_ssize_tin a dictionary in Python.
In all practical cases (64 bit machines aka. probably you), you will run out of memory before you max out Py_ssize_t. Trying dict.fromkeys(range(sys.maxsize + 5)) never got there, it just slowed my computer down.