Pyomo Compatibility with Python 3.11
Question:
Is there a known compatibility issue with pyomo and python 3.11?
Just trying out the latest/greatest python release and importing pyomo is failing with pyomo v 6.4.2:
Python 3.11.0 (v3.11.0:deaf509e8f, Oct 24 2022, 14:43:23) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyomo.environ import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/environ/__init__.py", line 79, in <module>
_import_packages()
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/environ/__init__.py", line 61, in _import_packages
_do_import(pname)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/environ/__init__.py", line 16, in _do_import
importlib.import_module(pkg_name)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/__init__.py", line 43, in <module>
from pyomo.core import expr, util, kernel
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/util.py", line 21, in <module>
from pyomo.core.base.var import Var
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/__init__.py", line 34, in <module>
from pyomo.core.base.label import (CuidLabeler,
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/label.py", line 19, in <module>
from pyomo.core.base.componentuid import ComponentUID
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/componentuid.py", line 25, in <module>
from pyomo.core.base.reference import Reference
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/reference.py", line 17, in <module>
from pyomo.core.base.set import SetOf, OrderedSetOf, _SetDataBase
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/set.py", line 4208, in <module>
DeclareGlobalSet(_AnySet(
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/set.py", line 4199, in DeclareGlobalSet
_set.__class__.__setstate__(_set, obj.__getstate__())
^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/component.py", line 787, in __getstate__
state = _base.__getstate__()
^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/indexed_component.py", line 316, in __getstate__
state = super(IndexedComponent, self).__getstate__()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/component.py", line 471, in __getstate__
state[key] = val
~~~~~^^^^^
TypeError: 'tuple' object does not support item assignment
>>>
Pyomo Version check:
~ % pip3 show pyomo
Name: Pyomo
Version: 6.4.2
Summary: Pyomo: Python Optimization Modeling Objects
Home-page: http://pyomo.org
Author:
Author-email:
License: BSD
Location: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
Requires: ply
Required-by:
~ %
Answers:
Pyomo 6.4.2 isn’t compatible with python 3.11. You need to update to 6.4.3, which was released on 28th November 2022.
Previous answer – before 6.4.3 update
Verion 6.4.2 does some stuff with pickle and the __getstate__/__setstate__ functions. However, Python 3.11 introduced a default implementation for these functions that does not match Pyomo’s expectations. So the whole thing falls over.
See this note on the new default implementation for __getstate__.
Classes can further influence how their instances are pickled by overriding the method getstate(). It is called and the returned
object is pickled as the contents for the instance, instead of a
default state. There are several cases:
-
For a class that has no instance __dict__ and no __slots__, the default state is None.
-
For a class that has an instance __dict__ and no __slots__, the default state is self.__dict__.
-
For a class that has an instance __dict__ and __slots__, the default state is a tuple consisting of two dictionaries:
self.__dict__, and a dictionary mapping slot names to slot values.
Only slots that have a value are included in the latter.
-
For a class that has __slots__ and no instance __dict__, the default state is a tuple whose first item is None and whose second
item is a dictionary mapping slot names to slot values described in
the previous bullet.
Changed in version 3.11: Added the default implementation of the __getstate__() method in the object class.
In the v6.4.2 code base you can see it does a check to see if __getstate__ is implemented or not. https://github.com/Pyomo/pyomo/blob/6.4.2/pyomo/core/base/component.py#L467
In the main branch, this code has recently been changed. And there is a recent commit saying that the code is now 3.11 compatible. So just hang tight, and wait for the new version to be release.
Or if you really cannot wait and just want to experiment, then install the development version of 6.4.3 with git:
eg.
pip install git+https://github.com/Pyomo/pyomo.git@main
NB. Obviously, you’ll need git installed to be able to do this. And do not use this to install pyomo for production code. This is code that is under development and has not yet been released.
Is there a known compatibility issue with pyomo and python 3.11?
Just trying out the latest/greatest python release and importing pyomo is failing with pyomo v 6.4.2:
Python 3.11.0 (v3.11.0:deaf509e8f, Oct 24 2022, 14:43:23) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyomo.environ import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/environ/__init__.py", line 79, in <module>
_import_packages()
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/environ/__init__.py", line 61, in _import_packages
_do_import(pname)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/environ/__init__.py", line 16, in _do_import
importlib.import_module(pkg_name)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/__init__.py", line 43, in <module>
from pyomo.core import expr, util, kernel
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/util.py", line 21, in <module>
from pyomo.core.base.var import Var
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/__init__.py", line 34, in <module>
from pyomo.core.base.label import (CuidLabeler,
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/label.py", line 19, in <module>
from pyomo.core.base.componentuid import ComponentUID
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/componentuid.py", line 25, in <module>
from pyomo.core.base.reference import Reference
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/reference.py", line 17, in <module>
from pyomo.core.base.set import SetOf, OrderedSetOf, _SetDataBase
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/set.py", line 4208, in <module>
DeclareGlobalSet(_AnySet(
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/set.py", line 4199, in DeclareGlobalSet
_set.__class__.__setstate__(_set, obj.__getstate__())
^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/component.py", line 787, in __getstate__
state = _base.__getstate__()
^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/indexed_component.py", line 316, in __getstate__
state = super(IndexedComponent, self).__getstate__()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyomo/core/base/component.py", line 471, in __getstate__
state[key] = val
~~~~~^^^^^
TypeError: 'tuple' object does not support item assignment
>>>
Pyomo Version check:
~ % pip3 show pyomo
Name: Pyomo
Version: 6.4.2
Summary: Pyomo: Python Optimization Modeling Objects
Home-page: http://pyomo.org
Author:
Author-email:
License: BSD
Location: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
Requires: ply
Required-by:
~ %
Pyomo 6.4.2 isn’t compatible with python 3.11. You need to update to 6.4.3, which was released on 28th November 2022.
Previous answer – before 6.4.3 update
Verion 6.4.2 does some stuff with pickle and the __getstate__/__setstate__ functions. However, Python 3.11 introduced a default implementation for these functions that does not match Pyomo’s expectations. So the whole thing falls over.
See this note on the new default implementation for __getstate__.
Classes can further influence how their instances are pickled by overriding the method getstate(). It is called and the returned
object is pickled as the contents for the instance, instead of a
default state. There are several cases:
For a class that has no instance
__dict__and no__slots__, the default state isNone.For a class that has an instance
__dict__and no__slots__, the default state isself.__dict__.For a class that has an instance
__dict__and__slots__, the default state is a tuple consisting of two dictionaries:
self.__dict__, and a dictionary mapping slot names to slot values.
Only slots that have a value are included in the latter.For a class that has
__slots__and no instance__dict__, the default state is a tuple whose first item is None and whose second
item is a dictionary mapping slot names to slot values described in
the previous bullet.Changed in version 3.11: Added the default implementation of the
__getstate__()method in the object class.
In the v6.4.2 code base you can see it does a check to see if __getstate__ is implemented or not. https://github.com/Pyomo/pyomo/blob/6.4.2/pyomo/core/base/component.py#L467
In the main branch, this code has recently been changed. And there is a recent commit saying that the code is now 3.11 compatible. So just hang tight, and wait for the new version to be release.
Or if you really cannot wait and just want to experiment, then install the development version of 6.4.3 with git:
eg.
pip install git+https://github.com/Pyomo/pyomo.git@main
NB. Obviously, you’ll need git installed to be able to do this. And do not use this to install pyomo for production code. This is code that is under development and has not yet been released.