Python: reload component Y imported with 'from X import Y'?
Question:
In Python, once I have imported a module X in an interpreter session using import X
, and the module changes on the outside, I can reload the module with reload(X)
. The changes then become available in my interpreter session.
I am wondering if this also possible when I import a component Y from module X using from X import Y
.
The statement reload Y
does not work, since Y is not a module itself, but only a component (in this case a class) inside of a module.
Is it possible at all to reload individual components of a module without leaving the interpreter session (or importing the entire module)?
EDIT:
For clarification, the question is about importing a class or function Y from a module X and reloading on a change, not a module Y from a package X.
Answers:
If Y is a module (and X a package) reload(Y)
will be fine — otherwise, you’ll see why good Python style guides (such as my employer’s) say to never import anything except a module (this is one out of many great reasons — yet people still keep importing functions and classes directly, no matter how much I explain that it’s not a good idea;-).
reload()
module X
,
reload()
module importing Y
from X
.
Note that reloading won’t change already created objects bound in other namespaces (even if you follow style guide from Alex).
First off, you shouldn’t be using reload at all, if you can avoid it. But let’s assume you have your reasons (i.e. debugging inside IDLE).
Reloading the library won’t get the names back into the module’s namespace. To do this, just reassign the variables:
f = open('zoo.py', 'w')
f.write("snakes = ['viper','anaconda']n")
f.close()
from zoo import snakes
print snakes
f = open('zoo.py', 'w')
f.write("snakes = ['black-adder','boa constrictor']n")
f.close()
import zoo
reload(zoo)
snakes = zoo.snakes # the variable 'snakes' is now reloaded
print snakes
You could do this a few other ways. You could automate the process by searching through the local namespace, and reassigning anything that was from the module in question, but I think we are being evil enough.
Answer
From my tests, the marked answer, which suggests a simple reload(X)
, does not work.
From what I can tell the correct answer is:
from importlib import reload # python 2.7 does not require this
import X
reload( X )
from X import Y
Test
My test was the following (Python 2.6.5 + bpython 0.9.5.2)
X.py:
def Y():
print "Test 1"
bpython:
>>> from X import Y
>>> print Y()
Test 1
>>> # Edit X.py to say "Test 2"
>>> print Y()
Test 1
>>> reload( X ) # doesn't work because X not imported yet
Traceback (most recent call last):
File "<input>", line 1, in <module>
NameError: name 'X' is not defined
>>> import X
>>> print Y()
Test 1
>>> print X.Y()
Test 1
>>> reload( X ) # No effect on previous "from" statements
>>> print Y()
Test 1
>>> print X.Y() # first one that indicates refresh
Test 2
>>> from X import Y
>>> print Y()
Test 2
>>> # Finally get what we were after
from modulename import func
import importlib, sys
importlib.reload(sys.modules['modulename'])
from modulename import func
If you want to do this:
from mymodule import myobject
Do this instead:
import mymodule
myobject=mymodule.myobject
You can now use myobject in the same way as you were planning (without the tiresome unreadable mymodule references everywhere).
If you’re working interactively and want to reload myobject from mymodule you now can using:
reload(mymodule)
myobject=mymodule.myobject
assuming you used from X import Y
, you have two options:
reload(sys.modules['X'])
reload(sys.modules[__name__]) # or explicitly name your module
or
Y=reload(sys.modules['X']).Y
few considerations:
A. if the import scope is not module-wide (e,g: import in a function) – you must use the second version.
B. if Y is imported into X from another module (Z) – you must reload Z, than reload X and than reload your module, even reloading all your modules (e,g: using [ reload(mod) for mod in sys.modules.values() if type(mod) == type(sys) ]
) might reload X before reloading Z – and than not refresh the value of Y.
Just to follow up on AlexMartelli’s and Catskul’s answers, there are some really simple but nasty cases that appear to confound reload
, at least in Python 2.
Suppose I have the following source tree:
- foo
- __init__.py
- bar.py
with the following content:
init.py:
from bar import Bar, Quux
bar.py:
print "Loading bar"
class Bar(object):
@property
def x(self):
return 42
class Quux(Bar):
object_count = 0
def __init__(self):
self.count = self.object_count
self.__class__.object_count += 1
@property
def x(self):
return super(Quux,self).x + 1
def __repr__(self):
return 'Quux[%d, x=%d]' % (self.count, self.x)
This works just fine without using reload
:
>>> from foo import Quux
Loading bar
>>> Quux()
Quux[0, x=43]
>>> Quux()
Quux[1, x=43]
>>> Quux()
Quux[2, x=43]
But try to reload and it either has no effect or corrupts things:
>>> import foo
Loading bar
>>> from foo import Quux
>>> Quux()
Quux[0, x=43]
>>> Quux()
Quux[1, x=43]
>>> reload(foo)
<module 'foo' from 'foo__init__.pyc'>
>>> Quux()
Quux[2, x=43]
>>> from foo import Quux
>>> Quux()
Quux[3, x=43]
>>> reload(foo.bar)
Loading bar
<module 'foo.bar' from 'foobar.pyc'>
>>> Quux()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foobar.py", line 17, in __repr__
return 'Quux[%d, x=%d]' % (self.count, self.x)
File "foobar.py", line 15, in x
return super(Quux,self).x + 1
TypeError: super(type, obj): obj must be an instance or subtype of type
>>> Quux().count
5
>>> Quux().count
6
>>> Quux = foo.bar.Quux
>>> Quux()
Quux[0, x=43]
>>> foo.Quux()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foobar.py", line 17, in __repr__
return 'Quux[%d, x=%d]' % (self.count, self.x)
File "foobar.py", line 15, in x
return super(Quux,self).x + 1
TypeError: super(type, obj): obj must be an instance or subtype of type
>>> foo.Quux().count
8
The only way I could ensure the bar
submodule was reloaded was to reload(foo.bar)
; the only way I access the reloaded Quux
class is to reach in and grab it from the reloaded sub module; but the foo
module itself kept holding onto the original Quux
class object, presumably because it uses from bar import Bar, Quux
(rather than import bar
followed by Quux = bar.Quux
); furthermore the Quux
class got out of sync with itself, which is just bizarre.
If you’re working in a jupyter environment, and you already have from module import function
can use the magic function, autoreload
by
%load_ext autoreload
%autoreload
from module import function
The introduction of the autoreload
in IPython is given here.
In Python, once I have imported a module X in an interpreter session using import X
, and the module changes on the outside, I can reload the module with reload(X)
. The changes then become available in my interpreter session.
I am wondering if this also possible when I import a component Y from module X using from X import Y
.
The statement reload Y
does not work, since Y is not a module itself, but only a component (in this case a class) inside of a module.
Is it possible at all to reload individual components of a module without leaving the interpreter session (or importing the entire module)?
EDIT:
For clarification, the question is about importing a class or function Y from a module X and reloading on a change, not a module Y from a package X.
If Y is a module (and X a package) reload(Y)
will be fine — otherwise, you’ll see why good Python style guides (such as my employer’s) say to never import anything except a module (this is one out of many great reasons — yet people still keep importing functions and classes directly, no matter how much I explain that it’s not a good idea;-).
reload()
moduleX
,reload()
module importingY
fromX
.
Note that reloading won’t change already created objects bound in other namespaces (even if you follow style guide from Alex).
First off, you shouldn’t be using reload at all, if you can avoid it. But let’s assume you have your reasons (i.e. debugging inside IDLE).
Reloading the library won’t get the names back into the module’s namespace. To do this, just reassign the variables:
f = open('zoo.py', 'w')
f.write("snakes = ['viper','anaconda']n")
f.close()
from zoo import snakes
print snakes
f = open('zoo.py', 'w')
f.write("snakes = ['black-adder','boa constrictor']n")
f.close()
import zoo
reload(zoo)
snakes = zoo.snakes # the variable 'snakes' is now reloaded
print snakes
You could do this a few other ways. You could automate the process by searching through the local namespace, and reassigning anything that was from the module in question, but I think we are being evil enough.
Answer
From my tests, the marked answer, which suggests a simple reload(X)
, does not work.
From what I can tell the correct answer is:
from importlib import reload # python 2.7 does not require this
import X
reload( X )
from X import Y
Test
My test was the following (Python 2.6.5 + bpython 0.9.5.2)
X.py:
def Y():
print "Test 1"
bpython:
>>> from X import Y
>>> print Y()
Test 1
>>> # Edit X.py to say "Test 2"
>>> print Y()
Test 1
>>> reload( X ) # doesn't work because X not imported yet
Traceback (most recent call last):
File "<input>", line 1, in <module>
NameError: name 'X' is not defined
>>> import X
>>> print Y()
Test 1
>>> print X.Y()
Test 1
>>> reload( X ) # No effect on previous "from" statements
>>> print Y()
Test 1
>>> print X.Y() # first one that indicates refresh
Test 2
>>> from X import Y
>>> print Y()
Test 2
>>> # Finally get what we were after
from modulename import func
import importlib, sys
importlib.reload(sys.modules['modulename'])
from modulename import func
If you want to do this:
from mymodule import myobject
Do this instead:
import mymodule
myobject=mymodule.myobject
You can now use myobject in the same way as you were planning (without the tiresome unreadable mymodule references everywhere).
If you’re working interactively and want to reload myobject from mymodule you now can using:
reload(mymodule)
myobject=mymodule.myobject
assuming you used from X import Y
, you have two options:
reload(sys.modules['X'])
reload(sys.modules[__name__]) # or explicitly name your module
or
Y=reload(sys.modules['X']).Y
few considerations:
A. if the import scope is not module-wide (e,g: import in a function) – you must use the second version.
B. if Y is imported into X from another module (Z) – you must reload Z, than reload X and than reload your module, even reloading all your modules (e,g: using [ reload(mod) for mod in sys.modules.values() if type(mod) == type(sys) ]
) might reload X before reloading Z – and than not refresh the value of Y.
Just to follow up on AlexMartelli’s and Catskul’s answers, there are some really simple but nasty cases that appear to confound reload
, at least in Python 2.
Suppose I have the following source tree:
- foo
- __init__.py
- bar.py
with the following content:
init.py:
from bar import Bar, Quux
bar.py:
print "Loading bar"
class Bar(object):
@property
def x(self):
return 42
class Quux(Bar):
object_count = 0
def __init__(self):
self.count = self.object_count
self.__class__.object_count += 1
@property
def x(self):
return super(Quux,self).x + 1
def __repr__(self):
return 'Quux[%d, x=%d]' % (self.count, self.x)
This works just fine without using reload
:
>>> from foo import Quux
Loading bar
>>> Quux()
Quux[0, x=43]
>>> Quux()
Quux[1, x=43]
>>> Quux()
Quux[2, x=43]
But try to reload and it either has no effect or corrupts things:
>>> import foo
Loading bar
>>> from foo import Quux
>>> Quux()
Quux[0, x=43]
>>> Quux()
Quux[1, x=43]
>>> reload(foo)
<module 'foo' from 'foo__init__.pyc'>
>>> Quux()
Quux[2, x=43]
>>> from foo import Quux
>>> Quux()
Quux[3, x=43]
>>> reload(foo.bar)
Loading bar
<module 'foo.bar' from 'foobar.pyc'>
>>> Quux()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foobar.py", line 17, in __repr__
return 'Quux[%d, x=%d]' % (self.count, self.x)
File "foobar.py", line 15, in x
return super(Quux,self).x + 1
TypeError: super(type, obj): obj must be an instance or subtype of type
>>> Quux().count
5
>>> Quux().count
6
>>> Quux = foo.bar.Quux
>>> Quux()
Quux[0, x=43]
>>> foo.Quux()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foobar.py", line 17, in __repr__
return 'Quux[%d, x=%d]' % (self.count, self.x)
File "foobar.py", line 15, in x
return super(Quux,self).x + 1
TypeError: super(type, obj): obj must be an instance or subtype of type
>>> foo.Quux().count
8
The only way I could ensure the bar
submodule was reloaded was to reload(foo.bar)
; the only way I access the reloaded Quux
class is to reach in and grab it from the reloaded sub module; but the foo
module itself kept holding onto the original Quux
class object, presumably because it uses from bar import Bar, Quux
(rather than import bar
followed by Quux = bar.Quux
); furthermore the Quux
class got out of sync with itself, which is just bizarre.
If you’re working in a jupyter environment, and you already have from module import function
can use the magic function, autoreload
by
%load_ext autoreload
%autoreload
from module import function
The introduction of the autoreload
in IPython is given here.