Proper way to reload a python module from the console
Question:
I’m debugging from the python console and would like to reload a module every time I make a change so I don’t have to exit the console and re-enter it. I’m doing:
>>> from project.model.user import *
>>> reload(user)
but I receive:
>>>NameError: name 'user' is not defined
What is the proper way to reload the entire user class? Is there a better way to do this, perhaps auto-updating while debugging?
Thanks.
Answers:
Unfortunately you’ve got to use:
>>> from project.model import user
>>> reload(user)
I don’t know off the top of my head of something which will automatically reload modules at the interactive prompt… But I don’t see any reason one shouldn’t exist (in fact, it wouldn’t be too hard to implement, either…)
Now, you could do something like this:
from types import ModuleType
import sys
_reload_builtin = reload
def reload(thing):
if isinstance(thing, ModuleType):
_reload_builtin(thing)
elif hasattr(thing, '__module__') and thing.__module__:
module = sys.modules[thing.__module__]
_reload_builtin(module)
else:
raise TypeError, "reload() argument must be a module or have an __module__"
You can’t use reload() in a effective way.
Python does not provide an effective support for reloading or unloading of previously imported
modules; module references makes it impractical to reload a module because references could exist in many places of your program.
Python 3 has removed reload() feature entirely.
As asked, the best you can do is
>>> from project.models.user import *
>>> import project # get module reference for reload
>>> reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2
it would be better and cleaner if you used the user module directly, rather than doing import * (which is almost never the right way to do it). Then it would just be
>>> from project.models import user
>>> reload(user)
This would do what you want. But, it’s not very nice. If you really need to reload modules so often, I’ve got to ask: why?
My suspicion (backed up by previous experience with people asking similar questions) is that you’re testing your module. There are lots of ways to test a module out, and doing it by hand in the interactive interpreter is among the worst ways. Save one of your sessions to a file and use doctest, for a quick fix. Alternatively, write it out as a program and use python -i. The only really great solution, though, is using the unittest module.
If that’s not it, hopefully it’s something better, not worse. There’s really no good use of reload (in fact, it’s removed in 3.x). It doesn’t work effectively– you might reload a module but leave leftovers from previous versions. It doesn’t even work on all kinds of modules– extension modules will not reload properly, or sometimes even break horribly, when reloaded.
The context of using it in the interactive interpreter doesn’t leave a lot of choices as to what you are doing, and what the real best solution would be. Outside it, sometimes people used reload() to implement plugins etc. This is dangerous at best, and can frequently be done differently using either exec (ah the evil territory we find ourselves in), or a segregated process.
You could also try twisted.python.rebuild.rebuild.
IPython can reload modules before executing every new line:
%load_ext autoreload
%autoreload 2
Where %autoreload 2reloads “all modules (except those excluded by %aimport) every time before executing the Python code typed.”
See the docs:
For python3.4+, reload has been moved to the importlib module. you can use importlib.reload(). You can refer to this post.
>>> import importlib
>>> import project # get module reference for reload
>>> importlib.reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2
For python3 versions before 3.4, the module to import is imp (instead of importlib)
from test_reload import add_test
where test_reload is a module, and add_test is a function
if you changed the function add_test, of course you need to reload this function.
then you can do this:
import imp
imp.reload(test_reload)
from test_reload import add_test
this will refresh the function add_test.
so you need to add
imp.reload(test_reload)
from test_reload import add_test --add this line in your code
As of Python 3.4 you can use importlib.reload(module)
>>> from importlib import reload
>>> from project.model import user
>>> reload(user)
from matplotlib.cbook import maxdict
# importing package
import matplotlib.pyplot as plt
import numpy as np
# create data
x = sd_p.index.values
var=(["train","val1",'val2'][1]) # change to data name list
val1 = sd_p.val1
#maxd_steps
#MAXD
maxd_steps=2
MAXD=1
# plot lines
plt.plot(x, sd_p.train, label = "train" )
plt.plot(x, sd_p.val1, label = "val#1" )
plt.plot(x, sd_p.val2, label = "val#2" )
plt.axvline(x = maxd_steps, color = 'b', label = 'maxd_steps',linestyle ="--")
plt.axhline(y = MAXD, color = 'b', label = 'maxd_steps',linestyle ="--")
plt.legend()
plt.show()
I’m debugging from the python console and would like to reload a module every time I make a change so I don’t have to exit the console and re-enter it. I’m doing:
>>> from project.model.user import *
>>> reload(user)
but I receive:
>>>NameError: name 'user' is not defined
What is the proper way to reload the entire user class? Is there a better way to do this, perhaps auto-updating while debugging?
Thanks.
Unfortunately you’ve got to use:
>>> from project.model import user
>>> reload(user)
I don’t know off the top of my head of something which will automatically reload modules at the interactive prompt… But I don’t see any reason one shouldn’t exist (in fact, it wouldn’t be too hard to implement, either…)
Now, you could do something like this:
from types import ModuleType
import sys
_reload_builtin = reload
def reload(thing):
if isinstance(thing, ModuleType):
_reload_builtin(thing)
elif hasattr(thing, '__module__') and thing.__module__:
module = sys.modules[thing.__module__]
_reload_builtin(module)
else:
raise TypeError, "reload() argument must be a module or have an __module__"
You can’t use reload() in a effective way.
Python does not provide an effective support for reloading or unloading of previously imported
modules; module references makes it impractical to reload a module because references could exist in many places of your program.
Python 3 has removed reload() feature entirely.
As asked, the best you can do is
>>> from project.models.user import *
>>> import project # get module reference for reload
>>> reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2
it would be better and cleaner if you used the user module directly, rather than doing import * (which is almost never the right way to do it). Then it would just be
>>> from project.models import user
>>> reload(user)
This would do what you want. But, it’s not very nice. If you really need to reload modules so often, I’ve got to ask: why?
My suspicion (backed up by previous experience with people asking similar questions) is that you’re testing your module. There are lots of ways to test a module out, and doing it by hand in the interactive interpreter is among the worst ways. Save one of your sessions to a file and use doctest, for a quick fix. Alternatively, write it out as a program and use python -i. The only really great solution, though, is using the unittest module.
If that’s not it, hopefully it’s something better, not worse. There’s really no good use of reload (in fact, it’s removed in 3.x). It doesn’t work effectively– you might reload a module but leave leftovers from previous versions. It doesn’t even work on all kinds of modules– extension modules will not reload properly, or sometimes even break horribly, when reloaded.
The context of using it in the interactive interpreter doesn’t leave a lot of choices as to what you are doing, and what the real best solution would be. Outside it, sometimes people used reload() to implement plugins etc. This is dangerous at best, and can frequently be done differently using either exec (ah the evil territory we find ourselves in), or a segregated process.
You could also try twisted.python.rebuild.rebuild.
IPython can reload modules before executing every new line:
%load_ext autoreload
%autoreload 2
Where %autoreload 2reloads “all modules (except those excluded by %aimport) every time before executing the Python code typed.”
See the docs:
For python3.4+, reload has been moved to the importlib module. you can use importlib.reload(). You can refer to this post.
>>> import importlib
>>> import project # get module reference for reload
>>> importlib.reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2
For python3 versions before 3.4, the module to import is imp (instead of importlib)
from test_reload import add_test
where test_reload is a module, and add_test is a function
if you changed the function add_test, of course you need to reload this function.
then you can do this:
import imp
imp.reload(test_reload)
from test_reload import add_test
this will refresh the function add_test.
so you need to add
imp.reload(test_reload)
from test_reload import add_test --add this line in your code
As of Python 3.4 you can use importlib.reload(module)
>>> from importlib import reload
>>> from project.model import user
>>> reload(user)
from matplotlib.cbook import maxdict
# importing package
import matplotlib.pyplot as plt
import numpy as np
# create data
x = sd_p.index.values
var=(["train","val1",'val2'][1]) # change to data name list
val1 = sd_p.val1
#maxd_steps
#MAXD
maxd_steps=2
MAXD=1
# plot lines
plt.plot(x, sd_p.train, label = "train" )
plt.plot(x, sd_p.val1, label = "val#1" )
plt.plot(x, sd_p.val2, label = "val#2" )
plt.axvline(x = maxd_steps, color = 'b', label = 'maxd_steps',linestyle ="--")
plt.axhline(y = MAXD, color = 'b', label = 'maxd_steps',linestyle ="--")
plt.legend()
plt.show()