How do you reload a Django model module using the interactive interpreter via "manage.py shell"?

Question:

I know how to reload a regular Python module within a regular Python interpreter session. This question documents how to do that pretty well:

How do I unload (reload) a Python module?

For some reason, I am having trouble doing that within Django’s “manage.py shell” interpreter session. To recreate my issue, start the basic Django tutorial found here:

Writing your first Django app, part 1

After creating the “polls” application and “Poll” class, start up the interpreter via “manage.py shell” and import the “polls” app into it.

import polls.models as pm

Create a new “Poll” object:

p = pm.Poll()

All is well and good so far. Now go back to your source and add any arbitrary method or attribute. For example, I’ve added:

def x(self):
    return 2+2

Now go back to the interpreter and “reload” the module:

reload(pm)

Now try to use your new method or attribute:

p1 = pm.Poll()
p1.x()

You’ll get this message:

'Poll' object has no attribute 'x'

What gives? I’ve also tried rerunning the import command, importing the module using different syntax, deleting all references to any “Poll” objects or to the “Poll” class. I’ve also tried this with both the IPython interpreter and with the plain Python (v2.6) interpreter. Nothing seems to work.

Using the same techniques with an arbitrary Python module in a regular interpreter session works perfectly. I just can’t seem to get it to work in Django’s “shell” session.

By the way, if it makes any difference, I’m doing this on a Ubuntu 9.04 machine.

Asked By: Chad Braun-Duin

||

Answers:

ipython console does a deep reload with each reload() expression; and of course adds a lot of other useful stuff.

Answered By: lprsd

Well, I think I have to answer to this. The problem is that Django caches its models in a singleton (singleton like structure) called AppCache. Basically, to reload Django models you need to first reload and re-import all the model modules stored in the AppCache. Then you need to wipe out the AppCache. Here’s the code for it:

import os
from django.db.models.loading import AppCache
cache = AppCache()

curdir = os.getcwd()

for app in cache.get_apps():
    f = app.__file__
    if f.startswith(curdir) and f.endswith('.pyc'):
        os.remove(f)
    __import__(app.__name__)
    reload(app)

from django.utils.datastructures import SortedDict
cache.app_store = SortedDict()
cache.app_models = SortedDict()
cache.app_errors = {}
cache.handled = {}
cache.loaded = False

I’ve put all of this in a separate file called reloadmodels.py in the root directory of my Django site. Using IPython I can reload everything by running:

%run ~/mysite/reloadmodels.py
Answered By: Chad Braun-Duin

Assuming your project is set up this way

  • project name : bookstore
  • app name : shelf
  • model name : Books

first load

from bookstore.shelf.models import Books

subsequent reloads

import bookstore;reload(bookstore.shelf.models);from bookstore.shelf.models import Books
Answered By: surtyaar

As far as I’m concerned, none of the above solutions worked on their own, also this thread didn’t help much on its own, but after combining the approaches I managed to reload my models in shell_plus:

  1. Make changes to the model (MyModel)
  2. remove models.pyc
  3. Clean Django model cache (like here):

     from django.db.models.loading import AppCache
     cache = AppCache()
     from django.utils.datastructures import SortedDict
     cache.app_store = SortedDict()
     cache.app_models = SortedDict()
     cache.app_errors = {}
     cache.handled = {}
     cache.loaded = False
    
  4. Reload model like here

    reload(project.app.models)
    from project.app.models import MyModel
    
Answered By: Tomasz Zieliński

I wasn’t able to get any of the above solutions to work, but I did come up with a workaround for reloading any other non-models module in my django project (e.g. a functions.py or views.py module).

  1. Create a file called reimport_module.py. I stored it in the local/ folder of my django project on my dev machine.

    # Desc: Imports the module with the name passed in, or imports it for first
    #       time if it hasn't already been imported.
    #
    #       Purpose of this script is to speed up development of functions that
    #       are written in an external editor then tested in IPython.
    #
    #       Without this script you have to exit & reenter IPython then redo
    #       import statements, definitions of local variables, etc.
    #
    #       Note: doesn't work for Django models files, because Django caches
    #       them in a structure called AppCache.
    #
    # Args: module to reload (string)
    
    import sys
    
    module_to_reload = sys.argv[1]
    
    # Attempt to pop module
    try:
        sys.modules.pop(module_to_reload)
        print 'reimporting...'
    except KeyError:
        print 'importing for first time...'
    
    # (re)import module
    import_str = 'from {0} import *'.format(module_to_reload)
    exec(import_str) 
    
  2. Launch shell plus (which uses an embedded IPython shell):

    python manage.py shell_plus

  3. Use the following to import the module you are developing:

    %run local/reimport_module.py 'your.module'

  4. Use IPython to test functions in your module.

  5. Make changes to the module in an external editor.
  6. Use the following to reimport the module without having to exit & reenter IPython:

    %run local/reimport_module.py 'your.module'

    Note: this command was already used in step 3, so you can type %run then the up arrow to autocomplete it.

Answered By: Scott Foubister

You can also use django-extensions project with the following command:

manage.py shell_plus --notebook

This will open a IPython notebook on your web browser instead of the IPython shell interpreter. Write your code there, and run it.

When you change your modules, just click on the web page menu item ‘Kernel->Restart’

Re-running the code now uses your modified modules.

Answered By: mpaf

Enable IPython autoreload extension before importing any code:

%load_ext autoreload
%autoreload 2

I use it with the regular django shell and it works perfectly, although it does have some limitations:

*Caveats:

Reloading Python modules in a reliable way is in general difficult, and unexpected things may occur. %autoreload tries to work around common pitfalls by replacing function code objects and parts of classes previously in the module with new versions. This makes the following things to work:

  • Functions and classes imported via ‘from xxx import foo’ are upgraded to new versions when ‘xxx’ is reloaded.
  • Methods and properties of classes are upgraded on reload, so that calling ‘c.foo()’ on an object ‘c’ created before the reload causes the new code for ‘foo’ to be executed.

Some of the known remaining caveats are:

  • Replacing code objects does not always succeed: changing a @property in a class to an ordinary method or a method to a member variable can cause problems (but in old objects only).
  • Functions that are removed (eg. via monkey-patching) from a module before it is reloaded are not upgraded.
  • C extension modules cannot be reloaded, and so cannot be autoreloaded.*

source: https://ipython.org/ipython-doc/3/config/extensions/autoreload.html#caveats

Another great option is to write your code in a separate script and send it to django shell, like this:

manage.py shell < my_script.py
Answered By: Alex Matos

My solution on 2016 (in future it may be changed)

1.Install django_extension

2.Add next settings:

SHELL_PLUS = 'ipython'

IPYTHON_ARGUMENTS = [
    '--ext', 'autoreload',
]

3.Run shell

./manage.py shell_plus

See results:

model example

class Notification(models.Model):

    ........

    @classmethod
    def get_something(self):

        return 'I am programmer'

In shell

In [1]: Notification.get_something()
Out[1]: 'I am programmer'

Made changes on model

@classmethod
    def get_something(self):

        return 'I am Python programmer'

In shell

# shell does not display changes
In [2]: Notification.get_something()
Out[2]: 'I am programmer'

In shell. This is a magic

# configure extension of ipython
In [3]: %autoreload 2

In shell

# try again - all worked
In [4]: Notification.get_something()
Out[4]: 'I am Python programmer'

Made changes again

    @classmethod
    def get_something(self):

        return 'I am full-stack Python programmer'

In shell

# all worked again
In [5]: Notification.get_something()
Out[5]: 'I am full-stack Python programmer'

Drawback:
1. Need manually run code

%autoreload 2

since django_extension 1.7 has not support for run arbitrary code. May be in future release it has this feature.

Notes:

  1. Django 1.10
  2. Python 3.4
  3. django_extension 1.7.4
  4. Based (primary) on https://django-extensions.readthedocs.io/en/latest/shell_plus.html and http://ipython.readthedocs.io/en/stable/config/extensions/autoreload.html
  5. Caution. It is may be produce an error, if you try change a code where used super().
Answered By: PADYMKO

From the answers of Seti Volkylany and pv

  1. Install IPython: pip install ipython
  2. Run python manage.py shell : the symbol at the beginning of a line should now be In [1]: (in cmd it was >>>)
  3. Run ipython profile create
  4. Go in ~/.ipython/profile_default/ipython_config.py and open it in a text editor and add these two lines at the end:

    c.InteractiveShellApp.extensions = [‘autoreload’]
    c.InteractiveShellApp.exec_lines = [‘%autoreload 2’]

You can now run python manage.py shell, edit your models without having to write %autoreload 2

Answered By: MagTun
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.