"no such column" after adding a field to the model

Those migration issues bother me too when i have to update my app with new columns, so i’ve made a small bash with some python to search for troublesome columns, remove them, migrate, re-add them and migrate again.

Here is the resolve.sh that calls python and migrations (has to be placed in the same folder as your manage.py file, like the .py files) :

python manage.py shell < check.py

read -p "Does it seem ok ? (y for ok/n for remigrate)" migrateok

if [ $migrateok = "n" ]
then
    python manage.py shell < rm_badcolumns.py

    python manage.py makemigrations
    sleep 1
    python manage.py migrate

    python manage.py shell < resume_badcolumns.py

    python manage.py makemigrations
    sleep 1
    python manage.py migrate

    echo It should work now..
else
    echo No changes were made..
fi

check.py checks if there are issues with some models in the database :

from <app>.models import *
import os, sys
from shutil import move
from django.db import connection

tables = connection.introspection.table_names()
seen_models = connection.introspection.installed_models(tables)

errorColumn = []
is_fine = "EVERYTHING LOOKS FINE!"
#looping through models to detect issues
for model in seen_models:
    try:
        print(model.objects.all())
    except:
        is_fine = "WARNING SOME MODELS ARE CORRUPTED"

print(is_fine)

This check will tell the user if some models are not sync with your database.

If the user chooses to “re-migrate”, it calls rm_badcolumns.py :

from <app>.models import *
import os, sys
from shutil import move
from django.db import connection


tables = connection.introspection.table_names()
seen_models = connection.introspection.installed_models(tables)

errorColumn = []

for model in seen_models:
    try:
        model.objects.all()
    except:
        errorColumn.append(str(sys.exc_info()[1])[30::]+' =')
        #this weird expression just get the column that causes trouble
        #you may have to adapt indexes depending on error output in exc_info

os.chdir('./<app>/')

#removing columns from models.py based on the error pattern
def replace(pattern, subst):
    with open('models_sub.py','w') as fout:
        with open('models.py','r') as models:
            for line in models:
                fout.write(line.replace(pattern, subst))

    os.remove('models.py')
    move('models_sub.py','models.py')

#applying this func to each error, and commenting out columns with an error mark to catch them back later
for errorStr in errorColumn:
    replace(errorStr, '#error_mark '+errorStr)

print 'Removing troublesome column for re-entering..'

Then resolve.sh will start another migration.
We uncomment the troublesome columns with resume_badcolumns.py to get them migrated again :

from <app>.models import *
import os, sys
from shutil import move
from django.db import connection


os.chdir('./<app>/')

#same search and replace func but returning True if sthg was replaced, and False otherwise
def replace(pattern, subst):
    a = False
    with open('models_sub.py','w') as fout:
        with open('models.py','r') as models:
            for line in models:
                fout.write(line.replace(pattern, subst))
                if pattern in line:
                    a = True
    os.remove('models.py')
    move('models_sub.py','models.py')

    return a

a = True
#while error_marks are replaced, we go on looping
while a:
    a = replace('#error_mark ', '')

print 'Re-adding troublesome columns..'

We make a final migration and everything should be fine.

This is a kind of heavy and dirty artillery for a small problem but i wanted it to be automatized.
A few points may need some rework, such as the way i identify which column is not synced, and i’d appreciate comments about this.

This can happen if you are referencing your model at the root level of your app

This happened to me when I was updating the app mapping_master. I was adding a new field like so:

class MappingMaster(models.Model):

    ...

    # New field that was being added
    statement = models.CharField(max_length=20, choices=STATEMENT_CHOICES, default='PNL', blank=True, null=True)

Gave me the following stacktrace:

D:Adwaithcodebaseunitapp>python manage.py makemigrations
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "C:Python27libsite-packagesdjangocoremanagement__init__.py", line
338, in execute_from_command_line
    utility.execute()
  File "C:Python27libsite-packagesdjangocoremanagement__init__.py", line
312, in execute
    django.setup()

....
....

  File "C:Python27libsite-packagesdjangoappsconfig.py", line 198, in impor
t_models
    self.models_module = import_module(models_module_name)
  File "C:Python27libimportlib__init__.py", line 37, in import_module
    __import__(name)
  File "D:Adwaithcodebaseunitapptrial_balance_entrymodels.py", line 5, in <
module>
    from unitapp import docclass
  File "D:Adwaithcodebaseunitappunitappdocclass.py", line 139, in <module>
    sample_train_type(type_classifier)
  File "D:Adwaithcodebaseunitappunitappdocclass.py", line 6, in sample_trai
n_type
    for mapping in MappingMaster.objects.all():

....
....

  File "C:Python27libsite-packagesdjangodbbackendssqlite3base.py", line
318, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such column: mapping_master_mappingmaster.statement

Turns out that my problem was in another file entirely. It was in the trial_balance_entry app:

...
# The important line is below
from unitapp import docclass


class TrialBalanceEntry(models.Model):
    ...

And inside docclass.py, I had:

import re, csv, os
from mapping_master.models import MappingMaster


def sample_train_type(cl):
    for mapping in MappingMaster.objects.all():
        cl.train(mapping.entry, mapping.type)


def sample_train_category(cl):
    for mapping in MappingMaster.objects.all():
        cl.train(mapping.entry, mapping.category)

...

Turns out that the MappingMaster model instances were being referenced at the root of the app (since I imported it at the start of the file in the models file in trial_balance_entry.

I fixed that by moving the import to one of the inner methods of my TrialBalanceEntry model. This made sure I didn’t have any hidden circular dependencies.

P.S. From next time, please provide a stacktrace from the console so we can debug it more easily.

Keeping it short and precise, give that field a default value.

For example:

nickname = models.CharField(max_length=12, default=" ") 

and not default=True

If that doesn’t work delete the migrations in your app’s migration folder except

__init__.py file and type both the commands again.
python manage.py migrate
python manage.py makemigrations appname
python manage.py migrate(again)

Well the problem is NOT with your makemigrations command or models.py. It is because you have probably imported your class in the model (your database in this case) in on of your views.py files and the problem is with that. If you read the all of the error message then you can understand that easily.

Just try commenting that importing part and run your python.manage.py makemigrations and python manage.py migrate commands then you can uncomment your import in your views.py file

Hope this was useful for others as well

Easyes Solution:
RENAME the field
makemigrations
migrate
RENAME back
makemigrations
migrate

All, that u need: download «DB Browser for SQLite» and:

1. Copy your ‘db.sqlite3’ to different Folder.
2. Delete your ‘db.sqlite3’.
3. Run ‘python3 manage.py make migrations’, then ‘python3 manage.py migrate’.
4. Open your new ‘db.sqlite3’ in «DB Browser for SQLite» and find table ‘account_profile’
5. Then do like 1st image. U should export ‘account-profile’ table to CSV file.
6. Delete your new ‘db.sqlite3’ and return you old ‘db.sqlite3’
7. Open your DB in «DB Browser for SQLite», then «File—>Import—>Tables from file CSV..» and choose ‘account_profile.csv’
8. Then U will see, that in this table u should change columns ‘field1’ to ‘id’ and ‘field2’ to ‘user_id’. I did it on 2nd and 3rd images.
9. Last, what should u do: Look at ‘id’ of your admin in the table ‘auth_user’ and change on this value in the ‘account_profile table’
10. Delete your Admin-Model in DB(auth_user table)

Yeah, 5-10 minutes to do it, but all is work)

I fixed this issue by commenting out all the imports of the model from views, then trying to migrating. Deleting the db.sqlite3 file and all these steps weren’t necessary, because most of the time this error gets triggered when you try to import a model that doesn’t have migrations for certain fields.