Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrations that remove field return column does not exist error when running pytest #1045

Open
JaspervanderSter opened this issue Jan 31, 2023 · 0 comments

Comments

@JaspervanderSter
Copy link

TLDR of error:
Add a migration that removes a field using the django makemigrations flow. Running migrations using django is fine, no errors returned.
Running pipenv run pytest sample/tests.py returns the error ERROR sample/tests.py::TestModelTestCase::test_example - django.db.utils.ProgrammingError: column sample_testmodel.original_text does not exist.

I run into an error when testing a django application with pytest, more specifically when running tests if I have done a migration that removes a field used in an earlier migration. Doing so outside of tests gives no error.

To reproduce:

models.py
from django.db import models

class TestModel(models.Model):
    # We started with this field, but now longer need it
    # original_text = models.CharField(max_length=255)
    # We added this field and then moved over to it
    new_text = models.CharField(max_length=511)
migrations/0001_initial.py
from django.db import migrations, models


class Migration(migrations.Migration):

    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='TestModel',
            fields=[
                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('original_text', models.CharField(max_length=255)),
            ],
        ),
    ]
migrations/0002_testmodel_new_text.py
from django.db import migrations, models


def move_text_to_other_field_and_append_new(apps, schema_editor):
    TestModel = apps.get_model("sample", "TestModel")

    for test_model in TestModel.objects.all():
        test_model.new_text = test_model.original_text + " new"
        test_model.save()

class Migration(migrations.Migration):

    dependencies = [
        ('sample', '0001_initial'),
    ]

    operations = [
        migrations.AddField(
            model_name='testmodel',
            name='new_text',
            field=models.CharField(default='', max_length=511),
            preserve_default=False,
        ),
        migrations.RunPython(move_text_to_other_field_and_append_new, migrations.RunPython.noop)
    ]
migrations/0003_remove_testmodel_original_text.py
from django.db import migrations


class Migration(migrations.Migration):

    dependencies = [
        ('sample', '0002_testmodel_new_text'),
    ]

    operations = [
        migrations.RemoveField(
            model_name='testmodel',
            name='original_text',
        ),
    ]
tests.py
from django.test import TestCase


class TestModelTestCase(TestCase):

    def test_example(self):
        self.assertEqual(1, 1)

This is a minimal setup to reproduce the error. Now when running pipenv run pytest sample/tests.py I get the error ERROR sample/tests.py::TestModelTestCase::test_example - django.db.utils.ProgrammingError: column sample_testmodel.original_text does not exist. In the error stack it clearly states the following:

sample/migrations/0002_testmodel_new_text.py:9: in move_text_to_other_field_and_append_new
    for test_model in TestModel.objects.all():

And then:

self = <django.db.backends.utils.CursorWrapper object at 0x7f3e53a40e80>, sql = 'SELECT "sample_testmodel"."id", "sample_testmodel"."original_text", "sample_testmodel"."new_text" FROM "sample_testmodel"', params = ()
ignored_wrapper_args = (False, {'connection': <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f3e5d7e5d90>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7f3e53a40e80>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
                # params default might be backend specific.
                return self.cursor.execute(sql)
            else:
>               return self.cursor.execute(sql, params)
E               django.db.utils.ProgrammingError: column sample_testmodel.original_text does not exist
E               LINE 1: SELECT "sample_testmodel"."id", "sample_testmodel"."original...
E                                                       ^

../../../.local/share/virtualenvs/sprancher-wR5ufyMh/lib/python3.8/site-packages/django/db/backends/utils.py:84: ProgrammingError

This has been bugging me for a long time. It only goes wrong with the tests. Commenting out the RunPython line in 0002 has worked, but is obviously not a long term solution. Running the tests before adding 0003 outputs green lights, no issues up to that point. The order of execution seems incorrect to me or something, but I can't really wrap my head around it.

My question is, why does this happen and how do I prevent this from happening?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant