Handling Unmanaged Models in Pytest-Django

Sandeep Rawat — Thu, 26 Dec 2024 10:11:52 +0000

The Challenge of Testing Unmanaged Models

In Django projects, we occasionally encounter unmanaged models—models that don’t have managed = True in their meta options. These models can make testing tricky, especially when your test setup involves a mix of managed and unmanaged models or multiple databases (e.g., one with managed models and another with unmanaged models).

This blog post explores approaches to testing unmanaged models with pytest-django, highlighting pros, cons, and workarounds to help you manage these scenarios effectively.

Approach 1: Mark All Models as Managed

One straightforward way to handle unmanaged models during testing is to temporarily mark them as managed. Here’s how you can do it:

# Add this to conftest.py
@pytest.hookimpl(tryfirst=True)
def pytest_runtestloop():
    from django.apps import apps
    unmanaged_models = []
    for app in apps.get_app_configs():
        unmanaged_models += [m for m in app.get_models()
                             if not m._meta.managed]
    for m in unmanaged_models:
        m._meta.managed = True

Note: For this approach to work, you need to add a --no-migrations option to your pytest settings (or pytest.ini)

Reference: Stack Overflow

Pros:

Simple to implement.

Cons:

Skips migration testing, which can cause issues when multiple developers are working on the same project.

Approach 2: Create Unmanaged Models Manually

Alternatively, you can manually create unmanaged models during the test setup. This approach ensures that migrations are tested:

@pytest.fixture(scope="session", autouse=True)
def django_db_setup(django_db_blocker, django_db_setup):
    with django_db_blocker.unblock():
        for _connection in connections.all():
            with _connection.schema_editor() as schema_editor:
                setup_unmanaged_models(_connection, schema_editor)
        yield

def setup_unmanaged_models(connection, schema_editor):
    from django.apps import apps

    unmanaged_models = [
        model for model in apps.get_models() if model._meta.managed is False
    ]
    for model in unmanaged_models:
        if model._meta.db_table in connection.introspection.table_names():
            schema_editor.delete_model(model)
        schema_editor.create_model(model)

Pros:

Tests migrations as part of your test cases.

Cons:

Slightly more complex.
transaction=True doesn’t work with this approach (discussed in the next section).

Understanding Transactional Tests

Pytest-django provides a database fixture: django_db and django_db(transaction=True). Here’s how they differ:

django_db: Rolls back changes at the end of a test case, meaning no actual commit is made to the database.

django_db(transaction=True): Commits changes and truncates the database tables after each test case. Since only managed models are being truncated after every test, this is the reason unmanaged models require special handling during transactional tests.

Example Test Case

@pytest.mark.django_db
def test_example():
    # Test case logic here
    pass

@pytest.mark.django_db(transaction=True)
def test_transactional_example():
    # Test case logic here
    pass

Making Transactional Tests Work with Unmanaged Models

Since transactional tests truncate only managed models, we can modify unmanaged models to be managed during the test run. This ensures they are included in truncation:

def setup_unmanaged_models(connection, schema_editor):
    # As before
    for model in unmanaged_models:
        # As before
        model._meta.managed = True  # Hack: Mark as managed

Avoiding transaction=True with on_commit Hooks (iff possible)

In scenarios involving on_commit hooks, you can avoid using transactional tests by capturing and executing on_commit callbacks directly, using fixture django_capture_on_commit_callbacks from pytest-django(>= v.4.4):

@pytest.mark.django_db
def mytest(django_capture_on_commit_callbacks):
   with django_capture_on_commit_callbacks(execute=True):
      # Test logic here
      # `on_commit` hooks will execute immediately

References

Do you have other approaches or tips for handling unmanaged models? Share them in the comments below!

Forem: Sandeep Rawat