Django, Pytest and data used in Migrations

Background

Today, I wanted to increase performance of a pytest test suite, which is running in the gitlab CI on 4 parallel workers. I already discovered earlier that the load of these 4 workers is not evenly distributed. Thus, the straight-forward idea was to execute the tests in random order. Prior to this modification, tests that were located in the same directory were frequently run on the same worker.

The rapid appearance of numerous test faults caught me off guard. I eventually fixed them all, but it took more effort than I thought.

Today I Learned

The test suite needs data from DB migrations. However, in a few tests people were using the TransactionTestCase. They were actually used indirectly through the @pytest.mark.django_db mark.

As explained in the docs, the TransactionTestCase, does not retain data from data migrations. In our case, just using the db fixture in the tests (they did not really need transaction support) fixed some test failures.