Using a seeded Faker instance with Recipes breaks in versions 1.3.3 and greater

[epicserve]

Describe the issue
When using a seeded faker instance to generate strings (e.g. username) the assertions randomly fail, in versions 1.3.3 and greater of Model Bakery.

To Reproduce
You can view my example project that has tests that reproduce the bug using tox to run the tests in multiple versions of Model Bakery and Django. It also has a detailed README.md with everything that has been tried.

But you can reproduce it with a test like the following and running it multiple times using something like: for i in {1..20}; do pytest tests/test_recipes.py::test_faker_recipe; done

import pytest
from faker import Faker
from model_bakery.recipe import Recipe

@pytest.mark.django_db
def test_faker_recipe(self):
    faker = Faker()
    faker.seed_instance(10)

    user_recipe = Recipe(
        "auth.User",
        username=faker.user_name,
        email=faker.email,
    )

    user = user_recipe.make()
    assert user.username == "pattersonbelinda"
    assert user.email == "stevenhenry@example.com"

Expected behavior
It would be expected that no matter how many times you run the tests, they would pass every time and not randomly fail.

Versions

• Python: 3.9
• Django 3.2, 4.0
• Model Bakery >=1.3.3

[amureki]

Hey @epicserve thanks for raising the issue!

I am not familiar with Faker, but happy to try to understand what is going on here.
Definitely kudos for providing test scenario and even making an example repository! 🏅
I tried to run it and indeed sometimes it fails.

May I ask why would you not define faker methods in recipe as callable?

    user_recipe = Recipe(
        "auth.User",
        username=faker.user_name(),
        email=faker.email(),
    )

So far, that's the way of using methods in the recipes. However, you might be affected by the issue described in #49. Is that the case?

Best,
Rust

[epicserve]

Hi @amureki,

Thank you for looking into this issue. The test repo is just a reproduction of the issue we're having in our main product and doesn't really capture the use case. Our product has exports where users can export their data. To test the export, we run a create data function that creates records using Model Bakery and Faker. When we create the data the first time, we create a snapshot, and we use faker.seed_instance(10) to make sure when we run the test again, the Faker methods we use to create the same values every time instead of random data.

As to your direct question ... why don't we call the faker methods? If we called the faker method when the recipe was initialized, it wouldn't be seeded yet and would be random every time. Or at least I think that is the case.

#49 may be related, I can do some more digging on it.

[amureki]

@epicserve okay, I think, I found where the issue comes from:

model_bakery/model_bakery/baker.py

Line 428 in 912d4a8

for field in self.get_fields():

self.get_fields() is not ordered (set), so sometimes email field comes before username, so with the given seed, faker generates pattersonbelinda@example.org as an email and rli as a username.

We could consider reworking this bit and making it ordered (so for example, simply wrapping it in sorted()), but I need to think on the possible consequences...

[epicserve]

@amureki, THANK YOU SO MUCH! I tried your PR in the bug project, and it seems to have solved it. I'll also try it out in our product to see if it solves it there. It seems like all the tests pass in the PR. Any chance this could get merged and get a release pushed out? This bug is blocking us on some work we're trying to get done.

[amureki]

@epicserve https://pypi.org/project/model-bakery/1.8.0/ is out :)
Looking forward to getting your feedback from the product test. 👍

[epicserve]

@amureki, it seems to be working in my project as well!