Fix inconsistent processing of model docstring formfeed char

[MaxwellPayne]

Problem

Generating openapi specs for models with a formfeed (\f) in the docstring yields inconsistent results. For example, in the case of:

class Address(BaseModel):
    """
    This is a public description of an Address
    \f
    You can't see this part of the docstring, it's private!
    """
    ...

Sometimes the openapi description of Address correctly renders as This is a public description of an Address\n. But other times, you get the entire docstring even though the \f should be keeping the remainder of the docstring private.

The bug seemed to be in the following function:

def get_model_definitions(
    *,
    flat_models: Set[Union[Type[BaseModel], Type[Enum]]],
    model_name_map: Dict[Union[Type[BaseModel], Type[Enum]], str],
) -> Dict[str, Any]:
    definitions: Dict[str, Dict[str, Any]] = {}
    for model in flat_models:
        m_schema, m_definitions, m_nested_models = model_process_schema(
            model, model_name_map=model_name_map, ref_prefix=REF_PREFIX
        )
        definitions.update(m_definitions)
        model_name = model_name_map[model]
        if "description" in m_schema:
            m_schema["description"] = m_schema["description"].split("\f")[0]
        definitions[model_name] = m_schema
    return definitions

The issue is that the definitions dict has its entries updated in two different places:

1. definitions.update(m_definitions)
2. definitions[model_name] = m_schema

When a model definition is written in line (2), the proper formfeed escape of the docstring is persisted. But if the same model is subsequently written by line (1), the previous docstring fix is clobbered over and we see the entire docstring.

This is a non-deterministic error, since flat_models is an unordered set. Depending on order of models in iteration, you could end up with either (1) or (2) as the final edit to a given model.

Solution

This fix works around the bug by handling the formfeed escape at the end of model processing. By that point our definitions dict will be final, so any edits to the models are guaranteed to make it into the return value.

Related issues

• ✨ Support Python internal description on Pydantic model's docstring #3032

Questions & Alternatives

• Should Pydantic handle this on the model_process_schema() function instead? (also asked in ✨ Support Python internal description on Pydantic model's docstring #3032)
• Might there be other pre-existing, undiscovered bugs resulting from the fact that definitions is updated in two different ways?

[github-actions]

📝 Docs preview for commit 8a90482 at: https://63f7b86c8f57d70058148cad--fastapi.netlify.app

[github-actions]

📝 Docs preview for commit 0eef5f7 at: https://63f7ba9bcac94d0068451d8b--fastapi.netlify.app

[github-actions]

📝 Docs preview for commit 3a15194 at: https://63f7bcb8cac94d008f451d7c--fastapi.netlify.app