FIX: validation on model attribute with nested Literal breaks

[DBCerigo]

Change Summary

Nested literals, e.g. Literal['foo', Literal['bar']] are legal syntax per PEP 586. mypy correctly resolves them, as does pydantic.mypy for model attributes.

Nested literals break pydantic model attribute validation at run time. For a comprehensive example case, including pydantic.mypy output and runtime error output see example script.

This PR implements a fix for this by implementing a function to flatten nested literals down to a a list of their raw values.

Bonus notes:

• I put 85bf860, adding test for the flatten function in its own commit, in case testing internals as such wasn't wanted
• The tuple cast was so that these changes would be back compatible with the literal tests in test_types.py.
  • I think it would better to use the set (allowed_choices_set ) instead of the tuple (permitted_choices ) in https://github.com/samuelcolvin/pydantic/blob/aaec3c9c15581d2de31bcda0fbf096638386ee20/pydantic/validators.py#L397 but this would have required altering those literal tests test_types.py, and I wanted to keep the scope as small as possible.
  • Those tests test_types.py look quite stale though (have skip markers, and don't use the proper python version switches used in the rest of pydantic). So maybe it would be worth removing them for these ones in test_validators.py`? Lmk in if so for any case - happy to make amends as wanted.

Checklist

• Unit tests for the changes exist
• Tests pass on CI and coverage remains at 100%
• Documentation reflects the changes where applicable
• changes/1364-DBCerigo.md file added describing change

[codecov]

Codecov Report

Merging #1364 into master will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master     #1364   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           21        21           
  Lines         3772      3774    +2     
  Branches       749       750    +1     
=========================================
+ Hits          3772      3774    +2     

Impacted Files	Coverage Δ
pydantic/typing.py	100.00% <100.00%> (ø)
pydantic/validators.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 58b95b7...6949f5b. Read the comment docs.

[DBCerigo]

Hm, the breaks & the loss of code coverage are from imports from other pydantic modules than the one altered in this PR 🤔

Any hints at how you think they are best resolved appreciated. (I would write a test for literal_values, but there is no test_typing.py file yet, so makes me think making such a file isn't the proper solution...!)

[PrettyWood]

Hi @DBCerigo ! Well it tries to run your tests without extra dependencies (so without typing_extensions) meaning the tests on python < 3.8 fail
You should just add a skipif like it's done for test_interval_validation_error

@pytest.mark.skipif(not Literal, reason='typing_extensions not installed')
def test_interval_validation_error():

[PrettyWood]

Btw imo the method to retrieve all the values of a Literal should be next to literal_values and could be simplified like this:

def all_literal_values(type_: Type[Any]) -> Tuple[Any, ...]:
    """
    This method is used to retrieve all Literal values as
    Literal can be used recursively (see https://www.python.org/dev/peps/pep-0586)
    e.g. `Literal[Literal[Literal[1, 2, 3], "foo"], 5, None]`
    """
    if not is_literal_type(type_):
        return type_,

    values = literal_values(type_)
    return tuple(x for value in values for x in all_literal_values(value))

[DBCerigo]

@PrettyWood thanks for guidance, much appreciated.

I've added the test skips, and used your improved implementation, and moved the location of the function to the typing module.

The only bit I think is still not up to scratch is the location of testing all_literal_values - currently this is still in test_validators.py which isn't very appropriate, but there doesn't seem to yet be a test_typing.py - should I make that file? Or is there a better place for that test to live.

Thanks again.

(For my own understanding, what is reasoning for having just the Ubuntu CI run without external deps and thus differ from the osx and windows? Or have I misunderstood why it was just the Ubuntu builds that failed before?)

[PrettyWood]

@DBCerigo Well I can just tell you what I would have done but @samuelcolvin may think otherwise. I would have done a new file test_typing.py to test properly the new all_literal_values method and just an extra test in test_validators.py or test_main.py with one example for nested Literal.

As for the CI, I reckon the main reason is just that it's useless to do all checks possible on every OS. It's quite common to make heavy tests on ubuntu (here we test that everything works with extra dependencies, without, without compiled binaries...for all supported python versions). Don't know if it's relevant to do everything possible on every OS. Checks on mac and windows are already quite complete. Maybe there is another reason though 🤷‍♂️

[DBCerigo]

@DBCerigo Well I can just tell you what I would have done but @samuelcolvin may think otherwise. I would have done a new file test_typing.py to test properly the new all_literal_values method and just an extra test in test_validators.py or test_main.py with one example for nested Literal.

Cool, as this was my thinking also then I'll go for it and @samuelcolvin can say to correct as required.

As for the CI, I reckon the main reason is just that it's useless to do all checks possible on every OS. It's quite common to make heavy tests on ubuntu (here we test that everything works with extra dependencies, without, without compiled binaries...for all supported python versions). Don't know if it's relevant to do everything possible on every OS. Checks on mac and windows are already quite complete. Maybe there is another reason though 🤷‍♂️

Thanks for that. Having now looked at the CI steps in more detail I see that the build is of course tested with deps on ubuntu, but is also tested in those other settings (without deps etc.) only on ubuntu, hence the difference - so all clear to me now :)

Last thing just to check; I won't re-jig the commit history for this PR too much, as it looks like all PRs are merged with squash by default. Let me know if this is wrong, or it should have a tidier history in any case.

[PrettyWood]

@DBCerigo I like to have a clean history to help the reviewers but you are right all PRs are squash and merged. Since this PR is very small the whole diff is enough for a review so no need to worry about the history too much

[DBCerigo]

@PrettyWood yep - if it was more substantial I would make the commit history proper to enabling stepping through the commits. Thanks for all the help again!

[samuelcolvin]

otherwise looks great, sorry for the very slow response.

[DBCerigo]

@samuelcolvin thanks for the review. Hopefully good to go this time :)

[samuelcolvin]

Thanks so much.