Add exclude as field parameter

[daviskirk]

Change Summary

• Add "exclude" as a field parameter so that it can be configured using model config instead of purely at .dict / .json export time.
• Main reference used is this comment: add exclude to Config #660 (comment)

Related issue number

• Closes add exclude to Config #660

Checklist

• Unit tests for the changes exist
• Tests pass on CI and coverage remains at 100%
• Documentation reflects the changes where applicable
• changes/<pull request or issue id>-<github username>.md file added describing change
  (see changes/README.md for details)

[codecov]

Codecov Report

Merging #2231 (508209d) into master (619ff26) will not change coverage.
The diff coverage is 100.00%.

❗ Current head 508209d differs from pull request most recent head be897bc. Consider uploading reports for the commit be897bc to get more accurate results

@@            Coverage Diff            @@
##            master     #2231   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           25        25           
  Lines         5109      5120   +11     
  Branches      1050      1045    -5     
=========================================
+ Hits          5109      5120   +11     

Impacted Files	Coverage Δ
pydantic/fields.py	100.00% <100.00%> (ø)
pydantic/main.py	100.00% <100.00%> (ø)
pydantic/utils.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 619ff26...be897bc. Read the comment docs.

[samuelcolvin]

No changes here, just the change description ATM.

[Bobronium]

Happy to see it's finally getting implemented, thank you!

[daviskirk]

Implementation wise, this is just about the way I'd like it.
Since there are probably multiple opinions on how the logic and strategy of merging the various parts should work, it would be great to have some other eyes on this just to make sure I didn't make any stupid assumptions.

A few key points here:

• I consolidated a lot of the "merging" logic from ValueItems. Since I was having to merge in many places the older logic (with 2 mutliple merge functions depending on the keys and types) were to much for my brain to process. If this was done for performance reasons and has to stay, I can try and reimplement it. The main changes here are:
  • Only one merging method (ValueItems.merge)
  • During processing only use dictionaries instead of checking everywhere if the values are dictionaries or sets (or combinations).
• Merge config settings, field level settings and explicit settings. The order of preference here is:
  • Explicit exclude/include on dict/json.
  • Field level settings ala ``Field(..., exclude=..., include=...)```
  • Config level settings via "fields": fields = {"a": {"exclude": ..., "include": ...}}
• Merging logic:
  • Merging of exclude settings happens as a "union" so overriding a field level exclude of {"a"} with dict level exclude of {"b"} will exclude: {"a", "b"}
  • Merging of include settings happens as an "intersection" so overriding a field level include of {"a", "b"} with a dict level include of {"b"} will include: {"b"}.
  • These rules are the most consistent with the way fields are handled on the objects (when I include, ONLY the fields I set in the "include" setting are included, when I exclude everything EXCEPT the fields I set are included). It also should help with not having users to "accidentally" expose a field that was supposed to be kept unexported (for example having set a "mysecretfield" field to "exclude").

In summary: reviews are very welcome

[Bobronium]

• Merging of include settings happens as an "intersection" so overriding a field level include of {"a", "b"} with a dict level include of {"b"} will include: {"a"}.

I am probably missing something, but why {"a", "b"} | {"b"} should result in {"a"} rather than {"b"}?

[daviskirk]

I am probably missing something, but why {"a", "b"} | {"b"} should result in {"a"} rather than {"b"}?

nope, not missing anything. Typo on my part, sorry. I updated the comment.

[Nemesis7]

@samuelcolvin when can we expect this functionality to be released?

[PrettyWood]

@daviskirk Do you think it could be possible to also set a callable as a value?
For example:

class User(BaseModel):
    name: str
    friends: list[str] | None = Field(None, exclude=lambda x: x is None)

assert User(name="Alone", friends=None).json() == '{"name": "Alone"}'
assert User(name="Popular", friends=["Bob", "Alice"]).json() == '{"name": "Popular", "friends": ["Bob", "Alice"]}'

WDYT?

[daviskirk]

@daviskirk Do you think it could be possible to also set a callable as a value?

I think that would be possible... I do think that it would open up a pretty large field of possibilities for users to "use"/"misuse" that I'm not sure I can predict right now.

Given all the larger changes and v2 coming along I'm not sure if we want to commit to such a broad feature without lot's of users actually really wanting it.

For example, I can behavior the behavior of "exclude" (as implemented here already with being able to drill down into nested fields) being popular enough that we might want to expand it to other fields like
I can imagine that if we allow something like this there will quickly be people that would want the callable to take the entire object as an argument as well (similar to how validators work with the optional "values"), or using this in weird ways using "nested" values as flags and stuff like that.

Not that I'm against this... I think it would be a pretty fancy feature, just trying to look out for you and samuel who might be stuck supporting it ;)

[PrettyWood]

You're absolutely right. I just shared a recent need and got away easily with custom dict. I didn't have the nesting in mind when I shared the idea but it will surely be expected if a callable support is added. Let's not do that now 😬
The feature is already super nice and complete 👍
Thanks for the detailed answer

[samuelcolvin]

otherwise this looks great.

[samuelcolvin]

This is awesome. Thank you so much @daviskirk 🎉

[rickerp]

Any release predictions on this?