schema ref_template #1480

Kilo59 · 2020-05-05T22:26:51Z

Change Summary

Adds a ref_template option to schema generation that takes a string.Template and substitutes the model_name when generating $refs to other schemas.

my_model.schema(ref_template=string.Template("/foo/${model_name}.json#/bar"))

Edit:

Updated to just use a regular .format() template.
#1479 (comment)

my_model.schema(ref_template="/foo/${model}.json#/bar")

Related issue number

#1479

Checklist

Unit tests for the changes exist
Tests pass on CI and coverage remains at 100%
Documentation reflects the changes where applicable
changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)

change to regular string.format() template

codecov · 2020-05-05T22:45:42Z

Codecov Report

Merging #1480 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1480   +/-   ##
=======================================
  Coverage   99.90%   99.90%           
=======================================
  Files          21       21           
  Lines        4000     4003    +3     
  Branches      798      799    +1     
=======================================
+ Hits         3996     3999    +3     
  Misses          3        3           
  Partials        1        1

Impacted Files	Coverage Δ
pydantic/schema.py	`100.00% <100.00%> (ø)`
pydantic/typing.py	`96.85% <0.00%> (ø)`
pydantic/datetime_parse.py	`100.00% <0.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a43c2d2...1a3f30e. Read the comment docs.

Kilo59 · 2020-05-05T22:54:14Z

pydantic/schema.py

+        if ref_template:
+            m_schema = {'$ref': ref_template.substitute(model_name=model_name)}
+        else:
+            m_schema = {'$ref': ref_prefix + model_name}


This could probably be done more elegantly.

Some additional things I could do.

regular ref_prefixs could also be resolved using the same string.Template method. "#/components/schemas/${model_name}". Could avoid this if else conditional.

better error message if a bad template is passed.

coerce a regular string into a template instead of taking an actual string.Template object.

Kilo59 · 2020-05-05T23:00:41Z

pydantic/schema.py

@@ -69,6 +70,7 @@ def schema(
    title: Optional[str] = None,
    description: Optional[str] = None,
    ref_prefix: Optional[str] = None,
+    ref_template: Optional[string.Template] = None,


With no extra handling...

If str is passed instead of a string.Template an AttributeError will be raised.
AttributeError: 'str' object has no attribute 'substitute'

If string.Template is passed that doesn't have the $model_name identifier a KeyError will be raised.
KeyError: 'foo_bar'

https://docs.python.org/3/library/string.html#template-strings

Kilo59 · 2020-05-05T23:01:34Z

tests/test_schema.py

+def test_schema_with_ref_template():
+    class Foo(BaseModel):
+        a: str
+
+    class Bar(BaseModel):
+        b: Foo
+
+    class Baz(BaseModel):
+        c: Bar
+
+    model_schema = schema([Bar, Baz], ref_template=string.Template('/schemas/${model_name}.json#/'))
+    assert model_schema == {
+        'definitions': {
+            'Baz': {
+                'title': 'Baz',
+                'type': 'object',
+                'properties': {'c': {'$ref': '/schemas/Bar.json#/'}},
+                'required': ['c'],
+            },
+            'Bar': {
+                'title': 'Bar',
+                'type': 'object',
+                'properties': {'b': {'$ref': '/schemas/Foo.json#/'}},
+                'required': ['b'],
+            },
+            'Foo': {
+                'title': 'Foo',
+                'type': 'object',
+                'properties': {'a': {'title': 'A', 'type': 'string'}},
+                'required': ['a'],
+            },
+        }
+    }
+
+


I'm happy to add more tests.

samuelcolvin · 2020-06-27T12:53:57Z

sorry for the slow reply, as I mentioned on the issue #1479, I think a normal str should be used not Template.

It's not necessary, and Template is very rarely used so will confuse many people.

Kilo59 · 2020-06-27T13:06:20Z

sorry for the slow reply, as I mentioned on the issue #1479, I think a normal str should be used not Template.

It's not necessary, and Template is very rarely used so will confuse many people.

Makes sense, and thanks for the reminder. I'll try to update this over the weekend.

Kilo59 · 2020-07-04T18:35:44Z

pydantic/schema.py

@@ -76,6 +77,7 @@ def schema(
    title: Optional[str] = None,
    description: Optional[str] = None,
    ref_prefix: Optional[str] = None,
+    ref_template: Optional[str] = None,


@samuelcolvin I was just following the existing pattern here.
But should I set the default_template as the default argument for ref_template?

ref_template: Optional[str] = '#/definitions/{model}',

I think better to switch to ref_template: str = '#/definitions/{model}',, the others are only optional because they really can be optional.

The only advantage of making this optional would be that ref_template could take priority over ref_prefix if it's not None.

Done.
I don't know if it would have been better to keep a REF_TEMPLATE constant and use that as the default for all the ref_template defaults, but it looked like doing that prevented the editor help text from showing what the actual value was.

Kilo59 · 2020-07-04T19:51:33Z

Getting a failure on test_constrained_set_default, same failure that is on master~~that I don't understand.~~
https://github.com/samuelcolvin/pydantic/runs/837388454

>           raise validation_error
E           pydantic.error_wrappers.ValidationError: 1 validation error for ConSetModelMax
E           v
E             value is not a valid set (type=type_error.set)

pydantic/main.py:346: ValidationError

samuelcolvin · 2020-07-04T20:34:41Z

See #1682

Kilo59 · 2020-07-04T22:17:06Z

@samuelcolvin I still need to update the schema-customization section docs.
https://pydantic-docs.helpmanual.io/usage/schema/#schema-customization

I'll do that tonight or tomorrow.

samuelcolvin

Just need documentation and the tweak discussed here, otherwise looks good.

samuelcolvin · 2020-09-06T15:01:13Z

pydantic/schema.py

@@ -76,6 +77,7 @@ def schema(
    title: Optional[str] = None,
    description: Optional[str] = None,
    ref_prefix: Optional[str] = None,
+    ref_template: Optional[str] = None,


I think better to switch to ref_template: str = '#/definitions/{model}',, the others are only optional because they really can be optional.

The only advantage of making this optional would be that ref_template could take priority over ref_prefix if it's not None.

use a string.Template instead of a ref_prefix to allow for more varied`$ref`s to be created. Template string is expected to have $model_name `identifier `

if no `ref_prefix` provided, use the `template_default`

test name change test for key error

samuelcolvin · 2020-10-08T18:35:43Z

This is looking great, but you still need to:

fix coverage (there are two cases) which are currently not covered
add something to docs.

Kilo59 · 2020-10-10T21:53:29Z

This is looking great, but you still need to:

fix coverage (there are two cases) which are currently not covered

What are the cases that need to be covered?

samuelcolvin · 2020-10-18T19:01:08Z

look at the codecov report above.

samuelcolvin · 2020-10-25T16:24:43Z

thanks so much.

Kilo59 changed the title ~~Ref template~~ schema ref_template May 5, 2020

Kilo59 mentioned this pull request May 5, 2020

Support more json schema $ref patterns #1479

Closed

Kilo59 force-pushed the ref_template branch 2 times, most recently from 41e3eb5 to de3e038 Compare May 5, 2020 22:42

Kilo59 commented May 5, 2020

View reviewed changes

Kilo59 marked this pull request as ready for review May 5, 2020 23:00

Kilo59 commented May 5, 2020

View reviewed changes

Kilo59 force-pushed the ref_template branch 3 times, most recently from 0fc49f4 to 82631c8 Compare July 4, 2020 18:29

Kilo59 commented Jul 4, 2020

View reviewed changes

Kilo59 force-pushed the ref_template branch 5 times, most recently from 50378db to cf5154e Compare July 4, 2020 19:45

samuelcolvin mentioned this pull request Sep 6, 2020

depreciate ref_prefix in v2 #1908

Closed

samuelcolvin reviewed Sep 6, 2020

View reviewed changes

Kilo59 added 5 commits September 6, 2020 17:45

ignore Pipfile, .lock files

4a711a4

add ref_template option

7f2ed76

use a string.Template instead of a ref_prefix to allow for more varied`$ref`s to be created. Template string is expected to have $model_name `identifier `

formatting / linting

f63540a

add changes

326fef5

typo

a4c8cc8

Kilo59 added 5 commits September 6, 2020 17:47

use string.format instead of string.Template

9876bb7

remove ref_prefix default

0e642da

if no `ref_prefix` provided, use the `template_default`

use ref_template in field_singleton_schema

13b27c5

fix test_schema_with_ref_template

d6eec34

add parameters for test_schema_with_refs

7db1940

test name change test for key error

Kilo59 force-pushed the ref_template branch from 9a4fafa to 261f184 Compare September 6, 2020 21:56

provide ref_template default argument

a3e1ca6

Kilo59 force-pushed the ref_template branch from 261f184 to a3e1ca6 Compare September 7, 2020 00:03

samuelcolvin mentioned this pull request Oct 8, 2020

feat(schema): support custom title, description and default for enums #1749

Merged

4 tasks

samuelcolvin added 6 commits October 25, 2020 14:34

Merge branch 'master' into Kilo59-ref_template

f7bf6c1

fix linting

b93f04b

fix mypy and coverage

49954bb

docs and correct model_schema usage

82b328c

fastAPI tests actualy caught an error

b3922fa

linting

1a3f30e

samuelcolvin merged commit d14731f into pydantic:master Oct 25, 2020

Kilo59 deleted the ref_template branch June 10, 2022 13:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

schema ref_template #1480

schema ref_template #1480

Kilo59 commented May 5, 2020 •

edited

codecov bot commented May 5, 2020 •

edited

Kilo59 May 5, 2020 •

edited

Kilo59 May 5, 2020

Kilo59 May 5, 2020

samuelcolvin commented Jun 27, 2020

Kilo59 commented Jun 27, 2020 •

edited

Kilo59 Jul 4, 2020 •

edited

samuelcolvin Sep 6, 2020

Kilo59 Sep 7, 2020 •

edited

Kilo59 commented Jul 4, 2020 •

edited

samuelcolvin commented Jul 4, 2020

Kilo59 commented Jul 4, 2020 •

edited

samuelcolvin left a comment

samuelcolvin Sep 6, 2020

samuelcolvin commented Oct 8, 2020

Kilo59 commented Oct 10, 2020 •

edited

samuelcolvin commented Oct 18, 2020

samuelcolvin commented Oct 25, 2020

schema ref_template #1480

schema ref_template #1480

Conversation

Kilo59 commented May 5, 2020 • edited

Change Summary

Edit:

Related issue number

Checklist

codecov bot commented May 5, 2020 • edited

Codecov Report

Kilo59 May 5, 2020 • edited

Choose a reason for hiding this comment

Kilo59 May 5, 2020

Choose a reason for hiding this comment

Kilo59 May 5, 2020

Choose a reason for hiding this comment

samuelcolvin commented Jun 27, 2020

Kilo59 commented Jun 27, 2020 • edited

Kilo59 Jul 4, 2020 • edited

Choose a reason for hiding this comment

samuelcolvin Sep 6, 2020

Choose a reason for hiding this comment

Kilo59 Sep 7, 2020 • edited

Choose a reason for hiding this comment

Kilo59 commented Jul 4, 2020 • edited

samuelcolvin commented Jul 4, 2020

Kilo59 commented Jul 4, 2020 • edited

samuelcolvin left a comment

Choose a reason for hiding this comment

samuelcolvin Sep 6, 2020

Choose a reason for hiding this comment

samuelcolvin commented Oct 8, 2020

Kilo59 commented Oct 10, 2020 • edited

samuelcolvin commented Oct 18, 2020

samuelcolvin commented Oct 25, 2020

Kilo59 commented May 5, 2020 •

edited

codecov bot commented May 5, 2020 •

edited

Kilo59 May 5, 2020 •

edited

Kilo59 commented Jun 27, 2020 •

edited

Kilo59 Jul 4, 2020 •

edited

Kilo59 Sep 7, 2020 •

edited

Kilo59 commented Jul 4, 2020 •

edited

Kilo59 commented Jul 4, 2020 •

edited

Kilo59 commented Oct 10, 2020 •

edited