Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RecursionError when validating schema with anyOf and draft 2019-09 #847

Closed
Zeckie opened this issue Oct 1, 2021 · 12 comments
Closed

RecursionError when validating schema with anyOf and draft 2019-09 #847

Zeckie opened this issue Oct 1, 2021 · 12 comments
Labels
Bug Something doesn't work the way it should.

Comments

@Zeckie
Copy link

Zeckie commented Oct 1, 2021

A python project i'm working on recently started failing with a RecursionError. We are using jsonschema to validate quite a complex yaml file, but i've been able to cut that down to few lines.

Issue occured when using jsonschema==4.0.1 on Python 3.9.6, but did not occur with jsonschema==3.2.0

from jsonschema.validators import validate
validate(
    instance={"foo": "bar"},
    schema={
        "$schema": "https://json-schema.org/draft/2019-09/schema",
        "anyOf": [{"type": "object"}],
    },
)

Gives:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 963, in validate
    cls.check_schema(schema)
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 182, in check_schema
    for error in cls(cls.META_SCHEMA).iter_errors(schema):
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 224, in iter_errors
    for error in errors:
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_validators.py", line 359, in allOf
    yield from validator.descend(instance, subschema, schema_path=index)
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 224, in iter_errors
    for error in errors:
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_validators.py", line 295, in ref
    yield from validator.descend(instance, resolved)
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 224, in iter_errors
    for error in errors:
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_validators.py", line 329, in properties
    yield from validator.descend(
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 224, in iter_errors
    for error in errors:
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_validators.py", line 295, in ref
    yield from validator.descend(instance, resolved)
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 224, in iter_errors
    for error in errors:
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_legacy_validators.py", line 113, in items_draft6_draft7_draft201909
    yield from validator.descend(item, items, path=index)
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 224, in iter_errors
    for error in errors:
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_legacy_validators.py", line 224, in recursiveRef
    yield from validator.descend(instance, subschema)
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 224, in iter_errors
    for error in errors:
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_legacy_validators.py", line 224, in recursiveRef

... (those last 3 keep repeating) ...

  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_legacy_validators.py", line 224, in recursiveRef
    yield from validator.descend(instance, subschema)
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 224, in iter_errors
    for error in errors:
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\_legacy_validators.py", line 216, in recursiveRef
    lookup_url, next_target = validator.resolver.resolve(each)
  File "C:\...\Python\Python39\lib\site-packages\jsonschema\validators.py", line 775, in resolve
    uri, fragment = urldefrag(url)
  File "C:\...\Python\Python39\lib\urllib\parse.py", line 602, in urldefrag
    s, n, p, a, q, frag = urlparse(url)
  File "C:\...\Python\Python39\lib\urllib\parse.py", line 393, in urlparse
    splitresult = urlsplit(url, scheme, allow_fragments)
  File "C:\...\Python\Python39\lib\urllib\parse.py", line 458, in urlsplit
    url, scheme, _coerce_result = _coerce_args(url, scheme)
  File "C:\...\Python\Python39\lib\urllib\parse.py", line 120, in _coerce_args
    str_input = isinstance(args[0], str)
RecursionError: maximum recursion depth exceeded while calling a Python object
@Julian
Copy link
Member

Julian commented Oct 1, 2021

Hi there. This probably indeed is a bug, but just to clarify, you mean you created a new example different than the one you had previously been using correct?

jsonschema 3.2.0 didn't support draft2019-09, so the schema you have here wouldn't have done anything on that version.

@Zeckie
Copy link
Author

Zeckie commented Oct 3, 2021

I didn't realize that draft 2019-09 was not supported on 3.2.0. We have been using that combination for a while, and presumably the metaschema was being ignored. The instance was still being validated against the schema.

Our actual project contains a lot more in the instance and schema, and gets the data for both from yaml files. When posting this issue, I cut out everything that what was not required to reproduce the error.

I have tried a few different versions (with both the cut down code posted here, and our full project), and looks like this issue only occurs with https://json-schema.org/draft/2019-09/schema.

@Zeckie Zeckie changed the title RecursionError when validating schema with anyOf and no recursion RecursionError when validating schema with anyOf and draft 2019-09 Oct 3, 2021
@robherring
Copy link
Contributor

FWIW, I'm seeing what appears to be the same problem if using draft 2019-09 meta-schema.

@Zeckie
Copy link
Author

Zeckie commented Oct 4, 2021

@robherring does your schema use "anyOf"?

@robherring
Copy link
Contributor

@Zeckie Yes, it does. I would think the problem exists for other cases as well. 'allOf' has the same meta-schema for example.

@Julian Julian added the Bug Something doesn't work the way it should. label Oct 6, 2021
@mlucas-NU
Copy link

Facing the same issue with a known working schema/instance pair. In my case, the schema contains a oneOf, but not anyOf.

@Julian
Copy link
Member

Julian commented Oct 7, 2021

Given that this (recursiveRef) is a documented gap in 2019-09 support which I plan to fix but requires work, a PR fully gutting what's there would likely solve this issue (in the sense that it would stop blowing up the stack, even if it didn't apply the recursive validation).

If someone cares to send one that deletes the body of the implementation alongside a test that exercises this case, as I say, it'd certainly be welcome.

@olegshtch
Copy link

I suppose https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-00#appendix-C tree example couldn't be validated with the same error?

rpcope1 added a commit to cope-systems/bottle-swagger that referenced this issue Dec 15, 2021
* Pinned jsonschema to 3.2.0 to solve a serious issue with the interaction
  between bravado core and jsonschema (see: python-jsonschema/jsonschema#847)
* Fixed a bug in the order of route handling when it is desired to serve the UI and to allow
  subpaths in the Swagger API that aren't strictly defined by the specification.
rpcope1 added a commit to cope-systems/bravado-core that referenced this issue Dec 15, 2021
…used with

newer versions of jsonschema (4.0.0+); this seems to be a problem with jsonschema
itself (python-jsonschema/jsonschema#847). This commit pins
jsonschema to the last known compatible version for bravado-core until the
upstream issue is addressed.
@andylamp
Copy link

andylamp commented Apr 5, 2022

Just to chip in on this - does the issue occur with the Draft 2020-12 or just the "old" 2019-09?

Using the latest version of the package, namely v4.4.0 seems to... work without crashing...?

from jsonschema.validators import validate

# this is OK...?
validate(
	instance={"foo": "bar"},
	schema={
		"$schema": "https://json-schema.org/draft/2020-12/schema",
		"anyOf": [{"type": "object"}],
	},
)

# this exhibits the error described in the thread.
validate(
	instance={"foo": "bar"},
	schema={
		"$schema": "https://json-schema.org/draft/2019-09/schema",
		"anyOf": [{"type": "object"}],
	},
)

Can anyone confirm this behavior?

@mg-christian-axelsson
Copy link

Can anyone confirm this behavior?

Yes. I was hit by this last week and just found out about this issue.

@jrodriguezs92
Copy link

Facing the same issue in version 4.3.1 and 4.5.1, using "$schema": "https://json-schema.org/draft/2019-09/schema" and with schema containing allOf and minContains
Does this have any solution yet?

@Julian
Copy link
Member

Julian commented May 13, 2022

As I mentioned:

#847 (comment)

@Julian Julian closed this as completed in 19595b5 Jun 28, 2022
andir added a commit to andir/ipv6.watch that referenced this issue Aug 2, 2022
NixOS 22.05 does have a newer version of jsonschema that has issues with
the `anyOf` node that we are using the schema.json. Since the `anyOf`
node seems to be part of some draft that isn't fully supported there
isn't an upstream fix just yet.

The upstream `jsonschema` bug is being tracked here: python-jsonschema/jsonschema#847
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something doesn't work the way it should.
Projects
None yet
Development

No branches or pull requests

8 participants