RefResolver's base_uri changes between 4.14.0 and 4.15.0?

[kratsg]

Thanks for all the work so far. We had a user catch an issue in our project that depends on jsonschema which I've already figured out a fix for (scikit-hep/pyhf#1976), but I just want to make sure I understand why . Looking at the release and the diff (v4.14.0...v4.15.0#diff-2c623f3c6a917be56c59d43279244996836262cb1e12d9d0786c9c49eef6b43c), it's not clear where in this diff the base_uri is treated differently. For us, the change is very simple (for Draft6):

From the old code:

    resolver = jsonschema.RefResolver(
        base_uri=f"file://{variables.schemas}/",
        referrer=f"{version}/{schema_name}",
        store=variables.SCHEMA_CACHE,
    )

to this new code

    resolver = jsonschema.RefResolver(
        base_uri=f"file://{variables.schemas}/{version}/",
        referrer=f"{schema_name}",
        store=variables.SCHEMA_CACHE,
    )

where we've moved the {version} up into the base-uri instead. In 4.14.0 -- we have no issues finding the schemas locally, but in 4.15.0, the old code crashes. In both 4.14.0 and 4.15.0 -- the new code is fine. Any ideas what causes this difference? I can't track it down to a specific change, other than maybe what's highlighted in the release notes.

For example, when resolving 1.0.0/workspace.json:

{
  "$schema": "http://json-schema.org/draft-06/schema#",
  "$id": "https://scikit-hep.org/pyhf/schemas/1.0.0/workspace.json",
  "$ref": "defs.json#/definitions/workspace"
}

it will try to look for <path>/schemas/defs.json locally. This indicates that only the base_uri is used to search through, meaning the $id is ignored (as in the release notes).

[Julian]

Hi! bf92aed is the commit which fixed this on draft 6 if that's what you're looking for? In the schema you're showing, the specified behavior is that your $id should be ignored there, so it doesn't affect the resolution URI. In newer drafts though it'd work.

(Note by the way also that though it probably doesn't matter, you're not passing the right type of object to referrer when constructing a RefResolver. The argument is supposed to be the referrer document, i.e. a schema -- the one you want the precise URI f"file://{variables.schemas}/{version}/" to resolve to. If you never actually use that URI and it's just used as a base URI, then yeah it doesn't matter, but just pointing out it's not supposed to be a string).

[kratsg]

Ok, I think then I want to update to a newer draft. But then this gets me confused about about local vs remote. These schemas we have hosted online on GitHub pages (so the $id should refer to those for people using them outside of our python package). However, since we ship them in our python package, it wouldn't make sense to ship two sets, one with "$id": "https://...." and one with "$id": "file:///...." (when the latter depends on where the package is installed). Is base_uri meant for local resolving or remote resolving?

The goal here is we want to make sure offline access works with our package (which is why it was written this way).

The argument is supposed to be the referrer document, i.e. a schema -- the one you want the precise URI f"file://{variables.schemas}/{version}/" to resolve to

So if we wanted https://scikit-hep.org/pyhf/schemas/1.0.0/defs.json to point to the local file, we should instead do

resolver = jsonschema.RefResolver(
    base_uri=f"file://{variables.schemas}/{version}/defs.json",
    referrer=load_schema(f"{version}/defs.json"),
    store=variables.SCHEMA_CACHE,
)

(here: load_schema looks for the schema locally.) It sounds like $id isn't really used for resolving, and only $ref is meant to be used with respect to some "base" (referrer) document. Which is fine, if defs.json was a referrer document. It's more like we have schemas workspace.json, model.json, etc.. that all point to defs.json. So we wanted a RefResolver that will correctly only load locally, and never remotely. So RefResolver is only meant to resolve $ref in workspace.json and model.json that refer only to defs.json? Or am I misunderstanding the documentation? I don't know why it's all very confusing.

[Julian]

I don't know why it's all very confusing.

Don't worry, it's not you :), it's just confusing. Any doc suggestions you have are of course welcome though regardless they're in the midst of an overhaul to add more examples (eventually, need to get some other things done the next week first).

it wouldn't make sense to ship two sets, one with "$id": "https://...." and one with "$id": "file:///...."

$id is a canonical URI. It basically says "this schema is identified by this URI". It does not say that that's where the schema was retrieved from, nor does it even say that it can be retrieved from there, though of course it's common for it to be able to.

Is base_uri meant for local resolving or remote resolving?

Both -- it's the way to set which URL to resolve any relative reference against in the root schema if no $id is found first -- if that URL is local it's local, if it's remote it's remote.

So if we wanted https://scikit-hep.org/pyhf/schemas/1.0.0/defs.json to point to the local file, we should instead do

If you're trying to make sure all your schemas are retrieved straight from the shipped files really all you need to do is make sure store is a dict whose keys are all the URIs and values are the schemas. But those $ids certainly can be HTTP URIs. I.e. that argument is you caching what all the URIs are meant to point to. So in pseudocode all_schemas = {schema["$id"]: schema for schema in importlib.resources.files(mypackagedir)}; resolver = RefResolver(base_uri="file://dev/null", referrer={}, store=all_schemas}) should even do.

I have to do some other things this morning so may have to get back to the rest of your comment if the above isn't clear, but hopefully it's a start.

[kratsg]

This is a start. So first, I went with Draft202012Validator and I have this:

    resolver = jsonschema.RefResolver(
        base_uri=f"file://{variables.schemas}/{version}/{schema_name}",
        referrer={},
        store=variables.SCHEMA_CACHE,
    )   
    validator = jsonschema.Draft202012Validator(
        schema, resolver=resolver, format_checker=None
    )   

where the variables.SCHEMA_CACHE is currently empty dictionary. If I run this on the model.json schema

{
  "$schema": "http://json-schema.org/draft-06/schema#",
  "$id": "https://scikit-hep.org/pyhf/schemas/1.0.0/model.json",
  "$ref": "defs.json#/definitions/model"
}

I get the following error (no internet)

Traceback (most recent call last):
  File "/Users/kratsg/.pyenv/versions/pyhf-dev/lib/python3.8/site-packages/jsonschema/validators.py", line 885, in resolve_from_url
    document = self.store[url]
  File "/Users/kratsg/.pyenv/versions/pyhf-dev/lib/python3.8/site-packages/jsonschema/_utils.py", line 28, in __getitem__
    return self.store[self.normalize(uri)]
KeyError: 'https://scikit-hep.org/pyhf/schemas/1.0.0/defs.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "resolution.py", line 31, in <module>
    pyhf.schema.validate(spec, 'model.json')
  File "/Users/kratsg/pyhf/src/pyhf/schema/validator.py", line 39, in validate
    return validator.validate(spec)
  File "/Users/kratsg/.pyenv/versions/pyhf-dev/lib/python3.8/site-packages/jsonschema/validators.py", line 302, in validate
    for error in self.iter_errors(*args, **kwargs):
  File "/Users/kratsg/.pyenv/versions/pyhf-dev/lib/python3.8/site-packages/jsonschema/validators.py", line 277, in iter_errors
    for error in errors:
  File "/Users/kratsg/.pyenv/versions/pyhf-dev/lib/python3.8/site-packages/jsonschema/_validators.py", line 294, in ref
    scope, resolved = validator.resolver.resolve(ref)
  File "/Users/kratsg/.pyenv/versions/pyhf-dev/lib/python3.8/site-packages/jsonschema/validators.py", line 876, in resolve
    return url, self._remote_cache(url)
  File "/Users/kratsg/.pyenv/versions/pyhf-dev/lib/python3.8/site-packages/jsonschema/validators.py", line 890, in resolve_from_url
    raise exceptions.RefResolutionError(exc)
jsonschema.exceptions.RefResolutionError: HTTPSConnectionPool(host='scikit-hep.org', port=443): Max retries exceeded with url: /pyhf/schemas/1.0.0/defs.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x135210c70>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))

So I guess the point is that one would need to prepopulate variables.SCHEMA_CACHE with an entry for https://scikit-hep.org/pyhf/schemas/1.0.0/defs.json to make this work as expected. I guess the point here is that the $id in model.json is being used to locate defs.json alongside at https://scikit-hep.org/pyhf/schemas/1.0.0/ which is ignoring the base_uri specified in RefResolver because $id is specified [not fully what I expected but ok, that's fair].

[plondino]

I don't know why it's all very confusing.

Don't worry, it's not you :), it's just confusing. Any doc suggestions you have are of course welcome though regardless they're in the midst of an overhaul to add more examples (eventually, need to get some other things done the next week first).

it wouldn't make sense to ship two sets, one with "$id": "https://...." and one with "$id": "file:///...."

$id is a canonical URI. It basically says "this schema is identified by this URI". It does not say that that's where the schema was retrieved from, nor does it even say that it can be retrieved from there, though of course it's common for it to be able to.

Is base_uri meant for local resolving or remote resolving?

Both -- it's the way to set which URL to resolve any relative reference against in the root schema if no $id is found first -- if that URL is local it's local, if it's remote it's remote.

So if we wanted https://scikit-hep.org/pyhf/schemas/1.0.0/defs.json to point to the local file, we should instead do

If you're trying to make sure all your schemas are retrieved straight from the shipped files really all you need to do is make sure store is a dict whose keys are all the URIs and values are the schemas. But those $ids certainly can be HTTP URIs. I.e. that argument is you caching what all the URIs are meant to point to. So in pseudocode all_schemas = {schema["$id"]: schema for schema in importlib.resources.files(mypackagedir)}; resolver = RefResolver(base_uri="file://dev/null", referrer={}, store=all_schemas}) should even do.

I have to do some other things this morning so may have to get back to the rest of your comment if the above isn't clear, but hopefully it's a start.

This answer was extremely helpful, and I think it should be called out in the docs somewhere - I imagine it's a very common use case to define the mapping between the $id field in the schema to the resolved schema where the $id doesn't represent a downloadable location.

[Julian]

This interface is changing in the next few weeks once the referencing branch is merged, which uses the new referencing library for ref resolution, and is generally a lot simpler to configure. Once that happens, RefResolver will be deprecated entirely, so all of this won't apply.

I'll be writing up docs for how to use referencing in that time as well, though of course improvements / PRs are and will be welcome.