refactor: create json schema ref resolver

[ivan-tymoshenko]

Fix #506

I want to return a custom JSON schema reference resolver with some changes. It will only resolve JSON schema references only at serializer compilation time. All refs used for validation still will be resolved by Ajv. It's a proof of concept. I need to read the JSON schema doc carefully before merging it.

[ivan-tymoshenko]

@adrai Can you confirm that these changes fix your case?

[adrai]

Yes, seems much better, but still slower than before.

v4.1.0 156ms
v4.2.0 687ms
v4.2.0 with this fix 241ms

[adrai]

fyi: https://github.com/adrai/fastify-slower-schema-resolving/pull/1

[ivan-tymoshenko]

Yes, seems much better, but still slower than before.

Another optimization that I want to do is to add external schemas to the RefResolver only once instead of adding them for each route. But it would be a part of another PR. I want to move the fast-json-stringify-compiler to the fast-json-stringify.

I guess the problem is not in refResolver.getSchema. It should work as fast as refFinder worked. @adrai Can you check that, please?

[adrai]

Do you mean this?

https://github.com/adrai/fastify-slower-schema-resolving/blob/main/simple-benchmarks/4.2.0/getSchema.output

https://github.com/adrai/fastify-slower-schema-resolving/blob/main/simple-benchmarks/4.1.0/findRef.output

[mcollina]

lgtm

[Eomm]

I would try to check the ajv's readonly properties.
This does not trigger the compiling and we could avoid the deep/equal logic

(x.schemas[schemaKey] || x.refs[schemaKey]) === undefined

const Ajv = require('ajv')

const x = new Ajv()

const externalSchema = {
  $id: 'root',
  definitions: {
    subschema: {
      $id: 'subschema',
      definitions: {
        anchorSchema: {
          $id: '#anchor',
          type: 'string'
        }
      }
    }
  }
}

x.addSchema(externalSchema)

{
  const id = 'subschema'
  console.log(x.schemas[id] || x.refs[id])
}

{
  const id = 'root'
  console.log(x.schemas[id] || x.refs[id])
}

{
  const id = 'subschema#anchor'
  console.log(x.schemas[id] || x.refs[id])
}

[ivan-tymoshenko]

The only concern I have is that refs and schemas is not a part of public API and can be changed without major release.

[Eomm]

Isn't it public?
https://github.com/ajv-validator/ajv/blob/master/lib/core.ts#L281

They would have declared as private instead

[Eomm]

I meant that this check should be enough to replace the new RefResolver holder as both (ajv and refresolver) are storing the same values

I will try it locally

[ivan-tymoshenko]

It won't work for nested schemas like 'subschema' and 'subschema#anchor'. I will work if we want to know if Ajv has a schema, but we can't get it.

[ivan-tymoshenko]

Oh, I see. Ajv resolves references in that cases.

Another reason why I want to have a separate abstraction for this is because Ajv does two things: validates schemas at runtime and resolves references at compile time. It's not good. So in #504 I will have to create two Ajv instances (for schema validation and reference resolving), because there would be two different syntaxes.

And since we own this code it's easy to optimize it. For example in case if we want to reuse external schemas.

[ivan-tymoshenko]

@Eomm I rebased #504 to this branch. You can see there that RefResolver/Validator have different responsibilities and different reasons to be changed.

[mcollina]

@Eomm is this good to land?

[Eomm]

Lgtm

[adrai]

Will this land in the next fastify version?

[mcollina]

Ideally yes.