feat(core): support JSON ruleset validation #2062

P0lip · 2022-02-15T14:12:04Z

A small subset of changes from #2026
The only change shere are:

Added JSON Schema definitions that are suitable for JSON/YAML rulesets - going to be useful internally, particularly in the case of the ruleset-migrator package. Due to that, assertValidRuleset accepts a second parameter, namely format.
Moved JSON Schema definitions describing rulesets from src/meta to src/ruleset/meta
Setup package exports so it's easier to import the validator

The remaining changes such as the addition of AggregateError, changes around typings, some bug fixes, etc. aren't a part of this PR.

Checklist

Tests added / updated
Docs added / updated

Does this PR introduce a breaking change?

Yes
No

P0lip · 2022-02-15T14:12:53Z

packages/core/src/ruleset/validation/errors.ts

@@ -0,0 +1,65 @@
+import { ErrorObject } from 'ajv';


this is not a new code, it was copied over from packages/core/src/ruleset/validation.ts

P0lip · 2022-02-15T14:13:17Z

packages/core/src/ruleset/validation/assertions.ts

@@ -0,0 +1,33 @@
+import { isPlainObject } from '@stoplight/json';


copied over from packages/core/src/ruleset/validation.ts, the only change is the addition of format in assertValidRuleset

P0lip · 2022-02-15T14:13:48Z

packages/core/src/ruleset/validation/ajv.ts

@@ -0,0 +1,66 @@
+import Ajv, { _, ValidateFunction } from 'ajv';


Largely copied from packages/core/src/ruleset/validation.ts with a few subtle adjustments to account for 2 different rulesets formats.

Nezteb · 2022-03-02T15:55:45Z

@P0lip @mmiask This PR is also somewhat old; do we have a priority on it?

…eset-validation

mallachari

Looks good!
If it's still relevant.

…eset-validation

# [@stoplight/spectral-core-v1.12.0](https://github.com/stoplightio/spectral/compare/@stoplight/spectral-core-v1.11.1...@stoplight/spectral-core-v1.12.0) (2022-04-18) ### Features * **core:** support JSON ruleset validation ([#2062](#2062)) ([aeb7d5b](aeb7d5b))

stoplight-spectral-bot · 2022-04-18T12:14:11Z

🎉 This PR is included in version @stoplight/spectral-core-v1.12.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

tylerlong · 2022-04-25T16:27:50Z

I believe this PR breaks the lib.

Now it complaints "node_modules/@stoplight/spectral-core/dist/ruleset/validation/assertions.js:10
throw new Error('Provided ruleset is not an object');"

Ruleset never changed. And the previous version of @stoplight/spectral-core works well

@P0lip @Nezteb

P0lip · 2022-04-25T16:49:10Z

@tylerlong I'm sorry to hear that! Would you mind sharing your ruleset with us?

tylerlong · 2022-04-25T16:52:14Z

Here you go:

extends: ["spectral:oas", "spectral:asyncapi"]
rules:
  ensure-paths-kebab-case:
    description: Paths must be in kebab-case.
    message: '{{description}} (lower case and separated with hyphens)'
    formats: [oas3]
    type: style
    severity: error
    given: $.paths[*]~
    then:
      function: pattern
      functionOptions:
        match: "^(\/|[a-z0-9-.]+|{[a-zA-Z0-9]+})+$"

P0lip · 2022-04-25T17:01:46Z

Hm, I don't seem to be able to reproduce the issue using the latest.
Would you mind sharing the JS piece of code that you have? Or, if that's possible, a reproducible sample?

tylerlong · 2022-04-25T18:17:43Z

Do you know how to run a vscode extension: https://github.com/ringcentral/vscode-openapi-linter
In this project, if you upgrade the lib to latest version you will reproduce the issue.

If you don't want to run it, you may read the code here:
https://github.com/ringcentral/vscode-openapi-linter/blob/main/server/src/server.ts

tylerlong · 2022-04-25T21:13:01Z

Let me find some time to create a simple demo project.

tylerlong · 2022-04-25T21:30:35Z

Here is the simple project to reproduce it https://github.com/tylerlong/spectral-demo

How to run the project?

yarn install
yarn test

After upgrading to "@stoplight/spectral-core": "^1.12.0", the first issue is:

/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:820
    return new TSError(diagnosticText, diagnosticCodes);
           ^
TSError: ⨯ Unable to compile TypeScript:
src/index.ts:45:23 - error TS2345: Argument of type 'Ruleset' is not assignable to parameter of type 'RulesetDefinition | Ruleset'.
  Type 'import("/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-ruleset-bundler/node_modules/@stoplight/spectral-core/dist/ruleset/ruleset").Ruleset' is not assignable to type 'import("/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-core/dist/ruleset/ruleset").Ruleset'.
    Property '#private' in type 'Ruleset' refers to a different member that cannot be accessed from within type 'Ruleset'.

45   spectral.setRuleset(customRules);
                         ~~~~~~~~~~~

    at createTSError (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:820:12)
    at reportTSError (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:824:19)
    at getOutput (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1014:36)
    at Object.compile (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1322:43)
    at Module.m._compile (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1454:30)
    at Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1458:12)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) {
  diagnosticCodes: [ 2345 ]
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I changed line 45 from spectral.setRuleset(customRules); to spectral.setRuleset(customRules as any); to workaround the compiling issue.

Then there is this issue:

yarn test
yarn run v1.22.18
$ ts-node src/index.ts
/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:820
    return new TSError(diagnosticText, diagnosticCodes);
           ^
TSError: ⨯ Unable to compile TypeScript:
src/index.ts:3:9 - error TS2305: Module '"@stoplight/spectral-ruleset-bundler"' has no exported member 'bundleAndLoadRuleset'.

3 import {bundleAndLoadRuleset} from '@stoplight/spectral-ruleset-bundler';
          ~~~~~~~~~~~~~~~~~~~~

    at createTSError (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:820:12)
    at reportTSError (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:824:19)
    at getOutput (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1014:36)
    at Object.compile (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1322:43)
    at Module.m._compile (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1454:30)
    at Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1458:12)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) {
  diagnosticCodes: [ 2305 ]
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Root cause is TypeScript: microsoft/TypeScript#33079

I workaround it by deleting "exports" section from package.json of @stoplight/spectral-ruleset-bundler and import {xxx} from @stoplight/spectral-ruleset-bundler/dist/loader/node.

Then the error is:

yarn run v1.22.18
$ ts-node src/index.ts
Error: Provided ruleset is not an object
    at assertValidRuleset (/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-core/src/ruleset/validation/assertions.ts:11:11)
    at new Ruleset (/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-core/src/ruleset/ruleset.ts:63:25)
    at Spectral.setRuleset (/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-core/src/spectral.ts:101:59)
    at main (/Users/tyler.liu/src/ts/spectral-demo/src/index.ts:45:12)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Which I believe is a new bug cause by this PR.

Thanks for your time.

@P0lip

P0lip · 2022-04-26T12:16:20Z

Hey @tylerlong.
Thank you so much for getting back to me with a very thorough example.

Speaking of the first issue, you can run npx yarn-deduplicate --scopes @stoplight && yarn to get rid of it.
as any shouldn't needed once the commands above get executed.

Furthermore, it's likely the error you're experiencing will be resolved altogether, since I'm able to run the code just fine

yarn run v1.22.17
$ ts-node src/index.ts
[
  {
    "code": "oas3-server-trailing-slash",
    "message": "Server URL must not have trailing slash.",
    "path": [
      "servers",
      "0",
      "url"
    ],
    "severity": 1,
    "source": "spec.yml",
    "range": {
      "start": {
        "line": 13,
        "character": 9
      },
      "end": {
        "line": 13,
        "character": 42
      }
    }
  },
  {
    "code": "ensure-paths-kebab-case",
    "message": "Paths must be in kebab-case. (lower case and separated with hyphens)",
    "path": [
      "paths",
      "/rest_api"
    ],
    "severity": 0,
    "source": "spec.yml",
    "range": {
      "start": {
        "line": 17,
        "character": 12
      },
      "end": {
        "line": 36,
        "character": 26
      }
    }
  }
]
Done in 2.74s.

LMK if that did the trick.

tylerlong · 2022-05-24T21:27:22Z

I think I have found the root cause. It can only be reproduced in some scenarios.

spectral/packages/core/src/spectral.ts

Line 101 in 3baba29

this.ruleset = ruleset instanceof Ruleset ? ruleset : new Ruleset(ruleset);

ruleset instanceof Ruleset is problematic.

Especially when you need to compile the code to js first (or even more complicated, bundle the code using some kind of tools like webpack).

Especially when you rule set is loaded by @stoplight/spectral-ruleset-bundler/dist/loader/node which has different class definition for the RuleSet.

Yes npx yarn-deduplicate --scopes @stoplight && yarn does fix the issue.

feat(core): support JSON ruleset validation

885fa2c

P0lip added the enhancement New feature or request label Feb 15, 2022

P0lip requested a review from mmiask February 15, 2022 14:12

P0lip self-assigned this Feb 15, 2022

P0lip commented Feb 15, 2022

View reviewed changes

P0lip mentioned this pull request Mar 7, 2022

feat(core): improve ruleset validation #2026

Merged

4 tasks

Merge remote-tracking branch 'origin/develop' into feat/core/json-rul…

e08ba8b

…eset-validation

Nezteb requested review from Amjcraft, dotslashderek, jasonmgillhub and mallachari April 4, 2022 19:06

mallachari previously approved these changes Apr 14, 2022

View reviewed changes

Merge remote-tracking branch 'origin/develop' into feat/core/json-rul…

49be390

…eset-validation

P0lip dismissed mallachari’s stale review via 49be390 April 14, 2022 12:43

P0lip requested a review from a team as a code owner April 14, 2022 12:43

P0lip merged commit aeb7d5b into develop Apr 14, 2022

P0lip deleted the feat/core/json-ruleset-validation branch April 14, 2022 12:47

stoplight-spectral-bot added the released label Apr 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(core): support JSON ruleset validation #2062

feat(core): support JSON ruleset validation #2062

P0lip commented Feb 15, 2022

P0lip Feb 15, 2022

P0lip Feb 15, 2022

P0lip Feb 15, 2022

Nezteb commented Mar 2, 2022

mallachari left a comment

stoplight-spectral-bot commented Apr 18, 2022

tylerlong commented Apr 25, 2022 •

edited

P0lip commented Apr 25, 2022

tylerlong commented Apr 25, 2022

P0lip commented Apr 25, 2022

tylerlong commented Apr 25, 2022

tylerlong commented Apr 25, 2022

tylerlong commented Apr 25, 2022

P0lip commented Apr 26, 2022 •

edited

tylerlong commented May 24, 2022

		@@ -0,0 +1,33 @@
		import { isPlainObject } from '@stoplight/json';

		@@ -0,0 +1,66 @@
		import Ajv, { _, ValidateFunction } from 'ajv';

feat(core): support JSON ruleset validation #2062

feat(core): support JSON ruleset validation #2062

Conversation

P0lip commented Feb 15, 2022

P0lip Feb 15, 2022

Choose a reason for hiding this comment

P0lip Feb 15, 2022

Choose a reason for hiding this comment

P0lip Feb 15, 2022

Choose a reason for hiding this comment

Nezteb commented Mar 2, 2022

mallachari left a comment

Choose a reason for hiding this comment

stoplight-spectral-bot commented Apr 18, 2022

tylerlong commented Apr 25, 2022 • edited

P0lip commented Apr 25, 2022

tylerlong commented Apr 25, 2022

P0lip commented Apr 25, 2022

tylerlong commented Apr 25, 2022

tylerlong commented Apr 25, 2022

tylerlong commented Apr 25, 2022

P0lip commented Apr 26, 2022 • edited

tylerlong commented May 24, 2022

tylerlong commented Apr 25, 2022 •

edited

P0lip commented Apr 26, 2022 •

edited