Failing Tests from JSON-Schema-Test

[colearendt]

I code-generated a bunch of testthat tests that seem to be failing when I would expect them not to. First check is probably to be sure that the underlying is-my-json-valid passes these tests as well.

Test 1 - Pseudo-Array

context('an array of schemas for items')

schema <- '{
    "items": [
        {
            "type": "integer"
        },
        {
            "type": "string"
        }
    ]
}
'

test_that("JavaScript pseudo-array is valid",{ 

 json <- '{
    "0": "invalid",
    "1": "valid",
    "length": 2
}
'
 expect_true(json_validate(json,schema,verbose=TRUE)) 
})
##1. Failure: JavaScript pseudo-array is valid (@test-items.R#130) ------------------------
##json_validate(json, schema, verbose = TRUE) isn't true.

Test 2 - ref sibling keywords

context('ref overrides any sibling keywords')

schema <- '{
    "definitions": {
        "reffed": {
            "type": "array"
        }
    },
    "properties": {
        "foo": {
            "$ref": "#/definitions/reffed",
            "maxItems": 2
        }
    }
}
'


test_that("ref valid, maxItems ignored",{ 

 json <- '{
    "foo": [
        1,
        2,
        3
    ]
}
'
 expect_true(json_validate(json,schema,verbose=TRUE)) 
})
##2. Failure: ref valid, maxItems ignored (@test-ref.R#282) -------------------------------
##json_validate(json, schema, verbose = TRUE) isn't true.

Test 3 - ref inside a tree

context('Recursive references between schemas')

schema <- '{
    "id": "http://localhost:1234/tree",
    "description": "tree of nodes",
    "type": "object",
    "properties": {
        "meta": {
            "type": "string"
        },
        "nodes": {
            "type": "array",
            "items": {
                "$ref": "node"
            }
        }
    },
    "required": [
        "meta",
        "nodes"
    ],
    "definitions": {
        "node": {
            "id": "http://localhost:1234/node",
            "description": "node",
            "type": "object",
            "properties": {
                "value": {
                    "type": "number"
                },
                "subtree": {
                    "$ref": "tree"
                }
            },
            "required": [
                "value"
            ]
        }
    }
}
'

test_that("invalid tree",{ 

 json <- '{
    "meta": "root",
    "nodes": [
        {
            "value": 1,
            "subtree": {
                "meta": "child",
                "nodes": [
                    {
                        "value": "string is invalid"
                    },
                    {
                        "value": 1.2
                    }
                ]
            }
        },
        {
            "value": 2,
            "subtree": {
                "meta": "child",
                "nodes": [
                    {
                        "value": 2.1
                    },
                    {
                        "value": 2.2
                    }
                ]
            }
        }
    ]
}
'
 expect_false(json_validate(json,schema,verbose=TRUE)) 
})

##3. Failure: invalid tree (@test-ref.R#471) ----------------------------------------------
##json_validate(json, schema, verbose = TRUE) isn't false.

[colearendt]

Update - it does not look like these tests in particular are addressed in is-my-json-valid's implementation of Json-Schema-Test. So that is probably the first step.

Note that all are from the most recent draftv4

[richfitz]

thanks for looking into this

it may turn out that we need to update the copy of is-my-json-valid within the package; that can be done with rerunning ./build.sh I believe (it's been a while). The current version must be at least a year old now.

I'm about to head away from internet for the next couple of weeks, so can't deal with any investigation until July.

If there is a better validator than is-my-json-valid we can include that, too

[colearendt]

No worries! Looks like the current version of is-my-json-valid fails on these tests. Will see if we can get the tests passing for is-my-json-valid and then update the copy as you describe.

Once we get the tests passing on is-my-json-valid, I will submit a PR with the testthat tests included.

[colearendt]

As far as another validator, perhaps worth looking at ajv

[richfitz]

Merely one year later 🙃, I'm doing some work here on getting the package working with ajv

[richfitz]

See #17 for an update on this!

[richfitz]

Fixed in #17