Help with JSON Node - Schema validation

Hey together,
i wasn't sure if this problem would fit inside https://discourse.nodered.org/t/json-node-schema-validation/2574/13

So i would like to validate json against a json schema.
The JSON node comes with the abillity to pass a schema via msg.schema
But somehow it seems not to work correctly for me. At least i got the feeling it can not always compile a valid schema.
I am using examples from https://json-schema.org/learn/miscellaneous-examples.html

{
  "$id": "https://example.com/person.schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Person",
  "type": "object",
  "properties": {
    "firstName": {
      "type": "string",
      "description": "The person's first name."
    },
    "lastName": {
      "type": "string",
      "description": "The person's last name."
    },
    "age": {
      "description": "Age in years which must be equal to or greater than zero.",
      "type": "integer",
      "minimum": 0
    }
  }
}

The error i am getting is "JSON Schema error: failed to compile schema"
But the interesting thing is... The very first time i pasted it in it worked.

So what i did:

  • i created the flow below. Tested it and it worked.
  • Then i changed the inject node and extend it with a ,"required": ["firstName"],
    This was the firstTime the error message displays.
  • Then i copied the example again from the page and pasted it inside the inject node.

Nothing seems to change it back.
Does the schema validation do not recover from compile errors or am i doing something wrong ?
The payload is from another example but i expect to get schema validation errors and not compile errors.

Also i took a look inside the core node. The schemes added inside there is only draft-06.
Probably they can be updated to newer schemes. Somewhere i saw also a topic from the devs from this april about the json node but i can't find it for linking.

My Environment:

NodeRed 1.3.3
Node.js v14.16.0
Windows_NT 10.0.18363 x64 LE

flow:

[{"id":"28adefd8.91d96","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"fef64023.e8edf","type":"json","z":"28adefd8.91d96","name":"","property":"payload","action":"obj","pretty":false,"x":410,"y":180,"wires":[["660c0a7b.7bd554"]]},{"id":"7399e62e.fe6558","type":"inject","z":"28adefd8.91d96","name":"","props":[{"p":"schema","v":"{\"$id\":\"https://example.com/person.schema.json\",\"$schema\":\"https://json-schema.org/draft/2020-12/schema\",\"title\":\"Person\",\"type\":\"object\",\"properties\":{\"firstName\":{\"type\":\"string\",\"description\":\"The person's first name.\"},\"lastName\":{\"type\":\"string\",\"description\":\"The person's last name.\"},\"age\":{\"description\":\"Age in years which must be equal to or greater than zero.\",\"type\":\"integer\",\"minimum\":0}}}","vt":"json"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"longitude\":0,\"latitude\":90}","payloadType":"json","x":250,"y":180,"wires":[["fef64023.e8edf"]]},{"id":"660c0a7b.7bd554","type":"debug","z":"28adefd8.91d96","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":590,"y":180,"wires":[]},{"id":"f40a8e83.298b6","type":"catch","z":"28adefd8.91d96","name":"","scope":null,"uncaught":false,"x":240,"y":400,"wires":[["798295a5.ba882c"]]},{"id":"7275d6ac.7c4868","type":"debug","z":"28adefd8.91d96","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":640,"y":400,"wires":[]},{"id":"798295a5.ba882c","type":"function","z":"28adefd8.91d96","name":"","func":"msg.payload=msg.error.message\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":420,"y":400,"wires":[["7275d6ac.7c4868"]]}]

EDIT:
so i just deleted the $ chars inside the schema in the inject node. And the compile error is gone.

{
    "id": "https://example.com/person.schema.json",
    "schema": "https://json-schema.org/draft/2020-12/schema",
    "title": "Person",
    "type": "object",
    "properties": {
        "firstName": {
            "type": "string",
            "description": "The person's first name."
        },
        "lastName": {
            "type": "string",
            "description": "The person's last name."
        },
        "age": {
            "description": "Age in years which must be equal to or greater than zero.",
            "type": "integer",
            "minimum": 0
        }
    }
}

But adding a required generic keyword inside, brings up the compile error again,

{
    "id": "https://example.com/person.schema.json",
    "schema": "https://json-schema.org/draft/2020-12/schema",
    "title": "Person",
    "required": [
        "firstName",
        "lastName"
    ],
    "type": "object",
    "properties": {
        "firstName": {
            "type": "string",
            "description": "The person's first name."
        },
        "lastName": {
            "type": "string",
            "description": "The person's last name."
        },
        "age": {
            "description": "Age in years which must be equal to or greater than zero.",
            "type": "integer",
            "minimum": 0
        }
    }
}

Try with out first 3 properties

{
    "type": "object",
    "required": [
        "firstName",
        "lastName"
    ],
    "properties": {
        "firstName": {
            "type": "string",
            "description": "The person's first name."
        },
        "lastName": {
            "type": "string",
            "description": "The person's last name."
        },
        "age": {
            "description": "Age in years which must be equal to or greater than zero.",
            "type": "integer",
            "minimum": 10
        }
    }
}
1 Like

Well strange but yes it works. But only the first time again.
image
So i changed my payload to match the schema:

{"firstName":"hey","lastName":"ho"}

and the result is again the compile error
image

Indeed strange, as it working fine for me.

[{"id":"7399e62e.fe6558","type":"inject","z":"28adefd8.91d96","name":"correct","props":[{"p":"schema","v":"{\"type\":\"object\",\"required\":[\"firstName\",\"lastName\"],\"properties\":{\"firstName\":{\"type\":\"string\",\"description\":\"The person's first name.\"},\"lastName\":{\"type\":\"string\",\"description\":\"The person's last name.\"},\"age\":{\"description\":\"Age in years which must be equal to or greater than zero.\",\"type\":\"integer\",\"minimum\":10}}}","vt":"json"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"firstName\":\"john\",\"lastName\":\"wills\",\"age\":10}","payloadType":"json","x":260,"y":180,"wires":[["fef64023.e8edf"]]},{"id":"fef64023.e8edf","type":"json","z":"28adefd8.91d96","name":"","property":"payload","action":"obj","pretty":false,"x":410,"y":180,"wires":[["660c0a7b.7bd554"]]},{"id":"660c0a7b.7bd554","type":"debug","z":"28adefd8.91d96","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":590,"y":180,"wires":[]},{"id":"11ba9e86.d5b189","type":"inject","z":"28adefd8.91d96","name":"incorrect","props":[{"p":"schema","v":"{\"type\":\"object\",\"required\":[\"firstName\",\"lastName\"],\"properties\":{\"firstName\":{\"type\":\"string\",\"description\":\"The person's first name.\"},\"lastName\":{\"type\":\"string\",\"description\":\"The person's last name.\"},\"age\":{\"description\":\"Age in years which must be equal to or greater than zero.\",\"type\":\"integer\",\"minimum\":10}}}","vt":"json"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"gfirstName\":\"john\",\"glastName\":\"wills\",\"age\":9}","payloadType":"json","x":270,"y":240,"wires":[["fef64023.e8edf"]]}]
1 Like

so i imported your flow on a new tab...
executed correct and incorrect -> both work as expected.
then i went back to my flow... executed it and it worked also!

Maybe the JsonNode does not recompile the schema on change?
Also i don't get it why you have to remove the first 3 key pairs

  "$id": "https://example.com/geographical-location.schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Longitude and Latitude Values",

EDIT:
maybe i was confused with another node doing the same.
i retried all steps:

  • copy from json-schema.org
  • removing only "ID" and "Schema" key pairs
  • Adding required...

works.

So i will mark your post as the solution when someone can answer me why you have to remove the ID and the schema key pair

They are not part of the scheme for your object, and are not required as far as I can see, not played or fully read the documentation so it may just be one of those properties that causes the issue, either way they are not required.

from the docs and good practices:

8.2.1.1. Identifying the root schema

The root schema of a JSON Schema document SHOULD contain an "$id" keyword with an absolute-URI (containing a scheme, but no fragment).

The ones you used are examples from the docs and hold no scheme for validation
from json schema.org $id

If present, the value for this keyword MUST be a string, and MUST represent a valid URI-reference. This URI-reference SHOULD be normalized, and MUST resolve to an absolute-URI (without a fragment). Therefore, "$id" MUST NOT contain a non-empty fragment, and SHOULD NOT contain an empty fragment.

Please correct me if im wrong, but i understand it from the specification so that "$id" must only be a valid URI. Nothing is beeing validated or downloaded from the URI neither it must exist there.
It is only there to identify the schema.
If you are using hypermedia for automatic processing of schemas you would use the link reference.

Link: <https ://example.com/my-hyper-schema>; rel="describedby"

9.1.1. Initial Base URI

RFC3986 Section 5.1 defines how to determine the default base URI of a document.

Informatively, the initial base URI of a schema is the URI at which it was found, whether that was a network location, a local filesystem, or any other situation identifiable by a URI of any known scheme.

If a schema document defines no explicit base URI with "$id" (embedded in content), the base URI is that determined per RFC 3986 section 5.

If no source is known, or no URI scheme is known for the source, a suitable implementation-specific default URI MAY be used as described in RFC 3986 Section 5.1.4. It is RECOMMENDED that implementations document any default base URI that they assume.

If a schema object is embedded in a document of another media type, then the initial base URI is determined according to the rules of that media type.

Unless the "$id" keyword described in the next section is present in the root schema, this base URI SHOULD be considered the canonical URI of the schema document's root schema resource.

9.1.2. Loading a referenced schema

The use of URIs to identify remote schemas does not necessarily mean anything is downloaded, but instead JSON Schema implementations SHOULD understand ahead of time which schemas they will be using, and the URIs that identify them.

When schemas are downloaded, for example by a generic user-agent that does not know until runtime which schemas to download, see Usage for Hypermedia.

Implementations SHOULD be able to associate arbitrary URIs with an arbitrary schema and/or automatically associate a schema's "$id"-given URI, depending on the trust that the validator has in the schema. Such URIs and schemas can be supplied to an implementation prior to processing instances, or may be noted within a schema document as it is processed, producing associations as shown in appendix A

A schema MAY (and likely will) have multiple URIs, but there is no way for a URI to identify more than one schema. When multiple schemas try to identify as the same URI, validators SHOULD raise an error condition.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.