Latency on JSON schema validation with ajv module

episensor · 19 February 2021 10:29

Hi all,

I'm importing the ajv module (for JSON schema validation) into NodeRED from npm to the GlobalContext as follows:

functionGlobalContext = { 
    ajv:require(/path/to/node_modules/ajv'),

and then using it in a function node:

const Ajv = global.get("ajv.default");

// JSON schema

const commandsSchema = {
    "$schema": "http://json-schema.org/schema#",
    "type": "object",
    "properties": {
    ...}
};

const ajvValidator = new Ajv();

var validate = ajvValidator.compile(commandsSchema);

// incoming msg.payload contains the JSON to validate

msg.result = validate(msg.payload);

return msg;

but the average latency to validate the payload is 3 - 4 seconds

there is an example here of a NodeRED node using Ajv for validation. It pulls the schema from a HTTP request and caches it locally, so presumably passing the schema to Ajv inline (as per the example above) would be just as fast - but the performance of Ajv when it's used from globalContext is much worse.

github.com

alexandrainst/node-red-contrib-json-multi-schema/blob/master/json-multi-schema-validator.js

/* jshint esversion:8, node:true, strict:true */
/**
 * Node-RED node that validates a JSON payload against a specified JSON Schema URL.
 * JSON Schemas are automatically downloaded and cached on disk the first time they are needed.
 * JSON Schema validators are cached in memory.
 */

//Ajv: Another JSON Schema Validator
const Ajv = require('ajv');
const util = require('util');

module.exports = RED => {
	"use strict";

	function JsonMultiSchemaNodeValidator(config) {
		RED.nodes.createNode(this, config);
		const node = this;

		let lastStatusError = true;
		node.status({ fill:'grey', shape:'ring', text:'Uninitialized', });

This file has been truncated. show original

Could the difference in performance be that the ajv module loaded once on startup with the node-red-contrib-json-multi-schema node, but loaded every time a message is received when it's used from globalContext?

If so, any thoughts on how to improve the performance?

Thanks!

knolleary · 19 February 2021 11:16

Hi @episensor

As a guess, it may be because it is having to recompile the schema for every message.

Did you know the built-in JSON node can do schema validation? If you set msg.schema to the schema you want it to use, it will validate the payload for you. If there is an error with the validation, I believe it will log an error that can be handled with the catch node. Some more information is available in the sidebar help for the node.

episensor · 19 February 2021 11:16

Ah! didn't realise the JSON node could do schema validation. 6ms latency now Thanks Nick!