Add a "Tests" tab to function nodes, allowing developers to define multiple tests for their node.
Each test consists of an input and an expected output, both defined as JSON objects.
This feature enables developers to confidently update function nodes without breaking the overall flow, which is particularly useful when working with nodes or flows they did not originally create.
Key details:
Each test entry contains input and expected output JSON objects.
The Tests tab includes a play button next to each test for individual execution, and a global play button to run all tests at once.
Test run states are stored in the client only and are reset whenever the function code changes, alerting the developer that tests need to be rerun.
Defined tests are serialized alongside the node as an array of { input, output } objects, or { function } for advanced tests, in a property called tests.
When NODE_RED_EXECUTE_FUNCTION_NODES_TESTS=true, all tests run automatically before the flow starts. If any test fails, the flow will not start and the developer will be informed of the failing test.
tests state that run are reset upon changing the function to let developer be aware he has to rerun them
the admin cli must have a new command to let developers run function node tests in CI environments
Advanced tests allow developers to write full JavaScript test code to validate side-effects, such as changes to context, external services, or other nodes, going beyond simple input/output assertions. When writting code, node-red-helper-tests is available in the context.
jsonata could be used to bind env variables, but I would prefer writting tests with js if I ever need to inject env context in my tests. [OPTIONAL]
Take inspiration from postman to enable advanced tests
Be good for one of the core devs to start a design note for this. It probably does need some thought to make sure it isn't adding too many overheads or too much complexity to the function node.
I like the idea that only "input - output" pairs are defined, no logic, no complexity. Allowing JSONata - even though it provides flexibility, the extra complexity (understanding tests if one isn't so firm on JSONata produces complexity) is a non-niceness!
Strangely I already suggested something similar --> Visual Unit Tests
Not to blow my own whistle, but I've been working with my assert nodes for a while now and I've grown to enjoy the approach. Using copy and paste, it's pretty simple to duplicate test cases and alter as needed.
It also naturally fits into the Node-RED flow based programming paradigm. Hence I've certainly gone off creating extra visual elements (as I suggested in my earlier suggestion).
@knolleary
I’m not entirely satisfied with the current design because it feels like it was made for developers, not low-code users. It tries to test higher-level abstractions (flows) using lower-level mechanisms, essentially mimicking how we write tests in JavaScript. While that level of control can be powerful, it’s unnecessarily complex for users who aren’t programmers.
In my view, focusing on flows and function nodes would be enough to improve reliability and make it easier for new maintainers to ramp up.
If we could test entire flows as if they were individual nodes — treating them as black boxes where a message goes in and another comes out — that would cover most real-world testing needs. The same applies to function nodes.
Think of it as if we were testing lambda functions in AWS console. There we are only interested in what happens when a given input is provided.
I also don’t see much value in testing core or installed nodes individually. The most meaningful tests are integration-level ones that validate: given a specific input, the flow consistently produces the expected output.
If users want to assert side effects or custom configurations, that’s where writing code becomes necessary.
Flow/Subflows Tests
Flows and Subflows tests are composed of a list of tests that can be a pair of input and output, or a js function. Each test is asserting that given an input X I will always end up with output Y. And when side effects tests are needed, a js function entry must be used.
Flow/subflow tests tab is located in the editing pannel that appears when double clicking on the tab.
Function Nodes Tests
Function nodes tests are composed of a list of tests that can be a pair of input and output, or a js function. Each test is asserting that given an input X I will always end up with output Y. And when side effects tests are needed, a js function must be used.
@knolleary which issue(s) do you foresee if this proposal is implemented instead of the one found in the design doc?
Very similiar indeed. I just don't like the idea of writting tests using another node-red flow. I would prefer defining inputs/outputs pairs, and use js functions for more complex use cases. Im sure that most important tests just want to garantee that given an input X to this flow/subflow/function node, it will always return an output Y
