Each node would have an extra 'plus in circle' marker that is grey. Clicking on the marker would open a dialog with two columns where test cases could be added. Test cases would consist of an test message and an expected message with the dialog to enter these similar to the inject dialog. The marker would then have a green plus if the test cases succeed or red if they are failing. A white plus would indicate that no test cases have been added to the node.
I know this won't cover all possibilities for testing but it would be a simple start for adding test cases visually. Plus the test cases would be within the flow itself, less testing outside of Node-RED itself.
Is anyone working on something like this? Would this be of interest? I would test more if test definitions could be done within Node-RED itself.
I don't know how much effort (or rather I can guess!) this would be ... there would be the visual component and then extending nodes (somehow) with the test cases within the flows.json - although this could be an extra attribute which contained an array of test-pairs (thus the current format can be kept unmodified).
I think it would. Though I wonder if it would be better to have a container, rather like a native group and have the success/failure checkmark on that? Rather than trying to make a change on every node. That way, you could still test a single node but you could also test a group of nodes.
To come up with the final design would be the job of a UI designer Certainly having an extra '+' on each node would be annoying, so then there might an extra option to turn that off.
Personally I would like to have the "indicator"(^1) for the unit test'ness (i.e. whether a node has tests or not) at the top level, i.e., front and center. This would make unit testing a first class citizen which would be a good thing. It would make "not testing" a conscious act of toggling a setting to hide the indicator.
As for unit test entry box - there could be something like a treelist in the property sidebar instead, it doesn't have to be as shown above. I came up with the above design to a) make it clear for which node the unit tests are for and b) to make testing a kind of layer above the flow - i.e. the UI for unit testing is outside of the standard UI to avoid confusion. Testing is also something that is done parallel to the actual coding, so that should be indicated by the UI. Perhaps greying out the workspace window (just in the case of selecting nodes for catch node) and only highlighting the node for which the unit tests are for would be an alternative(^2).
Important is that the user is shown a list of all unit tests and each is highlighted green/red to indicate success or failure. Hence the UI should show all defined unit tests. After all unit testing is about diagnosing problems as fast as possible, so having a UI that aids that, makes unit testing in Node-RED a pleasure not a struggle!
^1=be it an '+' in a circle or whatever - again something for a UI designer.
^2=Which gives me an idea how to implement - a basic version - using a node and the property window - i.e. extra unit test node for which one could select the node for which the unit tests are defined in the property sidebar - does this not already exist as a node? This would require no extra UI changes and could be integrated into the existing flows.json format (since the new node is just that: a node even though it would be a kind of meta-node).
I know you are trying to build it into the ui, but have you seen things like node-red-contrib-flowtest (node) - Node-RED that go some way to helping create tests. Maybe something useful there to pick up from.
Ideally yes but those nodes look like what I was thinking of as an initial solution, i.e., visually defining test cases - that's more the point than have the perfect UI for testing (which would be the best!).
I hadn't really looked, thanks for the tip - there is a lot of stuff to found from the documentation of the nodes.
I have another more technical question and I know I could go through the source code but is it possible to "clone" a node on the server-side? I was thinking that RED.nodes.getNode(..) on the server gives me the node with all its connections and sending it a message would trigger the flow beyond it. So can I do something like assert( messageResponse, RED.nodes.getNode(..).clone().trigger('input',testMessage)) to avoid triggering the nodes connected to the node to be tested?
@gregorius before you get too deep into this, we have had an outline for a design around flow testing for some time. Progress on it has somewhat stalled, but it is something I'd really like to see some more effort applied to.
Whilst you can of course go and do your own thing, I'd really like to try to build some collaboration into the existing work.
No. There is no easy way to do that. For the testing framework design I've linked to above the thinking was to use the hooks api to insert custom code around each node that can be used to either inject custom messages, stub the whole node behaviour, and/or verify the output of the node.
That's a great document, thanks for the headsup! I guess what I was solely thinking of was the single node testing as a type of unit testing but I see there are lot of more things to consider, e.g. environment setup.
I know that I tend to go off do my own thing but in this case it does seem to be better to ask the professionals - thanks for all the feedback, much appreciated and sort after
I would love to collaborate on this, since I see inbuilt testing as something that would set Node-RED apart from other node-based programming applications. Perhaps documentation i.e. design patterns, best practices, for how to do testing in Node-RED would also be a good start.
Are there any go-to documents for how to doing testing in Node-RED? What does the community generally do when something should/needs-to be tested? And I admit, testing isn't on the top of my list of todos, so please excuse me if there is are best practices and I haven't found them yet.
That approach (clone nodes in the backend), after reading the design document, is a bit naive, I hadn't thought about test setups and cleanups for example.
Having said all that, I would take a pragmatic approach and start with something just to get a feel for how something like testing could be integrated. Throwing together some nodes (which a few people have already done) and seeing what works might also be a good way of getting a feel for more expansive features.
After all, testing in general began with unit testing and then expanded out to integration testing, frontend testing and things like selenium. (When I studied computer science, testing wasn't an major topic ... ;))
@gregorius:
Sorry, I'm a bit late to this thread: Nick is right (as usually ) yet it's definitely do-able.
The node-red-mcu-plugin replaces or wraps the runtime code of a node to completely change its behaviour when the flow runs on a connected MCU. Of course this is transparent to the user and reversible.
Thus it might be possible to create a shell around a node (or a group of nodes) to implement unit testing.
but in this case it does seem to be better to ask the professionals - thanks for all the feedback, much appreciated and sort after 
