Sorry for rejoining late to this conversation. I am again impressed by the valuable and productive discussion with the developers.
I am in for option 2 as well, sounds like the most backward-compatible and easiest solution.
As for cloning and message throughput. At work our messages often contain NodeJS buffers for decoding/encoding raw data from Siemens PLCs sent over TCP. Depending on the size of the project, there are a lot of messages going around, so cloning could become an issue in certain cases.
I often try to avoid for messages to get cloned in the first place (keep flows straight, no branches if possible), so having this new option around will certainly help!
While I understand the reasons and will ultimately accept the final decision (which will be the right decision), the most backwards compatible method would be to add the extra optional parameter BUT have it default to current implementation.
If necessary, a means of waning the user could be implemented e.g.
This warning could be eliminated by a user who understands the issue by either adding the additional parameter or disabling it (through settings for example).
Hi Nick, the warning could be generated at flow loading or first call of node.send(msg) (it need not be every call).
But I think it is fair to say that the most backwards compatible method is to not change the default implementation.
At the end of the day I know you guys (the architects) have far more insight and knowledge of the nuts and bolts and will ultimately do what you believe to be the best for the majority. I am happy either way I just wanted to put forward my thoughts on the matter and how I might handle this situation.
I think perhaps, if someone's trying to handle that level of message flow, they should look at other options. As efficient as modern JavaScript is, it's never going to be as efficient as carefully written code for a compiled language such as C. (For info - I spend most of my time developing and maintaining a very high data / message rate API.)
Going back to the original discussion, even as a very new Node-RED user, I can see the benefits of this proposal, and it would seem like a good way to go.
No-one is saying you have to move to 1.0 - if you want to be 100% compatible with 0.2.8 - stay on 0.2.8... I still have a box running 0.0.9 - because it works...
I also believe that kind of change is not an easy call to make.
While I still believe that the user is responsible for cloning the message where necessary (that's why RED.util.cloneMessage(..) was made available to the function node in the first place), Nick is just trying to find a solution that fits the majority of users.
As long as advanced users can still enforce the old behavior, I am good with that change.
I wouldn't disagree and you will find a number of responses where I've said this to people.
However, I would like to see Node-RED become more useful in more use-cases so if there is an option not to slow things down, or options to improve performance, I will certainly be supporting them if possible.
Of course, and nobody will disagree with that. Though I do find it interesting just how far people have managed to push JS, it continues to improve in performance and it can now be considered for applications that even a couple of years ago would have been infeasible.
The point for me at least about Node-RED and JavaScript is that they are more accessible to more people. People like myself who are technical but not developers. There are relatively few developers but there are a great many people who have the logicl skills required to automate processes and manipulate information. Those are people that Node-RED can particularly benefit in my view.
Anyway, this is drifting well off topic now and if we want to continue the conversation, it should be in a different thread I think.
Extreme possibilities in document section Why change the default behaviour at all
When this issue came up, there were two possible possible approaches we could take.
Just wondering, after all discussions about clones, what is the recommended way to end a function?
I mean, I have code typically like
//some code
node.send(msg)
where node.send(msg) is the last line. With the new approach the msg will be cloned. What happens then? Will this data remain in memory or will memory be freed up due to garbage collection since this is the last line in the function node that then terminates and there are no other references to the cloned object? Or should all code snippets in function nodes be terminated by a simple return for this to happen? Like
//some code
node.send(msg)
return
What is best, not only for performance reson but for memory management? Is there any difference?
The recommended way is return msg. That will not clone unless multiple wires are attached in which case it will clone for the second and further wires.
Whether messages are cloned or not, JavaScript objects are released when no further references to them exist and will be garbage collected in the fullness of time.
That I did not ask about, it is already understood, as well as node.send(msg, false). My concern was related to,,,well what I wrote
I hope it is like that, when the code finishes execution in the (function) node, it does not leave open references to cloned messages (maybe this can be better and more professional expressed but hopefully you understand what I mean)
There is nothing for you to do in this area. If there was, the docs would cover it. When your function returns, and variables in side the function will no longer be in scope and they will be garbage collected. Having an empty return statement at the end of a function doesn't do anything different to not having one at all.
@krambriw the examples I gave in the blog post are flows that would break if we didn't clone by default.
The question from @cymplecy is for an example flow that will break because we have started cloning.
@cymplecy off the top of my head, one example would be a Function node that creates its own network socket connection and passes that socket down the flow for another Function node to do something with. The socket object cannot be cloned. As you imagine, this is very much an edge case scenario (in my opinion).
Another would be a flow that is handling very large objects - like a Buffer with 50mb of data in. Cloning that will slow down the flow and increase memory usage.
For every "advanced user" in (relatively small numbers of) larger organisations where they have the luxury of having individuals fixate on a narrow aspect of programming in order to achieve "advanced" status, I would suggest there will be many like me who, while being far from nprogramming novices, may have missed the fact that a msg object can get re- used even while we are still in a function, especially those who would not necessarily have gone down the JS route had it not been for the inmmensely useful Node-Red. I believe I understand as of this morning that NR WILL continue as it was all the way up to but not including the 1.0 beta, is it safe to assume that another beta (or 1.0 official) will emerge real soon?
That indeed has been the thrust of the discussion and, I think, Nick's choice of the way forwards. Minimise unexpected behaviour for novice/non-expert users and document any exceptions.
