I am trying to read a file and process each record as shown below. But before I process the file, I want to get the current processing time and add it to the message object so that I can later retrieve it and use later so that all of my processed records have the same processing time. As you can see below, I can successfully compute the current date time but I can not read it after the next node. What am I doing wrong? Or is there a better way to do it?
Cool, I was able to do this in one node
var x = (new Date()).toISOString();
flow.set("x", processDateTime);
and get to it in a different node:
var processDateTime = flow.get("x");
Thanks.
You have to be careful using that approach. A problem arises if another message arrives at the first function before you pick it up later, as the flow variable will have been overwritten by the second message before you pick up the original value. Using the message property is usually a much better option. I have just tested the File In node and it does not overwrite the property for me. I wonder if you are using an old version of node-red. Which version are you using? You can see that at the bottom of the dropdown menu in the editor.
If it is not an old version then if you want to investigate then put the old code back and change the debug nodes to output Complete Message and run it, then expand the debug output enough that we can see what is there.
1 Like
I am using 1.3.1. What version did you test for the message property not to be overwritten?
Node red v1.3.5
Try it with this flow. Put the name of an existing file in the File In node.
[{"id":"b22a62a9.62166","type":"file in","z":"84405ff5.25fa6","name":"","filename":"/path/to/file","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":330,"y":1940,"wires":[["5b5e6595.00999c"]]},{"id":"a1e3dacf.a2a45","type":"inject","z":"84405ff5.25fa6","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"},{"p":"processDateTime","v":"testing","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":170,"y":1940,"wires":[["b22a62a9.62166"]]},{"id":"5b5e6595.00999c","type":"debug","z":"84405ff5.25fa6","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":490,"y":1940,"wires":[]}]
The reason I asked you earlier to change the Debug nodes to output Complete Message was to check that there was not a typo or similar in the property name. By showing the complete message you will immediately see that.
I think I figured out what is happening. In the file-in node, you selected "a single utf8 string" in Output box. In that case, the custom properties from the original msg object are being passed from one node to the other. Where as I selected "a msg per line" and this causing a new msg object to be created per line without custom properties.
- Is this a bug in the file in node (not to copy all the custom properties) into the new msg object?
- What exactly is the difference between both of these options? ("a single utf8 string" and "a msg per line")?
If there are 20 lines in your text file, "a msg per line" mode will output 20 msg's
If there are 20 lines in your text file, "a single utf8 string" mode will output a single msg containing the full 20 lines.
You can always chose "a single utf8 string" mode then grab the extra value from the msg object then use a split node to generate 20 msgs
I don't want to choose "a single utf8 string" because my file could be 100MB file and I don't want to incur the memory penalty. I like to choose "a msg per line" so that I can my process requirements really small (with streaming model) and have my msg properties propagated. Looks like the issue opened by Colin is going to account for what I want.
In the meantime, msg.topic does get passed through to every msg so you could put your time stamp there for now.
2 Likes
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
