Multiple lines of serial in as single file

[Boerwors]

hey all i am once again in need of some help. i've got a bunch of arduinos sending data to a single arduino which then prints the serial data in individual lines.

in short, i want to write both lines (and more) including a single timestamp at the top to a file lets say every hour. but each time it writes it over writes the previous file.

the issue i'm having is that

1. when i was parsing the data with (line 70) "," (e.g.:
   
   
   )

it only outputs one of the lines (the last one):

it also does this for the text node i.e. only prints the timestamp and second reading.

2. if i remove the "\n" (line 71)between the two print lines:

it then writes almost! what i want but it doesn't write the messages as one message per line (except for the timestamp) e.g. board 1 reading: 10V, board 2 reading: 2V rather than one under the other

any suggestions or help would be appreciated. i would put more pictures but i'm limited to 2 atm

[TotallyInformation]

It would probably help if you shared a screenshot of you flow to begin with so that we can see what you are doing.

I suspect that you are sending each received input to the file individually. Though even this should sort-of work if you have kept the default file out settings:

But you may want to accumulate the data into a context variable and then write to file and clear down periodically.

[Boerwors]

Hey, thanks for the reply. In regards to posting the flow on the forum, how would I go about that?

Apologies I'm really inexperienced with node red

As for your remarks, it does "behave" if I keep it as append. However ideally what I want is a whole new text file to be made every hour overwriting the old one with all of the data given.

Currently it's repeated strings of data sent through serial from a duino to node red so all information is coming from a single serial in. I was thinking if there was a way to basically make a buffer before writing to file? So kind of as you suggest, let's say 5 transmitters put their data through, node will only write the file when all 5 have arrived and thus write them as a single output to the file...

(Apologies again if my explanation is confusing)

[TotallyInformation]

@Boerwors

Hey, thanks for the reply. In regards to posting the flow on the forum, how would I go about that?

In Node-RED, select the nodes you want to export (the whole tab will export if you have nothing selected). Then use the hamburger menu and chose "Export". Click on the JSON tab and click the "Compact button". Then copy to clipboard.

Now in a forum post, click on the pre-formatted text icon:

and paste between the 2 lines containing ```

@Boerwors

Apologies I'm really inexperienced with node red

No worries, we all start somewhere

@Boerwors

However ideally what I want is a whole new text file to be made every hour overwriting the old one with all of the data given.

I thought that might be the case.

@Boerwors

I was thinking if there was a way to basically make a buffer before writing to file?

Well there may be other ways, but the simplest is probably to use a function node. Something like this (untested):

// Get the cycle var from this node's context store or set to 0 if it doesn't yet exist
let cycle = context.get('cycle') ?? 0

// Get the data buffer or an empty array if not exists yet
const databuff = context.get('databuff') ?? []

// Add the new data to the buffer
databuff.push(msg.payload)

// Increment the cycle count
cycle++

if (cycle >= 5) {
    // Output the buffer (join the array as a string)
    node.send({payload: databuff.join('/n')})
    // reset the buffer
    context.set('databuff', [])
    // reset the counter
    context.set('cycle', 0)
} else {
    context.set('databuff', databuff)
    context.set('cycle', cycle)
}

[Boerwors]

hi again, ah thanks for that here is the flow then for my setup. i'm aware it's not that forth coming as the data is coming from an arduino to the serial input node. but prior to this flow, the serial input just outputs a message saying 'board x reading: x V' but each message is sent one after another so i guess node red reads it as 2 payloads?

i'll have a work with your suggestion

[{"id":"e485991ccf3e0822","type":"serial in","z":"11abe06410824819","name":"","serial":"6929e66dfb37b765","x":180,"y":240,"wires":[["4bfbe73e0ce0a23d"]]},{"id":"4bfbe73e0ce0a23d","type":"function","z":"11abe06410824819","name":"add time","func":"msg.payload = new Date().toString() + msg.payload;\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":380,"y":160,"wires":[["245ca36fe017d921","b4e0a61848e30042"]]},{"id":"245ca36fe017d921","type":"file","z":"11abe06410824819","name":"test file","filename":"C:\\Users\\223131446\\documents\\testfile","filenameType":"str","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":780,"y":200,"wires":[[]]},{"id":"b4e0a61848e30042","type":"debug","z":"11abe06410824819","name":"debug 1","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":780,"y":60,"wires":[]},{"id":"6929e66dfb37b765","type":"serial-port","name":"Nano IN","serialport":"COM5","serialbaud":"9600","databits":"8","parity":"none","stopbits":"1","waitfor":"","dtr":"none","rts":"none","cts":"none","dsr":"none","newline":"\\n","bin":"false","out":"char","addchar":"","responsetimeout":"10000"}]

[Boerwors]

hello again, it more or less works! however it does behave quite strange:

That is with the cycle count set to == 3 in your code stack.. also do you know if there is a way to get each entry in the buffer on a new line? i would've thought that would have been what the '/n' (should it be '\n'?) is for in databuff.join() but it doesnt seem to do what i would expect. thank you greatly for your help, i'm on the cusp of success because of it!

[Boerwors]

hi again, added update i've done some tinkering and i managed to make it work for my bespoke backward data acquisition method!!! see:

1. the arrangement of the flow:

[{"id":"e485991ccf3e0822","type":"serial in","z":"11abe06410824819","name":"","serial":"6929e66dfb37b765","x":180,"y":240,"wires":[["dad545f1b5f6476c"]]},{"id":"8d630393845bd583","type":"ui_text","z":"11abe06410824819","group":"e926a6330eaa482e","order":0,"width":"0","height":"0","name":"","label":"","format":"{{msg.payload}}","layout":"row-spread","className":"","style":true,"font":"","fontSize":16,"color":"#000000","x":830,"y":220,"wires":[]},{"id":"4bfbe73e0ce0a23d","type":"function","z":"11abe06410824819","name":"add time","func":"var stamp;\nmsg.payload = new Date().toString() + '\\n' + msg.payload;\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":660,"y":140,"wires":[["245ca36fe017d921","d1156230815e2561","8d630393845bd583"]]},{"id":"245ca36fe017d921","type":"file","z":"11abe06410824819","name":"test file","filename":"C:\\Users\\223131446\\documents\\testfile","filenameType":"str","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":860,"y":140,"wires":[[]]},{"id":"d1156230815e2561","type":"debug","z":"11abe06410824819","name":"debug 1","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":820,"y":40,"wires":[]},{"id":"dad545f1b5f6476c","type":"function","z":"11abe06410824819","name":"payload buffer","func":"// Get the cycle var from this node's context store or set to 0 if it doesn't yet exist\nlet cycle = context.get('cycle') ?? 0\n\n// Get the data buffer or an empty array if not exists yet\nconst databuff = context.get('databuff') ?? []\n\n// Add the new data to the buffer\ndatabuff.push(msg.payload)\n\n// Increment the cycle count\ncycle++\n\nif (cycle == 2) {\n // Output the buffer (join the array as a string)\n node.send({payload: databuff.join('')})\n // reset the buffer\n context.set('databuff', [])\n // reset the counter\n context.set('cycle', 0)\n} else {\n context.set('databuff', databuff)\n context.set('cycle', cycle)\n}","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":400,"y":160,"wires":[["4bfbe73e0ce0a23d"]]},{"id":"6929e66dfb37b765","type":"serial-port","name":"Nano IN","serialport":"COM5","serialbaud":"9600","databits":"8","parity":"none","stopbits":"1","waitfor":"","dtr":"none","rts":"none","cts":"none","dsr":"none","newline":"\\n","bin":"false","out":"char","addchar":"","responsetimeout":"10000"},{"id":"e926a6330eaa482e","type":"ui_group","name":"Arduino Monitor Page","tab":"f95d506942394920","order":1,"disp":true,"width":"6","collapse":false,"className":""},{"id":"f95d506942394920","type":"ui_tab","name":"Home","icon":"dashboard","disabled":false,"hidden":false}]

2. my output file (that is overwritten each time it is written with data!:
   

thank you so much mate!!