Is it possible to ask how, or if not currently included, ask to add, the ability to set this node with an option to write to top of file instead of always (only) appending to end? This would be very helpful with log files (that can get long).
Do you have examples of other logfiles to which data is prepended rather than appended?
How do they handle multi-line log data?
I can't think of any examples, but maybe it's a thing in Windows or MAC?
On Linux I tackle large logfiles by using tail -100 logfile or tail -f logfile or even vi logfile followed by G to go to the last line.
Appending will always be on the end as it is too expensive to insert it at the top (in terms of I/O, cpu).
You can do a tail or use tac (reverse of cat).
eg,
# to get the the last 10 lines in the file
tac /somepath/file.txt | head -10
# more efficient:
tail -10 /somepath/file.txt | tac
# or add linenumbers in front of each line and sort it in reverse
nl /somepath/file.txt | sort -nr
tac is installed by default on most linux distributions.
1 Like
I guess I'm asking because I'd prefer to avoid creating a bunch of .sh files and running a bunch of bash exec nodes for all the logging (15 log files). Would rather use the NR node, since that's kinda the point of NR.
Besides, I didn't even ask the other favour ... to also include the ability to limit # of lines. lol
Well you could look to using this from a function node… prepend-file - npm But yes you will def need to limit the number of lines.
In which case if not too large you may be off better just holding an array of lines in memory. Unshifting to the front of that, then writing it all out , overwriting the existing file.
A worthy suggestion, I'll investigate.
I don't understand where exec nodes come in, are you intending to view/analyse the log files within Node-red, on a dashboard maybem or is this to generate the log files?
No I'm not. The suggestion was to use Linux commands by having the processes run an exec node do the save each time (instead of a NR node) which infers needing to implement those.
Well you don't have to create 'a bunch of bash files', it can be run directly in the exec node and you can create the command via node-red as well and list all 15 logfiles directly in one go.
Unless I am missing something, @bakman2's suggestions and the commands I listed allow you to inspect the bottom of a logfile, not to build a logfile by adding lines at the top.
Can you explain what method you use to view logfiles that would be easier if the newest lines were at the top?
Here is a very inelegant work-around to maintain a second, reversed copy of each logfile:
[{"id":"b50d0e96ebb14c83","type":"inject","z":"faa13ab8c3e03edc","name":"Write something to log file","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":150,"y":360,"wires":[["22ce53081d0ffdaf"]]},{"id":"22ce53081d0ffdaf","type":"file","z":"faa13ab8c3e03edc","name":"","filename":"/home/pi/logtwo.gol","filenameType":"str","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":370,"y":360,"wires":[[]]},{"id":"7fb54d6e4ee3a26a","type":"exec","z":"faa13ab8c3e03edc","command":" ","addpay":"payload","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"Do it","x":650,"y":420,"wires":[[],[],[]]},{"id":"f9db08676855add5","type":"complete","z":"faa13ab8c3e03edc","name":"","scope":["22ce53081d0ffdaf","9db35fa02d666815"],"uncaught":false,"x":90,"y":420,"wires":[["0223575f56b0f3c6"]]},{"id":"24079cc28b05f9bc","type":"inject","z":"faa13ab8c3e03edc","name":"Write something to log file","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":150,"y":320,"wires":[["9db35fa02d666815"]]},{"id":"9db35fa02d666815","type":"file","z":"faa13ab8c3e03edc","name":"","filename":"/home/pi/logone.gol","filenameType":"str","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":370,"y":320,"wires":[[]]},{"id":"0223575f56b0f3c6","type":"template","z":"faa13ab8c3e03edc","name":"tac {{{filename}}} > $(basename {{{filename}}} .gol).log","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"tac {{{filename}}} > $(basename {{{filename}}} .gol).log","output":"str","x":380,"y":420,"wires":[["7fb54d6e4ee3a26a"]]}]
