# Help with file reading and message formatting

**URL:** <https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467>\
**Category:** General\
**Created:** [5 February 2021 23:53 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467 "2021-02-05T23:53:34Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Trying\_to\_learn](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/trying_to_learn/32/28400_2.png) [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)\
**Post date:** [5 February 2021 23:53 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/1 "2021-02-05T23:53:34Z")

</div>

I have files which are created every time the machine is booted.

With `fs-ops` node/s I get a list of said files:

```auto
["Powered on at 2021-02-01 212108.db","Powered on at 2021-02-02 073139.db","Powered on at 2021-02-02 114601.db","Powered on at 2021-02-02 165843.db","Powered on at 2021-02-02 171120.db","Powered on at 2021-02-02 172557.db","Powered on at 2021-02-03 122336.db","Powered on at 2021-02-04 064446.db","Powered on at 2021-02-04 124940.db","Powered on at 2021-02-04 190643.db","Powered on at 2021-02-05 065420.db","Powered on at 2021-02-05 115718.db","Powered on at 2021-02-05 185952.db","Powered on at 2021-02-05 215415.db","Powered on at 2021-02-05 222908.db","Powered on at 2021-02-06 063001.db"]

```

What I want to do is make a _list_ (as in a list displayed on the screen) of these files with the format:  
the date and time of the file name + the first line of the file.

As the list is _contained_ in `msg.payload[x]` I am at a loss how to split it up and spit out all those as lines which than can then be put through a `file in` node to read the first line.

I'm sure after my posting this, I will have that _ah ha!_ moment, but as many of you who know me: It won't happen until I post this.

But any help would be appreciated.

# Update:

(I told you)

Ok, so now I have this as the _message_:  
`up 4 hours, 28 minutesLast alive 2021-02-06 10:57:59`

_Expanded_ it has a `\n` in it. Or so I think.

(See picture)

![Screenshot from 2021-02-06 11-01-34](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/4/4/4480978ce775ada4614cb12e6950f0666eeca892.png)

I tried the `string` node and `strip right` `\n` but that doesn't work.

# Further update:

This is a working flow, but I'm just curious if there is a nicer/neater way I could do it for formatting the message/s.

_Foreign nodes_  
`fs-ops-dir`  
`ring-buffer`  
(Not important in the workings.)

```auto
[{"id":"f6edc695.44c1a","type":"split","z":"8bb4de19.f72c88","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":2080,"y":1330,"wires":[["5affc1e5.4dd268"]]},{"id":"5affc1e5.4dd268","type":"change","z":"8bb4de19.f72c88","name":"set msg.filename","rules":[{"t":"move","p":"payload","pt":"msg","to":"filename","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":2165,"y":1330,"wires":[["9da1bb26.a6eb58"]],"l":false},{"id":"a72d260.586b358","type":"change","z":"8bb4de19.f72c88","name":"set msg.files","rules":[{"t":"move","p":"files","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1995,"y":1330,"wires":[["f6edc695.44c1a"]],"l":false},{"id":"9da1bb26.a6eb58","type":"template","z":"8bb4de19.f72c88","name":"","field":"filename","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"/home/me/.node-red/public/history/{{filename}}","output":"str","x":2260,"y":1330,"wires":[["33e7cf5b.473a98"]]},{"id":"5da9bf43.2ee768","type":"fs-ops-dir","z":"8bb4de19.f72c88","name":"List files","path":"/home/me/.node-red/public/history","pathType":"str","filter":"*","filterType":"str","dir":"files","dirType":"msg","x":1890,"y":1330,"wires":[["a72d260.586b358"]]},{"id":"33e7cf5b.473a98","type":"file in","z":"8bb4de19.f72c88","name":"Log","filename":"","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":1880,"y":1380,"wires":[["6830cf4c.36f878"]]},{"id":"a5dec011.09a168","type":"inject","z":"8bb4de19.f72c88","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":1900,"y":1260,"wires":[["5da9bf43.2ee768"]]},{"id":"668363a0.da2484","type":"ui_button","z":"8bb4de19.f72c88","name":"Create history list","group":"8a97b9d.4e64d48","order":1,"width":"2","height":"1","passthru":false,"label":"Show","tooltip":"","color":"","bgcolor":"","icon":"","payload":"","payloadType":"str","topic":"","x":1920,"y":1210,"wires":[["5da9bf43.2ee768"]]},{"id":"6830cf4c.36f878","type":"string","z":"8bb4de19.f72c88","name":"filename","methods":[{"name":"getRightMost","params":[{"type":"str","value":"/"}]},{"name":"delRightMost","params":[{"type":"str","value":"."}]}],"prop":"filename","propout":"filename","object":"msg","objectout":"msg","x":2030,"y":1380,"wires":[["a0199d1e.23bd58"]]},{"id":"a0199d1e.23bd58","type":"string","z":"8bb4de19.f72c88","name":"","methods":[{"name":"delRightMost","params":[{"type":"str","value":"minutes"}]},{"name":"chompLeft","params":[{"type":"str","value":"up "}]}],"prop":"payload","propout":"payload","object":"msg","objectout":"msg","x":2170,"y":1380,"wires":[["d600953e.5b1e4"]]},{"id":"d600953e.5b1e4","type":"template","z":"8bb4de19.f72c88","name":"Structure","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"{{filename}} for {{payload}} minutes","output":"str","x":2310,"y":1380,"wires":[["b355ebea.1d25a"]]},{"id":"b355ebea.1d25a","type":"ring-buffer","z":"8bb4de19.f72c88","name":"Cache","capacity":16,"order":"new-to-old","sendOnlyIfFull":false,"pushAfterClear":false,"extra":false,"perTopic":false,"x":2450,"y":1380,"wires":[["dfebd910.415f88"]]},{"id":"3ddb8752.20aff8","type":"change","z":"8bb4de19.f72c88","name":"Wipe","rules":[{"t":"set","p":"clear","pt":"msg","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":2255,"y":1210,"wires":[["b355ebea.1d25a"]],"l":false},{"id":"dfebd910.415f88","type":"ui_template","z":"8bb4de19.f72c88","group":"8a97b9d.4e64d48","name":"Hisotry list","order":3,"width":"9","height":"8","format":"<table id=\"table\" border=\"1\">\n <tr>\n <th>History</th> \n </tr>\n <tbody>\n <tr ng-repeat=\"row in msg.payload track by $index\">\n <td class=\"text\" >{{row}}</td>\n </tr>\n </tbody>\n</table>\n","storeOutMessages":true,"fwdInMessages":true,"resendOnRefresh":true,"templateScope":"local","x":2630,"y":1380,"wires":[[]]},{"id":"d57a1888.e83428","type":"ui_button","z":"8bb4de19.f72c88","name":"Create history list","group":"8a97b9d.4e64d48","order":2,"width":"2","height":"1","passthru":false,"label":"Wipe","tooltip":"","color":"","bgcolor":"","icon":"","payload":"blah","payloadType":"str","topic":"","x":2130,"y":1210,"wires":[["3ddb8752.20aff8"]]},{"id":"8a97b9d.4e64d48","type":"ui_group","name":"Group 1","tab":"718742c7.3798cc","order":1,"disp":true,"width":"9","collapse":false},{"id":"718742c7.3798cc","type":"ui_tab","name":"Machine Usage","icon":"dashboard","order":1,"disabled":false,"hidden":false}]

```

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [6 February 2021 09:31 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/2 "2021-02-06T09:31:28Z")

</div>

> [@Trying\_to\_learn](#):
>
> With `fs-ops` node/s I get a list of said files

Is that a list of file names or the contents of a file, or what?

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [6 February 2021 09:55 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/3 "2021-02-06T09:55:15Z")

</div>

Colin, fs-ops is a set of nodes for the file system. one of them (`dir`) gives to an array of filenames/dir names in the specified directory. II find the fs-ops nodes to be pretty decent and well thought out for getting lists of files, getting file properties, copying, moving, etc.

@Trying_to_learn  
Just a thought, left of field, instead of saving your logs as plain text then trying to parse them, you could use the `flogger` node to save JSON - then its pretty easy to pick out the parts you want.

Another possibility, write your logs to a database? or write to syslog?

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [6 February 2021 10:14 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/4 "2021-02-06T10:14:44Z")

</div>

If they are “live” logs and this is just for display of one line then rather than go via the file why not save the current line in context at the same time it is written now to the log, so it can then just be grabbed.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [6 February 2021 10:20 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/5 "2021-02-06T10:20:58Z")

</div>

> [@Steve-Mcl](#):
>
> Colin, fs-ops is a set of nodes for the file system.

Yes, I know that, I just couldn't believe that the files are named as shown in the array. To give them names like that is just making life difficult.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [6 February 2021 10:24 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/6 "2021-02-06T10:24:40Z")

</div>

> [@Colin](#):
>
> I just couldn't believe that the files are named as shown in the array.

~~Its done like that so you can send the array of file names to the next fs-ops node - e.g. to get permissions or properties for all files in a directory - it works really well TBH.~~

My bad - just re-read yours and Andrews posts - haha yes, those file names are somewhat 🤔 unusual 😉

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [6 February 2021 10:28 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/7 "2021-02-06T10:28:30Z")

</div>

> [@Steve-Mcl](#):
>
> instead of saving your logs as plain text then trying to parse them, you could use the `flogger` node to save JSON

[Flogger](https://github.com/x821938/node-red-contrib-flogger) is the way to go @Trying_to_learn because it takes care of the housekeeping, for example it will rotate and compress your log file, preventing it from getting too large.  
It also adds a timestamp automatically.

---

<div class="post-metadata">

**Author:** ![Trying\_to\_learn](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/trying_to_learn/32/28400_2.png) [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)\
**Post date:** [6 February 2021 10:45 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/8 "2021-02-06T10:45:31Z")

</div>

I believe I have tried `flogger` and last time I tried, it didn't work.

Ok, it is working - only just - but good enough for what I want.

This is a screen grab of what I get:

 ![Screenshot from 2021-02-06 21-38-15](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/e/d/ed236e90256b9753c8e8f75c246482fe725f51f9.jpeg)

Which shows me that the machine was booted at.... (first part of message) and was on for (second part of the message).

This kind of shows me how much time per day I am sitting here getting _square eyes_.

The filenames - which are a bit _weird_ are created when the machine is booted.  
It gets the last log file and renames it to a filename which shows _now_.  
YYYY-MM-DD hhmmss  
That file has two lines.  
The first is how long it has been up and...... ~~I forget what the second~~ The second is a time stamp of when it was last up.

That is handy so I can see when it was last _alive_.

I was just wanting to tidy up the line but worked around it with the `string` nodes.  
Though it would be nice to know how to strip the carriage return as shown in the first post and indicated by the red line.

But I stripped to the right of `minutes` and restored the word in the `template` node.  
So I guess all is good at the end of the day.

I may try `flogger` again soon.

Maybe it wasn't it didn't work, but maybe something else _stuck out_ as a problem for me.

I can't remember. I'll go back and see if I can remember.

Thanks though to all replies.

# Update:

Ok, it was `logger` I had. It came with a mismatch of what it does to what it claims.

Got `flogger` now and shall try to start applying it to machines tomorrow.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [7 April 2021 10:46 UTC](https://discourse.nodered.org/t/help-with-file-reading-and-message-formatting/40467/9 "2021-04-07T10:46:25Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.