Making Sense of Syslog Messages

General
1

My first problem is that Node-red sort off randomly restarts every day some times once sometimes more times have a look at the log

Node Red System Startup at Jul 13, 2020, 2:40 AM
Node Red System Startup at Jul 14, 2020, 1:17 AM
Node Red System Startup at Jul 14, 2020, 4:54 PM
Node Red System Startup at Jul 15, 2020, 3:55 AM
Node Red System Startup at Jul 15, 2020, 7:59 AM
Node Red System Startup at Jul 16, 2020, 2:28 AM
Node Red System Startup at Jul 16, 2020, 3:02 PM
Node Red System Startup at Jul 16, 2020, 7:07 PM
Node Red System Startup at Jul 17, 2020, 11:41 AM
Node Red System Startup at Jul 17, 2020, 6:18 PM
Node Red System Startup at Jul 18, 2020, 12:39 PM
Node Red System Startup at Jul 18, 2020, 9:38 PM
Node Red System Startup at Jul 19, 2020, 9:19 PM
Node Red System Startup at Jul 20, 2020, 9:02 PM
Node Red System Startup at Jul 21, 2020, 10:35 AM
Node Red System Startup at Jul 22, 2020, 6:14 AM
Node Red System Startup at Jul 22, 2020, 6:20 AM
Node Red System Startup at Jul 23, 2020, 2:03 AM
Node Red System Startup at Jul 24, 2020, 3:58 AM
Node Red System Startup at Jul 24, 2020, 12:47 PM
Node Red System Startup at Jul 24, 2020, 2:56 PM
Node Red System Startup at Jul 24, 2020, 5:01 PM
Node Red System Startup at Jul 24, 2020, 6:14 PM
Node Red System Startup at Jul 25, 2020, 2:17 AM

After reading and searching I found that the var/log/syslog file could indicate the problems.I had a look at the file and there is a lot record pertaining to node-red. They are running in a loop and basically looks like this

Jul 24 14:56:57 farm-geluk-server Node-RED[8312]: 24 Jul 14:56:57 - [info] [function:2e90710e.099eae] start storednull
Jul 24 14:56:57 farm-geluk-server Node-RED[8312]: 24 Jul 14:56:57 - [info] [function:2e90710e.099eae] tele/rpi/freememory
Jul 24 14:56:57 farm-geluk-server Node-RED[8312]: 24 Jul 14:56:57 - [info] [function:2e90710e.099eae] stored[object Object]
Jul 24 14:56:58 farm-geluk-server Node-RED[8312]: 24 Jul 14:56:58 - [info] [function:Collect Data and Send] start54
Jul 24 14:56:58 farm-geluk-server Node-RED[8312]: 24 Jul 14:56:58 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:56:58 farm-geluk-server Node-RED[8312]: 24 Jul 14:56:58 - [info] [function:Collect Data and Send] stat/victron/battvolts
Jul 24 14:56:58 farm-geluk-server Node-RED[8312]: 24 Jul 14:56:58 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:02 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:02 - [info] [function:Collect Data and Send] startsend
Jul 24 14:57:02 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:02 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:02 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:02 - [info] [function:Collect Data and Send]
Jul 24 14:57:03 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:03 - [info] [function:Collect Data and Send] start1
Jul 24 14:57:03 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:03 - [info] [function:Collect Data and Send] start storednull
Jul 24 14:57:03 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:03 - [info] [function:Collect Data and Send] stat/victron/battamps
Jul 24 14:57:03 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:03 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] start498
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] stat/victron/solarwatts
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] start5
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] stat/victron/pvCurr
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] start65
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] stat/victron/battwatts
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] start500
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] stat/victron/maxDischarge
Jul 24 14:57:05 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:05 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df467c9-cdad-11ea-b9a6-f0def1b20d03 847
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df471c2-cdad-11ea-b9a9-f0def1b20d03 9014
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df47462-cdad-11ea-b9aa-f0def1b20d03 8960
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df479db-cdad-11ea-b9ac-f0def1b20d03 8821
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df479b3-cdad-11ea-b9ab-f0def1b20d03 8859
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df46f5f-cdad-11ea-b9a8-f0def1b20d03 9131
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4807a-cdad-11ea-b9ae-f0def1b20d03 8719
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4858a-cdad-11ea-b9af-f0def1b20d03 8598
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df47f26-cdad-11ea-b9ad-f0def1b20d03 8789
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df46b94-cdad-11ea-b9a7-f0def1b20d03 9293
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4a72c-cdad-11ea-b9b6-f0def1b20d03 16138
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4b1ad-cdad-11ea-b9bb-f0def1b20d03 16037
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4b4a0-cdad-11ea-b9bc-f0def1b20d03 15998
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4bcb4-cdad-11ea-b9c1-f0def1b20d03 15760
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4b18c-cdad-11ea-b9ba-f0def1b20d03 16170
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4a198-cdad-11ea-b9b3-f0def1b20d03 16282
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df487db-cdad-11ea-b9b0-f0def1b20d03 16980
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4c20f-cdad-11ea-b9c3-f0def1b20d03 15850
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4a52f-cdad-11ea-b9b5-f0def1b20d03 16231
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df48c6e-cdad-11ea-b9b1-f0def1b20d03 16893
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4a52b-cdad-11ea-b9b4-f0def1b20d03 16691
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4ab8a-cdad-11ea-b9b7-f0def1b20d03 16102
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4a0c1-cdad-11ea-b9b2-f0def1b20d03 16407
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4abe9-cdad-11ea-b9b8-f0def1b20d03 16124
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4b764-cdad-11ea-b9be-f0def1b20d03 16329
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4ae5f-cdad-11ea-b9b9-f0def1b20d03 16088
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4c0cc-cdad-11ea-b9c2-f0def1b20d03 15624
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4ba72-cdad-11ea-b9bf-f0def1b20d03 16348
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4c513-cdad-11ea-b9c4-f0def1b20d03 16108
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4b4a9-cdad-11ea-b9bd-f0def1b20d03 15792
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4bcb4-cdad-11ea-b9c0-f0def1b20d03 16387
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df50e12-cdad-11ea-b9dc-f0def1b20d03 22218
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4c78a-cdad-11ea-b9c5-f0def1b20d03 24071
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4cfa4-cdad-11ea-b9c8-f0def1b20d03 23883
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4dc60-cdad-11ea-b9cc-f0def1b20d03 23595
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4df1e-cdad-11ea-b9cd-f0def1b20d03 23624
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4d30b-cdad-11ea-b9c9-f0def1b20d03 23929
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4ca70-cdad-11ea-b9c6-f0def1b20d03 24174
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4eef3-cdad-11ea-b9d2-f0def1b20d03 23248
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4d9f9-cdad-11ea-b9cb-f0def1b20d03 23619
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4e2e0-cdad-11ea-b9ce-f0def1b20d03 23643
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df501ee-cdad-11ea-b9d8-f0def1b20d03 22881
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df50865-cdad-11ea-b9da-f0def1b20d03 22741
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df50c0f-cdad-11ea-b9db-f0def1b20d03 22661
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4fb28-cdad-11ea-b9d6-f0def1b20d03 23130
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4fee6-cdad-11ea-b9d7-f0def1b20d03 23065
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df50651-cdad-11ea-b9d9-f0def1b20d03 22899
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4f597-cdad-11ea-b9d4-f0def1b20d03 23197
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4f811-cdad-11ea-b9d5-f0def1b20d03 23279
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4ec2a-cdad-11ea-b9d1-f0def1b20d03 23289
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4cccc-cdad-11ea-b9c7-f0def1b20d03 23900
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4f1b1-cdad-11ea-b9d3-f0def1b20d03 23264
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4e8b7-cdad-11ea-b9d0-f0def1b20d03 23187
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4e59c-cdad-11ea-b9cf-f0def1b20d03 23574
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2df4d616-cdad-11ea-b9ca-f0def1b20d03 23664
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2e004028-cdad-11ea-b9dd-f0def1b20d03 991
Jul 24 14:57:07 farm-geluk-server influxd[956]: [httpd] 127.0.0.1 - farmgeluk [24/Jul/2020:14:57:07 +0200] "POST /write?db=bandwidth&p=%5BREDACTED%5D&precision=n&rp=&u=farmgeluk HTTP/1.1" 204 0 "-" "-" 2e004497-cdad-11ea-b9de-f0def1b20d03 9460
Jul 24 14:57:07 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:07 - [info] [function:Collect Data and Send] start103
Jul 24 14:57:07 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:07 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:07 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:07 - [info] [function:Collect Data and Send] stat/victron/pvVolt
Jul 24 14:57:07 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:07 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:09 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:09 - [info] [function:Collect Data and Send] start14
Jul 24 14:57:09 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:09 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:09 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:09 - [info] [function:Collect Data and Send] stat/victron/grid
Jul 24 14:57:09 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:09 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:09 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:09 - [info] [function:Collect Data and Send] start362
Jul 24 14:57:09 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:09 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:09 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:09 - [info] [function:Collect Data and Send] stat/victron/inverter
Jul 24 14:57:09 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:09 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:14 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:14 - [info] [function:Collect Data and Send] start54
Jul 24 14:57:14 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:14 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:14 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:14 - [info] [function:Collect Data and Send] stat/victron/battvolts
Jul 24 14:57:14 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:14 - [info] [function:Collect Data and Send] stored[object Object]
Jul 24 14:57:16 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:16 - [info] [function:2e90710e.099eae] start2.5
Jul 24 14:57:16 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:16 - [info] [function:2e90710e.099eae] start stored[object Object]
Jul 24 14:57:16 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:16 - [info] [function:2e90710e.099eae] tele/rpi/cpuload
Jul 24 14:57:16 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:16 - [info] [function:2e90710e.099eae] stored[object Object]
Jul 24 14:57:16 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:16 - [info] [function:2e90710e.099eae] start42
Jul 24 14:57:16 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:16 - [info] [function:2e90710e.099eae] start stored[object Object]
Jul 24 14:57:16 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:16 - [info] [function:2e90710e.099eae] tele/rpi/cputemp
Jul 24 14:57:16 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:16 - [info] [function:2e90710e.099eae] stored[object Object]
Jul 24 14:57:17 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:17 - [info] [function:Collect Data and Send] startsend
Jul 24 14:57:17 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:17 - [info] [function:Collect Data and Send] start stored[object Object]
Jul 24 14:57:17 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:17 - [info] [function:Collect Data and Send]
Jul 24 14:57:17 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:17 - [info] [function:2e90710e.099eae] startsend
Jul 24 14:57:17 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:17 - [info] [function:2e90710e.099eae] start stored[object Object]
Jul 24 14:57:17 farm-geluk-server Node-RED[8312]: 24 Jul 14:57:17 - [info] [function:2e90710e.099eae]
Jul 24 14:57:17

As you can see it relates to influxdb and what I am doing is writing bandwidth data from my Mikrotik router into influxdb and I am writing about 65 records into the DB every 10 seconds. It seems like each line in Syslog relates to the writing of each record

I am also not sure if the flow will be of any help but if need be I can post that as well.

On the front end, everything seems to be going well and the data in influxdb also seems to be good. So I am not sure that this is even a problem but it does seem a bit strange that there is a log for every record that is written to influxdb.
Any ideas or am I looking at the wrong info?

2

Dealing first with the influxdb messages, those are generated by influxd when data is written to the database (presumably from node-red). To remove logging of http post operations then, in influxdb.conf, which on Ubuntu/Raspbian is /etc/influxdb/influxdb.conf, in the section [http] set suppress-write-log true. Assuming that you are using influxdb V8 that is.

The node-red messages are coming from nodes in your flow. The first ones are from a Function node with id 2e90710e.099eae. If you use the Search feature and search for that string it should take you to that node. The rest are from a function node named Collect Data and Send, if you don't know where that is then again the Search feature should find it. They are likely from node.warn statements or similar in the Function node, presumably put in for diagnostic purposes.

As for the restart, if node-red is actually restarting, then in syslog you should be able to find the Welcome to node-red message generated when it starts, and before that you should be able to see why it stopped.

1 Like
3

Man this is so cool. After using node red for more than three years I just learned something completely new! Thank you @Colin!
If it will help someone else. I copied some code and it had a "node.log" in the function so servers me for not making sure I know what I copied :slight_smile:

I will now monitor the Syslog file for the next time node-red restart/full deploy by itself.
I am not sure what I should call it. But I have a inject node that will publish a once after deploy and that happens once or twice a day by it self.
Here is the flow that I use.

[{"id":"8ad69df1.4503e","type":"inject","z":"713e8f81.5b7f3","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":"30","topic":"","payload":"Node Red System Startup","payloadType":"str","x":190,"y":3840,"wires":[["3813c293.08d1ee"]]},{"id":"3813c293.08d1ee","type":"function","z":"713e8f81.5b7f3","name":"Set Date","func":"// Set Time\nvar options = { year: 'numeric', month: 'short', day: 'numeric',hour: 'numeric' ,minute:'numeric'};\nvar dt = new Date().toLocaleString('en-GB',options ,{ timeZone:'Africa/Johannesburg' });\nmsg.payload =msg.payload + \" at \"+dt;\nnode.status({text:\"Last:\"+dt});\nflow.set(\"lastDt\",dt)\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":430,"y":3840,"wires":[["69c50037.09b1"]]},{"id":"5f5f23d5.0c2cfc","type":"debug","z":"713e8f81.5b7f3","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":970,"y":3840,"wires":[]},{"id":"69c50037.09b1","type":"file","z":"713e8f81.5b7f3","name":"","filename":"nodeRedStart.log","appendNewline":true,"createDir":true,"overwriteFile":"false","encoding":"none","x":630,"y":3840,"wires":[["5f5f23d5.0c2cfc"]]}]

Not sure I you can maybe see if I am missing something

4

If you run
grep -i "welcome to node-red" /var/log/syslog
that will find all the node-red restarts in the latest syslog, and if you run
grep -i "welcome to node-red" /var/log/syslog.1
you will find restarts from the previous syslog, which will usually be the previous day. Then you can look in the file to see what caused the restart. Possibly a node is crashing node-red under some circumstances. Or perhaps the system is re-booting.

5

Assuming that you are running a version of Linux that uses systemd, then it is much better to use journalctl to show log messages rather than syslog. All services started from systemd will log to the journal (in memory by default) and the journal may also output to syslog. If you look in the templates included in my alternate installer, you will see all the details and the command needed to view the Node-RED log. You can simply change the name in the journalctl command to get the influxdb and other logs. You can have live output as well and can even have different formats such as JSON which lets you process the logs if you like.

I include npm scripts so that I don't have to remember the journalctl commands all the time, you could use BASH alias's as well if you wanted to.

6

What would be the equivalent journalctl command to find all the restarts over the last couple of days please?

7

This will look at the last 50,000 records for the NR version which should be the first properly logged message (with timestamp) as long as you have set your NR logging to the default info level or more.

sudo journalctl -u nrmain -n 50000  -o cat --no-hostname | grep "\[info\] Node-RED version"

Note that my systemd entry for starting node-red is called nrmain, yours will be different if you used Dave's install script.

This is similar but will only show all error entries:

sudo journalctl -u nrmain -n 50000  -o cat --no-hostname | grep "\[error\]"

Lots of different control options, check out the man page:

https://www.freedesktop.org/software/systemd/man/journalctl.html

You can also combine multiple logs as in this example showing both node-red and influxdb logs together:

sudo journalctl -u nrmain -u influxdb -n 500
8

I don't see any great advantage in this particular case over looking in syslog.
One would have to put a lot of effort into looking up the exact command, unless one uses it regularly.
@Paulf007 if you want to use the command @TotallyInformation suggests then if you installed using the Pi/Ubuntu install script then use -u nodered instead of -u nrmain.

9

The main advantage is that syslog isn't necessary and doesn't actually have to be used by systemd services. Typically it is but that isn't a requirement. In the future, syslog might disappear from Linux altogether. Indeed, with the systemd journal, there is no requirement to log to file at all if you don't need logs to persist. That can be a tremendous advantage on a Pi to reduce slow output to SD-Card.

You also don't need to know where log files are either, especially when you need the output from just one service.

Other potential advantages come from the other settings you can leverage in journalctl that don't exist. The ability to filter to a specific set of services or sub-services, having a unified command for outputting logs for both kernel and other services. Output in structured form rather than just text, easily filter by date/time, ....

Oh, and I nearly forgot - you can use journalctl to include logs from multiple devices remotely.

And one trick that does use syslog - add something like this at the start of a bash script to output to systog:

# Redirect stdout to syslog - show with: `sudo journalctl -t jk_kou`
exec 1> >(logger -t jk_kou -p local0.info)  # -t $0)
# redirect stderr to syslog
exec 2> >(logger -t jk_kou -p local0.err)

And use sudo journalctl -t jk_kou to show the output.

10

I agree those are potential advantages, but I don't see how they apply in this case. I followed the Ubuntu discussion a couple of years ago, I think, when there was a suggestion of removing syslog but there was a lot of opposition due to exactly the point I made, that for users who don't use the command regularly the over-complexity of journalctl makes it difficult to use.

A further question about how it would be used in this case. Having used the command to determine that, for example, node-red restarted unexpectedly at 03:30 this morning and it appeared that it might be caused by a system reboot, how would one look to find evidence for what caused the system reboot?

11

It is what it is. When using either BASH or npm, the complexity is pretty irrelevant as a simple alias or script takes care of that. The over-simplicity of syslog has many issues.

Regardless, in this case it is much easier to see what is going on by looking at the correct log which isn't syslog.

One would then know when the reboot happened and so you would filter the log around that date/time. By looking at what happened in Node-RED prior to the reboot firstly. Then if that didn't show anything useful, widening the scope to see what else was happening on the device at the same time.

If nothing relevant appears in any of the logs. I would be looking at the general status of the device. Is any swap activity happening (indicating excessive use of RAM)? Is CPU peaking? Are filing system read/write errors showing up? Or network errors?

To do this last step really needs something to record the ongoing status of the device and OS. The best tool there that I've found is a combination of Telegraf, InfluxDB and Grafana. Though of course, more dedicated tools exist as well. But those 3 tools give you an excellent view of system activity over time.

If none of that helps, I would set up a new installation on a new SD-Card, copy over any flows and settings, install and configure any supporting tooling like Mosquitto, etc and see if that helps.

12

That can't be done in the context of this forum as the aliases are not standard so we can't suggest the user uses them. Therefore we would have to provide relatively complex commands, prone to typos either on our part or the OP.

The first thing I would do is cat /var/log/syslog then scroll back to the restart time, and in maybe 20 seconds would generally get a good idea of what happened. If you are going to have to run a number of journalctl commands to work it out then it will, I imagine, take significantly longer, and again require a number of non-standard aliases, or the use of relatively complex commands.

Also I was wondering where you would see the equivalent of syslog just stopping, often with a block of null characters on the end, which shows that everything just stopped without warning, such as you get from a PSU issue for example.

13

Sorry, I don't understand that. Yes, the OP needs to understand what the service is called but they either know that because they set it up themselves (as with my alt. installer) or it is standard because they used Dave's script. They are already delving into the command line and this really is no more complex. Also, if they edit their package.json file:

  "scripts": {
    "restart": "sudo systemctl restart nrlive",
    "logs": "sudo journalctl -u nrlive -e -f -n 50"
  },

They can now simply run npm run logs (having changed the service name if required of course). But actually, you can see that the command is not really any more complex than a tail -f -n 50 /some/path/to/a/log/file > more and you get the benefit of more standard formatting, colour formatting, automatic paging and it continues to work even if you decide to turn off logging to file.

Using syslog, they have to know where the file is, the formatting is more complex, logs may be mixed together and so hard to follow.

The hardest thing about using journalctl is remembering the command name (is it ctl on the end or ctrl?). The hardest thing about using syslog is remembering where it is and then picking out what you want form all of the other irrelevant output.

I started by trying to point out an alternative, I've done that and it is up to the OP and any other readers of this thread to use whatever is most convenient for them. It makes no difference to me.

On my older pi, syslog is 14k lines long. 19k lines on my new home server. It takes several seconds for the cat to finish on either device (presumably network speed being the limitation). Most of the output is irrelevant. On some terminals, that amount of output might well overflow the terminal buffer and so lose the first lines. You can pipe the cat output to a pager of course - but that adds more complexity to the command though it may then let you at least search through the output - if you can remember vi commands! With jounalctl, unless you've set NR to produce masses of output. I can probably see when it restarted on 1 screen. That gives me the context.

The Node-RED log alone will tell you if it stopped without closing down as in the example you give. No need to wade through system logs unless you need to.

Anyway, you've now managed to draw me into a fairly pointless argument about which command is "better". Everyone is allowed to do it their own way. But please lets not try to force people down one way of thinking when there are other options. Let them make their own minds up.

14

I think I have to finish by saying that is not balanced. I made one suggestion, but it was yourself that argued that "it is much better to use journalctl to show log messages rather than syslog", thereby arguing that one way is much better than another. Also at no time have I tried to force anyone to do anything.

15

Hey , @Colin and @TotallyInformation thank you for all the input and various ways. If I may share an opinion.
It does not matter how to problem is solved or how you got there but if you got there. The reason why I created my own log file was to "pinpoint" when the event happened and then go to the log file and have a look at the events during that time. One "problem" was that there was a lot of "noise" in the data that was caused by influx and that is sorted so now I just need to wait for the reset.

If I may, just one more question, is Syslog the correct file to look at?

16

You can either use the journalctl command or look in syslog, or a combination of both, whichever you find more convenient.

1 Like
17

@Colin - Ok now I am going somewhere, here is the info just before node-red fails and restart

Jul 26 14:00:16 farm-geluk-server Node-RED[22056]: 26 Jul 14:00:16 - [red] Uncaught Exception:
Jul 26 14:00:16 farm-geluk-server Node-RED[22056]: 26 Jul 14:00:16 - Error: Device timeout
Jul 26 14:00:16 farm-geluk-server Node-RED[22056]:     at fn.ontimeout (/home/farmgeluk/.node-red/node_modules/castv2-client/lib/senders/platform.js:56:26)
Jul 26 14:00:16 farm-geluk-server Node-RED[22056]:     at Object.onceWrapper (events.js:421:28)
Jul 26 14:00:16 farm-geluk-server Node-RED[22056]:     at fn.emit (events.js:315:20)
Jul 26 14:00:16 farm-geluk-server Node-RED[22056]:     at Timeout.ontimeout [as _onTimeout] (/home/farmgeluk/.node-red/node_modules/castv2-client/lib/controllers/heartbeat.js:44:10)
Jul 26 14:00:16 farm-geluk-server Node-RED[22056]:     at listOnTimeout (internal/timers.js:549:17)
Jul 26 14:00:16 farm-geluk-server Node-RED[22056]:     at processTimers (internal/timers.js:492:7)
Jul 26 14:00:16 farm-geluk-server systemd[1]: nodered.service: Main process exited, code=exited, status=1/FAILURE
Jul 26 14:00:16 farm-geluk-server systemd[1]: nodered.service: Failed with result 'exit-code'.
Jul 26 14:00:16 farm-geluk-server systemd[1]: nodered.service: Service hold-off time over, scheduling restart.
Jul 26 14:00:16 farm-geluk-server systemd[1]: nodered.service: Scheduled restart job, restart counter is at 21.
Jul 26 14:00:16 farm-geluk-server systemd[1]: Stopped Node-RED graphical event wiring tool.
Jul 26 14:00:16 farm-geluk-server systemd[1]: Started Node-RED graphical event wiring tool.
Jul 26 14:00:17 farm-geluk-server Node-RED[26876]: 26 Jul 14:00:17 - [info]
Jul 26 14:00:17 farm-geluk-server Node-RED[26876]: Welcome to Node-RED

I am still looking at the info but is there may be something that pops out?

18

I assume that this is the culprit?

Node-RED[22056]: at fn.ontimeout (/home/farmgeluk/.node-red/node_modules/castv2-client/lib/senders/platform.js:56:26)

The only node that uses is google API (That is what I got from the net so far) is " googlehome-notifier-offline"
So I would then assume that there could be a bug some where.

19

castv2-client is throwing an exception which is crashing node-red. Are you using a contrib node for casting or are you using castv2-client manually? If it is a contrib node then there is a bug in that node as it should catch errors thrown by modules it uses.

20

Not that I can find. I have looked at the node installed and as far as I could see none is using castv2-client. Is there a way of "disabling" that module , maybe renaming it and check if any of the nodes gives an error.