Flogger - the advanced logging node!

Hi there,

If anyone is interested I created an advanced logging module for Node-RED. Often the debug window gets too crowded if you want do log many different things at the same time. With my newly created node you can do long time logging to different files. Now it's suddently peace of cake to monitor many flows at the same time.

You can find the Node with full documentation and examples here: https://flows.nodered.org/node/node-red-contrib-flogger

Comments and ideas for improvements are welcome

Best regards,
Alex

8 Likes

Would it be greedy to hope for a future update that added output to a web page with formatted output? :slight_smile:

It would be quite simple to output to some formattet html files. Then you have to manually open those in a browser. I don't know if that's what you were imagining?

Best regards,
Alex

Yes, you will see an old example I did way back on the flows site. Here is what I think my ideal would look like:

  • Dedicated, known URL for flogger output
  • Formatted JSON output - ideally including folding
  • Ability to switch between a log view and a per node/per topic view so that you could see the latest log output for each specific node and/or for each specific topic. Sometimes this is a more useful view than simply having everything dumped in a long and possibly hard to follow list.

It is, of course, the last point that would really set this apart from the separate page view of the normal debug output. Along with the formatting features you are including.

Anyway, perhaps food for thought or maybe random twitterings - as always, feel free to ignore. :smile:

I have been thinking a little about your suggestion. I think all those features belongs more in the nodered core instead. It could easily grow into something big and bulky if i follow that path.

I was trying to keep my solution very lightweight for doing long time logging.
I think it would be better to build a real log analyzer on top of these logs instead. Maybe the ELK stack for example.

Best regards,
Alex

1 Like

No problem. I never know when to stop!

TotallyInformation,

hehe. Splunk!

I know it won't run on a Pi. But, I am able to run enterprise on my ubuntu desktop machine.

Ha! That's rather excessive even for me!

If you really want something for analysing the log output you are producing, you could try the cloud service Sentry.

But even cheaper - if a little more work - is to use Telegraf and InfluxDB. Telegraf has a log-parser plugin. You can then use Grafana to produce a nice dashboard.

https://www.influxdata.com/blog/telegraf-correlate-log-metrics-data-performance-bottlenecks/

Just an FYI. If logging less than 500gb, a developer license for splunk is...... free.

Yup, but I can run Node-RED, Mosquitto, InfluxDB, Telegraf and Grafana all on a single Pi :smile:

Hi!

I'm a very inexpereinced user and I'm trying to use flogger to log an issue I'm having.
The issue I'm having is that I'm getting a "cant write file!" error on the flogging node.

See my config below:


Any ideas? That "MMMlogs" directory doesn't exist, I wasn't sure if flogger would create it or not.I have no idea where that directory is to go and create it myself.

The directory you point to has to be created first.
For example if you specify "MMMlogs" in the Log Dir settings, it will point to a directory in your home folder. On a linux system that folder will be "/home/your_username/MMMlogs". Remember that you have to create it first.

Don't use the ~ in the Log Dir because it's implied. You could put a full path like "/var/log/MMMlogs" - but remember to create the directory first.

1 Like

Perfect thanks! Worked straight away.

Is there a way to tell flogger to use Daylight savings time? If it tries to log at 12:00, it gets logged as 11:00. (I'm assuming this is a DST issue)The Linux system time is correct.

There is a way now if you upgrade to version 1.0.5 :slight_smile:

Excellent! Works perfect, thanks!

Jusy FYI (doesn't bother me, but just in case you didn't know), it changes the time format:

OLD FORMAT:
2019-06-25T20:20:11.672Z INF SLGO:ON

NEW FORMAT(LOCAL TIME):
6/25/2019, 9:26:18 PM INF SLGO:ON

I know - I could do some extra formatting if I wanted.
I just used to formats javascript provided by default.

Thanks for writing a node. I've just checked similar nodes, so it might come handy)

Hi, just spent 10 mins with your excellent new logging node. I like where its going and would like to give you some feedback and suggestions. I hope you dont mind.

Small issue - no time in log if only 'Local Time' selected

As there is no info on the side-bar for the config node, I assumed selecting Local Time (without selecting timestamp) would put local time in log - there was no time in logs (just a space). Perhaps "Local Time" should be disabled if "Timestamp" is not checked? (alternatively, a drop down to choose "none", "timestamp/UTC", "Local Time" etc might be worth considering)?

Additionally, when Local Time and Time Stamp are selected, the log time is presented in mm/dd/yyyy format & is not respecting the servers locale (I have it set for UK format).
Perhaps consider ISO format yyyy/mm/dd. This ISO format can never be confused and as a side bonus, is sort-able :slight_smile:

Use typedInput for log message

For the log message property, perhaps consider using a typed input that lets you choose msg. flow. global. and "full msg object". I feel that would be more flexible, intuitive and remove the need for users to put a change node before the flogger node when logging flow or global context.

msg pass though

Would you consider permitting pass though of messages? (e.g. have an output on the flogger node so that messages can logged and simply pass through untouched). In-line logging if you will!
Reasoning...
When adding debug nodes / flogger nodes, we often have to branch off in the middle of a flow. This causes Node-RED to create clones of msg & sometimes there are side effects.

Log Level

  • Would you consider changing log level to the following common values...
    • ERROR
    • WARN
    • INFO
    • DEBUG
    • TRACE
  • Would you consider redefining the selected "log level" property as a default and allow the user to send an override this in msg (e.g. if msg.loglevel is null/undefined then use default, otherwise use msg.loglevel)
    • Reasoning...
    • One example - if I use a function node to node.send() debug/info/error messages to the log, I have no way of indicating its severity unless I have multiple function outputs and multiple flogger nodes.

Log Format options

Would you consider changing (or adding an option) to have all log entries as JSON similar to NLOG
e.g.
//logging msg.payload
{ "time": "2010-01-01 12:34:56.0000", "level": "ERROR", "message": "hello, world" }
//logging msg
{ "time": "2010-01-01 12:34:56.0000", "level": "ERROR", "message": {"topic": "Greeting", "payload": "hello, world"} }

Additional thoughts on NON JSON formatted logs...
How about permitting the user to use mustache formatting for "log msg"? Similar to the built in template node & http nodes?
This would permit a user to log something like Pinging {{payload.IPAddress}} responded with a TTL of {{payload.TTL}} - (Server name = {{computerName}})

Syslog

Have you considered adding "send to syslog"?
Not sure what would be involved but might be useful?

Stretch goal - a complete replacement for the built in debug node?

How about an option for "Send to debug side bar" and / or "Send to Console"?
Would be great to have a debug node that also logs to file

Hi Steve,

Thanks for a lot of constructive input. I will start implementing a lot of your changes.
Look out for a new version soon.

2 Likes