Remote syslog input node

Developing Nodes
1

Hi Everyone,

TL;DR

  • Working on a syslog node that remotely receives syslog messages
  • Input and feedback welcome

So I was looking for a simple way to have my NUT UPS monitoring software send event data into node-red so I can be alerted when my UPS switches between battery and mains power. After some digging around, I noticed that NUT, like most UNIX/Linux service-type software, logs its activity to files via the local syslog service.

This got me thinking. There is so much useful data going into syslog, a more reusable approach might be for node-red to consume data logged via syslog generally.

Now Syslog is very old, dating back to the 1980s, but it does include remote log transmission. It was formally documented as RFC3164 - The BSD syslog Protocol in 2001, and while it has been superseded by RFC5424, implementations of RFC3164 pervade even today.

Practically anything that resembles a UNIX-like operating system still uses Syslog today. This includes all sorts of devices, including Raspberry Pi's and even home internet routers.

The original implementation of BSD Syslog delivered log messages used UDP. More recent implementations use TCP for improved reliability and even TLS for encryption.

So I have started experimenting with a new input node that receives RFC3164 protocol-formatted syslog messages, using UDP, TCP or TLS.

The following are some of the use cases I can see, but I am sure there are others:

  1. My NUT UPS monitoring problem - obviously. I am aware of node-red-contrib-nut-ups, but from what I can tell, the node is designed to poll the NUT server, rather than listen for events.
  2. Critical system errors, like you know, when your computer's HDD is failing, and all those nasty SATA timeout errors are emitted to the syslog that you don't notice until its too late
  3. The state of practically any system service on any machine (running, stopped, crashed etc)
  4. Computer reboots or crashes

Simply take a look at your syslog files /var/log on most Linux/BSD systems to see what is recorded. All sorts of useful data hides there. :slight_smile:

In case I have made a mistake, or otherwise missed an existing solution, please let me know.

Or if people have any suggestions on how such a node might work, likewise please share.

Here is what the configuration dialog looks like for the current experimental prototype.

10%20pm
Screen Shot 2018-11-23 at 10.44.10 pm.png1626Ă—780 48.4 KB

For the listening IP address, I was thinking of making that a dropdown list of network interface names and IP address in brackets along side, rather than a box to type in an IP address.

The data structure emitted in the payload property of the messages its receives currently looks like the following, which is a time sychronisation log event.

{
  "payload": {
    "facility": 9,
    "severity": 6,
    "tag": "systemd-timesyncd[11993]",
    "timestamp": "2018-11-23T02:55:58.000Z",
    "hostname": "localhost",
    "address": "::ffff:192.168.147.131",
    "family": "IPv6",
    "port": 34840,
    "size": 117,
    "msg": "Synchronized to time server 91.189.94.4:123 (ntp.ubuntu.com)."
  },
  "topic": "topic",
  "_msgid": "3982b135.e2b30e"
}

There are a number of syslog implementations around, but the two most common are rsyslog, and syslog-ng. Raspbian uses rsyslog, as does debian, ubuntu and also centos. To configure outbound remote logging using rsyslog, you just need to add something like the following to the /etc/rsyslog.conf file:

*.*  action(type="omfwd" target="192.168.147.1" port="20514" protocol="tcp"
            action.resumeRetryCount="-1"
            queue.type="linkedList" queue.size="50000")

Cheers,
Damo.

2 Likes