Node-RED custom logger to a dynamic web page

I responded to a thread the other day where Nick reminded me that Node-RED has a custom logger capability and I realised that I'd not made use of it at all - how rude of me! :crazy_face:

So for some light relief, I thought I'd have a go. Now, because I'm doing some heavy development (read pulling apart and putting back together in a more sensible way!) of uibuilder, I often want to see the trace-level logs for uibuilder but not for everything else (that's a LOT of lines).

The custom logger is great for this.

The other thing I wanted was a more colourful and easier to read output than in a file or the console. So I've revisited something I did way back in the earlier days of Node-RED - create a web-based logger (you can probably find my old attempt in the flows site). And, of course, it is nice to throw in some MQTT - so I did!

So here is a quick and fairly dirty creation of a web-based logger that uses an MQTT broker as the message queue. Of course, since the data is going to MQTT, you could use a completely different interface if you wanted to and even this example allows you to use a flow to output the log entries as well as a direct connection from the web page to MQTT. So you can use it either way.

Note a slight limitation however, this approach doesn't work so well if Node-RED itself is taking part in the output of the log since you will likely miss the early log output. I've used my live instance of node-red to host the web-page but I want the log output from my dev environment so this isn't a problem for me. However, you could easily do away with Node-RED for the web page, either use a stand-alone node.js/Python web server or NGINX/Caddy/Apache.

You need to install an mqtt client into your userDir: cd ~/.node-red && npm install mqtt.

This code goes in the logging section of your settings.js:

        mqttLog: {
            level: 'trace',
            metrics: false,
            audit: false,
            handler: function(settings) {
                const nrLogLevels = {
                    10: 'FATAL', 20: 'ERROR', 30: 'WARN ', 40: 'INFO ', 50: 'DEBUG', 60: 'TRACE', 98: 'AUDIT', 99: 'MTRIC'
                }

                const mqtt = require('mqtt')
                const client  = mqtt.connect('mqtt://home.knightnet.co.uk')

                return function(msg) {
                    if ( msg.level < 51 || msg.msg.includes('[uibuilder') || msg.msg.startsWith('+-') || msg.msg.startsWith('| ') || msg.msg.startsWith('>>') ) {
                        client.publish( 'nrlog/dev', JSON.stringify(msg) )
                    }
                }
            }
        },

For the web front-end, I just added a uibuilder node with no inputs or outputs. This is rather overkill of course and you could easily use just a pair of http-in/out nodes with a template for the code.

Note that direct DOM manipulation is used - no framework needed. I started with uibuilder's basic, blank template.

index.js

/* jshint browser: true, esversion: 6, asi: true */
/* globals uibuilder */
// @ts-nocheck

const nrLogLevels = {
    10: 'FATAL', 20: 'ERROR', 30: 'WARN ', 40: 'INFO ', 50: 'DEBUG', 60: 'TRACE', 98: 'AUDIT', 99: 'MTRIC'
}

// Row tracker
let row = 1

function insertLogMsg(msg) {
    row++
    const level = nrLogLevels[msg.level]
    //console.log(topic, payload)

    const dLog = document.getElementById('log')
    dLog.insertAdjacentHTML(
        'beforeend',
        `<pre id="r${row}" class="${level}"><span>${level}| </span><span>${msg.msg}</span></pre>`
    )
    document.getElementById(`r${row}`).scrollIntoView()
}

// run this function when the document is loaded
window.onload = function() {
    // Connect to MQTT
    // Note the use of ws over TLS - you need to configure your broker for this or use non-secure connection
    const client = mqtt.connect('wss://home.knightnet.co.uk:8883')
    // Subscribe to the correct MQTT topic
    client.subscribe("nrlog/dev")

    // Listen to messages direct from MQTT broker
    client.on("message", function (topic, payload) {
        // MQTT messages can only be strings so decode that here
        payload = JSON.parse(payload)
        // Insert to the DOM UI
        insertLogMsg(payload)
    })
    
    // Start up uibuilder
    uibuilder.start()

    // Listen for incoming messages from Node-RED
    uibuilder.onChange('msg', function(msg){
        insertLogMsg(msg.payload)
    })
}

index.html

<!doctype html>
<html lang="en"><head>

    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <title>Node-RED UI Builder - Blank template</title>
    <meta name="description" content="Node-RED UI Builder - Blank template">
    <link rel="icon" href="./images/node-blue.ico">

    <link type="text/css" rel="stylesheet" href="./index.css" media="all">

</head><body>
    
    <h1>uibuilder Node-RED Logger</h1>

    <div id="log"></div>

    <script src="../uibuilder/vendor/socket.io/socket.io.js"></script>
    <script src="https://unpkg.com/mqtt/dist/mqtt.min.js"></script>
    <script src="./uibuilderfe.min.js"></script>
    <script src="./index.js"></script>

</body></html>

index.css

#log {color:white;background-color:black;}
pre { margin:0;}

/* Log level colours */
.FATAL {color: red;}
.ERROR {color: red;}
.WARN  {color: orange;}
.INFO  {color: yellow;}
.DEBUG {color: green;}
.TRACE {color: cyan;}
.AUDIT {color: grey;}
.MTRIC {color: grey;}

The output looks like this (yes, I know the colours aren't great - it is just something quick and dirty after all!):

3 Likes

I am new to node red and am about to embark on a 'rejig' of my home website (gates, weather, 2 x pi's working relays etc) centred around node red. I'm running node-red in a docker so I want to make sure I can easily see logs as I build up the complexity.

I cannot get index.js to connect to my broker; node-red-contrib-aedes running with tls. I can make my own publish / subscribe nodes and they work fine using ‘localhost.”

I have tied:
'wss://servername.local:8883' mqtts port
'wss://servername.local:4000' wss port
'wss://serverIP:8883'
'wss://serverIP:4000'
'wss://localhost:8883' (works with aedes broker node)
'wss://localhost:4000'

But each one gives me a….. mqtt.min.js:1 WebSocket connection to 'wss://serverIP.local:8883/' failed:

I hope someone is able to point me in the right direction as I have spent hours on this.

Thank you
Ian

So..... I couldn't get the settings.js code to send mqtt data but this seems to to work (partly I suspect because i use logon credentials, plus I wanted the whole NR log):

mqttLog: {
            level: 'trace',
            metrics: false,
            audit: false,
            handler: function(settings) {
                const nrLogLevels = {
                    10: 'FATAL', 20: 'ERROR', 30: 'WARN ', 40: 'INFO ', 50: 'DEBUG', 60: 'TRACE', 98: 'AUDIT', 99: 'MTRIC'
                }

                const mqtt = require('mqtt')
                const url = 'mqtt://IP_of_NR_server:1883'
                const options = {
                    connectTimeout: 4000,
                    // Authentication
                    username: "myuser",
                    password: 'my password',
                  }
                const client  = mqtt.connect(url, options)

                return function(msg) {
                    if ( msg.level < 51 || msg.msg.startsWith('+-') || msg.msg.startsWith('| ') || msg.msg.startsWith('>>') ) {
                        client.publish( 'nrlog/dev', JSON.stringify(msg) )
                    }
                }
            }
        }

The above produces a flow of data on the MQTT server. My ws settings in index.js are

const client = mqtt.connect('ws://IP_of_MQTT_Server:9001')

As I'm using uibuilder 6.0 I ammended (and installed mqtt in uibuilder) the scripts in my index.js are as follows:

        <script src="../uibuilder/uibuilder.iife.min.js">/* REQUIRED. NB: socket.io not needed */</script>
        <script src="./index.js">/* OPTIONAL: Put your custom code here */</script>
        <script src="../uibuilder/vendor/bootstrap/dist/js/bootstrap.min.js"></script>
        <script src="../uibuilder/vendor/mqtt/dist/mqtt.min.js"></script>

My webpage gives the following error

Firefox can’t establish a connection to the server at ws://IP_of_NR_server:1880/uibuilder/vendor/socket.io/?EIO=4&transport=websocket&sid=4AgWyxDRSaGmPYbTAAAL.

I'm not clear why my uibuilder page is trying to establish a ws connection with the NR server on the NR port.

Ian

The uibuilder client library establishes a separate connection to Node-RED over websockets in order to be able to exchange messages between Node-RED and the browser seamlessly. This is exactly the same as Dashboard.

Can you check the network tab on your browser's dev tools? You should see something like this:

The ones that start with ?EIO are the socket.io connections. Normally, you get several of type xhr and one that is type websocket with a status of 101. The others should all have status of 200.

The other thing you can do since you are using the new client library is to add this to the start of your index.js: uibuilder.logLevel = 'trace'. That should give us some more clues hopefully.

Refreshed the website this morning and no error message !!!!!

Thinking it through I should have added credentials to index.js as follows:

    const url = 'ws://IP_of_MQTT_server:9001'
    const options = {
        connectTimeout: 4000,
        // Authentication
        username: "my user name",
        password: 'my password',
      }
    const client  = mqtt.connect(url, options)

I now have a working log page, thank you Julian