MQTT Explorer v0.2.0 - MQTT swiss-army-knife

General
1

I'm a huge fan of Node-Red. I use it a lot for Home-Automation and use MQTT to integrate other services like zigbee2mqtt, but also to decouple the UI from device integrations.

Maintaining a nice topic hierarchy is a key requirement to maintain control over your project.

Last few weeks I conjured up a nice tool to du just that:
See what is going on, delete old retained topics, refactor the topic hierarchy.

A little bit displeased how traditional tools like MQTT.fx or MQTTLens work, I conjured up a little tool to help with the effort. The main goal was to visualize the topic hierarchy and make it navigatable.


ui-test%20(3)
ui-test (3).gif1024Ă—700 3.41 MB

I hope you enjoy and you'll forgive my off-topicness :wink:

11 Likes
2

Interesting. I'll give it a go when I get time. Though it will need to be very interesting to displace my usual MQTT tool - MQTTspy.

3

Excellent!

4

I love the history graph, well done @thomasnordquist.

1 Like
5

Nicely done!

6

MQTTspy has its own perks and purposes, MQTT-Explorer will most certainly not try to mimic the feature set of this or other tools.

MQTT-Explorer is mainly intended for visualization, maintenance, topic refactorings, device/service integration and debugging purposes.
There are still features missing, but the app is growing and new features are coming regulary :wink:

Also: Feature request and bug reports are always welcome

1 Like
7

What does the electron-telemetry library do ?

1 Like
8

Since this is a sensitive topic I'll explain it in great detail:

It helps me to keep track of software quality and "acceptance".
This App is for Windows, Linux and Mac users. As I learned quite early: if something works for one person it most certainly fails for others.

electron-telemetry is a small project that records errors, events, the used architecture and operating system, how much memory the app uses (so I can detect memory leaks). The information recorded is limited to technical data (for instance not even the IP is logged).
It's source can be found here: https://github.com/thomasnordquist/electron-telemetry

Firebase is "the standard" for mobile apps, for electron apps there is not really a good free product on the market to ensure that your users are crash free. I used "nucleus" before but since version 0.1.1 I decided to do the reporting myself.

This is the processed view of the data using Metabase:

50
Bildschirmfoto 2019-01-27 um 19.19.50.png2880Ă—1558 435 KB

This is a snapshot from the server logs

{ system: { arch: 'x64', platform: 'darwin' },
  appVersion: '0.0.7',
  events: { HELLO_EVENT: [ 1547714886134 ] },
  now: 1547714886135,
  transactionId: '1767d251-f492-4f2c-aa62-88add3acc26b' }
{ errors:
   [ { time: 1547714887921,
       message: 'He\'s dead Jim!',
       stack:
        'Error: He\'s dead Jim!\n    at ./src/tracking.ts.exports.default (./mqtt-explorer/app/build/bundle.js:142765:11)\n    at new Promise (<anonymous>)\n    at Object../src/tracking.ts (./mqtt-explorer/app/build/bundle.js:142764:1)\n    at __webpack_require__ (./mqtt-explorer/app/build/bundle.js:20:30)\n    at Object../src/index.tsx (./mqtt-explorer/app/build/bundle.js:142618:1)\n    at __webpack_require__ (./mqtt-explorer/app/build/bundle.js:20:30)\n    at ../backend/node_modules/charenc/charenc.js.charenc.utf8.stringToBytes (./mqtt-explorer/app/build/bundle.js:84:18)\n    at ./mqtt-explorer/app/build/bundle.js:87:10' } ],
  now: 1547714887921,
  transactionId: '53bf9aac-e695-40cc-9a81-b1cf3398843d' }

The transactionId is a "Universal Unique Identifier", the transactionId is used do avoid duplicate entries. A event like HELLO_EVENT or CONNECTION_SET_CONNECTED ONLY contains a timestamp, and no other data, so I know the order of events.

So now I can see: "Oh, there have been two crashes the last day".
Then I can click on the log entry below and see something like: CONNECTION_SET_CONNECTED, CONNECTION_FAILED, SHOW_ERROR, CONNECTION_SET_DISCONNECTED and a stacktrace (of bundled JavaScript, which is still kind of hard to understand).
This then allows me to instantly fix the error if I figured out what is going on: https://github.com/thomasnordquist/MQTT-Explorer/commit/e0d7f45ab88a23650e7f4b40950f488886299289

This is the beauty of open source. I could always simply tell you "I do not store personal data".
BUT It's all open for everyone to see :wink:

P.S.: It also motivates if I see that someone uses my software :yum:

2 Likes
9

Looks very useful. Also appreciate the detailed explanation about the support data you're collecting (electron-telemetry).

I have been wanting to create a nearly identical tool, but using Node-RED itself and likely Julian's uibuilder for the UI (particularly to achieve the topic hierarchy view). But I have simply been too busy with other work to get around to it.

1 Like
10

:smile:

I've thought about this myself as well. Now that VueJS has reached a reasonable level of maturity, it is now my favourite front-end library for this kind of thing. You can even use built components with Vue and uibuilder now so that you can build a proper VueJS app with uibuilder providing the links to and from Node-RED. See the WIKI for using webpack and Vue with uibuilder.

You will soon (time permitting) be able to edit your front-end files (index.html|js|css, etc) from within the Node-RED admin ui as well making life even easier. I also intend to build in the ability to run a build step from within Node-RED. Check out the design nodes in Issue 43.

11

Very useful tool :slight_smile:
Its let me quickly visualise some dead topics on my own broker

image
image.png1022Ă—519 29.5 KB

12

MQTT Explorer becomes mature.
Version 0.2.0 brings many new features:

  • Recursive deletion of retained topics. (Deletes a topic and all sub-topics with a click)
  • Can store different connection profiles
  • Allows custom subscriptions (not just the "#" wildcard root subscription)
  • Configurations persist across app starts
  • Setting to disable the activity indicator (blinky topics)

There also have been some bugfixes and the binary size has been reduced by ~60%.

And there is now also a small demo video (3.5MB)

3 Likes
13

Can you do a wildcard delete? That would be useful. It is a pain, as you know, to remove a whole bunch of retained topics.

14

Well done again @thomasnordquist, excellent tool!

1 Like
15

That is basically what it does.
Selecting a topic ("sensors" for instance) and deleting its child topics would be like deleting "sensors/#"

You can also first filter the topics with the search function and then delete all topics in the filtered tree.
This becomes even more useful once I added some more filter options. (regex, match topic only, match only messages etc.)

The recursive delete feature is however limited to delete 50 topics per click, which should be enough for most use-cases. I added the limit so one can not simply "nuke" a complete broker by accident. I might remove the limit if people complain.

16

Neat, thanks. I'll download the new version.

17

Nice project, thanks. My devices only seem to show some unreadable data ? Is this an encoding issue ?

19
Screenshot 2019-02-22 at 06.13.19.jpg496Ă—1136 71.5 KB

18

Thanks for reporting @bakman2.
Which language is your Operating System?
What should the values read?
Are the values readable in any other MQTT client?
How did you install the app ? (.exe, .dmg, snap, AppImage)

19
  • MacOS
  • Values should be readable strings, I am using NR and can read them there.
  • DMG
20

Do you have some information for me what piece of software is publishing these topics.
It might help me to narrow the issue down, which format the data is encoded.

What do you mean with "NR"?

It is a little bit puzzeling to me that no characters seem to de decoded well. The data should be decoded as utf8 by default, utf8 is "kind-of" backward compatible to most popular string encodings. At least to some degree.