# Need ideas on how to Log and inspect changes to mqtt topics

**URL:** <https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707>\
**Category:** General\
**Created:** [12 April 2020 16:00 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707 "2020-04-12T16:00:05Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![hominidae](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hominidae/32/4174_2.png) [@hominidae](https://discourse.nodered.org/u/hominidae)\
**Post date:** [12 April 2020 16:00 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707/1 "2020-04-12T16:00:05Z")

</div>

Hi Folks,

I do have a system, that is using a mqtt broker as central state hub (it is an EV Charging station).  
The software used in there is quite unique and open source, as well as getting frequent updates.  
When I try out a beta version, things sometimes get messed up/unexpected.  
While the number of topics used is quite large but the debug log in the EV station is quite small, I thought of ...

1. a solution to continously record every topic change (maybe keeping the log for a 24-36hrs, not more)

2. ..and when things get wrong (often only discoverd later throughout the day), try and find/evaluate the state/topic change by some way of inspecting/replaying the log (in a nice UI).

I thought of storing things in an influx DB and then finding a way to present the information in a nice UI, when rolling over/replaying the timeline.  
Any Ideas, hints, pointers are very welcome!

happy easter and stay save!

regards,  
hominidae

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [12 April 2020 16:13 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707/2 "2020-04-12T16:13:37Z")

</div>

Would it be good enough just to log it to a file? Then you could use a text editor to examine it. This command will generate timestamps with topic and values which may be helpful. You could just redirect this to a file.  
`mosquitto_sub -t "some/topic/#" -h mqtt_broker_ip -v | perl -pe 'print scalar(localtime()), " ";' `

---

<div class="post-metadata">

**Author:** ![hominidae](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hominidae/32/4174_2.png) [@hominidae](https://discourse.nodered.org/u/hominidae)\
**Post date:** [12 April 2020 16:29 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707/3 "2020-04-12T16:29:00Z")

</div>

Thank you for your fast response.  
I thought about this as well, but I am a visual senses guy and sometimes you want to see events happening in context of the whole system.  
Is there a feature available somewhere that would present things in a graph/tree?  
The system is build on/comprises of about 270 mqtt topics.  
Every data point (based on the time when some topic(s) changed could (re-)render the graph with the changed topics highlighted....maybe even two graphs side-by-side (before & actual time-step examined?)

OK, I am maybe dreaming here 😉

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [12 April 2020 16:42 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707/4 "2020-04-12T16:42:13Z")

</div>

Perhaps that [mqtt explorer](http://mqtt-explorer.com) can help ?

---

<div class="post-metadata">

**Author:** ![hominidae](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hominidae/32/4174_2.png) [@hominidae](https://discourse.nodered.org/u/hominidae)\
**Post date:** [12 April 2020 17:34 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707/5 "2020-04-12T17:34:26Z")

</div>

...I know and use mqtt explorer...wonderful tool...and it offers a tree view, but only a "live mode".  
When being connected, you can see topics flash/light up that get updated.  
But as everything is happening so fast, navigation between topics and inspecting them is a challenge.  
Also there is no "freeze frame" mode....  
Hence I am thinking of storing the data in a DB and then navigating along, maybe in a tree view simlilar or enhanced (like displaying the topic value alongside the topic in the tree) to what mqtt explorer does..

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [12 April 2020 18:26 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707/6 "2020-04-12T18:26:28Z")

</div>

The dataflow can be paused, charts can be paused, every topic has a history and you can search. Fantastic tool to analyze your data.

It's worth to watch the features

[http://mqtt-explorer.com/video.mp4](http://mqtt-explorer.com/video.mp4)

---

<div class="post-metadata">

**Author:** ![hominidae](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hominidae/32/4174_2.png) [@hominidae](https://discourse.nodered.org/u/hominidae)\
**Post date:** [12 April 2020 18:58 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707/7 "2020-04-12T18:58:26Z")

</div>

I totally agree, it is a very, very nice tool when the use case is debugging live, including the ability to stop/pause monitoring.  
My Use Case is that I want to "debug/analyse" after the fact...sometimes many hours later.  
I am not the developer but a (Beta-)User so the basic use case is not testing but using a product, hence the wish to record the events and the ability to "replay"/view them later.

Maybe something like a hierachical map/tree is something that would work.  
I see each point in time as a snappshot of the topic-tree...how can I visualize it?  
Is there for example something in the influx/grafana ecosystem that could do that?  
My Google foo just spat this out: [https://cambridge-intelligence.com/keylines/features/](https://cambridge-intelligence.com/keylines/features/) looks nice but not open.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [11 June 2020 18:58 UTC](https://discourse.nodered.org/t/need-ideas-on-how-to-log-and-inspect-changes-to-mqtt-topics/24707/8 "2020-06-11T18:58:36Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.