# Live Flow Activity Monitoring with Data Preview for Node-RED

**URL:** <https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275>\
**Category:** Feature Requests\
**Created:** [10 October 2024 01:58 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275 "2024-10-10T01:58:45Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![ahakobyan](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@ahakobyan](https://discourse.nodered.org/u/ahakobyan)\
**Post date:** [10 October 2024 01:58 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/1 "2024-10-10T01:58:45Z")

</div>

Request a **"Live Flow Activity Monitoring"** feature in Node-RED that provides a real-time visual report of all active flows and processes. This dashboard-like view would include:

- Currently running flows with status indicators
- Active loops and their counts in specific flows
- Resource utilization per flow, offering a "task manager"-style overview
- **Flowing Data Preview** : The ability to select a flow and view the real-time data passing through it, aiding in monitoring and troubleshooting the flow’s content.

This feature enhances visibility into flow activities and data, making it easier to manage and debug complex workflows.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [10 October 2024 11:01 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/2 "2024-10-10T11:01:25Z")

</div>

I think you may have misinterpreted how Node-RED works.

The entire flow (all tabs) run on startup and continue to run. All deployed nodes are always live and ready to process incoming messages.

While we talk about flows in the context of what you see in the Editor, under the skin, there is only 1 flow really.

The resources used by Node-RED are mostly defined by the installed nodes and the deployed nodes. On top of that are the resources being managed as part of message flows.

Also worth noting that, although Node-RED talks about message flows and so makes them seem like a water pipe analogy, this is not really true. A message trigger (either a trigger node or some event node) creates a temporary chunk of data that is passed in turn to the message callback function of each connected node.

Having said all that, there are some nodes already that help with monitoring and debugging and you should start with those.

---

<div class="post-metadata">

**Author:** ![ristomatti](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ristomatti/32/5857_2.png) [@ristomatti](https://discourse.nodered.org/u/ristomatti)\
**Post date:** [12 October 2024 02:48 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/3 "2024-10-12T02:48:49Z")

</div>

To add/expand what @TotallyInformation already said, Node.js runs [single threaded](https://en.wikipedia.org/wiki/Thread_(computing)#Single-threaded_vs_multithreaded_programs), meaning it's only doing one thing at a time. It doesn't mean everything runs consecutively though. Node.js (and therefore Node-RED) can do native calls that will be handled by the OS or do calls to native code that's executed asynchronously, but this is not something you could meaningfully measure resource utlilization from Node.js appllication code itself. In contrast, Chrome can show you a task manager presentation of the tabs you have open, but this is because Chrome runs each tab in a separate process and Chrome itself is not written in JavaScript.

I definitely would be possible to visualize the data flow and the counts of the messages passing through and calculating statistis of the time taken at each step, but like already mentioned, you can already do using so 3rd party nodes like [this one](https://flows.nodered.org/node/node-red-contrib-msg-speed). The core team has stated many times in the past the the flow editor is not intended as a dashboad. I haven't been around much lately so I can't say that for sure, but checking out the available 3rd party nodes is likely you're best bet to achieve something like this.

---

<div class="post-metadata">

**Author:** ![hardillb](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hardillb/32/12373_2.png) [@hardillb](https://discourse.nodered.org/u/hardillb)\
**Post date:** [12 October 2024 12:45 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/4 "2024-10-12T12:45:58Z")

</div>

The Node-RED Editor is NOT a dashboard and it was never meant to be one

---

<div class="post-metadata">

**Author:** ![hardillb](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hardillb/32/12373_2.png) [@hardillb](https://discourse.nodered.org/u/hardillb)\
**Post date:** [12 October 2024 12:48 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/5 "2024-10-12T12:48:43Z")

</div>

> [@ristomatti](#):
>
> I definitely would be possible to visualize the data flow and the counts of the messages passing through and calculating statistis of the time taken at each step,

This is a REALLY bad idea. The editor already limits the size of messages shown in the debug sidebar so as not to overwhelm the websocket connection between the backend and the editor, adding an event for EVERY single message in the flow would cause the editor to grind to a halt for any flow under even minimal load.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [12 October 2024 13:30 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/6 "2024-10-12T13:30:38Z")

</div>

> [@hardillb](#):
>
> adding an event for EVERY single message in the flow would cause the editor to grind to a halt for any flow under even minimal load.

I do wonder though (just some random Saturday thoughts) whether it wouldn't be a good idea to have a standard for nodes to add metrics. After all, Node-RED itself DOES already contain a lot of details available in the logs if you turn up the level.

At the node runtime level, this might not be too great an overhead if done right.

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [12 October 2024 17:31 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/7 "2024-10-12T17:31:54Z")

</div>

Better observability of the flows is something I want to look at (along with everything else).

This is why I'm keen to get some interest in contribution to the core rather than doing everything via plugins. Whilst the hooks API does make it easier to do, I suspect there will be more efficient approaches when do in conjunction with the core.

As for the "editor is not a dashboard" argument, I don't think that applies here. This isn't about creating an end user dashboard as you would with the dashboard nodes. This is about having the tools in the editor to help develop your flows.

---

<div class="post-metadata">

**Author:** ![ristomatti](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ristomatti/32/5857_2.png) [@ristomatti](https://discourse.nodered.org/u/ristomatti)\
**Post date:** [12 October 2024 22:50 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/8 "2024-10-12T22:50:50Z")

</div>

It would not need to be 100% accurate to be useful. You'd typically sample the metrics data with a configurable frequency that fits the volume of the data or the level of accuracy required.

---

<div class="post-metadata">

**Author:** ![ristomatti](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ristomatti/32/5857_2.png) [@ristomatti](https://discourse.nodered.org/u/ristomatti)\
**Post date:** [12 October 2024 23:09 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/9 "2024-10-12T23:09:50Z")

</div>

> [@knolleary](#):
>
> This isn't about creating an end user dashboard as you would with the dashboard nodes. This is about having the tools in the editor to help develop your flows.

But would the Node-RED editor be the right place to inspect this information? Have you considered instrumenting the core with something like [https://opentelemetry.io](https://opentelemetry.io)? This way the heavy lifting of processing/analysing the data could be handled in whatever compatible system/service the user decides to use (or not to use).

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [13 October 2024 07:52 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/10 "2024-10-13T07:52:34Z")

</div>

> [@ahakobyan](#):
>
> Request a **"Live Flow Activity Monitoring"** feature in Node-RED that provides a real-time visual report of all active flows and processes. This dashboard-like view would include:

You might be interested in this [post](https://discourse.nodered.org/t/dynamic-live-debug-message-tracing/92319). I've implemented a message tracer with an extra feature to send messages to the debug panel. More details at the package [readme](https://flows.nodered.org/node/@gregoriusrippenstein/node-red-contrib-introspection).

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [13 October 2024 08:05 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/11 "2024-10-13T08:05:58Z")

</div>

> [@knolleary](#):
>
> This is why I'm keen to get some interest in contribution to the core rather than doing everything via plugins. Whilst the hooks API does make it easier to do, I suspect there will be more efficient approaches when do in conjunction with the core.

If this is the case, then have a look at what I did with the introspection msg tracing. I'm using as much inbuilt Node-RED as I can, something that would be nice would be a method for highlighting message in the debug panel. I put something together but it's based on doing CSS changes directly on the divs of the debug panel - not very Node-RED like.

> [@knolleary](#):
>
> This is about having the tools in the editor to help develop your flows.

Exactly! This what I'm trying to do but via a plugin because via a plugin it is far simpler to experiment and try things out. The latest version, [0.9.3](https://flows.nodered.org/node/@gregoriusrippenstein/node-red-contrib-introspection), I've started using `RED.utils.createNodeIcon(...)`, in combination with `treeList(..)` widget, the end result is a simple but elegant (IMHO) listing of nodes being called during flow execution:

 ![Screenshot 2024-10-13 at 09.58.02](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/3/d/3d86761bd0654e2a45a9b0f916d05c2fd2f51a5b.png)

Plugins are good for this type of prototyping - considering I started [three days ago](https://discourse.nodered.org/t/message-tracing-for-beginners/92287) with simple message tracing. I would not have been able to do the same prototyping having to update the NR debugger with each step.

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [13 October 2024 08:38 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/12 "2024-10-13T08:38:27Z")

</div>

> [@hardillb](#):
>
> an event for EVERY single message in the flow would cause the editor to grind to a halt for any flow under even minimal load.

It works surprisingly well - I was monitoring a 50ms loop (reason will remain unknown) and that was generating four messages every 50ms - my [tracer](https://www.npmjs.com/package/@gregoriusrippenstein/node-red-contrib-introspection) and Node-RED editor handled that without an issue.

Having said that, if I then dump _every_ message to the debug panel, then the editor dies, very quickly in fact. But just keeping a count of messages and tracing them in the editor works fine.

EDIT: I'm not advocating doing this for hours, but for a few minutes to check something, then it's fine. Since yes, the websocket communication is flooded but a backlog _does not_ seem to be created. But as always, your mileage _will_ vary.

---

<div class="post-metadata">

**Author:** ![bohtho](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bohtho/32/6153_2.png) [@bohtho](https://discourse.nodered.org/u/bohtho)\
**Post date:** [15 October 2024 08:06 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/13 "2024-10-15T08:06:03Z")

</div>

I’d go even further and say that the editor is not just an editor when you are bug tracing and putting debug nodes everywhere (manual observability).

So I too think it’s a question of framing, and I think it would be helpful for even just the «editing» if for example the user could turn on a setting for having the famous Node-RED wires crudely change coulour (intensity) depending on activity (resource utilisation). And then I’m not talking about a misleading animation like in some flow editors.

There’s no doubt that this would help the user’s understanding of the flow in what is a _visual_ flow editor, while not being too much of an unecessary resource hog or just too much bells and whistles for others.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [14 December 2024 08:06 UTC](https://discourse.nodered.org/t/live-flow-activity-monitoring-with-data-preview-for-node-red/92275/14 "2024-12-14T08:06:31Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.