Count messages in rolling window?

General
1

Hello,

I'm looking to count the number of messages in a rolling window, i.e. 15 minutes and output the number when a new message is received, is this possible with nodes, or would I need a custom function?

I've tried to write a simple function node but sadly couldn't achieve what I wanted...

2

did you look at https://flows.nodered.org/node/node-red-contrib-msg-speed ?

3

@ukmoose I did but assumed it was suitable, on second glance it might just be...

I was keen to understand how I could do this via a function node, but will have a look at the node now and see how it works out. Appreciate the heads up.

4

Well I’m sure the source is available so you could study it as well to see how/if it works

5

On it now actually, just learning about buffers, looks to be pretty straightforward...

6

Hi @swiftnesses,

As @ukmoose already mentioned, you could indeed have a look at the code of my speed node. It makes use of the circular-buffer NPM module.

Summarized:

image

  1. I create a ring buffer, with length = 1 cell for each second. E.g. when the user specifies "speed per minute", then I create 60 cells.
  2. Every second I count the number of messages that have arrived during the last second, and I store that count into the next cell.
  3. Then I move the pointer to the next cell.
  4. That process continues every second. After 1 minute you will start overwriting old values ...

So at any moment the sum of the values in ALL cells = total number of messages arrived during the last minute. When you divide that sum by 60, you have the average number of messages per second.

However, calculating the sum of all cells every second would be very inefficient. So I use a simple trick to calculate the sum:

  1. As mentioned above, the messages are counted every second.
  2. Then the sum is recalculated, based on that new count and the old count (which is stored currently in the cell and will be overwritten in the next step):
    sum = sum - old_counter + new_counter
  3. Store the new counter in the cell

So I remove old counters from the sum and add new counters to the sum. That way the sum always contains the sum of all cells, without having to loop all the cells. This is particulary interesting when you have a large number of cells ...
Bart

2 Likes
7

@BartButenaers That's very useful information, appreciate you taking the time to explain the logic!

I've implemented a rolling buffer now, but will take a look at circular-buffer too.

1 Like