# High throughput examples (for arguing for the platform)

**URL:** <https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651>\
**Category:** General\
**Created:** [12 November 2021 21:43 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651 "2021-11-12T21:43:42Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [12 November 2021 21:43 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/1 "2021-11-12T21:43:42Z")

</div>

Hi,

Does anyone have examples, use cases where NodeRED is handling **alot** of transactions/messages. I just need it for "ammunition" as conceptual proof the platform can handle a higher load. I fully understand there is a huge difference in how different scenarios would affect the capability here, but things like:

_We log all cars going in and out of our mall garage, and thats about 400 transactions an hour_

or

_We handle 300.000 transactions from our production tracking system every day to keep track of individual parts for quality control_

That would be super helpful

Thanks in advance 🙂

---

<div class="post-metadata">

**Author:** ![ScheepersJohan](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/scheepersjohan/32/39556_2.png) [@ScheepersJohan](https://discourse.nodered.org/u/ScheepersJohan)\
**Post date:** [12 November 2021 21:59 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/2 "2021-11-12T21:59:46Z")

</div>

Welcome @werkstrom to the forum

Your work load is a transaction every 9 sec on average.

What do you want the flow to do?

As what it does will influence greatly the performance you require.

A simple flow like  
 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/6/e/6eebbc2c2fbf71bb354a5130e08e7a4b80517496.png)

running every 9 sec, or some thing very complex calling on API and reading and writing to DB are two different things.

But bottom line Node-Red can handle very complex task with ease.

Also bear in mind that your PC needs to be up to the task as well, you can preform a lot with a PI and Node-red ( not saying this is the solution for you)

---

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [12 November 2021 22:18 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/3 "2021-11-12T22:18:55Z")

</div>

Hi @ScheepersJohan and thanks for answering my call.

To clerify a bit, We do use NodeRED and have been for some time in different scenarios for POC use. Right now we are for example experimenting on setting up load balanced instances in Azure Webapps. But we also have several other cases. The reason for asking is if anyone has examples of their personal usages with high throughput. To kind of give me some ammunition in comparison to other solutions we have. For example handling road tolls for a couple of million passes a day or large invoice flows are things we do using more traditional types of ETL tools or integration platforms and in some cases the flexibility and adaptability of NodeRED could be of use. I just want to be able to "bite the head of the snake" when others ask about scalability and performance. Preferably by being able to exemplify, and that's where I was hoping for som help/insight 🙂

---

<div class="post-metadata">

**Author:** ![ScheepersJohan](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/scheepersjohan/32/39556_2.png) [@ScheepersJohan](https://discourse.nodered.org/u/ScheepersJohan)\
**Post date:** [12 November 2021 22:19 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/4 "2021-11-12T22:19:14Z")

</div>

Here is flow that wright to a DB every few min - that is very light.

But the recall from the DB and mapping it can be a bit more, this is a few 100 points from a DB mapping tem on and map.

This is running on the smallest AWS EC2 instance

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/6/9/694da339b4791de5db4e9f14531da718c572de2d.png)

![6szGa1TJn2](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/6/d60aa5b139fe3c94c37dc6ee77a324521982aaa6.gif)

---

<div class="post-metadata">

**Author:** ![ScheepersJohan](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/scheepersjohan/32/39556_2.png) [@ScheepersJohan](https://discourse.nodered.org/u/ScheepersJohan)\
**Post date:** [12 November 2021 22:29 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/5 "2021-11-12T22:29:05Z")

</div>

> [@werkstrom](#):
>
> flexibility and adaptability

The uses cases are from IOT platforms to Industrial applications, home automation to animal tracking.

Search a bit on the forum and you will find 1000 of different examples.

[https://flows.nodered.org/search?type=flow](https://flows.nodered.org/search?type=flow)

---

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [12 November 2021 22:33 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/6 "2021-11-12T22:33:24Z")

</div>

Hi again,

Yes, I get that to 🙂 But I'm looking for user stories and experiences, not flows that does those thing. But, I very much appreciate your input and help. Thank you,

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [12 November 2021 22:45 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/7 "2021-11-12T22:45:42Z")

</div>

Quite a few folk round here are using it for video handling so that's fairly intensive for small devices.

I've certainly used it for simple mqtt transformations at low thousands per second on a decent laptop. By that I mean just in, simple function and back out. No database lookups or other async tasks.

I know we had some large financial organisations trying extreme scalability with large iron and things in parallel getting way up there, but of course a lot depends on how you architect the solution.

---

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [12 November 2021 23:03 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/8 "2021-11-12T23:03:15Z")

</div>

Thanks @dceejay, that's precisely the type of very high level exmples I would need. I fully understand that the type of workload you put in your flow will dramatically influence the load, but high volume loads and complex logic generally pose a problem anywhere 😉. The important thing is to know it is indeed possible to scale. Not necessarily to describe how. Thnk you again for your kind answer.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [12 November 2021 23:03 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/9 "2021-11-12T23:03:58Z")

</div>

One of my flows operates every 30 mins, grabbing data from over 220 robots (http), converts data from html to topics and payloads (for MQTT) and XML (for external system) and writes data to database (for historical access/reporting). That's over 10000 ops per day. And there is room to comfortably speed this up to run every 5 mins (which would be over 60000 ops per day building 60000 XML files, 60000 database writes and LOTS of mqtt messages)

---

<div class="post-metadata">

**Author:** ![davidz](https://avatars.discourse-cdn.com/v4/letter/d/a88e57/32.png) [@davidz](https://discourse.nodered.org/u/davidz)\
**Post date:** [12 November 2021 23:07 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/10 "2021-11-12T23:07:06Z")

</div>

I believe that our case is a good example of using Node-RED in industrial applications with high throughput.

Just took some acceleration data at 800Hz for four [ultra-low power wireless vibration sensors](https://www.broadsens.com/sensors/wireless-low-power-sensors/) in parallel. Here is the data review. We do this many times a day with sampling rates up to 25.6kHz.

 ![high speed wireless vibration data acquisition](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/e/de1ff51d0b1a95ed5d9c36266ab986ce595b464a.jpeg)

---

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [12 November 2021 23:07 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/11 "2021-11-12T23:07:06Z")

</div>

Awesome @Steve-Mcl. That is actually a very interesting scenario with direct application to some solutions made by "legcy" (C#) coding here today. Thanks so much for sharing. 🙂

---

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [12 November 2021 23:15 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/12 "2021-11-12T23:15:29Z")

</div>

While way above my pay grade to understand fully, that is really cool @davidz . A quick followup to understand, do you query the devices and get a batch of measurement data points back at a relatively unfrequent (several seconds at least) interval, or do you read in realtime the current (one) data value and sample that at such a high frequency from NodeRED (that would be seriously impressive). Or du you do it in some other (probably way smarter) way? Thanks alot, 😊

---

<div class="post-metadata">

**Author:** ![davidz](https://avatars.discourse-cdn.com/v4/letter/d/a88e57/32.png) [@davidz](https://discourse.nodered.org/u/davidz)\
**Post date:** [12 November 2021 23:22 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/13 "2021-11-12T23:22:56Z")

</div>

A good question. The above GUI is for data query. We do lots of vibration DAQs and analyses daily.  
The GUI for [wireless vibration sensor data acquisition](https://www.broadsens.com/sensors/wireless-low-power-sensors/) is here. Just finished another round of DAQ at 400Hz. Did higher frequency DAQs earlier, so that we are going down on the frequency to test some new features for future development.

 ![high speed wireless vibration sensor DAQ](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/7/6/76acfc58d67638cffdd93390a5076c8f12e3b2df.jpeg)

The above DAQ is done manually. We have a timer setup that takes data automatically multiple times a day too.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [13 November 2021 02:26 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/14 "2021-11-13T02:26:52Z")

</div>

I'm using NR for home automation and have a bunch of sensors reporting every 1 minute. NR enriches data and redistributes data back out to MQTT and outputs to InfluxDB.

Probably my heaviest processing is managing the data from the Drayton Wiser heating system. Because this doesn't have a published API, I have to get the full XML from the controller every minute, analyse what has changed and then redistribute that data to MQTT and InfluxDB.

I know that we've also seen people trying to do processing on the ms level. That can have mixed results due to the nature of Node.js and Linux - all the possible interuptions make accurate timing somewhat variable. But I think that we've certainly seen processing of thousands of messages per second even on single-board computers.

I think that the biggest constraints we see are memory and IO.

---

<div class="post-metadata">

**Author:** ![RootShell-coder](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/rootshell-coder/32/47167_2.png) [@RootShell-coder](https://discourse.nodered.org/u/RootShell-coder)\
**Post date:** [13 November 2021 04:24 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/15 "2021-11-13T04:24:44Z")

</div>

I had a task to integrate a virtual cash register for printing checks into an existing payment system. It was also required to send a notification to the phone and email. I cannot say that this is a high load. Performs about 1000 operations per day. I couldn't find anything easier. Undoubtedly it could have been done in nodejs. And... perhaps there are no other options. I am ashamed to admit, I first wrote the integration, and then began to deal more closely with node-red.

---

<div class="post-metadata">

**Author:** ![RootShell-coder](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/rootshell-coder/32/47167_2.png) [@RootShell-coder](https://discourse.nodered.org/u/RootShell-coder)\
**Post date:** [13 November 2021 04:47 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/16 "2021-11-13T04:47:49Z")

</div>

![df](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/5/3/53fa7720b30a8db63f0f254c36b51707cba0f33a.png)  
here's another small example

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [13 November 2021 08:49 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/17 "2021-11-13T08:49:54Z")

</div>

> but high volume loads and complex logic generally pose a problem anywhere

I think that is the key: what complex/logic operations are you planning to perform ? It can be bogged down by all kinds of factors, but they can be optimized as well. nodejs (what node-red runs on) by itself is very fast.

I don't think it is hard to emulate real-world data, launch a VM and run it against node-red in some test setup and measure/collect any findings. It is the only way to determine if it is the right tool for the job.

---

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [13 November 2021 09:28 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/18 "2021-11-13T09:28:45Z")

</div>

That's really interesting @TotallyInformation. Handling 1000 msgs/sec would add up to several million an hour. And if the case is really simple like reading, complement and adding a record to a database (like a log value or a license plate) would be feasible. Thanks for the info.

---

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [13 November 2021 09:31 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/19 "2021-11-13T09:31:12Z")

</div>

@RootShell-coder That is also a very good real world example that shows reliability. Thank you so much for sharing.

---

<div class="post-metadata">

**Author:** ![werkstrom](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/werkstrom/32/41798_2.png) [@werkstrom](https://discourse.nodered.org/u/werkstrom)\
**Post date:** [13 November 2021 09:33 UTC](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651/20 "2021-11-13T09:33:21Z")

</div>

@bakman2 Totally agree, and we are doing that. However, the power of being able to point to others cannot be underestimated. 🙂 But you are absolutely true when it comes to actually validate different real world use cases.

[Next page](https://discourse.nodered.org/t/high-throughput-examples-for-arguing-for-the-platform/53651.md?page=2)