# Where/how does "Delay" store data?

**URL:** <https://discourse.nodered.org/t/where-how-does-delay-store-data/41602>\
**Category:** General\
**Created:** [25 February 2021 07:11 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602 "2021-02-25T07:11:30Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![NickXML](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/nickxml/32/37300_2.png) [@NickXML](https://discourse.nodered.org/u/NickXML)\
**Post date:** [25 February 2021 07:11 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602/1 "2021-02-25T07:11:30Z")

</div>

I have a system where data arrives in waves. There are a potentially large number of sensors that report data every few minutes.

My node-red flow grabs this data and writes it to influxdb. I'm using a delay node as a rate limiter to even out the spikes.

It's working fine at the moment.

However, I was wondering about the details of _how_ Delay works.

Specifically, how and where does Delay store the delayed data. Is it in memory, on the disk, or ?

I'm asking because I'm running this on a Raspberry Pi 4, and I'd like to understand what might break the system as the number of sensors increases.

All feedback welcome.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [25 February 2021 08:30 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602/2 "2021-02-25T08:30:37Z")

</div>

> [@NickXML](#):
>
> Specifically, how and where does Delay store the delayed data. Is it in memory, on the disk, or ?

The rate limited messages are held in memory (in a queue so to speak) until its their time to be released.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [25 February 2021 09:06 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602/3 "2021-02-25T09:06:14Z")

</div>

If throughput is really an issue (do you know that it is?) you might be better to use the influxdb-batch node, which allows you to add a number of samples in one call, which is more efficient that many individual writes.

---

<div class="post-metadata">

**Author:** ![NickXML](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/nickxml/32/37300_2.png) [@NickXML](https://discourse.nodered.org/u/NickXML)\
**Post date:** [25 February 2021 13:06 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602/4 "2021-02-25T13:06:11Z")

</div>

Thank you. Very helpful.

---

<div class="post-metadata">

**Author:** ![NickXML](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/nickxml/32/37300_2.png) [@NickXML](https://discourse.nodered.org/u/NickXML)\
**Post date:** [25 February 2021 13:16 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602/5 "2021-02-25T13:16:11Z")

</div>

Hi Colin,

Good points. I don't know for a fact that it will be an issue, I'm just trying to plan for potential issues.

I'm using the Pi with an SSD, so it is quite fast. However, when loading the same type of data via CSV, I found that pushing more than 200 writes per second caused some data loss. That's really but a complaint -- I had no idea I could get that performance from a Pi.

I will definitely look at batch mode. Any idea what the limitations of batch mode are (broadly speaking)?

I'm blown away at how robust node-red (and this whole stack) seems to be. It's come a very long way in just a couple of years.

I also very much appreciate the quick and helpful responses from you and @Steve-Mcl. Thank you both!

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [25 February 2021 13:55 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602/6 "2021-02-25T13:55:20Z")

</div>

> [@NickXML](#):
>
> . However, when loading the same type of data via CSV, I found that pushing more than 200 writes per second caused some data loss

Not necessarily relevant when using influx as the db writes happen in a different process. However with that many to handle then the batch node is probably a good idea anyway.

> [@NickXML](#):
>
> Any idea what the limitations of batch mode are (broadly speaking)?

It is impossible to answer that really. Are you talking about how fast influx can write the data or how fast you can prepare it and send it to influx? It is probably impossible to predict which one of those might be the limiting factor never mind say what the limitation might be. The only way is to try it with your hardware and see how it goes.

---

<div class="post-metadata">

**Author:** ![NickXML](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/nickxml/32/37300_2.png) [@NickXML](https://discourse.nodered.org/u/NickXML)\
**Post date:** [25 February 2021 14:04 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602/7 "2021-02-25T14:04:58Z")

</div>

Got it.

I was just checking for any major gotchas that might not be apparent to a noob such as myself... Just the classic "here be dragons" stuff like "this works perfectly for everything except ." I hadn't seen any documented problems with batch, other than the minor issue with timestamps that I saw discussed elsewhere. I figured I'd ask just in case.

Thanks again!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [26 April 2021 14:05 UTC](https://discourse.nodered.org/t/where-how-does-delay-store-data/41602/8 "2021-04-26T14:05:18Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.