# ETL Pipelines with Node-RED?

**URL:** <https://discourse.nodered.org/t/etl-pipelines-with-node-red/84347>\
**Category:** Share Your Projects\
**Created:** [6 January 2024 19:09 UTC](https://discourse.nodered.org/t/etl-pipelines-with-node-red/84347 "2024-01-06T19:09:33Z")\
**Posts on this page:** 1\
**Showing post:** 25

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [8 January 2024 16:05 UTC](https://discourse.nodered.org/t/etl-pipelines-with-node-red/84347/25 "2024-01-08T16:05:18Z")

</div>

Just an update on this, I've got the following going:

![flatmem4](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/b/db43ae5a239ab2fcab2c174d54a4826a5da8b87e.gif)

What this shows is the web2disk node streaming 145MB from web to disk. The disk2stream node then unzips that file, sees a 1.8GB [JsonL](https://jsonlines.org) file and begins to stream that line-by-line generating 2M messages, that end at the debug node.

What the point here is that memory is complete flat the entire time, it's all being streamed. The other thing is that the whole thing took 90s to do.

That's a promising start ... 😉

---

_[View the full topic](https://discourse.nodered.org/t/etl-pipelines-with-node-red/84347)._
