I need a help regarding Node-red at enterprise level As I want to use Node-red to collect data from Siemens S7 PLC for my project I/O may vary from 10s to 1000s and I am gonna use Node-red on my desktop computer so I want to check anybody is working on Node-Red at enterprise level with such a large I/O and what are there opinion regarding Node-Red ?
1 Like
Hi, developer of node-red-contrib-s7 here. We use Node-RED to collect data from Siemens PLCs in the industry at levels like you mentioned very reliably. Te actual amount of variables will depend on their type, your network throughput and round-trip delay (the S7 communication protocol has a pretty big overhead), therefore I'd recommend to put your collecting device physically as close as possible to the PLC.
We develop and use ST-One for tasks like this, so you can have Node-RED running just beside the PLC, and from there on perform any analysis or send the data to wherever needed.
7 Likes
I use it in a large medical setting (handling routing for messaging, watchdogs, protocol translations, logging to database) and I've had quite a bit of succcess. One instance (we have several spun up) processes a couple million messages a month and chugs along just fine.
4 Likes
We use Node-Red as an Enterprise Service Bus (ESB) and Extract Transform and Load (ETL) tool to integrate 8 different cloud systems ... copying records between the systems via cloud APIs. Once we had our config down (there was some mucking about with PM2 on our Cloud VM), we've had zero issues with reliability[*see note].
As far as data volumes we mostly have to throttle node-red (delay node set to rate limit) so that we don't trigger usage restrictions on the webservices we are calling or hit issues with overwhelming databases with the number of transactions we were pushing through.
I am gonna use Node-red on my desktop computer
Note: you're not going to get "Enterprise" reliability running on a desktop computer but we had good luck running one of our process on a laptop for a couple of months with no reliability problems. If you are running it on a desktop ... I've had better luck running node-red on an Ubuntu Windows Subsystem for Linux (WSL) instance rather than the default windows install (but I think that was mostly due to the add on modules working better in that environment).
Might be even better to just get a Rasberry PI and a good power supply (lot's of discussion about that on this forum).
[* ] Note: while we've had near zero issues with Node-Red itself ... we've had some issues with modules added via the pallet not handling errors correctly (things like outages on the other side) ... so we couldn't rely on our catch nodes to catch everything and had to monitor the error out logs. In some cases we were able to fix the error handling in the module.
4 Likes
Wow, some great stories here. Really nice to see Node-RED being used in enterprise settings.
It would be great to see more blog posts about enterprise use of Node-RED so that more people had confidence in it. It would be particularly good to see posts describing their architecture and giving hints and tips. I know that isn't always possible due to commercial issues but still it would be great if a few more people could get the word out.
Using Node-RED as an ESB is an interesting idea and it would be good to know if you compared it against other tools before committing to it.
ETL use is a more common use for Node-RED I think?
Great all round and a nice thread.
I would also recommend using a Linux base OS for Node-RED as well. Generally fewer overheads and battle tested throughput.
2 Likes
Third-party modules may indeed be an issue. There are modules with poor documentation, stability issues, some affecting even the whole Node-RED. Therefore, whenever we need to use any new (for us, at least) node, we test it very well before putting anything in production. This same advice we give to our customers. Despite of that, Node-RED is awesome 
Another thing that would be very nice would be to have more Node-RED talks and events. There have been some talks already, but public talks about successful use-cases would encourage people to use it as an enterprise-level tool.
2 Likes
Hi,
I wonder how you, guys, solved the issue to automate the tests the flows before deploying. At enterprise level you need a robust automated testing strategies at every changes in your flows, especially for ETL applications.
Another story from Australia
We started using NR at my company - one of Australia's Largest Energy Companies in mid 2020. It was a journey to get NR endorsed as one of the enterprise integration tools in ~July 2020. Of course, we have a few more enterprise integration tools part of the endorsed list, however NR is being promoted having following benefits:
- Integration as a Self Service > this is a big one for us, as we as central IT provide a platform for LOB IT to build and optionally maintain integration logic
- Obviously, zero license cost
- Enterprise support for nodes (yes, we have built our own nodes for different aspects including - data source based nodes, dynamic configuration, etc)
- There are other aspects to it that we have built:
- CI/CD across different environments
- Security
- Design & Architecture
- Development patterns and best practices
I am happy to share more details if anyone is interested.
4 Likes
One thing that personally I'd like to see more of and am trying to promote in the organisations that I architect for is a better acceptance that, as open source software becomes more prevalent, we invest upstream in the tools.
It would be great to see more organisations investing in Node-RED, doesn't need to be money though doubtless an investment fund could be managed by the JavaScript Foundation, but could simply be some small amount internal resource time to spend on bug-fixes, documentation improvements, help and how-to articles, etc.
This would massively improve Node-RED and bring much needed resources to the project.
9 Likes
Totally agree. My team’s plan is to convince the leaders in the “value of open source” and what it means to benefit from an open source - how and why we need to invest back.
Obviously, like with many large organisations it is a journey that we need to take to make the business/ IT leaders comfortable with it.
However, any help/guidance from NR team would be appreciated.
1 Like
Might be worth starting a separate thread where we can discuss options and ideas maybe with a view to compiling a list of things that organisations can do?
It is nice to see that Enterprises are slowly, but steadily gain consciousness over the open source and choose to support people behind it in voluntary rather than forceful market ways (I had to consult some software companies whose client wanted an open source solution). I would say that from my experience with enterprise it is hard to convince stakeholders to allocate money for anything that is not directly influence the profits, and especially hard if the company doesn't have a decent digital+business+social culture that appreciates the passion and benefits of open source and understands the economic processes that happen behind.
However there are good examples that can be cloned and enhanced. For example, that's just a random related blog post that I happen to come about today.
There is also a permanent place to discuss best practices should an enterprise feel well enough to share its success with maintainers of underlying infrastructure.
Feel free to move it into a separate thread. I am vary amused to see S7 protocol disassembled enough to be used with JavaScript, and I'd be interested to see more examples of it. My former team is responsible for discovering Stuxnet, you know, and working with Siemens and open source I could not imagine more things that don't work together. )
4 Likes
We don't all work for commercial organisations so profits not always the issue with open source investment. More often for me its about the fact that resources are always far less than needed, open source is often seen (wrongly) as a shortcut to avoiding outgoings.
Interestingly, I've just come off a very useful call with GitHub and Microsoft where we discussed how we can help the UK health sector make better use of GitHub and lower costs. UK Government and the NHS are making increasing use of open standards and developing open source solutions. This is an acceleration of a process that has been on a very slow burn for maybe a decade or more. The need for very rapid change, innovation and collaboration due to COVID-19 has really changed the landscape a lot.
Although GitHub is offtopic, I just want to remind that the platform itself is not open source. On a national level it is better invest resources into federated solutions that are not tied to specific vendor. Healthcare is a quite sensitive topics, and doubt UK citizens would be comfortable that US has a legal authority to track the behaviour of their developers and healthcare professionals.
Thanks, we are of course very mindful of this. But GitHub is so central to the world of Open Source that this is one of the exceptions that break the rules. Thankfully, Microsoft also realise this and are not trying - yet anyway - to undermine GitHub's position and strength.
We are very clear about what can and can not go into GitHub. But the whole point is that we need to get used to working in the open instead of behind closed doors. The debacle with track & trace v1 really brought that home to a lot more people thankfully. v2 is entirely open source.
As for citizens, they seem, by-and-large, quite happy to share their innermost secrets with vendors who have the most appalling track records. Speaking both as a citizen and a tech leader in the NHS, I can only say that I am comfortable with what we are doing with GitHub and mostly comfortable with what we are doing with Microsoft.
It really isn't code and standards documentation that we need to worry about in regard to US access. I have plenty of other information areas that I have to worry about and ensure aren't in the wrong domain.
As for federated solutions. Yes, I would prefer that over a central solution. However, it is a much bigger problem trying to get the 20,000+ organisations and 1.7m people that make up the NHS to adopt open and common standards. So if there is a well recognised tool that helps with that, I'm going to advocate for it until something better comes along.
But you are right - we've strayed rather a long way off-topic, apologies. Happy to carry on the conversation elsewhere - as you can probably tell, this is an important part of my professional work not just part of my hobby. 
GitHub became central to the world of Open Source, and that's the reason Microsoft got hold on it. The acquisition had been preceded by several social justice scandals, which could be as well classified as attacks that presumably made the deal of selling the octocat more likely for the platform that was historically a Ruby shop with no prior connection to this controversial corporation.
By analogy with Node-Red, it is okay to advocate for Node-Red as one of tools to build the solutions that can be used on both on premise and in the cloud, but advertising for specific hosting provider for Node-Red is not something I'd expect from national healthcare campaigns, even if decision makers are comfortable with their offers.
Happy to carry on this discussion by PM or on another forum but we are too far off-topic here.
Hi there, I would be very interested in your setup mentioned here
Topics i am interested is the ones you mention below plus one more
- How you have ensured best performance of NR for multiple clients (if it applies in your setup)
- CI/CD across different environments
- Security
- Design & Architecture
- Development patterns and best practices
1 Like
I can answer this in two ways:
- performance of development container - NR is not multitenant. So we setup docker containers managed by ECS as development playgrounds for the devs. i.e. each dev is given a dedicated dev-container. These containers roughly cost $20/month if run 24*7
- performance of deployment container - We use CI/CD to deploy the flows into separate docker containers (one per project, not one per developer). these containers are also managed by AWS ECS and can be scaled* both vertically or horizontally - taking care of performance & scalability
* As a standard, all our flows are loose microservices i.e. exposed as a HTTP REST API
What is your question regarding these?
3 Likes
Great answer it seems that we have the same setup (ECS cluster etc). Do you have a specific process as to what users are able to do in terms of flow creation and modification? We are thinking that there should be a specific process before anything is pushed to production so we're thinking of controlling the flows.json updates through git PRs. Also in our case flows shouldn't be heavy so if you have a performance checking process I would be happy to hear it
