How/When do you tidy up your code/Fix up edge use workarounds?

Guys,

Just wondering what the general approach here is (as there are a great range of levels of programming ability on here) to going back and redoing code once it gets too messy/difficult to maintain ?

In my instance i have implemented a whole of house enegy monitoring and use system that has evolved over the last 4 years or so.

First we had solar so i tracked the output of that, then i implemented a whole of house energy monitoring system and tracked where we were using power and sequenced variable loads around solar output etc

The biggie was when we changed energy providers and implemented a battery solution, i have ended up with around 20 tabs or so that handle different aspects of the battery control and cater for edge use cases as the appear and it is starting to get to be messy and difficult to maintain/troubleshoot.

How do other people hande going back and tidying something up - i have everything pretty well documented on each page, as to why it exists, what it should do etc but it has turned into a bit of nightmare to track the logic and see what is setting global environment variables that are shared between the different flows etc

Craig

My "main" NR home automation system has 39 tabs, running on an old Lenovo laptop previously running Windows but know with a replaced HD to SSD running Debian

I prefer splitting up functionality on "many" tabs instead of having them "too messy". The search function is very useful, you can easily find where you use your context variables and others. Each flow has evolved during the years, when I found some better solution to particular needs or replaced a solution with other nodes. Today it is "stable", don't think I have changed any vital thing the last year

The "main" NR is the hart in my total system; in addition I'm running NR in a number of RPi's and a Nvidia Jetson Nano for various tasks and services, they are all part of the total system having their dedicated responsibility, communicating via MQTT. The principle is the same, I split up functionality on tabs to make it clear and easy understandable

Best regards, Walter

Yeah, I have the same issue .... around 50+ tabs... it was above 80 in peak times :wink:

The new feature to hide a tab did help me a lot and I usually have only 5 tabs visible.

Over the last 6months i did a wide cleanup activity to reduce the number of tabs ... I create a new tab where I optimize my flows ... and disable the old ones.

If the new solution works well I download the disabled tab to a file and remove it from my flow NR environment.

I'm now working on NR for over 2years and done optimization and improvements on a regular basis

Some of mey key learnings:

  • Avoid using GLOBAL variables (I'm managing all devices and states in things)
  • dont repeat or copy/paste flows or partial flows too often ... use SubFlows where possible .. it will simplify your flows
  • minimize the use of protocolls (e.g. I mostly use MQTT, and webservices)
  • use an database external database for historical information (e.g I use InfluxDB)
  • use grafana for historical diagrams ... the UI-Chart node was creating high cpu and memory utilisation
  • use TELEGRAM to send notifications (alarms, events, status information, pictures, videos) .. works seamlessly and fast
  • use TELEGRAM to send COMMANDS back to your system (no VPN/DuckDNS required) ... (i can switch on/off devices or request a status for sensors or even a recording from my surveilance system)
  • use an old laptop with ubuntu&docker to operate NR .. improves stability and availabilty during blackouts
  • use a NAS location for your config and user-data folder .. it simplifies backup & restore dramatically
  • use fixed-ip adresses for remote devices & sensors ... any reboot of your router may cause failures as IP adress can have changed
  • distribute your internal services where possible (I have MQTT on dedicated RPi, and Zigbe2MQTT on another RPi)
  • use VPN/DuckDNS if you need to access your system from outside ..(there might be better solutions)
1 Like

Well, like Walter, I started years ago with a single Pi that became two Pi's and then I started migrating to a 3rd. About then I decided that the Pi's we too complex and replaced them with an old Lenovo laptop with Debian. Didn't bother replacing the drive though since I'm not doing anything that needs any drive performance.

Along the way, I've slowly evolved my understanding of Node-RED, Node.js, JavaScript, IoT, MQTT, InfluxDB, HTML, CSS and doubtless a few other things. :grinning:

So every now and then, I'll look at a flow and think of a simpler way of doing it and work towards that - which might be quick or might take some time if I need to change underlying data structures. I'll often have things running in parallel for a while.

But for me, the big changes tend to come when I look at my flows and decide that I can't be bothered to make changes because everything is a mess. At that point, I'll stand up a new, clean instance of Node-RED and start to move functions over while cleaning them up.

Because I'm familiar with JavaScript, the other thing I'll tend to do is collapse multiple nodes into a single function node where that makes sense.

Actually, I do make use of them a lot. Typically, I've ended up with MQTT topic hierarchies that I also have as a global var. That way, I always have instant access as well as subscriptions for change events . If the context variable functions had change events on them I wouldn't need MQTT anywhere near as much and one day I really will get round to writing a suitable plugin!

Actually, I hardly even need to use SubFlows. But I do make a lot of use of link nodes which can not only greatly simplify the visuals of your flows but also let you minimise the number of repeat nodes. For example, I will typically have 2 MQTT-out nodes on a tab - one set to retain and one not. They have link-in nodes attached and are used by all flows on that tab. Not much efficiency gain probably but it really helps where you later decide that you want to include some standard metadata with your mqtt outputs because you now only have to do them in 1 place on each tab. I generally try to avoid link nodes across tabs as that can get a little confusing. But I always try to name my links anyway.

Yes, totally agree. And agree that Grafana is better than Node-RED for most historical dashboards.

I also use Telegraf to grab system monitoring information from the server and from other devices such as Wi-Fi APs and routers. I output from Telegraf to both InfluxDB and to MQTT so I have both instance and event data via MQTT as well as historic monitoring via Grafana.

Yes, possibly the easiest bot tools and has really good performance and reasonable security/privacy.

Agree. I don't often use this but it is a good way to add simple, Internet connected controls without having to faff with secure proxies or VPN's.

I back up Node-RED 7xdaily/5xweekly/12xmonthly simply to local folders. My NAS takes backups from the server. I've never bothered to back up InfluxDB as I don't really need that data, I'd just start from scratch if I lost it.

Personally, I wouldn't try to use remote NAS folders for config and userDir live use as that complicates things and adds something else that could go wrong. The NAS is perfectly capable of backing up the server folders and files.

Absolutely, this is a really nice tweak. I use my routers DHCP settings to fix IP addresses for known MAC addresses and I give things recognisable (to me) names. I also maintain a table of addresses (IP and MAC) which I can add descriptions and other data to via a uibuilder page. That table is also updated by a BASH script that runs periodically using nmap. That way, I know anything that has ever connected to my network and can map known devices and when they were last active.

The other nice tweak is to implement a local DNS that uses "hairpin DNS". This lets you use an external domain name internally. That, in turn, lets you implement TLS using Let's Encrypt certificates. It stops your browser from whinging at you that you are using "insecure" web pages.

Yes, either your router's VPN capability if you can be bothered to set it up. Or NGROK or Cloudflare Teams (their zero-trust service).

1 Like

As mentioned, I use THINGS .. there are few other THING nodes available .. it works like variables with hirarchies .. but you can have events on em as well as soon a value in the hirarchy is changing .. :wink: .. I only can recommend using it :wink:

Hey guys thanks for the feedback - i will look at the THIINGS link that as provided thanks

I have already done many of the things that are listed but there are some nice pointers in there re flows etc - so thanks for that

Anyone elese wants to chime in - i am all ears.

Craig

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.