[Feedback Wanted]: Save change history to session storage

Thanks for the clarification @GogoVega.
Indeed, this is of no value to me. :neutral_face:
I do [add node, deploy, change, deploy, add, deploy ... ] (not quite so rigorously but that's the idea)
This is the approach I learned with Word in Windows 3.1 - save it or BSOD it.

I think I'd expect to lose undeployed work if my browser crashes.
Contrariwise, if the Node-red process crashes I'd expect everything undeployed to be safe in the browser.

This is already the case, except that there's no persistence; a refresh will make everything disappear.

The goal of this discussion is to find a native way for NR to retain history. Browser storage is one possibility, but I don't yet know the technical limitations (about size etc) but that limits the functionality to the browser.

@GogoVega this lib seems ideal https://rxdb.info/

If it does reach core because of personal reasons you could write a plugin "node-red-crash-protection-plugin"

We'll see about the plugin. The whole problem is optimizing the history as much as possible because it's not easily reversible.

It's second task on my list; I'm looking into a very old concept that I'd like to revive. No spoilers :shushing_face:

Give us spoilers!!! :pleading_face:

1 Like
Here you go 🤫
3 Likes
I won't say any more 😁

microsoft-windows-windows

I will share some of the secrets. Preview here

If the production environment is that critical then the expectation would be that you have (at least) two systems - one for production and one for dev/test. The dev one being local enough that you can deploy as often as you wish (eg on a laptop), to test and debug, before moving it to the live instance.

4 Likes

@dceejay you should open the preview I shared and tell us what you think

So how would effect a SD card on a Pi?

How much space would it use?

What effect would have on the life of a SD card?

I think we are talking about storing it in the browser, so assuming you are not running the browser on the pi then the answer is none. Even if it were on the pi then the effect would be negligible.

1 Like

Nobody with any professionalism would work on something that couldn't be saved for several days. As Dave says, this isn't necessary anyway since they would be working on a development version, not anything live. Such systems would always have dev, test, UAT and live instances. Something that is easy to do with Node-RED.

In a dev environment, deploy often and have backups of the flow file, this is far more robust and does not require any "clever", complex additions to Node-RED.

1 Like

I can also imagine a better world but it would be nice to hear from folks who actually use NR in production and how they deal with these issues.

So far we are all just theorizing about how to solve these issues but unfortunately, no has spoken of their real-life experiences nor their approaches.

For example, recreating a development or staging environment that mirrors the production setup is usually a non-trivial activity - mainly because of external dependencies and/or production data in large databases. So it would be interesting to hear how others do this.

Also the possibility of exporting work-in-progress using the export interface is an alternative to setting up development & staging environments. This is an annoyingly manual task and error prone.

Agreed. Though the principals of good practice for development, testing and production hold true across all different development environments. Of course, my experience is from large enterprises and I'm certain that not all dev houses follow good practice. :frowning:

Yes, it would indeed. But the creation of dev, various test and production environments is a very well understood discipline and really not that hard to do, especially now that infrastructure as code is now so well established. Test data can be a little more tricky depending on the data, its complexity and sensitivity, but again, there are plenty of tools - including free ones - that can produce structured, semi-randomised data to a given schema.

Replicating a Node-RED environment is, of course, pretty trivial and easily automated without anything more than a bash script or 2. Or even a "controller" instance of Node-RED! :smiley:

The real danger here is not really professional developers but rather the ever present issue of "citizen developers". Something that Node-RED, of course, actually encourages. If setting up low-code services for non-developers in live environments, great care needs to be taken. Actually, this is probably worthy of some documentation. What do you think @Steve-Mcl? Should we create a design doc somewhere where we can get some contributors and agree an outline?

Perspective is often formed from experience. In my experience, true dev/prod environments where thousands of automation equipment's (PLCs, Robots, bespoke & proprietary hardware, industrial devices), creating fully representative environments becomes near impossible - or simply not feasible

Of course large swathes of infra can be shadowed, mocked or stubbed; however there is always a threshold where the simulation diverges so far from reality, it no longer serves its purpose (becomes a huge chore and ultimately provides a false sense of security)!

I'm not say "dont" have DEV/PROD setups, just be realistic. Break up larger instances into several (scale horizontally) and pick out the stuff that can be easily mocked/duplicated. Just be aware not every thing can be done this way.

My issue with "professional developers" (in the sense of those in the IS/IT depts), is they often do not have the necessary domain knowledge to truly understand the requirements of the business and so citizen developers are born.

This is something I talk about often (not usually on open forums mind).

I have personally experienced the snide comments and distrust from the IT depts but they often miss the point. the citizen developer has deep domain knowledge and XP (essential for "right first time solutions"), can move fast (productivity bonus) and form PoCs that should lead to IT supported systems (not always the case)

In my experience, the most successful orgs don’t suppress or demonise citizen developers - they embrace, guide, support and form colaborations.

This is what FlowFuse offers IMO. Permit orgs to (in some sense) take the reigns on the citizen devs and embrace it. Best of both worlds.


ok, sorry for the mini-rant above, it may be telling, but this is a topic I have a lot (25y) of experience of and feel quite strongly about.

Back to your question:

Its not an easy one to call Julian. for example how far does this go? Since "live environments" and "citizen developers" are mentiond, I will frame the following with "company"/"org" situation in mind.

If the IT team already embraces citizen devs then they should already be implementing best practices - which are mostly well documented in the developer/server/software space, something I know you yourself have extensive knowledge about (e.g. keeping things up to date, user permissions, backup procedures, std operation sheets, audit logs, etc).

In short, I guess my honest response would be utelise a professional platform for Node-RED and most of this is taken care of by the nature of doing so.

But even if they (the Node-RED advocate/user) want to use Node-RED without a professional platform offering, the first right step is to get the company & IT team on board. Get the the company and IT team to recognise the benefits of merging their experts with low-code tools like Node-RED and actually do what an IT team does best - provide a (safe & secure) breading ground for collaboration. IT folk get awareness and governance of the "shadow IT", workers become more engaged and produce inventive solutions. Knowledge remains (and grows) in-house (due to not using some expensive external non-domain-expert to do the citizen stuff). It is literally win win.

While not a direct answer, I am not against some "best practices" documentation that works for orgs/professionals/tinkerers alike - I think that would make a nice separate page addition for the node-red docs site.

Enterprises can also fail to follow best practices. In fact, it’s often much harder to enforce "current" best practices in large organizations due to complex organizational structures, internal politics, and resistance to changing the status quo. Best practices is not set in stone, it changes, specially in tech.

I've been thinking a lot to recreate node-red from scratch. I need a purpose in life anyways to get better. I also don’t like the way I've been treated in the repo. Even when I give good arguments, or ask right questions, it seems the answer is no because it is me. I confess i even changed my strategy to influence the project a bit using @GogoVega, with all respect of course. I also was trying to help the project improving the way nodes are created and had no feedback from knolleary or hardlib, despite all my efforts to caught their attention. Maybe i feel this way because im sensetive to rejection and i care too much of what people think about me. If it is truly not the case, sorry.

Allan, as a maintainer of Node-RED I read most issues. As far as I see, you have not been treated any differently and certainly not "because it is you". Please link to issues where you feel you have been treated wrongly (you can DM me if you wish)

We all have good ideas and often we can think our ideas are the best (because we want it). Sometimes we push and get what we want but it turns out not so great after all (e.g. making the node-edit panel more like a dialog, forcing users to click cancel was something I was convinced was the right thing to so - turns out I absolutely hate it!)

I guess I am saying, not everything a person suggests can get implemented and often is the case, should never be implemented, but this is not a rejection or personal.

We love that people are involved and passionate and improvement ideas are always welcome. Sometimes, there is just not enough hours in the day and things slip. I too have many unanswered posts, threads, issues (from everyone). It is not personal. Its life I'm afraid.

It is easy to fall into that trap. The way I handle these things is to remind myself there is always 2 paths. The "good path" and the "other path". The other path leads to self doubt, suspicion, unhappiness. The "good path" is the better path - so I take that one more often. I simply assume my unanswered questions or feedback is due to it being "valid but difficult to implement", "too large a task to give any credible feedback to at this time", or it "slipped their mind", or "its a terrible idea because of x/y/z, but the devs are too polite to say so" or any number of other "good reasons" - and I feel better for that.
When I assume the worst, it stings, so I mostly chose not to!

Your requests and suggestions may indeed be the "right thing to do" (or not), but just because they don't get implemented or supported does not mean you are being rejected.

5 Likes

As I most often open a function node to just look at the code, then wonder why I cannot click on something else, I'm now getting so used to "rage quitting" by clicking on cancel, that I have lost far more work to accidentally cancelling instead of closing, than I ever did with accidental edits.

@AllanOricil "sometimes even if it is broke, don't fix it "
:rofl:

There is nothing inherently wrong with the above idea, and it would be completely unnoticed by new users as it is logical, but "One man's meat is another man's poison."

OK enough proverbs, I still read your posts with interest, so even if I dislike the above, you are still valued as a member of the community. Look after yourself !

1 Like