Experimenting with the Admin API

General
1

Hello,
Been experimenting a bit with the admin API and I have some questions.

I can POST a simple as well as more complex flow (tab) to the server. Like this simple example:

{"id":"91ad451.f6e52b9","label":"My Sheet test","nodes":[],"configs":[]}

If I then retrieve the same using a GET action (using the id I got in the response to the POST action) and save it to file, it looks almost the same:

{"id":"44a96de370716611","label":"My Sheet test","nodes":[]}

The thing is, I would now like to edit the retrieved flow to add some stuff and then upload it again using PUT

But the format is not ready to be imported into the NR editor. Would it not be better that the retrieved tab flow would have the correct format so it can be imported?

The same for the PUT action. It should be able to take the from NR editor exported format?

2

For the serverless NR, I created a minimal viable backend for the Node-RED editor to provide it with an initial flow.

What I realised is that the editor expects the following format (found in this node):

msg.payload = {
    flows: msg.payload,
    rev: RED.util.generateId() + RED.util.generateId()
}

where msg.payload is an array of node/tab/config objects and rev (in this case fake) is the current revision of the flow on the server.

The editor code that handles this response is over here. If you replay that code with your flow, you could probably update the editor with your changes.

Update: in fact the initial flow loading code of the editor is this, the above is handling the results of a deploy.

1 Like
3

I continued my stumbling experimenting further, now with nodes kindly shared by @gregorius

In my nearby, local RPi, I have a flow that can send a request to a remote RPi to get the complete flow returned. Communication between the both is via a public MQTT cloud service (HiveMQ)

The returned flow is saved to disk. Then being imported for some editing, then exported to a another file, finally sent to the remote, again via MQTT

image
image1121Ă—370 35.5 KB

My local RPi

When the flow is arrived at the remote site, it is replacing the current flow

image
image1121Ă—370 26 KB

My remote RPi

So far so good. It seems to work. But I have to be careful, it is easy to damage the security setting of the MQTT broker since credentials are not transferred

Ideally I really would like to avoid updating the flow tab with the MQTT nodes, I would like to just leave them as is but I cannot select which tabs to send, only the full flow is possible to send since the sent flow will replace everything

1 Like
4

Oh dear, we've both done the same thing. I've created a HTTP<-->MQTT proxy, I asked in the forum and then went ahead and built it.

There is no file storage just direct communication with the target NR instance. For this to happen there is a flow on the target NR instance:

Screenshot 2024-12-03 at 14.35.12
Screenshot 2024-12-03 at 14.35.121388Ă—587 63 KB

It also proxies the websocket connection so that debug message actually appear in the dummy NR instance.

The proxy code is this:

Screenshot 2024-12-03 at 14.36.44
Screenshot 2024-12-03 at 14.36.441710Ă—796 104 KB

The top half is the http handling of the dummy NR instance. Basically what it does is store a reference to the http response object, send out a mqtt request to the target NR instance, obtain the response and send it out on the http response object stored in the flow.

The bottom half is the websocket handling, again just converting websocket requests to mqtt messages and waiting for responses.

The final part is the serverless NR instance that I point to the proxy. Loading it into the browser gives me the complete flow from the target NR instance in my browser.

Fun project and extremely useful for those that only have an MQTT connection to the their headless NR instance (somewhere in a dark, damp place!).

2 Likes
5

Impressive! Is a kind of personal "FlowFuse" setup alternative...

I think this is a pretty cool concept and I also like what you asked in the forum

Just for my understanding; in your case you are actually editing the full flow as if you would be on site and load a local editor?
There is no risk, as in my case, that you may mess up the local MQTT security settings so that the remote disconnects the MQTT connections?
I did myself some mistake so when I sent the updated flow, by mistake the MQTT connection on the remote stopped working (fixed since both the remote and local instances and RPi's are currently in my "lab", just got me worried)

Is this whole thing not actually what FlowFuse is trying to target? But at a cost...

Next step, to make a remote dashboard (1 & 2) work using the same principles via MQTT?

1 Like
6

Definitely not my next step, dashboard would require a lot more proxying of HTTP requests.

FlowFuse is managed deployment of Node-RED instances. Those instances - AFAIK - are accessed directly using a browser - there is no proxying of the HTTP connections and certainly not via MQTT.

Definitely and yes it can be easily broken but I make the assumption that adults are using this proxy! If I only saw the risks, I'd probably do nothing.

For the current project having access to remote NR instance which only communicates via MQTT is already a big win - if its only to see what is installed on the endpoint. Most certainly this requirement will grow as the project progresses but those are the bridges that need crossing in the future, not now.

7

Understood, of course, just joking, just to challenge...

I think I have a sort of understanding of your flows but what part should be placed where and there is where I struggle (my flows are not that populated as yours and I think yours is smarter). Would it be possible to ask if you could share a sample flow for each of the instances, one for the local and one for the remote and I can start trying it?

Best regards, Walter

8

One flow needs to be on the target NR instance (the top screenshot) and the second screenshot needs to be on a separate NR instance which acts as the proxy server. And a third serverless NR that points to the proxy server.

Since the flows are part of a paid project, I won't be sharing them - copyright, licenses, contracts and all that jazz.

But that does remind me of something else I was going to raise but I already know what the answer will be.

Stuff like which is - potentially - useful for many people and companies is developed by developers for free without anyone paying them and using a non-restrictive open source license. Which is a pity. So I was thinking: won't it be nice to have a platform where common problems are solved by companies coming together and sharing the costs for developers to develop open source software for the benefit of all.

Now I know that sounds crazy but strangely exactly such a platform did exist - BerliOS but of course it failed because companies weren't willing to share the cost and then not having the sole benefit.

Now that might well be the case for software that is business related (i.e. which company is going to share the cost of developing software for it's core business) but in the IoT world there is much that can be shared and provide a common basis for the higher level solutions that companies offer as paid services. For example: device protocols that each company ends up implementing themselves (those packet-based protocols where bit logic is important) or nodes for common APIs or solutions like these.

Of course there are other platforms (kickstarter & co. for one) that kinda offer this service but it doesn't seem to be a thing in the IoT/Node-RED world.

9

You tell me, is like kicking on already open doors. I contributed during some 12 years with nice, useful, feature rich plugins for another platform, EventGhost, originally a German initiative developed by the user Bitmonster in Python (if I remember correct)

I know thousands of users around the globe used them. Never got a single penny as thank you and I believe similar situation for most contributors

I fear that my questions now are getting close to the edge where it soon becomes inconvinient and where relevant answers may lead to competition with the payed alternative already mentioned but I might be totally wrong

As you mention, the lack of (financial) support is the biggest threat to such platforms that are depending on the willingness of devoted developers to spend night and days for free, just because they can and like the thrill. I asked a question related, how is this dependency, but got no answer. Obviously. Just look at the never ending questions coming up related to video streaming; a very complex thing where one key non-payed developer just disappeared, is a pitty but understandable

10

Beyond my knowledge and perhaps there are paid alternatives. However I would go as far as saying that the solution that I created does not exist since it requires a serverless NR which - until I created it - didn't exist!

So this solution is definitely unique but that doesn't mean that someone else hasn't already built a closed-source variation of it. Probably based on tunnelling the http requests through a VPN or something. (or HTTP over SSH or similar.)

The nice thing about implementing it is that you learn a bunch of stuff on the way NodeRED client interacts with its server - quite a learning.

11

There are many companies that offer solutions to allow remote access to the OT network. If you for example purchase a router from the Teltonika company you can use their RMS to access your devices behind the firewall in the router. What you get is a VPN connection so you can access your Siemens S7 PLC's and other devices (like a RPi running NR). Works fine but is tied to a subscription plan. OK for companies but not so much for private use

Anyway, this was an interesting learning. I do not have a personal need, I already have a VPN server at home, I just thought it was interesting to see what can be done