So does this include a server ? - Can that be a node that binds to the local express service ?
Hold yer horses Hoss! No, not yet 
This is just the local part. The basic server should be pretty easy as I can take that from the examples on the docs site.
I'll update the thread when things happen.
Amazingly, managed to get another update out between dealing with a ransomware & email problem at work!
Refactored the code and settings to make more sense. Also see updated flow and note that the inject with the array JSON doesn't actually update the db. I think it is something to do with the fact that they don't seem to like array's. Will need to add some logic to cater for that. Also some of the labels (actually almost all of them!) are wrong.
[{"id":"392af451.27573c","type":"gun-get","z":"908cedac.17439","name":"","gunconfig":"dbbc7abe.617de8","soul":"test3","x":180,"y":360,"wires":[["462a3caf.4daf54"]]},{"id":"462a3caf.4daf54","type":"debug","z":"908cedac.17439","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":490,"y":360,"wires":[]},{"id":"a883018d.a7203","type":"gun-set","z":"908cedac.17439","name":"","gunconfig":"dbbc7abe.617de8","soul":"test3","x":470,"y":420,"wires":[["d9652d9e.74b47"]]},{"id":"d9652d9e.74b47","type":"debug","z":"908cedac.17439","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":770,"y":420,"wires":[]},{"id":"d80d1361.42b9a","type":"inject","z":"908cedac.17439","name":"","topic":"","payload":"[1,2,3]","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":190,"y":420,"wires":[["a883018d.a7203"]]},{"id":"4644c454.96e8ac","type":"inject","z":"908cedac.17439","name":"","topic":"","payload":"{\"one\":1,\"two\":2}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":220,"y":500,"wires":[["a883018d.a7203"]]},{"id":"2054570f.b77cb8","type":"inject","z":"908cedac.17439","name":"","topic":"","payload":"Just some text","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":220,"y":460,"wires":[["a883018d.a7203"]]},{"id":"dbbc7abe.617de8","type":"gun-config","z":"","server":""}]
Also @dceejay, I think that Gun.js sets itself up as a peer automatically. If you check the log, you will see that it reports that it is multicasting on 233.255.255.255:8765 or some such.
1 Like
Nice work. interested to see where this goes. I was tempted to pick this up if no one took the bait (fortunately you did). I might load it up and get a feel for it - see what it might be useful for - maybe I can throw in some ideas (or spanners)?
So it is - wowser it does seem quite "chatty" on that multicast. So I guess it tries to do local syncing automatically. Hopefully you can specify a list of remote servers to sync to for all those out fo the local network... (and indeed turn off local multicast). No doubt we will find out ! Looks good so far.
Seems rather chatty on the log too 
Might raise an issue on that to see if we can get an option to turn off the silly startup log msg.
Well, you can now enter 1 anyway, I've not yet made it a list and it is currently entirely untested so please feel free 
you set it using the config node.
Thanks. I nearly didn't and I really shouldn't have - but hey, who can resist right!
Well now would be a good time.
Please do, Gun.js is quite complex in detail and not especially well documented. However, it looks like it might be really quite useful. At the moment, this is just a very simplistic proof of concept and there are a lot of options that need adding.
OK, enough for 1 day. Please see the latest update - I'm trying to remember to increment the dev release numbers as I go along to make it a bit easier to know when there is an update.
The final version for today gets a couple of additional nodes - hopefully reasonably obvious in use. However, I'm finding that there are a lot of subtleties to Gun.js that are not yet reflected in the code. Also, you can currently only add/retrieve top-level entities to a "soul" at the moment and that itself has some complexities. I've started to try and document these either in the code or in the test flow shared below.
I am particularly interested, now that we have a very basic working proof-of-concept, in your opinions on style (of use not visual style) and vision.
You must have v0.0.1-dev4 installed for this flow.
[{"id":"8c238839.7283e8","type":"tab","label":"Gun.js Tests","disabled":false,"info":""},{"id":"674ccc55.7b53d4","type":"gun-get","z":"8c238839.7283e8","name":"","gunconfig":"dbbc7abe.617de8","soul":"test3","x":120,"y":140,"wires":[["4e9892bc.55e42c"]]},{"id":"4e9892bc.55e42c","type":"debug","z":"8c238839.7283e8","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":710,"y":140,"wires":[]},{"id":"c9faf3c0.9378b","type":"gun-set","z":"8c238839.7283e8","name":"","gunconfig":"dbbc7abe.617de8","soul":"test3","x":410,"y":320,"wires":[["e850514.bfcacb"]]},{"id":"e850514.bfcacb","type":"debug","z":"8c238839.7283e8","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":710,"y":320,"wires":[]},{"id":"c046700e.b3845","type":"inject","z":"8c238839.7283e8","name":"","topic":"","payload":"[1,2,3]","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":320,"wires":[["c9faf3c0.9378b"]]},{"id":"29f88dcf.4d4672","type":"inject","z":"8c238839.7283e8","name":"","topic":"","payload":"{\"one\":10,\"two\":20}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":400,"wires":[["c9faf3c0.9378b"]]},{"id":"69e1e701.54fa38","type":"inject","z":"8c238839.7283e8","name":"","topic":"","payload":"Just some text","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":360,"wires":[["c9faf3c0.9378b"]]},{"id":"956f71ab.57077","type":"gun-put","z":"8c238839.7283e8","name":"","gunconfig":"dbbc7abe.617de8","soul":"test3","x":410,"y":480,"wires":[["3e00e185.de11ae"]]},{"id":"3e00e185.de11ae","type":"debug","z":"8c238839.7283e8","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":710,"y":480,"wires":[]},{"id":"95694e7c.77ae8","type":"inject","z":"8c238839.7283e8","name":"","topic":"","payload":"[1,2,3]","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":480,"wires":[["956f71ab.57077"]]},{"id":"3d410b2d.14a174","type":"inject","z":"8c238839.7283e8","name":"","topic":"","payload":"{\"one\":1,\"two\":2}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":560,"wires":[["956f71ab.57077"]]},{"id":"88515841.6a40b8","type":"inject","z":"8c238839.7283e8","name":"","topic":"","payload":"Just some text","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":520,"wires":[["956f71ab.57077"]]},{"id":"23b6eb2e.2a0ee4","type":"gun-get-once","z":"8c238839.7283e8","name":"","gunconfig":"dbbc7abe.617de8","soul":"test3","singleOut":true,"x":400,"y":220,"wires":[["765375bb.f2d2dc"]]},{"id":"765375bb.f2d2dc","type":"debug","z":"8c238839.7283e8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":710,"y":220,"wires":[]},{"id":"fcbd3552.b44518","type":"inject","z":"8c238839.7283e8","name":"","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":95,"y":220,"wires":[["23b6eb2e.2a0ee4"]],"l":false},{"id":"32811971.abb886","type":"comment","z":"8c238839.7283e8","name":"Set vs Put","info":"On a root/top-level document:\n\n* Set with an object creates a single document with 2 properties and random key - Put creates two documents with keys of the object top-level property names.\n\n* Put with a scalar will not do anything because no key is available. Set does work because it creates a random key.","x":440,"y":400,"wires":[]},{"id":"216039df.978596","type":"comment","z":"8c238839.7283e8","name":"TODO","info":"* use .off() to stop this from creating a listener and only get triggered on input.","x":190,"y":200,"wires":[]},{"id":"efac55f8.ab4cf8","type":"comment","z":"8c238839.7283e8","name":"TODO","info":"","x":230,"y":120,"wires":[]},{"id":"75041178.1b3","type":"comment","z":"8c238839.7283e8","name":"Info","info":"## Using `.on` instead of `.map.on`\n\n* returns a SINGLE record for the soul instead of 1 for each document.\n* Does NOT output anything when a document is added/changed because the whole soul would need to be changed to trigger an update.","x":570,"y":200,"wires":[]},{"id":"abd816af.1d08f8","type":"comment","z":"8c238839.7283e8","name":"Overall TODO","info":"* Enable inner changes/subscriptions instead of just the soul-level.","x":470,"y":60,"wires":[]},{"id":"dbbc7abe.617de8","type":"gun-config","z":"","server":""}]
By the way, if you want to collaborate on this, please shout. Plenty of coding to go round 
Same goes for other people.
1 Like
@TotallyInformation - so i had a bit play...
node-red 1 (localhost:1880)
node-red 2 (localhost:1881)
works quite well - minimal setup - really simple to get going locally. 
Initial thoughts (some are lack of gun knowledge, some are observations I'm certain will get resolved as it matures)...
- the config node shows
when Gun.js peer server is left blank - what is the difference between put and set?
- I did note PUT did not cause the GET to trigger
- does the GET node need an input and a means of querying? (not certain it it works that way? - I did try in a node console (as described here) but
gun.get("soul1-out").val()just complainedUncaught TypeError: Gun.log.once is not a function - do the PUT and SET need to be 2 separate nodes?
- if the gun node permitted an option of PUT / GET then this could be determined at runtime by a msg property (e.g.
msg.gunOp = "PUT") (Again, might not make sense when i understand the difference - time to go reading i suspect)
- if the gun node permitted an option of PUT / GET then this could be determined at runtime by a msg property (e.g.
- have you setup a Gun.js peer server yet? How did you do that? Do you see a "Gun.js peer" node being added in this package - permitting a user to run a peer server from node-red?
Final words
- gonna do some local MQTT performance testing as comparison. Perhaps send different data and larger sizes too (and perhaps run it in a loop for average, min and max timing)
EDIT - vs MQTT (simple test)
EDIT - 20kb string
I observed GUN being about 1/2 the speed of my local aedes MQTT broker...
... while this is my 2 NR servers (on one machine, interanal data only) - perhaps when externalised with many clients, GUN will prove superior?
Thanks Steve, interesting stuff.
Ah, knew there was something else to fix there - thanks I'll hit that one, should be blank and shouldn't say GET.
Did you load up the test flow? It has a brief explanation there. Not sure I fully understand it myself yet.
On a root/top-level document:
- Set with an object (of 2 props) creates a single document with 2 properties and random key - Put creates two documents with keys of the object top-level property names.
- Put with a scalar will not do anything because no key is available. Set does work because it creates a random key.
Not really sure what to call that node really. Started with GET but it is more of an "In" node like MQTT-IN, it is a subscription to changes on the soul and any of its hierarchy. Ultimately, I'll add something that lets you create a subscription on something further down a hierarchy. Probably using a simple dotted notation.
You would have to load Gun in a browser context for that. Also, don't forget that get() returns a Gun object which has a number of methods but that doesn't include val() - if that came from the video, I note that some of those seem to be well out of date - seems that Gun has moved fast. You need an .once() with a callback or a .map().once() (or on()) if you want separate outputs for each change.
Maybe not, perhaps a single node with an option to use one or the other would be better? The structure of the processing and settings is otherwise identical.
No. I'm thinking that another node will be needed for that.
However, note that Gun on its own does create a streaming multicast server. Have a look at your node-red startup log and you will see something like Multicast on 233.255.255.255:8765.
Of course, that is a randomly assigned port. So not terribly useful. But you can use it for testing - but then you know that as you've done it!
Absolutely. This should be easy to do in Node-RED since ExpressJS is already available. I'll probably use the existing user-facing ExpressJS server that is already used by the http, Dashboard and uibuilder nodes. Might add an option for a separate server later on in case you wanted to use separate security settings.
Hmm, interesting. I think that this probably relates to the "eventual sync" model that it uses. Something that is much harder (if indeed possible) to get right with MQTT. Also, the fact that you can update a single property deep in the hierarchy most likely will show big differences - Gun gives you a LOT more flexibility than MQTT.
So I'm not sure that server-to-server performance is really the right measure. Raw read and write performance is more likely to be interesting.
I'm also interested in the performance once your Gun datastore gets really big. Gun is really more comparable with MongoDB or CouchDB. But remember, the focus for Gun is on being a local store with sync and no need for a central db server.
There appears to be a lot of (hidden) subtlety in Gun that needs some detailed discovery.
Now that I've discovered some basics, I'm quite keen to have a go at building a Node-RED context store with it. I think it is a much better fit than other versions I've looked at. Imagine an offline capable context store that you can share between Node-RED instances - across the globe if you wanted to!
well no - they have assigned it to themselves - it's the same for all instances. - If I had another instance running locally I would expect them to talk automatically - (assuming no firewalls... bit of an assumption)
That does sound interesting
I followed the online documentation, which linked to a video so yeah I tried the .val() as described.
Ah, OK, I thought I had seen it change. My bad.
I guess I know what I'll be doing this weekend then - dependent on what Mrs K wants done 
3 Likes
Well - mine is the same :-), And indeed just running it up on another Pi - If I do the object PUT then two objects turn up on both GET(test3) nodes... neither the array nor simple text go anywhere.
Right, so learned a little more.
There appears to be no way to actually delete a top-level element! The best you can do is set it to null. This appears to be a "feature" though it seems very odd to me.
The trick appears to be to move everything down a level and treat the top level like a db (that you can't fully delete!). You can empty a top level object by putting 'null' to it. That will remove any lower level elements.
So back to plan A where the config node included what I'd been calling the "soul". In truth, the concept of a soul is also more subtle than I'd thought. So I plan to call the top level a "db" and so you will be able to create multiple db's using config nodes. Unless anyone else has a better plan?
hmm - not so obvious is it :-)... so test3 (in your previous flows) seems to be the "database".
If I'm talking old school then we would then typically use something like msg.topic as the key and the payload as whatever, and put anything without a topic under some "unknown" key. So then hopefully you could "subscribe" to topics etc... in the familiar way... (though I haven't seen anything about wild cards - but I guess you can get things per property level ? needs more digging 
1 Like
No, not at all obvious. But I think you are about right. Looks like you may be able to use forward-slashes as delimiters too similar to MQTT topics. No wild cards or anything like that.
The 'soul' concept confuses me a bit. It is a graph database with a top element in which related elements live.
'database' (or graph) as top element makes more sense to me.
Looking at their api, it is heavily based on chaining. The GET method is used to PUT elements/objects in relation to the element specified in the GET. The current PUT node allows for one of those elements. This should be more generalized using a PUT input property I think. The SET method is used to create groups (i read that as a related table name) - like a (root?) topic in MQTT world ?
In their examples I see:
gun.get('user/marknadal').get('wife').get('husband').get('name').once(
(name) => console.log( 'Marks wife husbands name is '+name)
)
This is quite cool, but how would it translate into a flow ?
Yes - I saw that ... and indeed it may not translate brilliantly... - but hey - if we can get 80% of the effect with just single get and set - and then just chain up nodes... - not as efficient - but.
Or maybe (for now) just accept it - and use it as a cool object store with replication and subscriptions. 2/3 isn't bad.
There can always be version 2.
1 Like
