yer but that's what kafka offers, it remembers the message a consumer has consumed and when the consumer comes back up, it can start receiving messages from there. At least that's Kafka used to do, who knows what it does now (without Zookeeper).
So a consumer acknowledges messages and Kafka takes note of that. The assumption is that a consumer does not acknowledge unless the message has been fully consumed.
I like to be controversial. I once wrote a piece on why it isn't that bad if node red packages aren't maintained. Basically because they live in a stable environment[1], they might not need maintenance. Think of unix tools - when was the last time cat or sed or awk or tee ... were updated? They just work and are completed. And so it is with node-red packages - because they provide very specific functionality. Do one thing and one thing only, and do it well.
Another case in point: what if a node implements a well defined and stable protocol - something like FTP. Is the node broken, bad, or shouldn't be used if it has been updated in 6 days/weeks/months/years? No. If it just implements the FTP protocol, then it probably won't be updated. And actually that's a good thing. And because NR hasn't changed, the node will probably continue to just work.
Those kind of node packages also exist, so it's not the same as a "npm" package were the used-by-date is the last time it was updated. Which is wholly a false assumption: just because something gets constantly updated it must be good. I come from an age where things could just be done and don't need constant updating.
Indeed, no NodeJS required nor included.
And yes there aren't that many libraries out there that make it easy to port nodes to Erlang but that's not really the port of the exercise. It's to see what is required to implement flow based programming (FBP) - remember that's the influence on NR. If you have not heard of FBP, then look it up. Erlang is far better suited to FBP than NodeJS and part of the point of Erlang-Red is to show that.
Second, being able to copy & paste flow code and have it run in either NodeJS or Erlang or both is simply very useful. If not slightly left-field. Thirdly, what are the concepts of Erlang that could be ported back to Node-RED - Supervisors? Behaviours? Processes?
I mean it would be really cool to have a series of X-Reds - Rust-Red, Ruby-Red, etc. Of course, these only make sense if you can also interface with the underlying programming language. That is something that many Erlang folks what to see in Erlang-Red - that you can interact with the existing processes and not only defined nodes. Then Erlang-Red becomes this introspection vehicle into an Erlang system - might be useful, who knows.
Sorry for being a bit of a dick with my comments - I think what you are doing is good but I also like to know why and whether there is another approach.
Then you'd be using TLS over a HTTP connection. Or if you want, encrypt the data. Basically what your saying is what every good mobile app has to do - secure their traffic and prevent replay attacks.
Things like SSL certificate pinning then happen. But much beyond secure protocols via SSL (MQTT supports - atleast EMQX - SSL encryption) there really isn't much that can be done. So you have a secure wire to the worker and with a shared key, that worker can encrypt the data before sending over the secure wire.
[1]=NR is very stable in the sense of the runtime and nodes that ran with version 1 can potentially be executed with version 4 - most certainly the flows can still be loaded.
I read your message this morning and it made me think that it would indeed be really useful to see the cluster state from the admin and view debug messages from all workers.
