Node-red is a fantastic tool in IoT programming, however, in order to make the IoT more intelligent, it has to process data and do some data science tasks. While Node-red is not good at handling huge size of data and analyze, I am using KNIME to do this.
However, how to put Node-red and KNIME seamlessly and work together is a difficult thing, any idea? or any alternative of KNIME that can work better with Node-red in data processing?
Looks like it uses its own Java-based server.
So you will need a way to communicate with it. What communications channels does it support? HTML REST API? TCP? MQTT? ...
Well, so you can connect to some databases, read/write files and make calls to REST API's but it doesn't have a listening API feature that I've managed to see in my quick review. No MQTT either that I could see.
So I think the best approach is probably to use a shared db or file where you use Node-RED to output to the db or file and use KNIME to analyse the data. Or, depending on the use-case, use KNIME to analyse and summarise bulk data and write it to a db or file then use that within Node-RED.
KNIME is clearly designed as a stand-alone tool and doesn't appear to support seamless integration in any way that I can tell.
KNIME is also a typical Java beast of an application with embedded Eclipse and Chromium - it starts up really slowly even on my i7 16GB Windows 10 workstation. So you will need some very significant compute capability to run it on.
There are other ways to handle very large data sets but it depends what type of analysis you want to do along the way.
Yes, this is what I am thinking for Node-Red to generate a shared db or file to KNIME. But how to activate KNIME to run the analysis when new data is appended to db or file? I don't think there is a rbe node in KNIME that can activate the process when something in db/file changed.
I don't know KNIME but I suspect that you will have to poll the db. Perhaps it has a forum you can ask on? Let us know.
No luck so far. I have been looking around for how KNIME or Google Colab can work with Node-Red in any situation, or how to improve "AI" of Node-Red except AWS, Watson API, etc so that easier to customize algorithm.
I have been struggling with the same problem for a long time, it seems Node-Red and KNIME are two different worlds.
The only way I can think of is KNIME restful API. However, it needs KNIME server (paid version) to run and I am using KNIME free version.
Another possible thinking is to install KNIME free version on cloud, say AWS EC2, but I am not sure which KNIME node can trigger KNIME to run the analysis when Node-Red updates the database. It should be something like "watch" node in Node-Red.