Stream data from C program to node-red dashboard?

a1ex · 18 February 2021 06:19

Say I want to stream the results of calculations from a C program to node-red for real-time visualization.

The C program and node-red run on the same PC so it seems I can establish a connection with virtual TCP/IP ports.

In node-red, I can use the TCP In node and configure it to receive this data. I've done this with the node as a server (listening on) and am comfortable with it.

But what about the C program? I want to say I'll need to create a TCP client object in my C executable, then connect to my node-red server node in order to stream my calculation results.

Is this right? Are there examples anywhere of this?

I'd like to avoid other connection types, and I'd like to keep my program outside of node-red. I'd also like to stick with TCP connections, if possible.

a1ex · 18 February 2021 06:27

It seems my C program will need to do the right (client) side of the schematic below? This would be for Windows.

krambriw · 18 February 2021 06:31

Yes, but most likely not here in this forum, you have to google around. Stackoverflow, Codeproject, tons of possible places.

Myself I would for sure use MQTT instead, you would get a lot of other benefits as well

a1ex · 18 February 2021 06:41

OK, so it seems I can set up an MQTT broker, then stream my .c results to it, and then from my MQTT broker to node-red? I trust the broker-to-node-red connection is straightforward, but is the .c-to-MQTT straightforward also?

krambriw · 18 February 2021 06:48

Depends what you mean with straight forward? If you think C is, then there should not be any problem for you to add MQTT support. But for this, you have to google around. Look for Paho MQTT C Client

Like here maybe: GitHub - eclipse/paho.mqtt.c: An Eclipse Paho C client library for MQTT for Windows, Linux and MacOS. API documentation: https://eclipse.github.io/paho.mqtt.c/

a1ex · 18 February 2021 18:01

Totally lost trying to add the Paho library to my mingw compiler. Why are there no instructions ever on Github? I mean, what do people do with that bunch of files included in a package? Does everyone just know how to go from there? How are newbies supposed to figure it out there is zero info on how to install a Github package? Oy.

Steve-Mcl · 18 February 2021 18:17

@a1ex what is this c application to do that node-red cannot? Is this an academic thing or do you have a real need for this?

a1ex · 18 February 2021 18:22

I'll be running a simulation in C. I have no choice. My data has to come from C. So to visualize that simulation data in node-red I have to communicate it from C. Eventually the simulation will be on a separate machine, so I will need to establish a remote connection, so that will be coming up. So for now I'm just testing whether I can in fact stream data from C to node-red. If I cannot, then node-red is out, because I have no control over the simulation side of things.

a1ex · 18 February 2021 18:29

Actually forget mqtt. This is clearly meant for developers who already know what they're doing, and there is zero practical help for newbies like me.

So I'm sticking with TCPIP, which is well established and better documented. So now my challenge is reduced to streaming creating a TCP client object in C to stream data to node-red. I guess this is all I'd like to do at this point: to create a "hello world" program in C that streams "hello world" to node-red. That's all. Any practical examples that newbies could follow for this?

Please don't just suggest I google things, because trust I've googled things, and please don't just point me to Github because there is zero practical info on how to actually use what's on Github.

Steve-Mcl · 18 February 2021 18:28

Did you check out the samples

a1ex · 18 February 2021 18:32

Thanks, Steve, but Github... I don't know what to make of it... What are the files, how do I install them on my machine... Am I supposed to just download and put them in some folder---what? Because nothing is said on Github. Either you know it or you don't. I can't waste my time with assumptions that are undocumented. Why don't they bother to write just two sentences on how to use the packages shared on Github? I mean, come on. It's like giving someone a car but leaving the keys two towns over and expecting the person to drive off the lot no problem.

Steve-Mcl · 18 February 2021 18:40

Adding a c lib to a project is something you learn if you are a c programmer so it is not surprising there are no hand holding instructions. (I mean you are looking to do native TCPIP comms so i imagine you are a C programmer?)

I personally avoid C like the plague, hated it when I had to use it and find every excuse to avoid it.

That said, adding PAHO is no different to adding any other lib. You include the .h file, write the code (as per examples) compile your objects / binaries.

Writing your own TCP comms channel, I am afraid you will face many issues (like handling disconnections / re-connections etc) that MQTT handles automagically. If I can sway you, put your efforts into getting MQTT up and running - you will benefit from the experience of the great minds that developed MQTT and the years of issues that PAHO lib has seen and fixed.

davidz · 18 February 2021 18:45

Is your simulation result a data file or live data?

Both TCP and MQTT work wonderfully for your case. Yes Node RED can visualize the data nicely too.

If you feed live data, then I will recommend MQTT for its simplicity and robustness. If you just send files, then I may prefer TCP.

dceejay · 18 February 2021 18:50

there is a simple C tcp sender here - 64 lines of fairly commented code so should be possible to take apart ok

gist.github.com

https://gist.github.com/suyash/2488ff6996c98a8ee3a84fe3198a6f85

client.c

#include <arpa/inet.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>

int main() {
	const char* server_name = "localhost";
	const int server_port = 8877;

This file has been truncated. show original

server.c

#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

/**
 * TCP Uses 2 types of sockets, the connection socket and the listen socket.
 * The Goal is to separate the connection phase from the data exchange phase.

This file has been truncated. show original

a1ex · 18 February 2021 18:50

Not a developer. Just an engineer. I'm simulating a machine and the simulation environment is slow. It's common practice in the field to deploy the simulation as C code, which runs at a more respectable rate. So I'm trying it.

And I would like to have real-time gauges like those I see in node-red. I've gotten those to work with TCPIP connections.

And right now I want to do just one thing: to stream "hello world" from C to node-red using either TCPIP or MQTT.

But I'm not a developer. I've gone through multiple instructions and installed multiple programs and it's become clear that I'm going on tangents to tangents to tangents... getting no closer to streaming from C to node-red, and in fact risking running out of time before I accomplish a thing with node-red.

I just want to say hello from C to node-red. Literally.

a1ex · 18 February 2021 18:57

Thanks, Steve. I know you mean well, but the challenge here is the same as before. I don't have all the libraries/header files included in the script and therefore it doesn't compile. I don't know how to install the libraries, which is what kept me from using the Paho QTT library. If I can't install, I can't include, I can't compile, and I can't run...

drmibell · 18 February 2021 19:03

This might not work in your case, but the first thing I would try is to launch the C program using an exec node. Then have the C program send its output to stdout, and get the result in NR from the standard output of the exec node. As long as everything is on one host, you shouldn't need mqtt or tcp.

a1ex · 18 February 2021 19:03

It probably matters that I'm on Windows, not on Linux or Unix.

davidz · 18 February 2021 19:07

Windows will work well too. Though Linux environment is better
Your TCP framework is correct. You just need to sit down and write the C code for the communication part. The reward is there.

a1ex · 18 February 2021 19:30

Thanks, David. I know the reward is in doing it yourself.

The problem is the info on how to do it is swiss cheese. Seems everyone has a blog or stackoverflow post or other web page outlining how to do this... but there is always something missing.

If I knew how to do this, I would just do it and feel like a champ for doing it. But I don't know how to do it, so I depend on info being available and complete to do it.

I'm not a psychic and at this stage my guess as to why things don't work is as good as no guess.