Node-red-node-pi-gpiod Read initial state of pin unreliable

I have frequently used the node module called node-red-node-pi-gpio to read and write to the GPIO bus. However, the input node did not work reliably on my Pi W 2, particularly on Deploys. So I decided to try node-red-node-pi-gpiod (note the 'd' at the end).
This has the advantage of using a service (pigpiod.service) to interface with the GPIO via the pigpio library. This means that the link to the GPIO bus is not affected by Deploys and Node Red restarts.
However, it is important to understand that this GPIO system detects changes in state of the input, not its current value (0 or 1). Therefore the input node does not return anything after start-up until a change in state occurs. Because this delay in reading the input value until it changes might be undesirable in some situations, there is a node configuration setting labelled "Read initial state of pin on deploy/restart?". When checked it is intended to force an initial read of the absolute input condition and send its state out to the connected node(s). Once the initial state is determined, subsequent transitions of state will be transmitted as the occur.
The file that performs this function is called pi-gpiod.js and is found in the .node-red/node_modules/node-red-node-pi-gpiod folder. In this file, at line 20 in the 0.4.0 version, you will find a function called GPioInNode. It interfaces with the pigpiod.service via the js-pigpio library, found in .node-red/node_modules/js-pigpio, and its primary purpose is twofold. One, to set up a callback function to output the current input status whenever the latter changes. And two, to optionally determine and transmit the initial input status, based on the setting of the above-mentioned switch.
The function that performs the latter is inside a SetTimeout Javascript function configured to fire 20mS after it is established. I assume the delay was found necessary to give the GPIO system time to stabilize, as it will execute whenever Node Red is started or reloaded.
Experimentation has indicated that 20mS is too short to give reliable initialization. I do not know what the optimum value is, and it may vary with Pi model, but 1000 (ie: 1s) works well in my situation, and a delay of 1 second before initialization is unimportant to my system.
The line is question is 58, and my lines 52 - 59 now look like this:

if (node.read) {
     setTimeout(function() {
         PiGPIO.read(node.pin, function(err, level) {
         node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
         node.status({fill:"green",shape:"dot",text:level});
        });
    }, 1000);  // was 20mS, now 1S
}

I do not know if there is a race condition between the callback function reporting state changes and the one initial read of the input by the setTimeout function, if a state change should occur just as it’s being read by the latter. But I have not observed any issues. I assume the service will queue multiple reads appropriately.
Of course, any direct edit of the file will be removed if the node is updated by the authors, unless they read this and make the appropriate source change. However, the node has not been updated in over 5 years, so likely is not going to see further releases.
I hope this proves helpful.

Colin

Some additional thoughts … if you are really concerned about a callback while the inputs are being initialized, you can put the callback set-up in its own setTimer function, triggered to fire a few seconds after the first. Of course, it is only necessary to do this if the node has been set to perform an initial input read.

Here’s the code to be replaced:

node.cb = PiGPIO.callback(node.pin, PiGPIO.EITHER_EDGE, function(gpio, level, tick) {
    node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
    node.status({fill:"green",shape:"dot",text:level});
});
if (node.read) {
    setTimeout(function() {
	  PiGPIO.read(node.pin, function(err, level) {
	    node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
	    node.status({fill:"green",shape:"dot",text:level});
	});
    }, 20);
}

Here is the suggested code to use instead:

if (node.read) {	// only need timers if we're doing an initial read
	setTimeout(function() {
		PiGPIO.read(node.pin, function(err, level) {
			node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
			node.status({fill:"green",shape:"dot",text:level});
		});
	}, 1000);	// Increased from 20 to give system time to stabilize
			// Was causing input to not be checked on start, per read input switch

	setTimeout(function() {
		node.cb = PiGPIO.callback(node.pin, PiGPIO.EITHER_EDGE, function(gpio, level, tick) {
		node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
			node.status({fill:"green",shape:"dot",text:level});
		});
	}, 3000);	// trigger after the initialization
}
else {    // No timers needed
	node.cb = PiGPIO.callback(node.pin, PiGPIO.EITHER_EDGE, function(gpio, level, tick) {
		node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
		node.status({fill:"green",shape:"dot",text:level});
	});
}

The 3 second wait to set up the callback is a bit arbitrary. If you have rapidly changing inputs then one might change between the initial read and the callback setup, causing a change to be missed. Make the 3 seconds shorter if this is an issue but not less than or equal to 1 second (1000).
As an aside, I am running Bookworm, and this comes with the pigpiod.service installed but not activated. Earlier Raspian releases do not have this service by default.

I recommend editing the config file with

sudo nanolib/systemd/system/pigpiod.service

as follows, assuming you are only going to access GPIO locally:

ExecStart=/usr/bin/pigpiod -l -n localhost

Save with ctrl/x and Y , then enable auto-start of the service with:

sudo systemctl daemon-reload
sudo systemctl enable pigpiod.service
sudo reboot

I have tested this on my system and it seems to work. YMMV. It is handling reading 5 inputs and driving 2 outputs.

some good digging going on here. A couple of questions if I may.

Is it really that the node itself is not able to read the IO that quickly ? or is the next node not yet ready to receive it and so misses it ? Or has the dashboard not yet connected so it's just the status that is not shown correctly ?

Can you try setting a debug node connected directly t the output and set to send its output to the console (and not just the sidebar) - and see if it logs the initial state OK.

I think I can answer that directly. I put “node.warn”s directly in the code to indicate that the initial input read state code was actually running. These normally output directly to the node-red-log. So this would be entirely independent of any problem with the following node (or NR itself) not being ready, I would think. With the original code and a 20mS setTimer it did not output anything to node-red-log. But setting it to 1sec and it worked. I do not know where the threshold is between those values.

I did put a debug node directly on the output and it reported nothing being sent with 20mS. But it would have gone to the sidebar so is perhaps inconclusive.

It would, of course, be better if the code could detect if NR and/or gpiod and/or js-gpio are ready to receive commands, before attempting to read the input, rather than just timing it, which is always a solution of last resort! Clearly the original author decided a time delay was needed, so must have encountered some issue that it solved at the time.

I will put the original code back in to my setup and add a debug node the way you describe, but I will be very surprised if it outputs anything. If it’s a case of NR not being ready, why would the Debug node not also be affected, even to the console? More later.

Hi again.

I have done as you suggested. With the original code I added a Debug NR node to the output of the furnace monitor input, and directed its output to System Console.

I also added this line:

node.warn("Pin:" + node.pin + " Level: " + level);

to the anonymous function code inside the PiGPIO.read function in setTimer.

if (node.read) {
setTimeout(function() {
PiGPIO.read(node.pin, function(err, level) {
node.warn("Pin:" + node.pin + " Level: " + level);
node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
node.status({fill:"green",shape:"dot",text:level});
});
}, 600);
}

Using a binary search I found that neither debug line sent output to the node-red-log with a time of 599mS or less. 600mS and above the log contained:

29 Nov 14:47:43 - [info] [debug:debug 5]
{
topic: 'pi/37',
payload: 1,
host: 'localhost',
_msgid: 'fbfaa6be5e653068'
}

and

29 Nov 14:59:56 - [warn] [pi-gpiod in:Furnace] Pin:26 Level: 1

These results were completely repeatable.

Of course, 600mS is a threshold and likely dependent upon the Pi model, so I will use 1000mS as it seems completely reliable on my Pi W2.

I hope this answers your question.
Colin

PS. I’m not super happy with my second double setTimer approach. It feels kludgy (as does the first actually; needing timers is never good!). It would be nicer if the code could detect when it’s OK to call PiGPIO.read, and also delay setting up the callback until it is done. Maybe a limited while loop that keeps trying to get the input until it succeeds or times out? But that’s getting into async issues, and is going beyond my self-imposed mandate to make my code work.:wink:

1 Like

Hi,

if you are up for it could you try replacing that read loop with

                        if (node.read) {
                            var loop = setInterval(function() {
                                PiGPIO.read(node.pin, function(err, level) {
                                    node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
                                    node.status({fill:"green",shape:"dot",text:level});
                                    clearInterval(loop)
                                });
                            }, 5);
                        }

To see if that behaves any better for you...
Thanks

Hi,

Curiously enough I have experiments with something very like that. I put some debug in the loop and saw it rotate through several times failing, and then a read succeeding. Much better than a fixed, guessed delay. :grinning_face:

I am generally not happy with the reliability of the edge sensing approach. It seems to miss edges and I don’t know why. The system seems sensitive to multiple development NR restarts, but does seem OK after a clean reboot.

Anyway, I am part way through enhancing the code to optionally use a polling system to read inputs, and it seems to work well. Of course, a change in state won’t be detected until the next poll. I am polling every 10s, which is fine for my purposes and does not make the Pi unacceptably busy (LA is 0.37 as I write.).

So here’s what my enhancement will do when it’s finished. I plan to make the node configuration allow for either edge or level (polled) mode of operation and provide a range or pre-assigned frequencies for the polling method.

EDGE MODE

If initial read is set:

  1. Loop the read, the way you describe at 100mS intervals.
  2. Put a loop counter in to limit how long it will take to get a reading, and create an error message and abandon the process when a predetermined threshold is reached. We don’t want loop lockups.
  3. Put the above loop in a Promise that fires on successful completion of the loop. This makes sure that edge detection does not start until the initial read completes successfully.
  4. The promise.then installs the callback.

If initial read is not set then the callback is just installed.

POLLED MODE

There is no value in implementing the optional initial read feature in this mode so its setting will be disabled/hidden in this mode. The code will always perform an initial read in 1 below, then start polling at the specified intervals on success.

  1. Loop the read, the way you describe at 100mS intervals.
  2. Put a loop counter in to limit how long it will take to get a reading, and create an error message and abandon the process when a predetermined threshold is reached. We don’t want loop lockups.
  3. Put the above loop in a Promise that fires on successful completion of the loop. This makes sure that polling does not start until the initial read completes successfully.
  4. Then a setInterval loop runs for ever to poll the input at a frequency set in the node configuration (probably a range of configuration values from 100mS to 10s, TBD, thoughts?).

Also handle cleanup of the Interval loops on shutdown in either mode.

Do you see any issues with the above?

I see that you are the maintainer of the node, so you may be interested in receiving the end results of my work for evaluating, testing and possible inclusion in a new release. If so, how would that be accomplished?

Colin

Hi,
I must admit I'm sort of scratching my head here - I have tried the tweak I sent on an old Pi ZeroW running Trixie and even when I had 8 or so pins configured I only saw it loop about 4 or 5 times max at 5 mS polling before all pins had reported OK. So I have no idea why you are needing to poll so long unless you have it really heavily loaded with other things going on.

Also are you saying it is missing edges when in normal operation (once started ok) ? I haven't seen that - is there anything special you do to provoke it ?

Yes happy to look a Pull request to the original code (note I did just update it with my fix above and to drag in the original libs to fix a vulnerability in one of the "old" ES5 shims - so you should work off that if possible)

Poll mode timing selection could just be a free numeric text box to allow any value (up to a sensible max).

All understood. I will do a pull.

My loop count is typically 1 with a 100mS repeat. I did not try 5mS. So the earlier numbers I report don’t really tally with that. :face_with_diagonal_mouth:

My W2 NR is pretty loaded and I don’t have a good feel for the inconsistency of the edge mode approach. Once the dust settles I will look at the more closely. It might even be an issue in js-gpio or pigpiod.

It’s going to take me little while to complete the coding, so I’ll keep you posted.

Not a Github expert, but went to

and did not see any indication of any recent changes to it. I have a Github account.

What am I missing?

You're missing nothing...... I forgot to push changes to the repository - all there now.
apologies

No problem at all. I am at the page indicated previously, but the Pull Request menu item looks like it’s going to pull the entire node-red-nodes repository. Something I’d rather not do. Cannot I not just pull the pigpiod directory? I clicked Pull Request but the resulting screen didn’t help clarify much. :roll_eyes:

As I said, no Git expert … I find Git a bit inscrutable.

Colin

Ah indeed it is part of that single repo. Hmm - that is going to be a pain if you don't want to get the whole lot - maybe you can just install the latest node and you'll get all the files anyway - and can work on that. I guess I can still then look at the code and we can work out how to merge later.

Sorry for the delay replying. NP on the GIT, I have downloaded it all, and when done will request a pull.

I may abandon the poll approach in favor of improving the edge trigger, which is better from a CPU load POV. We’ll see.

I think my observation of unreliability of the edge method is because I have one input which sometimes starts off low, and as NR’s flow starts running it goes high. The initiate read sees it low, but before the callback gets installed it goes high and the change gets missed. With a fast changing input there will always be a window for issues like this.

To minimize this I am going to try the following start-up flow.

  1. Loop doing reads until a response is obtained, or a timeout.
  2. If not timed out, throw away the read result and continue. Error out else.
  3. Install the callback and note if it gets called and the state sent, before the next step.
  4. If an initial read is configured, and #3 did not see a callback, do a read and send the result.
  5. Otherwise do nothing if there was a callback, as we already sent the initial state.

Be interesting to see if this helps. Might be a day or two before I can get back to this.

Colin

1 Like

My thoughts

  1. When you start reading too early you don't get a timeout - you get nothing back immediately. (hence my setTimeout loop)
  2. yes that is what my timeout loop does
  3. I install the callback before I start trying to read so it should already be in place. - If a reading comes in (IE it spots an edge) that is goodness.
  4. this is when I actually do start the reading loop until it is ok.

so I'm not sure where the gap is where it can miss the edge - unless setting a callback is asynchronous... and even if it is then I don't actually do the initial read until the first timeout tick happens (currently 5mS) .

Yes I agree with all that. But I think the initial (or any) setInterval loop should have a counter in it to abort the loop if it runs too many times. Otherwise a failure can result in it never exiting. It is exiting this way that I am calling a “timeout”. Not sure what you should about such a timeout. It suggests a failure to communication with gpiod so presumably the callback will never happen either. Maybe put the node into error status?

I will put your code into my working flow and see how it behaves. I agree that if a callback occurs before the first 5mS is up and a read happens, that’s no bad thing.

The approach I suggested might be overkill, but would have used a Promise to wait for the gpiod to start responding, then install the callback, and only do the read if called for by the config AND the callback as not already been called. Although the latter is only likely on fast changing inputs and so what if it reads it again?

I like your scheme as in your latest code, but do feel that an escape from the setInterval would be wise.

I’ll let you know my findings.

Colin

Hi,

We can certainly add a counter to the loop - but agree not sure what to do about it. As you say the main reason would be a failure to communicate with gpiod - but that is already handled by the initial setting up of the pin callback (line 37) - which itself retries the main connection. Only when that is OK do we install the edge detect callback then try to read the pin... so we would never get into this loop if the connection is in error...

you could add a counter like this maybe

if (node.read) {
                            var count = 0;
                            var loop = setInterval(function() {
                                PiGPIO.read(node.pin, function(err, level) {
                                    node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
                                    node.status({fill:"green",shape:"dot",text:level});
                                    clearInterval(loop);
                                });
                                count++;
                                if (count > 20) {
                                    clearInterval(loop);
                                    node.status({fill:"yellow",shape:"ring",text:"No initial value"});
                                }
                            }, 5);
                        }

Hi,

I’ve been programming for over 50 years, if you can believe that, and I’ve found that unterminated infinite loops will always manage to end in a situation I thought completely impossible where they never end. You see plenty of instances of that never-ending spinning “Wait” wheel to know that’s true. So I always put a loop timeout in regardless. Call it insurance. :grinning_face:

Here’s what I had:

if (node.read) {
    var count = 400;
    var loop = setInterval(function() {
        if (--count == 0) {
            node.status({fill:"red",shape:"ring",text:"Timeout "+node.host+":"+node.port});
            clearInterval(loop)
        }
        PiGPIO.read(node.pin, function(err, level) {
            node.send({ topic:"pi/"+node.pio, payload:Number(level), host:node.host });
            node.status({fill:"green",shape:"dot",text:level});
            clearInterval(loop)
            });
        }, 5);
}

Your code, with that one modification, seems to be working well. I do have an electronic issue with one input that has me perplexed, but I do not think it’s software-related. I find that the GPIO inputs on the W2 and not as rugged or sensitive as those on the W.

I’ll follow up.
Colin

What do you mean by sensitive?
Have you got a pullup or down on the input?

On the W a dry contact between an input and 0V with a pull-up resistor activated working totally reliable. Not so on the W2, which was a direct swap into the monitor. I have put an opto-isolator between the contact and the input, which is working much better, but not totally reliable. Due the wire resistance to the contact I may not be pulling quite enough current through the LED to completely switch the transistor, so will be looking at that and either reducing the LED current-limit resistor or put in a transistor before it.

However, I do not think this is a software issue, as many other inputs, also with opto-isolators from the switched source are working fine.

I’ll let you know.
Colin