Serial-port nodes failing to receive and transmit within the same flow

[123]

I require assistance to debug a problem I've encountered when using the serial port nodes.

My ultimate goal is to create a UPB to MQTT converter (Universal Powerline Bus; a lighting-control protocol). However, it's early days and at the moment I just want to establish serial communications between Node-RED and the UPB PIM (Powerline Interface Module a.k.a. CIM=Computer Interface Module; RS232 to UPB interface device).

Full details about the hardware and software versions being employed will follow my description of the problem. In a nutshell, the flow can either receive serial data or transmit it but it can't alternate between the two without first resetting the serial-in node.

Testing procedure

After confirming the flow receives UPB messages from the PIM (via the serial-in node), sending a UPB message (via the serial-out node) will cause the serial-in node to cease receiving all additional messages. In addition, the transmitted message is not received by the PIM.

To clear it up, I need to reset the serial-in node. The procedure I use is (discovered empirically):

• switching its configuration to use a spare, unused serial port (/dev/ttyUSB0)
• deploying the flow
• switching its configuration back to the desired serial port (/dev/ttyUSB3)
• re-deploying the flow

If, after resetting the serial-in node, I immediately proceed to send a UPB message via the serial-out node, it is successfully transmitted to, and received by, the PIM (and all subsequent transmitted messages). However, this will cause the serial-in node to "go deaf" and fail to receive incoming UPB messages.

To summarize, after resetting the serial-in node, you can either:

• Receive UPB messages (as long as you don't transmit one).
• Transmit UPB messages (but now you won't be able to receive any).

Any insight into the likely cause of this behavior would be greatly appreciated.

Screenshot of the flow

Screenshot of the serial port settings

Software

• Raspbian Stretch (with latest updates)
• Node-RED 1.0.3
• node-red-node-serialport 0.10.0
• nodejs 12.13.1
• npm 6.12.1

Hardware

• Raspberry Pi 3B+
• Digi Edgeport/4 (4-port USB to Serial converter)
• Simply Automated UPB-PIM (RS232 to UPB interface module)

---

EDIT
In case someone may thinking "Maybe this PIM gadget can't 'walk and talk' at the same time!", rest assured it can definitely handle bi-directional communications. There are several UPB software interfaces implemented in python, nodejs, Java, c++, etc and all are able to alternate between sending and receiving data with the UPB-PIM.

[dceejay]

This is indeed unusual. Is the input split character correct ? Maybe (for a test) set that to be a timeout instead.
Under the menu - configuration nodes - check that there is only one serial port configuration node eg like

(to ensure there is only one real connection being made.)

[123]

Thank you for your prompt reply. I have some results that I think you'll find interesting.

@dceejay

check that there is only one serial port configuration node

Node-RED reports serial port dev/ttyUSB3 is in use by 2 nodes:

The other one, dev/ttyUSB0 is not in use. It's sole purpose is to allow me to easily reset the serial nodes using the process I described above (toggling between the ports).

@dceejay

Is the input split character correct ?

Based on the protocol's documentation, it indicates \r (carriage return = hex 0D) is the correct terminating message delimiter. The flow is capable of receiving from the serial port and displaying the received commands via the output node. However, this ceases after I send a (valid UPB) message to serial-out.

Nevertheless, I tried your suggestion and, instead of using \r as a delimiter, I set a timeout value of 200 ms.

Interestingly, it eliminated the issue I described. The flow can now both send and receive UPB messages without fail. In other words, I can send a message without causing the serial-in node to cease reporting incoming messages.

Just for fun, I reconfigured the serial port to deliver binary buffers instead of ascii strings. Here are a few samples of received messages:

Maybe I'm misinterpreting the data but I would've expected a carriage return (\r) to be expressed as 0x0d (a double-byte value). However, what I'm seeing in the last buffer value is 0xd. I don't know if that's the equivalent form or if it's trying to tell me it's only a one-byte nibble value and not a doublefull -byte value?

---

EDIT
Correction. Mixed up my nibbles and bytes ...

[edje11]

I suggest to try it without the Digi Edgeport, it's not clear if this is a switching 4port USB to serial or a that it are 4 coms ports for the Pi. If this is a switching device, then it need commands to switch to one of the 4 ports.

I think that the UPB-PIM will echo the data he receives direct and for some reason this isn't handled correct by the Edgeport or the Pi.
The idea of echoing direct what you receive is that you can do collision detection, if you not receive what you just send there was somebody else also talking on the bus.

Can you show us the content of your Send UPD message function or better the whole flow.

[123]

@edje11

it's not clear if this is a switching 4port USB to serial or a that it are 4 coms ports for the Pi. If this is a switching device, then it need commands to switch to one of the 4 ports.

The Digi Edgeport/4 uses a host's USB port to provide four RS-232 ports. Once connected to the Pi (and loaded with the appropriate driver software) the four serial ports appear as dev/ttyUSB0 to dev/ttyUSB3.

---

@dceejay

I performed two more tests. I changed it back from timeout to using a delimiter. This time, instead of specifying \r, I tried 0xd and then 0x0d.

In both cases, one case the flow worked properly; it received and transmitted messages without any issues. I'm both pleased and perplexed; happy to have found a solution but unclear why using \r caused a problem whereas 0x0d (or 0xd) did not.

---

EDIT
I spoke too soon. I mishandled my test of 0x0d. All takes is one transmitted message and the serial-in node fails to report any additional received messages.

Basically, the results are telling me that \r = 0x0d does not work, but 0xd does. It doesn't explain why \r does work initially, and receives messages, but then fails after just one transmitted message. There's something happening under the hood that is unclear to me.

[dceejay]

totally weird as most of us would assume assume that 0x0d is the byte you need and 0xd is just 4 bits... but... but... but... if it works then err... go with it.

[123]

@dceejay

totally weird

At least I'm not alone in thinking someone unusual is occurring.

Just to confirm, when Node-RED reports a buffer value of 0xd it truly means a 4-bit nibble and it's not some kind of shorthand for 0x0d which is a full 8-bit byte? I want to make sure I fully understand how buffer values can be represented.

---

EDIT

I'm now believe that 0xd is not a nibble value but shorthand for a full byte. Which makes it all the more mysterious as to why 0x0D fails to work.

I'm basing this on the code I use to transmit a UPB message. The third line of code from the bottom appends a 0D to the balance of the command and then the next line converts it all to hex values in a buffer.

var str = '08749609FF2200C4';
var arr = [];
for (var i = 0, l = str.length; i < l; i ++) {
    var hex = Number(str.charCodeAt(i)).toString(16);
    arr.push(hex);
    }
var cmd = '14' + arr.join('') + '0D';
msg.payload = Buffer.from(cmd, 'hex');

return msg;

Here's how the resulting buffer appears in the debug window. Notice how the final byte is reported as 0xd and not 0x0d.

It suggests that 0xd is, in fact, the representation for hex 0d which means \r should work equally well for separating incoming messages (except it doesn't). Truly weird.

Anyway, I haven't found an explanation for the unusual behavior when using \r or 0x0d but did find a workaround (albeit inexplicable) in the form of 0xd. At least now I have a workaround and can proceed to develop my project.

Thanks for your help!

[jmorris644]

Are you sure that the default baud rate if 4800? I have worked with quite a number of PIM modules and they were all 96008N1. The edgeport could be playing tricks between the Pi at 4800 and the PIM at 9600. Causing the erratic behavior?

Joe

[123]

Don't know what to say other than I have one PIM made by PCS and another by Simply Automated (they refer to it as a CIM) and both suggested setting the port to 4800-N-8-1.

Here's an excerpt from PCS's App Note #126 (bottom of the first page; link to document available in an earlier post):

The home automation software I've been using for many years, which has a native UPB driver, defaults to using those parameters when communicating with the PIM.

[123]

@jmorris644

The edgeport could be playing tricks

I thought you might want to know that I have not yet eliminated the possibility that the source of this 'weirdness' is the Edgeport/4. Although the unit ran reliably on a Windows PC, this is the very first time I have ever used it with a Raspberry Pi. It's entirely possible that its Debian software driver isn't perfect.

The reason I say this is because, despite everything that's been discussed so far, I continue to see the original problem I described. So all my 'revelations' about nibbles, bytes, \r, 0xd, timeouts, blabla, etc may be worthless because the Edgeport has been leading me down a blind alley. Maybe. The current level of reliability is poor so I intend to replace the Edgeport/4 with an FTDI-based USB-RS232 cable and see if it makes a difference.

[jmorris644]

I too have used the edgeport very successfully on windows. I have not tried it on Pi. I think trying the USB-RS232 cable is worth the effort. I have used those with zero issues on Pis.

[123]

FWIW, I have experienced one incompatibility with the Edgeport (on a Windows PC). I have an RS232 to RS485 converter and the Edgeport refuses to work with it. I used its configuration utility to tweak a few parameters but no joy. So I bought a USB-RS232 cable with an FTDI chip and, out of the box, it also didn't work with the RS485 converter. However, the cable's configuration utility allows you to tweak a broader range of operating parameters and I eventually found the right combination to make it work. I intend to purchase the same cable for use with the Pi.

---

EDIT
I really want to make this work reliably. It'd be a damn shame if this serial port instability issue derails my entire project. I don't relish having to implement it using a traditional programming language, especially given the progress I've already made using Node-RED. It allows me to focus on the task of converting the UPB protocol as opposed to managing the needs of serial and MQTT communications.

[dceejay]

Indeed - it is most odd. My default test device is a USB GPS that is also 4800 baud but happily splits on \r or 0x0d
(that is PL2303 chip based)