Node-red + MCP23017 (x5) slowing down

Can I test with const log2console = false;?
I'm also baffled by the fact that I write and read one pin on every chip (5 chips), for seven hours now at 150ms and performance hasn't dropped. I'm starting to get scared that there is something else wrong with my original flow.
I wil let it run for at least 20hours and then if it bogs down I will change the logging to false and restart
You take your time, for me if logging = false work I don't care avout the checkbox, so you can withour problem take your time to change your conde :slight_smile:

Yes. const log2console should default to false and only be true when logging is required. Re your flow, you are on your own there :wink:

1 Like

Well I think I just wasted your time. I started my original flow and it bogged down again, using btop I noticed that a process is popping up and it is almost unseen, until it runs 3h+. Then you can see it stick around for a second. I took another look at my flow and I remembered that I'm pulling CPU load and memory stats. I disabled them both and the problem is gone. I'm three hour in and blinking is steady. I will leave it until tomorrow.
the nodes in question are

  • node-red-contrib-cpu
  • node-red-contrib-linux-memory
    In reality I can delete as it is some irrelevant info anyway
    I tried it with OrangePi 5, RPi3, RPi4, everytime the same thing

No worries. Let's hope this sorts out your problem.

That being said, this exchange has convinced me that logging should be off by default, with an option to turn it on, so thanks for that. I hope you are right and your system remains stable with those mods. That at least gives me some breathing space :slightly_smiling_face:

I have run in to an issue here that does not allow me to test local modifications (which is why the enable logging checkbox was not visible), because suddenly I get a cannot find i2c-bus module error, whereas I can clearly see it in my system. Funny thing is that installing the current release from palette manager does not have this problem. Anyway, I'll look around on this discourse to see if anyone else has run in to something similar, otherwise I'll start a new thread.

Thank you for such a prompt response and the time and will to adapt the code as needed. Truth be told I didn't even expect you (the author) to respond. A lot of these nodes are a bit outdated (but they work). So thank you for that

I don't understand, what errors?

Have you considered using the log level setting in settings.js to control the log?

So my flow has been running for a day and an hour and no performance dip and memory usage remains constant at 250mb. So it was the CPU and Memory nodes that were boging down the system. I will remove those two nodes and call it a success! So thank you for all the work you did and still doing . Your nodes are very important when dealing with SBCs!!
Oh and I also ran a SSD1306 OLED display and no issues :slight_smile:
Thank you so much!!!

1 Like

I know the feeling. What got me started going down this rabbit hole was the fact that I could not find a node to do what I wanted, nor one that I could easily modify because the code relating to the actual port expander was buried in some compiled c-code and thus not accessible. It wasn't until I ran into @PizzaProgram's repo that I found something I could work with because it included the config node, an inherent part of node-red. I worked with him on and off for about a year or so to bring the node to where it is now. Not being a programmer, many others on this forum helped me along the way.

And thanks to you I can improve it a little further :slightly_smiling_face:

Glad to hear things are running as desired.

2 Likes

Well it seems to me you are a very good programmer. I come from a world of STM32, Arduinos, EPS,.. i know C and C++ (this is all self taught and a hobby) and I saw your code and was amazedā€‹:ok_hand::ok_hand::ok_hand:

1 Like

I'll gladly take the complement, but I can tell you that my kids would disagree! It's a hobby for me too and it is only due to the help of many others on this discourse that I ever got as far as I did.

I want to bounce something off you: The good news is I have "enable logging" working. However, I have a test setup using 5 port expanders and the quick&dirty change I made set logging to whatever the last configuration chip "enable log" is set to. In other words, logging for all configuration chips (port expanders) is either enabled or not. It would seem to me that you want to enable logging per port expander, right? If you know a certain port expander is not responding as expected you would only want to see that port expander's log and not the others. If you do want all port expanders to log their in-and outputs, then you would enable logging per port expander.

Just trying to get my thinking straight here.

Yeah I would agree that logging or debugging should be enabled per configuration. SO like you said if you have a problem with one expander you can just show info on that node, if you want to debug all of them, just enable debugging on all config nodes.

1 Like

Yeah, I realized that I had answered my own question as I was asking it :wink:

It took me a while, but I now have logging working per chip:

  • When first deploying a flow the creation of all port expanders will be logged.
  • If none of the created port-expander configuration nodes has enable log checked, that will be all that gets logged.
  • If for one of the port expanders, the enable log option is checked, then the initial status off all input/output nodes on the node-red design panel connected to that chip will also be logged. As you can see below, 5 port expanders are created and only the initial states for the PCF8575 chip are shown, which was the only chip that had the enable log option checked:
MCP23008 chip initialization OK. BusNumber=1 Address=0X25  id:b4b58b6fa3698704  startAllHigh = false
  MCP23008 chip initialization OK. BusNumber=1 Address=0X20  id:6770c89fcf1b9f0b  startAllHigh = false
  PCF8574 chip initialization OK. BusNumber=3 Address=0X21  id:6625a8bbd4430917  startAllHigh = true
  MCP23017 chip initialization OK. BusNumber=3 Address=0X20  id:70c0610a20ba9fa0  startAllHigh = false
  PCF8575 chip initialization OK. BusNumber=3 Address=0X22  id:b9f353241e35b92f  startAllHigh = true
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=0  ID=a12265c207064734
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=0  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=a12265c207064734
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=1  ID=28eee134299acac0
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=1  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=28eee134299acac0
---
>>> Initializing  PCF8575 Output node >>  invert=true pinNum=2  ID=078f4ae6d6f42246
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=2  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=078f4ae6d6f42246
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=3  ID=81d6a6583b1b8aa3
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=3  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=81d6a6583b1b8aa3
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=4  ID=d84eac653ce115ea
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=4  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=d84eac653ce115ea
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=5  ID=790490f0e8847959
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=5  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=790490f0e8847959
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=6  ID=c13af33978dc2271
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=6  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=c13af33978dc2271
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=7  ID=ab35c44a7a9da555
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=7  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=ab35c44a7a9da555
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=8  ID=16a5b288c0912d88
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=8  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=16a5b288c0912d88
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=9  ID=3c2df2de770964bf
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=9  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=3c2df2de770964bf
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=10  ID=28e2bfcb0473dd75
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=10  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=28e2bfcb0473dd75
---
>>> Initializing  PCF8575 Output node >>  invert=true pinNum=11  ID=875ecaa15b5acb29
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=11  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=875ecaa15b5acb29
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=12  ID=c5d6776e20ab844f
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=12  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=c5d6776e20ab844f
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=13  ID=c734eec0367d9bff
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=13  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=c734eec0367d9bff
---
>>> Initializing  PCF8575 Output node >>  invert=false pinNum=14  ID=23013e5299916417
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=14  isInput=false  pullUp=false  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=23013e5299916417
>>>> DEBOUNCE = 20 ms and _parentChipNode.interval = 00 ms
---
>>> Initializing PCF8575  Input node >>  bitNum=15  pullUp=true  invert=true  id=b143d513a70cb591
    PCF8575 init-Pin started... i2cAddr = 0X22  pinNum=15  isInput=true  pullUp=true  startAllHigh=true  LastState=-2
    PCF8575 Now Setting ALL output pins to HIGH; Addr = 0X22
    PCF8575 First READ OK; allStates=1111111111111111
    PCF8575 Bit-initialization finished. Bus closed.
    ...status update >>  _onOffStatus: 1   globalState= 1  Node_id=b143d513a70cb591
    PCF8575 startChipTimer = 00 ms
    PCF8575  Timer interval is UNDEFINED or 0 ! Timer will not be started, old may be cleared. Exiting Timer.
20 Nov 14:36:24 - [info] Started flows

From that point on, only changes to ports of the PCF8575 are logged. Changes to nodes connected to other chips are not logged. I will do some more testing, but it looks like it is working as desired. A new release with this option will be forthcoming. Thanks for bringing this to my attention.

1 Like

Very nice! You really put alot of effort in to this. Thank you again :slight_smile: I have ran my flow for two days now without issues. So the only thing that was the problem was pooling the linux system every 5s for CPU and memory. I don't need that, that was just some kind of cherry on top :slight_smile: so this was all brought up by accident :smiley:

These are the best kind of accidents :smile:

FYI, version 3.3.0 is now available through the palette manager. This version includes the enable log option, we've been talking about. I love it and I think the lack of logging actually speeds up the node and makes it more efficient.

GL on the home automation project.

I still se some logs. is it intended?

Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=01111111  ipB00=01111111   ipALL =0111111101111111
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=01111100  ipB00=01111111   ipALL =0111111101111100
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=00001111  ipB00=00011000   ipALL =0001100000001111
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=01111111  ipB00=01111111   ipALL =0111111101111111
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=01111100  ipB00=01111111   ipALL =0111111101111100
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=00001111  ipB00=00011000   ipALL =0001100000001111
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=01111111  ipB00=01111111   ipALL =0111111101111111
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=01111100  ipB00=01111111   ipALL =0111111101111100
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=00001111  ipB00=01111000   ipALL =0111100000001111
Nov 20 23:14:15 DietPi node-red[3856]: MCP23017 Read success ipA00=01111111  ipB00=01111111   ipALL =0111111101111111
Nov 20 23:14:16 DietPi node-red[3856]: MCP23017 Read success ipA00=01111100  ipB00=01111111   ipALL =0111111101111100

Looks like you are regularly polling data, is that correct? I only use interrupts, so didn't check that area of the code. Will take a look and get back to you.

Yes I'm pooling for data. I didnt wire up interrups on my board :man_facepalming: every 100ms all 5 chips (maybe not the best way but cant change it now)

I think I've found the problem. Anyway I released version 3.3.1 with the bug fix. Take a look and let me know if this fixes your issue.

Ok give me a few days. I have the Pi at work (free time) and i'm going Monday. I will update and test. But from what I can tell you probably fixed it anyway :slight_smile: