New NR 0.20.5 and --safe mode

General
1

Well, here's one for anyone:

I updated some RPIs (and a Ubuntu NUC).
They worked.

One RPI (Raspberry Pi Model B Rev 2) is not playing the game.

Looking at it, it is 100% load on boot and sits at 100% for a long time.

Logging in with a CLI and:

cd .node-red
node-red-stop
node-red --safe

Works.

The load drops and I can load NR on the browser.

BUT!

Ok, I got a warning saying that the flows are stopped.

I can't deploy anything.

Kinda defeats the purpose of safe mode.

Stuck.

Ok, I tried dragging a new node onto the flow.

That now works.... But I can't edit any existing nodes.

Is this a bug in how safe mode works?

2

You will need to explain what you mean by "I can't deploy anything"/

  1. the deploy button never turns red - I can't deploy anything
  2. After making some changes, I click the deploy button and it appears to work but nothing changes - I can't deploy anything
  3. After making some changes, I click the deploy button and it reports an error - I can't deploy anything
  4. Something else entirely - I can't deploy anything

Have you checked the javascript console for any error messages?
Have you checked the node-red log for any error messages?

3

Ok, sorry. Its been a bit stressful just recently.

I start node-red in safe mode.

It starts. I load the browser page and see the warning.

I click and accept it.

I then try to edit any existing node and it complains that the node isn't deployed.
At this stage the deploy button is GREY!
Which doesn't make sense to me.

I can't edit any existing nodes.

What I did was drag a new node onto the flow.
Then the deploy button went RED.

Hurray!

I clicked it and it is deploying.
And I don't know how long it will take.

"Originally" after the update, it was not allowing me to look at the flows for about 10 minutes.
(Can't say for sure, but it was way longer than usual)

I looked at it with a VNC and saw the processor at 100% and staying there.
I then opened a CLI and stopped NR.

The load went to ...... stuff all.

Ok. Let's start it in safe mode.

So now I loop back to where I started.

I loaded it in safe mode. deploy button GREY.

I couldn't edit any EXISTING nodes. Only by adding a new node could I then get the deploy button to turn RED.

Pressed it and it is now a waiting game to how long it will be before the load is ..... less than 100%.

Should safe mode not have the deploy button as RED when the page loads?

It is 00:31 local time. {Bed} will occur soon.

I might leave it "overnight" and hope it has sorted out its problems in the mean time.

There didn't seem to be any when I started it in either safe mode or normal.
I'll check the log tomorrow sorry. I need sleep.

But that's where I am at with the safe mode.

4

ARGH!

I stopped/started it from a CLI in/on the VNC.

I can't copy/paste to this window!

I can't be sure but the error at the bottom could be because I closed the VNC, but I thought that using VNC got around that kind of problem....

Screenshot%20from%202019-06-18%2000-36-46

From a SSH CLI this is what happens:

pi@TimePi:~ $ cd .node-red
pi@TimePi:~/.node-red $ node-red --safe
18 Jun 00:39:32 - [info] 

Welcome to Node-RED
===================

18 Jun 00:39:32 - [info] Node-RED version: v0.20.5
18 Jun 00:39:32 - [info] Node.js  version: v8.12.0
18 Jun 00:39:32 - [info] Linux 4.14.79+ arm LE
18 Jun 00:39:38 - [info] Loading palette nodes
18 Jun 00:40:11 - [info] Dashboard version 2.15.4 started at /ui
18 Jun 00:40:17 - [info] Settings file  : /home/pi/.node-red/settings.js
18 Jun 00:40:17 - [info] HTTP Static    : /home/pi/.node-red/public
18 Jun 00:40:17 - [info] Context store  : 'default' [module=memory]
18 Jun 00:40:17 - [info] User directory : /home/pi/.node-red
18 Jun 00:40:17 - [warn] Projects disabled : set editorTheme.projects.enabled=true to enable
18 Jun 00:40:17 - [info] Flows file     : /home/pi/.node-red/flows_TimePi.json
18 Jun 00:40:18 - [info] Server now running at http://127.0.0.1:1880/
18 Jun 00:40:33 - [info] *****************************************************************
18 Jun 00:40:33 - [info] Flows stopped in safe mode. Deploy to start.
18 Jun 00:40:33 - [info] *****************************************************************

and pictures to show what I mean.

Shouldn't deploy be red?

Screenshot%20from%202019-06-18%2000-41-45
Screenshot from 2019-06-18 00-41-45.png1360Ă—768 114 KB
Screenshot%20from%202019-06-18%2000-41-48
Screenshot from 2019-06-18 00-41-48.png1360Ă—768 113 KB

5

I added the node and then the deploy button allowed me to deploy.

Screenshot%20from%202019-06-18%2000-43-56
Screenshot from 2019-06-18 00-43-56.png1360Ă—768 131 KB

Now waiting for it to finish.

Update:
Ok, it is "finished" deploying, but nothing is working.

The browser page won't load.

This is pretty much what has happened since the last paste of the log of starting NR.

(Picture as it may say more than I can say)

Note: Processor load @ 100% and it hasn't dipped.

Screenshot%20from%202019-06-18%2000-49-52
Screenshot from 2019-06-18 00-49-52.png1360Ă—768 415 KB

6

Once again - I dread to ask - this is a Pi... how exactly did you upgrade ?... (there is only one correct answer...)

1 Like
7

Yes it is a PI.

Updated with:
sudo npm install -g --unsafe-perm node-red
done from ~/.node-red

This was done with 3 other RPIs and they all work. They updated, I stopped/started NR and they loaded and I could see their NR pages.

And it worked for the Ubuntu machine. (But that's possibly off topic)

This one....... Not playing the game.
(I'm going to the bad place huh?)

8

When you open the editor in safe mode, the only difference to normal is it notifies you are in safe mode.

The deploy button is not enabled at that point in time, because you have not made any changes to deploy.

If your flow are causing 100% CPU usage then there must be a bug in your flows causing that. The reason to put it into safe mode is to let you edit your flows without them running and remove the bug. Starting in safe mode then instantly deploying a random change will just exit safe mode without you having done anything to address the reason you wanted to put it into safe mode to begin with.

9

Nick,

Well, yes and no.

I updated ..... 3 RPIs. (2 x 2B or 3B and 1 X RPZ(W)) They worked with the command stated above for the update to 0.20.5 from 0.20.3

Great. This new method works as to the older one which I used to use.

I log into this particular RPI and do the same.

Before the udpate it worked. Granted it is an old one and it takes a long time to boot.
I get that.
Maybe 3 minutes to get it up and stable.

But I do the update, I see that it worked when NR starts.

But it just "hangs" at 100% load. (Got this via VNC connection and saw it at 100% in the top bar)

Ok, what's going on? The only option was to check if NR was causing it.
(Yeah, don't get too far ahead of me)

So I started it in safe mode. Now I haven't used this often (or much).
But from memory, when it loaded previously...... It puts up the warning about safe mode and the deploy button is/was red.

I could simply there and then press it.

Now, I can't do that. All I can do to get the deploy button to go red is add another node.
Ok, I could have deleted a node also.....
I am sure I tried moving nodes to no avail before this. Nothing. Which is why I posted.
Then shortly after posting I realised I could add a node to activate the deploy button.

Given the flow worked/s and all I did was update NR, I don't get why it is going to 100% load for 10 (or so) minutes.

What I am going to do is call it a night, and tomorrow log in and let it do its thing and see if it will settle down eventually.

10

No. It does not do that. The Deploy button will only be active when you have a change to deploy.

If you are putting it into safe mode in order to open the editor and click deploy, then you may as well not bother with safe mode as you aren't using it for what it was meant for.

I fully acknowledge something is going on with this device to cause the 100% cpu. That is often caused by a flow stuck in a loop somewhere.

If you start node-red in safe mode, is the CPU already at 100%? Or does that only start when you hit deploy?

11

As you said this is an old Pi - you should really use the proper upgrade script as per the Pi upgrade docs... the command you are using may be OK if the version of node.js and npm is compatible - but - as your other issue showed you had one running node 8.11.1 - then the recommended script is safer even though it takes a lot longer.

12

killing/stopping NR the load goes to..... about 14%.

node-red --safe

It goes to 100% for about..... 1 icon width of the load monitor graph screen then goes back to abut 8, 14%

If I then DEPLOY, it goes to (and stays at) 100%

Screenshot%20from%202019-06-18%2007-17-38
Screenshot from 2019-06-18 07-17-38.png1360Ă—768 300 KB
Screenshot%20from%202019-06-18%2007-18-02
Screenshot from 2019-06-18 07-18-02.png1360Ă—768 291 KB

13

Something else I just noticed:

Ok, I'm trying things to see if I can narrow down what is going on.

I started NR on that machine. (ip .0.99)

I went to this machine and loaded/opened a NR page of this machine and injected a signal which once before helped me with times/delays and processor load.

This machine (ip .0.6)

MQTT is connected

Ok, that's a start as the host/broker is on .0.99 So that's a good sign.

I then go to that machine's web page and look.

MQTT connecting!

Ok, how does that work? (Semi rhetorical.)

Oh, and a good thing is the delay which gave me grief is at 5 seconds, not 0.2 seconds. So that's that problem out of the game.

Screenshot%20from%202019-06-18%2007-28-19
Screenshot from 2019-06-18 07-28-19.png1360Ă—768 119 KB
Screenshot%20from%202019-06-18%2007-28-27
Screenshot from 2019-06-18 07-28-27.png1360Ă—768 161 KB

Another update:
So, I started NR from a CLI in a VNC as I did last night.
(What ever)
Suddenly on this machine I couldn't talk to that machine. (NR)

I flip back to the VNC and..... I think what I saw last night is the problem as I think it happened again. It looks very much like the same error.

Screenshot%20from%202019-06-18%2007-37-23
Screenshot from 2019-06-18 07-37-23.png773Ă—516 291 KB

14

Why are you using VNC and not just SSH'ing into the machine? I seem to remember reading a post - not to long ago - that said VNC is a hog one resources.

If you want to get to the nodered of a maching just use a browser and enter IP.of.machine.runningNR:1880.

15

Zenofmud,

Why?

#declarations:
this machine: 192.168.0.6 NUC Ubuntu
RPI 192.168.0.99

Well, all being good, I see the RPI's processor load via NR on this machine.
Yesterday while trying to update NR, the RPI decided to spit the dummy. Though I had updated another 3 RPIs successfully with the same command.

After the update I couldn't get the NR page. So I can't get the processor load.

I can SSH into it and poke about but I don't know the command to see/show the processor load.

I started a VNC to allow me to see the desktop of the RPI. It was/is running at 100% load.
I accept that it takes a couple of minutes after boot before NR becomes available.
It was still at 100% 10 minutes later.
(Though now I see there is a problem I see if using --safe, but I'll get to that)

My home network is a bit hap-hazard just now. Which is annoying for me too, but not a direct cause of the problem. This morning I just spent an hour getting the main modem back up and working.

Anyway:
So I SSH into it and stop NR (on the RPI).
Processor load goes to abut 8%. (See via VNC)

I was hoping that I could log in with VNC, open a CLI and let it get on with its business overnight.
Doing it that way I thought would be better, as if I SSH to it and then turn off THIS machine, the command dies.

VNC allows me to open a CLI and do a command then shut down the VNC..... I may be wrong.

But the Gods were against me last night, and after 18+ hours of being awake, I wasn't too good for that sort of stuff.

I shut it all down and left it for today.

Alas (going back to what I just said) I have since lost an hour (or more) to getting the main internet modem working. The internet modem is going up and down "like a bride's panties on her honeymoon" - I think is the expression.
That is also killing me posting here and having internet (in general) access.

IP.of.machine.runningNR:1880 not working. It just sits there. Page won't load - if node-red is running "normal" mode.
Starting NR in safe mode I see the screen, but that's about it.
All nodes are visible, and all that stuff.

If I modify anything - saying that because I need to change something to then allow me to deploy the flows.

Remember: The flows - as they are - were working before the update.

Pressing deploy it just shoots to 100% processor load and after about.... (a good while) .....
it spits out the error I showed at the bottom of the previous post.

I'm stuck what to do.

16

Ok, 13:02 local time.

Given:

  1. NR used to work.
  2. The flows haven't changed.
  3. All I have done is update to the newer version.
  4. I can still see the flows.

Nothing can be broken.

For the sake of nothing to lose, I tried:
bash <(curl -sL https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered)

Though depricated.

It took a long time, but it completed.

(Good news/bad news)
It didn't really fix the problem.

I had to (many times) start NR in safe mode and try things.

After a few attempts I noticed that the MQTT wasn't connecting. That machine is the broker/host.

So how were the MQTT nodes on this machine connected?

A bit more poking in the dark looking at the MQTT nodes and nothing.

It is now up and working as before but is now 0.20.5

Go figure.

Lesson learnt:
Though I have been told there is only 1 way to update, the other way doesn't "break" it either.
I can't say it would work, but considering I had few choices to re-establish a working version I use the forbidden command to update it and it may (or not - I'll accept that) have gone towards helping getting it working again.

I'm not upset or angry at anyone for saying what ever.
But I have also become wise to this:
The only thing holding you back is the fear of failure.

I hope that is of help to someone.

I don't know what it was that fixed it. So please don't ask.

17

Maybe related, but......

When NR is running I am seeing this in the log a lot:

      pi : TTY=unknown ; PWD=/home/pi ; USER=root ; COMMAND=/sbin/hwclock -r
pam_unix(sudo:session): session opened for user root by (uid=0)
pam_unix(sudo:session): session closed for user root
18 Jun 13:40:51 - [warn] [function:Variable repeat time] Value set
      pi : TTY=unknown ; PWD=/home/pi ; USER=root ; COMMAND=/sbin/hwclock -r
pam_unix(sudo:session): session opened for user root by (uid=0)
pam_unix(sudo:session): session closed for user root
      pi : TTY=unknown ; PWD=/home/pi ; USER=root ; COMMAND=/sbin/hwclock -r

sudo hwclock -r is the command to read the hardware clock.

Why is it giving me this?

It is in a exec node and the time output is used in part of the flow.

Looking at all possibilities to what caused the problem.

18

spawn ping EAGAIN means you are hitting the process limit for the user.

Are you spawning a lot of processes while other spawns are still running ?

You can check by using ulimit -u while running the flows.

1 Like
19

Thanks.

I think I found that part of the problem.

I changed the default poll time to longer allowing it to get on its feet before it starts the regular tasks.

20

htop is what you're looking for