Have I been hacked?

General
41

Same issue here. Tuesday I've found new nodes in my main flow which were suppose to download a file from 185.216.71.153/bins/bins.sh . I've completely reinstalled RPI (used even new SD Card - not that it matters) I've changed my Password to the system and removed all SSH keys. It worked till now - Sunday. Few minutes ago I've received a request to merge changes in my flow and the nodes to the same IP address appeared + another one with command curl https://iplogger.com/2zRmf4 . The changes has been made at 15:37. I've want to say my instance is not exposed to the internet directly. I am using Remote.it and logs do not show any access to the instance today, also since reinstall I did not even assign static address to the instance. My NodeRED is running in docker container and I expose volume with config files through Samba. Also I am using ssh currently with password only, previously with a key. Apparently malicious script failed in execution so I am wondering, is my PC actually infected? Defender is negative about it, but probably will try some 3rd party AV.

Edit: One important note infected files are flow.json and flow.json.backup

42

@TotallyInformation, @knolleary, @Steve-Mcl
This is potentially a 3rd case in 7 days!

Itā€™s hard to say if this is all convenient and/or related, but maybe a sticky warning about the sudden influx to warn users?

The last report above is of interest to me.
Would it be an idea to gather the installed nodes that all 3 cases have installed?

1 Like
43

Hopefully it's complete. I've found it in my clipboard. However I've already removed these nodes and blocked IP address in RPI's firewall.

Admin edit - dangerous flow removed

44

I mean the nodes you have installed via the palette menu, not the flow.
Letā€™s wait for senior members to comment, before listing your installed nodes.

1 Like
45

I have removed that flow.

Is it not the case that in all cases the machine has, at some point, been opened to the internet?

1 Like
46

Extremely possible of course, just odd that this week seems to be active with the same threat found in flows.

I think colors is still fresh in my mind :roll_eyes:

47

To be honest, this looks like an opportunistic hack. As with previous examples, I think that someone came across Node-RED, realised there are a lot of people with minimal experience sharing things over the Internet, doing a search for standard ports and using some simple flows to trigger malware.

Possibly only a single person given the number of occurrences we are seeing. Probably someone on the forum.

I've already suggested a pinned note - I don't believe I have the admin level needed to do it myself though.

1 Like
48

I have seen at least three "How to attack Node-red" articles on the internet. So that's three authors plus anyone reading the pages, and their bots, who could be active.
It seems reasonable to assume that automatic scanners commonly include port 1880. I guess that entering and deploying the malicious flow has to be done by hand.

Given that Raspberry Pies are ideal for Node-red, designed for inexperienced users and by default have passwordless sudo, I'm not convinced that the "Do not expose your Node-red to the internet" is a strong enough warning.

49

I have not had a Node-red install hacked as yet but my Win10 laptop was hacked a few months ago. The trojan was a new one, Google found it first and protected my account, but the trojan was still active but no virus scanner would see it, 3 days later Windows Defender removed it, for those three days the laptop was isolated and only connected just to update virus profiles for the next scan. So even running virus scanners is not 100% proof against being hacked. So now I have set my router DNS to Quad 9 DNS servers, this blacklists hacker domains so even if you do get hacked their Trojan cannot connect and it also blacklists domains where the website contains Trojans. Current hit rate is 95% plus if the direct IP address is in the code Quad 9 will not block. However better than nothing but still not 100% but combined with good virus scanners and ports being blocked it should be close.

1 Like
50

Hi @kommando828.

This is only about malware which exploits Node-red systems insecurely exposed to the internet, eg by port forwarding port 1880.

A DNS service can offer some protection yes. But some of the malware we have seen posted reaches out to servers using IP address not domain name, so the DNS service would not be involved.

1 Like
51

Nothing is 100% safe, it isn't possible. But risks can be reduced to acceptable levels.

The use of quad 9 or similar 3rd-party DNS is a good idea - as long as your router isn't hacked :wink: - but if you can, set your router to transparently either reject or redirect DNS queries from the network to the 1 capability.

And of course, to protect from the worst situation, make sure that every useful piece of information on your devices is backed up to immutable storage or at least to storage that blocks large-scale rapid change (such as OneDrive). And have multiple backups for anything remotely important.

As long as you are careful with email and other links, the biggest threat on end-user devices is ADVERTISING. It is still poorly controlled by the advertising networks and remains both a major privacy issue and a major source of malware. So do yourself a favour and just block it all. Either at the network level using PiHole or similar or at the device level with AdGuard or similar. Or maybe both!

52

Quite right.

All of the hacks we've ever seen on Node-RED are very simply avoided.

DO NOT EXPOSE AN UNSECURED SERVICE TO THE INTERNET.

That's it, simple and effective. There are plenty of ways to have something accessible but not directly exposed so please make use of them and save yourself a lot of issues.

1 Like
53

Hi
I have been on the Cloudfare site and with my limited networking knowledge it is bewildering. I already have a domain name hosted by "Hostinger" but I don't think they provide any services like this. I have 2 locations that I need full access to from my phone. I've seen folk commenting on "Tailscale" and wonder would that be easier for me to embrace.
John

54

OK, can't deny that CF ZT can be confusing. Took me a while to get my head around it. In case it helps, I've dumped my setup notes below.

Tailscale, I think, is more a straight VPN service so you need a VPN client - I've not tried it myself though so I might be wrong.

CF ZT is several things: a web proxy service, SASE and zero-trust network. So yes, more complex, but it avoids the traditional VPN issues.

NGROK is the easiest option - as long as you follow my guide to getting a secure connection. It is also the most limited but it is enough to work with for short periods.

Getting ready

Get started Ā· Cloudflare Zero Trust docs

  1. Sign up for Cloudflare.
  2. Create a new domain or transfer control of an existing domain to CF.
    CF must know about your endpoint before ZT will work. It must also be managing your domain's DNS. You don't have to transfer the domain registration to CF only make the CF DNS servers the managing ones. But CF is almost always cheaper and better anyway so best to move the registration.
  3. Sign up for CF Zero Trust.

Configuration and monitoring is done from the Cloudflare Zero Trust portal. Specifically the "Access" section.

Create an access group

There are several ways to include ID's into access groups. By email for example or GitHub/Google OAuth. Access groups define who can connect to services and how.

Assign 1 or more identity providers to the group.

Create a tunnel

Cloudflare Tunnel Ā· Cloudflare Zero Trust docs

  1. Install cloudflared
    You can do this by going to the ZT portal, Access > Tunnel, create a tunnel. Name it and save, instructions are given on installing.
    You need to create a tunnel anyway - however, installation is better managed via the cloudflared repository.
    Local config is in ~/.cloudflared/config.yml but probably best to use remote (CF ZT portal) config.
  2. Configure the tunnel from the instructions given as you create the tunnel in the portal. If installing cloudflared from the repository, you only need the sudo cloudflared service install part.
  3. Part of setting up a tunnel is adding either/both public hostnames and private networks. When setting up a public hostname, create an application as well so that you can give/protect access to that endpoint.
    You can add multiple of these.

You should only create 1 tunnel for each network.

In the tunnel, you define hostnames which are publicly accessible (sub)domains with optional URL paths.

Each hostname connects to a "service". The service URN is relative to the server running cloudflared. e.g. ssh://localhost:22 or http://localhost:1880.

Create your applications

Node-RED

Create a Public Hostname on the tunnel. The "service" is the http(s) connection on localhost, e.g. https://localhost:1880. It cannot have a path.

To give/protect access to public hostnames,

  • Create a Self-hosted application.
  • The application domain should be the same as the tunnel's public hostname.
  • Make sure to assign 1 or more identity providers.
  • Add an access policy

SSH

Do not turn on "Protect with Access"

Create a Public Hostname on the tunnel. The "service" ssh://localhost:22.

Create a matching Application. Turn on the option to create a browser SSH terminal. Access via browser.

Tunnelling a specific web path

If you want to create a different login or other security for a specific Node-RED path (e.g. /ui for Dashboard), you will need to use a local proxy such as NGINX. Create an entry that turns the location (e.g. node-red path) into a new IP port.

How to Forward Request to Another Port in NGINX - Fedingo

Errors

Don't forget to check the cloudflared logs using sudo journalctl -u cloudflared.service

Already installed

home@home:~$ sudo cloudflared service install ......... 
2023-10-23T17:27:08Z INF Using Systemd  
2023-10-23T17:27:08Z ERR error generating service template error="cloudflared service is already installed at /etc/systemd/system/cloudflared.service; if you are running a cloudflared tunnel, you can point it to multiple origins, avoiding the need to run more than one cloudflared service in the same machine; otherwise if you are really sure, you can do `cloudflared service uninstall` to clean up the existing service and then try again this command"  
cloudflared service is already installed at /etc/systemd/system/cloudflared.service; if you are running a cloudflared tunnel, you can point it to multiple origins, avoiding the need to run more than one cloudflared service in the same machine; otherwise if you are really sure, you can do `cloudflared service uninstall` to clean up the existing service and then try again this command  
home@home:~$ sudo systemctl stop cloudflared.service  
home@home:~$ sudo systemctl disable cloudflared.service  
home@home:~$ sudo rm /etc/systemd/system/cloudflared*  
home@home:~$ sudo cloudflared service install .... 
2023-10-23T18:16:41Z INF Using Systemd  
2023-10-23T18:16:42Z INF Linux service for cloudflared installed successfully  
home@home:~$ sudo systemctl enable cloudflared.service  
home@home:~$ sudo systemctl start cloudflared.service  
home@home:~$

You have to stop and disable the service

ICMP - ping_group_range

How to allow unprivileged users to use ICMP (ping) - :woman_genie: Knowledge Base - OpenNMS Community

2023-10-23T18:27:56Z WRN The user running cloudflared process has a GID (group ID) that is not within ping_group_range. You might need to add that user to a group within that range, or instead update the range  
to encompass a group the user is already in by modifying /proc/sys/net/ipv4/ping_group_range. Otherwise cloudflared will not be able to ping this network error="Group ID 1000 is not between ping group 1 to 0"  
2023-10-23T18:27:56Z WRN ICMP proxy feature is disabled error="cannot create ICMPv4 proxy: Group ID 1000 is not between ping group 1 to 0 nor ICMPv6 proxy: socket: permission denied"

This command adds the user to the ping group (default is 1 0 which means that no user can do ICMP pings):

sysctl net.ipv4.ping_group_range='10000 10000'

To make permanent:

sudo nano /etc/sysctl.d/99-allow-ping.conf

Add the following line and save the file:

net.ipv4.ping_group_range=10000 10000

Load the settings without a reboot using:

sudo sysctl -p /etc/sysctl.d/99-allow-ping.conf

Probably sensible to set up a suitable group and give access to that group.

Bad gateway

  1. Check if Node-RED is configured to use https
    If so, check if this is a local cert - if so, turn off cert checking in the tunnel public hostname config.
  2. Check if you have moved the default root paths
    If so, make sure you are using the correct path in the url.
1 Like
55

Good gracious!! I wasn't expecting anything like this. Most of this is gobbly gook to me but I will go through it slowly. Thanks a lot.

1 Like
56

It was just my setup notes so there is every chance that it IS gobbly gook! :grinning:

57

It wouldn't hurt to change the default settings when installing NR, in my opinion. Perhaps even alert the user when updating Node-RED and no password/HTTPS settings are detected.

58

What in particular?

59

This should change:
By default, the Node-RED editor is not secured - anyone who can access its IP address can access the editor and deploy changes. This is only suitable if you are running on a trusted network.

I'm not an expert in this field and perhaps shouldn't comment. However, from a user stand-point, securing things has always been complicated. Meaning, maturity doesn't really care or has the time to care (and expect things to be secure by design). Asking the user to type a password to secure the editor should have been implemented long time ago, in my opinion - there are still risks of course. Also some brute force detection would be nice (maybe it's there already?).

A bigger problem which will arise probably soon is that people will jump to the conclusion that NR is not safe.

1 Like
60

I agree, some of the people for whom Node-red provides an easy introduction to programming the internet of things won't have the faintest idea what running on a trusted network means.

It's natural once you can turn on your TV from the kitchen to see if you can do it from the pub. How? Port forwarding obviously.

Surely there is a responsibility to make the product as safe as possible for these users?

Maybe it's all been considered and discussed, but this is the third (?) wave of people getting attacked because they made a mistake with securing Node-red.

Something must be done or risk the widespread opinion Node-red = unsafe.

3 Likes