Exchanging data between flows over VPN

General
21

Yes so what i was saying - i will illustrate with some network numbers

Lets say all your devices that are providing information are on a private subnet - lets call it

172.16.1.0/24 - default gateway to get off this subnet/vlan = 172.16.1.1

You assign your internal MQTT broker and IP address on this subnet of 172.16.1.100

All client devices that wish to pub/sub to MQTT will use this as the address - you would on this setup appropriate IPTables/UFW rules to only allow MQTT in from devices with (say) a range of 172.16.1.200 - 172.16.1.250 on this internal interface

You would have a 2nd interface on your MQTT broker - this would be in a seperate subnet on a seperate VLAN (or physical switch)

Lets say this interface has an IP Address of 172.16.2.100 - no default gateway

You put a client device on this same subnet - 172.16.2.101 and this has a default gateway of the firewall/switch for that VLAN/Subnet - lets call it 172.16.2.1

You would establish a VPN from that firewall through to your external broker in the cloud

Lets say the broker in the cloud is on a subnet with your provider with an IP address of 172.16.100.1

You will setup a VPN between this subnet and the 172.16.2.0 subnet in the office - firewall rules will only allow traffic between the 172.16.100.1 IP address and the 172.16.2.101 - client to the internal broker

You only allow stateful packets across this link initiated from the DMZ client at 172.16.2.101 i.e. the client can sub/pub to the MQTT broker but the broker can not initiate any communication back into the client machine

You then establish a 2nd VPN that will be for External Clients/Tablets etc - this VPN will only be able to talk to the broker at 172.16.100.1 and restrict what they can send - preferably only MQTT traffic.

If you draw this up with some pictures and the IP addresses laid out (i have just done this off the top off my head) it will give you a good starting point.

There are additional pieces that could be added to this along the way depending on how bulletproof you wanted to make the system.

Remember with security it should be like an onion with more and more layers - and i always keep in mind that there are smarter hackers out there than me - so you need as many layers as possible.

Craig

1 Like
22

Yes I understand this isn't a help desk but a discussion forum and I appreciate the ideas. Sometimes one needs to get the ball rolling in the correct direction and get more professional help.

I will be bouncing off variations of these suggestions with a networking professional with full context of the deployment.

I would never bitch about some mis-managed thing I built from forum posts and try to act like it was someone elses fault if it goes hay wire.

I understand it isn't an MQTT issue but ADDING mqtt as per our discussion does complicate network topology according to Craigs recommendations. Adding brokers is adding an attack vector no? I'm still delivering HTTPS NR via the dashboard and wanted to add another layer which in turn adds more 'doors' for people to break into, might I think of it as more ports = more risk in general terms? Additionally as Craig points out if I layer the network properly if they get past the first layer then you just keep putting up road blocks.

" "secure data centres" that turn out to be cupboards that the cleaners have access to and that have no protected power nor filtered aircon." -wow- also not that surprising but know I know what job to get to take down a data center :stuck_out_tongue: I see the same sort of thing at municipal buildings and open ports on managed switches that go into city networks. This project is nothing that dangerous as a far as exposure. Isolated VPS to single device on 4G lan. However I'd like to use it as learning experience either way.

I have it working in the original 'crappy' format after reading more about mqtt listeners, UFW, and port forwarding. (IE over the single VPN) I talked some with my consulting guy today and I sent him Craigs layout and hes taking a look, I don't know how to create vlans on ubuntu without another managed switch but this is fun learning. I'm going to study the layout that Craig has proposed more and think about what that would take to accomplish and test that.

I will continue to work on this and explore your suggestions and what this networking guy recommends as well. Cheers!

2 Likes
23

Good one - keep us updated on your progress and continue to ask questions if you need any help

Craig

1 Like
24

Hey guys, I'm still at this. Ill spare my woe is me rant long story short hes pointing me towards SD wan devices. Circling back here to get opinions. Industrial SD wan devices are expensive AF however it is very appealing to be able to push automated/manual updates all at once from a central location to multiple nodes. Managing the VPN is a headache and a real problem as we don't have an in house networking person. Do you have any thoughts on this? I have gotten assistance with topology and ways to get mqtt routed. I'm now thinking I can't use openvpn and leave it on devices. This is crossing fields into IT mait where it needs to be updating firmware, sw and client devices which is out of my scope. Was wondering what you all do if anyone is managing anything this large. Lets say 20+ customers each with 5-10 clients located throughout the entire continent. (for me just continental US). I don't see how I could possibly manage this.

25

Frankly, if you haven't the capacity to manage a VPN, you certainly should not be looking at SD-WAN unless you buy it as a managed service.

SD-WAN is great for enterprise WAN's and is certainly the future to get us away from expensive private circuits, MPLS and the like. Eventually, it should even lead to the replacement of stupidly expensive proprietary routers and firewalls with commodity compute hardware. I recently (well nearly a year ago now) specified it for our next-generation WAN and LAN service. BUT we are buying a managed service from BT and we still had to settle for expensive CISCO hardware which isn't quite as SD as I would like.

Implementing it without the expertise either in-house or as a service is, in my view, asking for trouble.

Nope, me neither.

For that kind of topology, I would possibly go for a secured cloud service in Azure or AWS or something similar. Then the networking is simpler and the infrastructure a lot less. I suspect this doesn't help you either though as you probably don't have the knowledge to architect that either (not getting at you, no reason why you should have those skills).

Which ever way you look at it, trying to set up a commercial IoT service without the skills to design, build and maintain it is again asking for trouble and I hope your insurance is good.

Sorry to appear negative but that's what I see.

26

"buy it as a managed service." yes, trying to push this off onto someone else haha

Was looking at Cisco and others who offer the service, I would not be managing this. I don't want to worry about the connection side and focus on developing in NR and the electrical systems.

I don't mind the 'negative' outlook I see it as realistic. I'm gonna pull something off with this weather you all think its a terrible idea or not haha. I at least have one professional to wrist slap me over here. I hope that the costs come down as it seems like great technology and if i can do python deploys I can learn it over time. 3 years ago I couldn't read a schematic now I can design controls systems so it will just take time. Thanks for the thoughts. *ups my insurance. :stuck_out_tongue: :stuck_out_tongue: :stuck_out_tongue:

1 Like
27

Buying an SD WAN as a managed service is not that big a thing to do though,

If you look at ZerotIer as an Example it is a managed service you are purchasing (if you go down the paid route) and you can then setup levels of Admin people (so you could for instance have an external IT person who managed the day to day additions etc on it)

I work with quite few clients many of whom are non-native english speakers (french primarily) and i have had no problems with getting them to self install ZT on a range of platforms with some simple email instructions with pictures to enable WFH during this pandemic

Once you have the basic topology/networking setup done there is minimal maintenance required on the ZT side.

Craig

1 Like
28

Craig, thank you for this. Zerotier pricing seems really, really reasonable. Out of curiosity what kind of hardware are you running with this? I'm skimming some of their docs atm. Btw you guys have really helped with my self awareness I WANT to know more about this stuff, I love to learn I just don't want to be pushed off a cliff and lose people money etc. The pressures of controls engineer space are redic what they expect me to do sometimes. Since we last rambled on here I built a CCNA lab (3 switches, 3 routers) that I plan on diving into when I have time. I know I need to be better at the fundamentals if I ever hope to grasp modern sd wan technology and be better versed on management and vpns. For now Ill take the hand hold. Best.

29

We run this off their servers for our clients.

The client systems run on a range of devices from Phones, laptops, tablets and Home PCs (with all 3 major operating systems)

Good work on the CCNA lab !!

Craig

30

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.