MQTT SSL/TLS connection

Hi All
I'm trying to configure an MQTT in to use SSL/TLS security.

I have created
ca, client, and server crt files
ca, client, and server key files.

We are using a Mosqitto MQTT broker, in which I have changed the conf file to use the above files, and restarted it (service mode)

In the Node Red MQTT in I have configured the tls-config to use
client.crt, client.key, and ca.crt files, and given the passphrase that was provided when creating the files. Port is set to 8883

The node transitions from not connected (red circle) to connecting (yellow circle), but doesn't connect.

Without TLS, we connect instantly.

I've either configured something wrong, or I've missed something.
I'm new to TLS...

Any ideas from the info provided?

Hello!

I am looking at this very setup in front of me, I have Node-RED talking to mosquitto and am currently bashing my head against the Paho Java Client!

I'd suggest we break the debugging into two stages:

  1. Server (mosquitto)
  2. Client (Node-RED)

Are you able to paste your mosquitto.conf here (removing any private data of course)?

If you're on a unix/linux environment, could you try issuing:
openssl s_client -connect <server>:8883 -prexit
and see what it comes out with, it should print the servers certificate and a negotiated TLS protocol, if that fails you definitely have a config problem in your server.

If you have client auth enabled in mosquitto, then you'll also need to specify the cert to openssl, but I'm going to assume you're using only server-auth for now.

Hi Andreas
Conf file attached as a txt file, couldn't upload a *.conf file
mosquitto.txt (38.2 KB)

I haven't edited that much.
Port for Extra Listeners
cafile, certfile and keyfile in
Certificate based SSL/TLS support
section below Extra Listeners.

I'm on a Win7 PC, but may need to duplicate on a Win 10 PC in the future.

Thanks for your time

Chris

Well the mosquitto.conf looks good, onwards!

How did you generate the certs and keys? Can you open the files and see a first line of
-----BEGIN RSA PRIVATE KEY-----
or
-----BEGIN CERTIFICATE-----
if not they're not in PEM format and need to be converted.

Did you specify a password for the broker key? I don't think mosquitto supports password enabled keys so you'd need to strip that out (easy if you have openssl: openssl rsa -in server.key -out server-nopass.key and then change the keyfile to be server-nopass.key in the conf.)

cheers,
Andreas

I should probably ask for completeness:

  • You did check the Node-RED and mosquitto logs for errors didn't you‽

***** EDIT ***** (since I'm a new discourse user I can't reply any more)
Add

log_type all
log_dest file C:\mosquitto\mosquitto.log

to your mosquitto config to enable logging, then restart

Hi Andreas

certs and keys generated by following

steps executed in Windows PowerShell, if that's relevant.

All files have a BEGIN **** line

As for checking logs, I'm afraid I haven't.
I'll have a look.

The mosquitto/log directory is empty
and .node-red doesn't have a log directory

I may need to configure them to produce log files?

ceejay has bumped my user level :slight_smile: I can reply again!

Morning Andreas
I've configured the logging for Mosquitto.
An initial log file attached.
mosquittoI.txt (19.5 KB)

Looking at it, it looks like I may need to focus on my 'client' device, a BLE Gateway from Minew.

Thanks for your help so far, much appreciated

Chris

I have seen this (likely) exact issue on the forum about a month ago. Went through a lot of things but never managed to solve it. Let me link it.

While it says solved, the original poster never got the answer to their problem.

Since you are on windows, should you not use double-backslashes in the paths for certificates etc?

forward slashes work pretty much everywhere in Windows and have done for years.

Morning Chris,

The Gateway looks fine (using unsecured port 1883), but the logging is being particularly helpful in saying

1570002289: New connection from 127.0.0.1 on port 8883.
1570002289: Socket error on client <unknown>, disconnecting.

when N-R connects :cry:

but since you used openssl to generate your certificates, you can use openssl to test mosquitto, can you issue:

openssl s_client -connect localhost:8883 -CAfile  C:\mosquitto\certs\ca.crt -prexit

please?

It'll attempt to create a TLS connection to mosquitto (it'll fail to create an mqtt connection, but we don't mind that), then it'll print out what it achieved.
e.g.:

CONNECTED(00000005)
depth=1 <CA Subject>
verify return:1
depth=0 <endpoint subject>
verify return:1
139641003102656:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1528:SSL alert number 40
---
Certificate chain
 0 s:<endpoint subject>
   i:<CA subject>
 1 s:<CA subject>
   i:<CA subject>
---
Server certificate
-----BEGIN CERTIFICATE-----
<PEM Cert>
-----END CERTIFICATE-----
subject=<endpoint subject>

issuer=<CA subject>

---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:gost2012_512+md_gost12_512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:gost2012_256+md_gost12_256:gost2001+md_gost94:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
---
SSL handshake has read 2403 bytes and written 646 bytes
Verification: OK
---
New, TLSv1.2, Cipher is AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-GCM-SHA384
    Session-ID:
    Session-ID-ctx:
    Master-Key: 1395B4B2EC84BEAF0EFEFE52F787C507F8625296C06D587A328B1A7A1BC1718C84A338A8082235DA0441E7E859114CC8
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1570007277
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

It'll print it twice since we passed "-prexit" (print status on exit), but without the -prexit it won't terminate the connection itself (avoiding the hardship of typing C-d)

The main thing to note is that it established an SSL-Session using a recognised protocol (in my case, Protocol : TLSv1.2). If openssl fails to do that then mosquitto is misconfigured. If it succeeds, the Node-RED is misconfigured and we'll move on!

cheers,
Andreas

OK, I just saw that the config file attached had single backslashes in the paths and thought that could cause a problem, I might be wrong though

Single backslashes seem to be the norm in the other windows-examples I've seen online, but if the openssl test fails, it's probably worth a try...

Use forward slashes in commands, it saves a lot of hassle. If you do use backslashes on the command line, you have to double them up and put the whole thing in quotes. That is, of course, because a backslash is an escape character for all command line terminals.

There are a few older Windows applications that can't cope with forward slashes but all command-line apps should support them.

Andreas
Here's the result from the openssl command requested

connection.txt (13.6 KB)

Chris

Lovely, your mosquitto connection is working just fine.

On to Node-RED!

Let's get some logs!

If you're using pm2 to run node-red, then simply do a pm2 logs command to retrieve the logs.
Otherwise if you're following https://nodered.org/docs/getting-started/windows to run on windows, you can see the suggestion:
If you want to access to the logs when running this way, you should amend the node-red.cmd file to redirect std and error outputs to a file (creating an alternative startup file would be better so that it isn’t overwritten on updates).
Thankfully windows has caught up with the rest of the sane world, so that redirect is simple. Add > C:\node-red.log 2>&1 to the end of the two red.js lines in node-red.cmd and restart node-red. Or just kill the started task and run node-red manually in a terminal.

I'd suggest disabling the Use legacy MQTT 3.1 support in the MQTT config as it might be holding you back (i don't think it's getting that far, but best to be safe)

In your TLS configuration, are you:

  1. Using Use key and certificates from local files? (I am)
  2. Specifying a passphrase (I'm not)
  3. Using Verify server certificate (I'm being lazy and am not)

If the answer to 1 is no, then try to enable it and specify the paths to the certificate files directly (since you're running on the server itself the certificates are already on there..)

If the answer to 2. is yes, double-check it's the right one:

openssl rsa -in C:\path\to\node-red.key -noout -text -passin <passphrase>

If you've enabled 3 then disable it for now to see if that's the bit that's failing.

fingers crossed we'll get there!
cheers,
Andreas

Hi Andreas
Made the changes to node-red.cmd - still no log files though.
Disabled legacy MQTT 3.1 support

TLS Config

  1. Wasn't using local files, changed it so now we are
  2. Specifying a passphrase - as provided one when generating certs
  3. Disabled Verify server certs

Updated, and deployed.

All connected, instantly.

I do believe you have cracked it.

Thank You so much for all your time and efforts
Chris

1 Like

Yay!

I'll have an experiment with those settings here, see if there's a bug somewhere!

cheers,
Andreas

FYI

Just been through the process of adding an extra listener to the mosquitto.conf at port 8884, and we're back to connecting (yellow circle)

Remove the extra listener, and we connect ok again

Thanks

Chris