MQTT IN payload type detection inconsistency

General
1
  1. MQTT IN payloads are now parsing as primitive types bool at least, assuming numerics and arrays would too but haven't tested) rather than as strings. Assuming this is due to the new "autodetect JSON payloads" it seems to do? Intended? Meaning, I'm getting true/false instead of "true"/"false", etc.
  2. EXISTING MQTT IN nodes AREN'T doing this, only the ones that are newly created are doing this, introducing an inconsistency.

Do I need to go through and change all my MQTT nodes to explicitly parse to string, or is this a bug? I couldn't find anyone else mentioning it in searches.

Sounds eerily similar to an issue I had on another project a long time ago where I assumed a JSON.parse() only parsed for actual objects, and didn't realize it also happily parsed primitives?

2

Whenever we introduce a change in behaviour in nodes, we try to minimise the impact to existing users and their flows.

This is why, when we introduced the new behaviour in 3.0 it only applies to newly added nodes. Existing nodes behave as they did before.

It is up to you whether to adopt the behaviour across your existing flows (with any necessary changes in logic due to the change in output), or configure new nodes to behave like your existing nodes.

3

So it IS the intent to parse not just JSON OBJECTS but primitive JS types from strings now? Did I miss that announcement then, as I didn't see it in the 3.0 changelogs?

To be clear here, its not parsing "JSON object, binary buffer or string" its parsing to "JSON object, binary buffer, string, number, float, array, bool, ...."

4

If you check the definition of JSON you will see that it does include primitive types, so, for example, the string "true" is valid JSON representing the Boolean true. So the new behaviour is correct when it does that conversion.

5

No, JSON is always a string. It is parsing from JSON to js object, array, or primitive.

This post should help to clarify the difference between Javascript objects and JSON.

6

Im assuming that's a "yes, intended behavior" then? I'm fine with that, it just wasn't a change I could find documentation for, and I couldn't figure out why two nodes that looked identical were returning different things.

For the record, then, old nodes don't always select the old deprecated "string or buffer" option when you open their settings, even though that's what they seem to USE still. Instead half the time they are selecting the NEW option still for me which makes them look IDENTICAL to the NEW nodes I'm creating, but are FUNCTIONING differently until I explicitly change them to a different parsing option. I haven't been able to find a commonality between the nodes that DO select the old value by default, vs the ones that don't, just seemed random.

I have already changed over all my nodes to explicit strings to avoid the issue, but imagine whatever bug does that to the settings screen will trip up others in the future.

7

Can you post a screen shot of the config of the two that appear the same but function differently?

8

Unfortunately, no, I've already corrected all of them, and I don't seem to be able to put it back into that state now through the interface. Wild guess, the configuration at one time didn't save a value for the output parsing (null, these were all mostly setup back in 1.x days probably) and while it falls back to the deprecated BEHAVIOR in this case, the UI did NOT handle it and so selects the "default"?

9

If you have a backup of the flow it would be very useful to see this.

There may be something we missed in v1->v3 upgrade to make this transparent and backwards compatible.

10

Ok I pulled the git repo and searched through the flows.json manually on an old commit... here's an example of two MQTT IN nodes that were right next to each other.

Im not positive this exhibits the issue, but one of these has a "dataType: auto" and one doesn't have ANY dataType specified... if I'm right that you're looking at a null issue, that fits the bill...

    {
        "id": "160a1b6d.d450c5",
        "type": "mqtt in",
        "z": "a21d7c7d.09f94",
        "g": "7b879bad.db6304",
        "name": "Current Temp",
        "topic": "ecobee/default/316595279306/actualTemperature",
        "qos": "0",
        "broker": "6050f6f3.5f8368",
        "inputs": 0,
        "x": 130,
        "y": 80,
        "wires": [
            [
                "e3b45ce4.56a05"
            ]
        ]
    },
    {
        "id": "82b314ba.106f28",
        "type": "mqtt in",
        "z": "a21d7c7d.09f94",
        "g": "53374b36.be37a4",
        "name": "Guest Room Occupancy",
        "topic": "ecobee/default/316595279306/sensor/GuestRoom/occupancy",
        "qos": "0",
        "datatype": "auto",
        "broker": "6050f6f3.5f8368",
        "inputs": 0,
        "x": 870,
        "y": 780,
        "wires": [
            [
                "57dd6d84.bb6d24"
            ]
        ]
    },

Specifically, this line (and the similar line in the v5 section above it) seems to default to "utf8 string" parsing if the dataType is null (ie, not even defaulting to string or buffer, but actually to explicitly utf8?):

Seems to make sense as its explicitly trying to default it to utf8 here if the config node doesn't have a specified value:

And of course the config UI is bound to the config value (which doesn't exist) so is defaulting to the first one in the list (auto-detect). Well, actually, its explicitly selecting that in the undefined case:

11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.