Adapting flow to another input

General
1

Hi guys,

I discovered nodered because I found 2 flows a guy kindly shared in a forum that does exactly what I need. So bear with me !

I installed nodered in my homeassistant instance (few days ago, so everything is up to date), I imported both his flows and I managed to get this working. I can tell because he included ingest nodes so we could troubleshoot and final data are sent to my endpoint.

Problem started when I tried to adapt his flows to my configuration.

The guy receives his input messages by connecting to SenseCAP cloud portal API, on the contrary, I own a gateway from this brand. This gateway has no API but it's configured to send data to my Home Assistant MQTT.

So I connected his flows to my MQTT and obviously, data are not formatted as the flow expects, so it fails. I've tried to understand the nodes and the guy's code but I'm no dev so I'm struggling.

I think I narrowed it to only 2 issues:

The guy can extract the "measureID" from the msg.topic but I can't. To try and fix this I replaced his code to get the measureId from the payload.......

var myArray = msg.topic.split("/");
msg.payload.measureId = msg.payload.object.messages[0][1].measurementId;
msg.payload.deviceId = myArray[3].toUpperCase();
return msg;

The flow goes on with my workaround but next failure comes when he process data in "Set data, considerIp and URL" node.

On his ingest node, data are at the same level:

{
    "value": [
        {
            "mac": "1C:57:3E:39:47:50",
            "rssi": "-71"
        },
        {
            "mac": "D4:F8:29:B3:2B:30",
            "rssi": "-90"
        },
        {
            "mac": "98:25:4A:2D:B2:50",
            "rssi": "-52"
        },
        {
            "mac": "68:3F:7D:C9:08:40",
            "rssi": "-70"
        }
    ],
    "timestamp": 1742821277000,
    "measureId": "5001",
    "deviceId": "2cg7f1c064900884"
}

In my MQTT message, data are spread in arrays or in the main structure of the message like so:

{
    "deduplicationId": "58d3b1f7-5700-47fb-8629-a25cc0092476",
    "time": "2025-03-24T14:01:20.019564078+00:00",
    "deviceInfo": {
        "tenantId": "52f14cd4-c6f1-4fbd-8f87-4025e1d49242",
        "tenantName": "SenseCAP",
        "applicationId": "053edc9b-4c47-4a2f-b0e8-414dacbada7a",
        "applicationName": "Home Assistant",
        "deviceProfileId": "3c6aa0b4-11cd-4a2d-abaa-6b3eb3e7edbf",
        "deviceProfileName": "Tracker",
        "deviceName": "Tracker Anaïs",
        "devEui": "2cf7f1c064900884",
        "tags": {}
    },
    "devAddr": "01235505",
    "adr": true,
    "dr": 5,
    "fCnt": 305,
    "fPort": 5,
    "confirmed": true,
    "data": "BwAAAABn4WWvaD99yQhAvZglSi2yUNgYYiyj08Wl/////////wEAABVU",
    "object": {
        "messages": [
            [
                {
                    "motionId": 0,
                    "type": "Event Status",
                    "measurementValue": [],
                    "measurementId": "4200",
                    "timestamp": 1742824879000
                },
                {
                    "type": "Wi-Fi Scan",
                    "measurementId": "5001",
                    "measurementValue": [
                        {
                            "rssi": -67,
                            "mac": "68:3F:7D:C9:08:40"
                        },
                        {
                            "rssi": -40,
                            "mac": "98:25:4A:2D:B2:50"
                        },
                        {
                            "mac": "18:62:2C:A3:D3:C5",
                            "rssi": -91
                        }
                    ],
                    "timestamp": 1742824879000,
                    "motionId": 0
                },
                {
                    "measurementId": "4097",
                    "measurementValue": 25.6,
                    "type": "Air Temperature",
                    "motionId": 0,
                    "timestamp": 1742824879000
                },
                {
                    "measurementValue": 21,
                    "motionId": 0,
                    "type": "Light",
                    "timestamp": 1742824879000,
                    "measurementId": "4199"
                },
                {
                    "motionId": 0,
                    "timestamp": 1742824879000,
                    "type": "Battery",
                    "measurementValue": 84,
                    "measurementId": "3000"
                }
            ]
        ],
        "valid": true,
        "err": 0,
        "payload": "070000000067e165af683f7dc90840bd98254a2db250d818622ca3d3c5a5ffffffffffffff0100001554"
    },
    "rxInfo": [
        {
            "gatewayId": "2cf7f110626000a2",
            "uplinkId": 1227594873,
            "rssi": -38,
            "snr": 14.8,
            "channel": 3,
            "location": {
                "latitude": 48.89110415553951,
                "longitude": 2.113258838653565
            },
            "context": "2ZrkFA==",
            "metadata": {
                "region_common_name": "EU868",
                "region_config_id": "EU_863_870_TTN"
            },
            "crcStatus": "CRC_OK"
        }
    ],
    "txInfo": {
        "frequency": 867100000,
        "modulation": {
            "lora": {
                "bandwidth": 125000,
                "spreadingFactor": 7,
                "codeRate": "CR_4_5"
            }
        }
    }
}

Here is the first flow the guy shared:

[
	{
		"id": "55a7380fb743d666",
		"type": "tab",
		"label": "SenseCAP",
		"disabled": false,
		"info": ""
	},
	{
		"id": "a452700489a0f506",
		"type": "mqtt in",
		"z": "55a7380fb743d666",
		"name": "",
		"topic": "/device_sensor_data/YOUR_ORGANIZATION_ID/+/1/+/+",
		"qos": "2",
		"datatype": "auto-detect",
		"broker": "270bf66dda334b51",
		"nl": false,
		"rap": true,
		"rh": 0,
		"inputs": 0,
		"x": 200,
		"y": 100,
		"wires": [
			[
				"c3bfe4d4202e4a13"
			]
		]
	},
	{
		"id": "c3bfe4d4202e4a13",
		"type": "function",
		"z": "55a7380fb743d666",
		"name": "set measureID & deviceId",
		"func": "var myArray = msg.topic.split(\"/\");\nmsg.payload.measureId = myArray[6]; \nmsg.payload.deviceId = myArray[3].toUpperCase(); \nreturn msg;",
		"outputs": 1,
		"timeout": 0,
		"noerr": 0,
		"initialize": "",
		"finalize": "",
		"libs": [],
		"x": 530,
		"y": 100,
		"wires": [
			[
				"9347ab5a76be7d81"
			]
		]
	},
	{
		"id": "9347ab5a76be7d81",
		"type": "switch",
		"z": "55a7380fb743d666",
		"name": "Lon - Lat - WiFi",
		"property": "payload.measureId",
		"propertyType": "msg",
		"rules": [
			{
				"t": "eq",
				"v": "4197",
				"vt": "num"
			},
			{
				"t": "eq",
				"v": "4198",
				"vt": "num"
			},
			{
				"t": "eq",
				"v": "5001",
				"vt": "num"
			}
		],
		"checkall": "true",
		"repair": false,
		"outputs": 3,
		"x": 760,
		"y": 100,
		"wires": [
			[
				"25c51ea149655734"
			],
			[
				"1d11a59570af2a39"
			],
			[
				"1d021e3fcec76e4a"
			]
		]
	},
	{
		"id": "25c51ea149655734",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "",
		"rules": [
			{
				"t": "set",
				"p": "parts.index",
				"pt": "msg",
				"to": "0",
				"tot": "num"
			},
			{
				"t": "set",
				"p": "parts.count",
				"pt": "msg",
				"to": "2",
				"tot": "num"
			},
			{
				"t": "set",
				"p": "parts.id",
				"pt": "msg",
				"to": "msg.payload.deviceId & msg.payload.timestamp",
				"tot": "jsonata"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 980,
		"y": 40,
		"wires": [
			[
				"c2bbb3ff2f6bb57f"
			]
		]
	},
	{
		"id": "1d11a59570af2a39",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "",
		"rules": [
			{
				"t": "set",
				"p": "parts.index",
				"pt": "msg",
				"to": "1",
				"tot": "num"
			},
			{
				"t": "set",
				"p": "parts.count",
				"pt": "msg",
				"to": "2",
				"tot": "num"
			},
			{
				"t": "set",
				"p": "parts.id",
				"pt": "msg",
				"to": "msg.payload.deviceId & msg.payload.timestamp",
				"tot": "jsonata"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 980,
		"y": 100,
		"wires": [
			[
				"c2bbb3ff2f6bb57f"
			]
		]
	},
	{
		"id": "c2bbb3ff2f6bb57f",
		"type": "join",
		"z": "55a7380fb743d666",
		"name": "",
		"mode": "auto",
		"build": "object",
		"property": "payload",
		"propertyType": "msg",
		"key": "topic",
		"joiner": "\\n",
		"joinerType": "str",
		"accumulate": true,
		"timeout": "",
		"count": "",
		"reduceRight": false,
		"reduceExp": "",
		"reduceInit": "",
		"reduceInitType": "",
		"reduceFixup": "",
		"x": 1150,
		"y": 100,
		"wires": [
			[
				"fe90207af88fc89e"
			]
		]
	},
	{
		"id": "7d2b45e11dbefe50",
		"type": "inject",
		"z": "55a7380fb743d666",
		"name": "Lon",
		"props": [
			{
				"p": "payload"
			},
			{
				"p": "topic",
				"vt": "str"
			}
		],
		"repeat": "",
		"crontab": "",
		"once": false,
		"onceDelay": 0.1,
		"topic": "/device_sensor_data/1111/ABCD/1/+/4197",
		"payload": "{\"value\":71.41331,\"timestamp\":1700230308000,\"measureId\":\"4197\"}",
		"payloadType": "json",
		"x": 330,
		"y": 40,
		"wires": [
			[
				"c3bfe4d4202e4a13"
			]
		]
	},
	{
		"id": "b22175ae4ea38b38",
		"type": "inject",
		"z": "55a7380fb743d666",
		"name": "Lat",
		"props": [
			{
				"p": "payload"
			},
			{
				"p": "topic",
				"vt": "str"
			}
		],
		"repeat": "",
		"crontab": "",
		"once": false,
		"onceDelay": 0.1,
		"topic": "/device_sensor_data/1111/ABCD/1/+/4198",
		"payload": "{\"value\":53.44913,\"timestamp\":1700230308000,\"measureId\":\"4198\"}",
		"payloadType": "json",
		"x": 330,
		"y": 160,
		"wires": [
			[
				"c3bfe4d4202e4a13"
			]
		]
	},
	{
		"id": "2b368c70b7d7bbe5",
		"type": "mqtt out",
		"z": "55a7380fb743d666",
		"name": "",
		"topic": "/tracking/add",
		"qos": "",
		"retain": "",
		"respTopic": "",
		"contentType": "",
		"userProps": "",
		"correl": "",
		"expiry": "",
		"broker": "35fd3a139e459f2a",
		"x": 2390,
		"y": 240,
		"wires": []
	},
	{
		"id": "2b12b1b2616ad781",
		"type": "http request",
		"z": "55a7380fb743d666",
		"name": "Get location from WiFi",
		"method": "POST",
		"ret": "obj",
		"paytoqs": "ignore",
		"url": "",
		"tls": "",
		"persist": false,
		"proxy": "",
		"insecureHTTPParser": false,
		"authType": "",
		"senderr": false,
		"headers": [],
		"x": 1040,
		"y": 360,
		"wires": [
			[
				"21c1ca15c0cdc4c3"
			]
		]
	},
	{
		"id": "3cfcea9eb8d59444",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "Replace property names",
		"rules": [
			{
				"t": "change",
				"p": "payload",
				"pt": "msg",
				"from": "mac",
				"fromt": "str",
				"to": "macAddress",
				"tot": "str"
			},
			{
				"t": "change",
				"p": "payload",
				"pt": "msg",
				"from": "rssi",
				"fromt": "str",
				"to": "signalStrength",
				"tot": "str"
			},
			{
				"t": "change",
				"p": "payload",
				"pt": "msg",
				"from": "value",
				"fromt": "str",
				"to": "wifiAccessPoints",
				"tot": "str"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 350,
		"y": 280,
		"wires": [
			[
				"8db6c0a63beeed03"
			]
		]
	},
	{
		"id": "1d021e3fcec76e4a",
		"type": "json",
		"z": "55a7380fb743d666",
		"name": "Convert to string",
		"property": "payload",
		"action": "str",
		"pretty": false,
		"x": 120,
		"y": 280,
		"wires": [
			[
				"3cfcea9eb8d59444"
			]
		]
	},
	{
		"id": "8db6c0a63beeed03",
		"type": "json",
		"z": "55a7380fb743d666",
		"name": "Convert to object",
		"property": "payload",
		"action": "obj",
		"pretty": false,
		"x": 590,
		"y": 280,
		"wires": [
			[
				"21a76a972e1ac698"
			]
		]
	},
	{
		"id": "21a76a972e1ac698",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "Set data, considerIp and URL",
		"rules": [
			{
				"t": "move",
				"p": "payload.timestamp",
				"pt": "msg",
				"to": "data.timestamp",
				"tot": "msg"
			},
			{
				"t": "move",
				"p": "payload.deviceId",
				"pt": "msg",
				"to": "data.id",
				"tot": "msg"
			},
			{
				"t": "set",
				"p": "payload.considerIp",
				"pt": "msg",
				"to": "false",
				"tot": "bool"
			},
			{
				"t": "set",
				"p": "url",
				"pt": "msg",
				"to": "\"https://www.googleapis.com/geolocation/v1/geolocate?key=\" & $env('googleToken')\t",
				"tot": "jsonata"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 850,
		"y": 280,
		"wires": [
			[
				"2b12b1b2616ad781"
			]
		]
	},
	{
		"id": "21c1ca15c0cdc4c3",
		"type": "switch",
		"z": "55a7380fb743d666",
		"name": "Check accuracy",
		"property": "payload.accuracy",
		"propertyType": "msg",
		"rules": [
			{
				"t": "lt",
				"v": "50",
				"vt": "num"
			}
		],
		"checkall": "true",
		"repair": false,
		"outputs": 1,
		"x": 1180,
		"y": 280,
		"wires": [
			[
				"a075c4ddced1bf40"
			]
		]
	},
	{
		"id": "dc046029d6ff210b",
		"type": "inject",
		"z": "55a7380fb743d666",
		"name": "WiFi",
		"props": [
			{
				"p": "payload"
			},
			{
				"p": "topic",
				"vt": "str"
			}
		],
		"repeat": "",
		"crontab": "",
		"once": false,
		"onceDelay": 0.1,
		"topic": "",
		"payload": "{\"value\":[{\"mac\":\"00:00:00:00:00:00\",\"rssi\":\"-91\"},{\"mac\":\"00:00:00:00:00:00\",\"rssi\":\"-90\"},{\"mac\":\"00:00:00:00:00:00\",\"rssi\":\"-86\"},{\"mac\":\"00:00:00:00:00:00\",\"rssi\":\"-74\"}],\"timestamp\":1700481455000,\"measureId\":\"5001\",\"deviceId\":\"ABCD\"}",
		"payloadType": "json",
		"x": 570,
		"y": 40,
		"wires": [
			[
				"9347ab5a76be7d81"
			]
		]
	},
	{
		"id": "3b73d15afdacc315",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "Set URL",
		"rules": [
			{
				"t": "set",
				"p": "url",
				"pt": "msg",
				"to": "\"https://maps.googleapis.com/maps/api/elevation/json?locations=\" & msg.data.lat & \",\" & msg.data.lon & \"&key=\" & $env('googleToken')\t",
				"tot": "jsonata"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 1660,
		"y": 180,
		"wires": [
			[
				"1ae88917e744764d"
			]
		]
	},
	{
		"id": "1ae88917e744764d",
		"type": "http request",
		"z": "55a7380fb743d666",
		"name": "Get elevation",
		"method": "GET",
		"ret": "obj",
		"paytoqs": "ignore",
		"url": "",
		"tls": "",
		"persist": false,
		"proxy": "",
		"insecureHTTPParser": false,
		"authType": "",
		"senderr": false,
		"headers": [],
		"x": 1770,
		"y": 260,
		"wires": [
			[
				"8eb4e0b032497b4f"
			]
		]
	},
	{
		"id": "fe90207af88fc89e",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "Prepare MQTT GPS data",
		"rules": [
			{
				"t": "move",
				"p": "payload[0].value",
				"pt": "msg",
				"to": "data.lon",
				"tot": "msg"
			},
			{
				"t": "move",
				"p": "payload[1].value",
				"pt": "msg",
				"to": "data.lat",
				"tot": "msg"
			},
			{
				"t": "move",
				"p": "payload[0].timestamp",
				"pt": "msg",
				"to": "data.timestamp",
				"tot": "msg"
			},
			{
				"t": "move",
				"p": "payload[0].deviceId",
				"pt": "msg",
				"to": "data.id",
				"tot": "msg"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 1370,
		"y": 100,
		"wires": [
			[
				"3b73d15afdacc315"
			]
		]
	},
	{
		"id": "a075c4ddced1bf40",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "Prepare MQTT WiFI data",
		"rules": [
			{
				"t": "move",
				"p": "payload.location.lng",
				"pt": "msg",
				"to": "data.lon",
				"tot": "msg"
			},
			{
				"t": "move",
				"p": "payload.location.lat",
				"pt": "msg",
				"to": "data.lat",
				"tot": "msg"
			},
			{
				"t": "set",
				"p": "data.hdop",
				"pt": "msg",
				"to": "1000",
				"tot": "num"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 1410,
		"y": 280,
		"wires": [
			[
				"3b73d15afdacc315"
			]
		]
	},
	{
		"id": "8eb4e0b032497b4f",
		"type": "switch",
		"z": "55a7380fb743d666",
		"name": "",
		"property": "statusCode",
		"propertyType": "msg",
		"rules": [
			{
				"t": "eq",
				"v": "200",
				"vt": "num"
			},
			{
				"t": "else"
			}
		],
		"checkall": "true",
		"repair": false,
		"outputs": 2,
		"x": 1870,
		"y": 180,
		"wires": [
			[
				"72e902c74e616e1b"
			],
			[
				"35eb4ab3ebe47a3b"
			]
		]
	},
	{
		"id": "72e902c74e616e1b",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "Set altitude",
		"rules": [
			{
				"t": "set",
				"p": "data.altitude",
				"pt": "msg",
				"to": "payload.results[0].elevation",
				"tot": "msg"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 2050,
		"y": 120,
		"wires": [
			[
				"35eb4ab3ebe47a3b"
			]
		]
	},
	{
		"id": "35eb4ab3ebe47a3b",
		"type": "change",
		"z": "55a7380fb743d666",
		"name": "Move data to payload",
		"rules": [
			{
				"t": "move",
				"p": "data",
				"pt": "msg",
				"to": "payload",
				"tot": "msg"
			}
		],
		"action": "",
		"property": "",
		"from": "",
		"to": "",
		"reg": false,
		"x": 2180,
		"y": 240,
		"wires": [
			[
				"2b368c70b7d7bbe5"
			]
		]
	},
	{
		"id": "270bf66dda334b51",
		"type": "mqtt-broker",
		"name": "SenseCAP",
		"broker": "sensecap-openstream.seeed.cc",
		"port": "1883",
		"clientid": "YOUR_CLIENTID",
		"autoConnect": true,
		"usetls": false,
		"protocolVersion": "4",
		"keepalive": "60",
		"cleansession": true,
		"autoUnsubscribe": true,
		"birthTopic": "",
		"birthQos": "0",
		"birthRetain": "false",
		"birthPayload": "",
		"birthMsg": {},
		"closeTopic": "",
		"closeQos": "0",
		"closeRetain": "false",
		"closePayload": "",
		"closeMsg": {},
		"willTopic": "",
		"willQos": "0",
		"willRetain": "false",
		"willPayload": "",
		"willMsg": {},
		"userProps": "",
		"sessionExpiry": ""
	},
	{
		"id": "35fd3a139e459f2a",
		"type": "mqtt-broker",
		"name": "YOUR_MQTT_SERVER_NAME",
		"broker": "1YOUR_MQTT_SERVER_URL",
		"port": "YOUR_MQTT_SERVER_PORT",
		"clientid": "",
		"autoConnect": true,
		"usetls": false,
		"protocolVersion": "4",
		"keepalive": "60",
		"cleansession": true,
		"birthTopic": "",
		"birthQos": "0",
		"birthPayload": "",
		"birthMsg": {},
		"closeTopic": "",
		"closeQos": "0",
		"closePayload": "",
		"closeMsg": {},
		"willTopic": "",
		"willQos": "0",
		"willPayload": "",
		"willMsg": {},
		"sessionExpiry": ""
	}
]

Here is the second flow:

[
    {
        "id": "4e134a97278a50c7",
        "type": "http request",
        "z": "1663510036786436",
        "name": "Create postion in Traccar",
        "method": "GET",
        "ret": "txt",
        "paytoqs": "ignore",
        "url": "",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [],
        "x": 470,
        "y": 920,
        "wires": [
            []
        ]
    },
    {
        "id": "e943624eef5477a5",
        "type": "mqtt in",
        "z": "1663510036786436",
        "name": "",
        "topic": "/tracking/add",
        "qos": "2",
        "datatype": "auto-detect",
        "broker": "35fd3a139e459f2a",
        "nl": false,
        "rap": true,
        "rh": 0,
        "inputs": 0,
        "x": 90,
        "y": 920,
        "wires": [
            [
                "620c805b0ccba766"
            ]
        ]
    },
    {
        "id": "620c805b0ccba766",
        "type": "function",
        "z": "1663510036786436",
        "name": "Set URL",
        "func": "msg.url=env.get(\"host\")+\":\"+env.get(\"portOsmand\")+\"/?id=\"+msg.payload.id+\"&lat=\"+msg.payload.lat+\"&lon=\"+msg.payload.lon+\"&timestamp=\"+msg.payload.timestamp\n\nif (msg.payload.speed != null)\n{\n    msg.url = msg.url + \"&speed=\" + msg.payload.speed\n}\n\nif (msg.payload.altitude != null)\n{\n    msg.url = msg.url + \"&altitude=\" + msg.payload.altitude\n}\n\nif (msg.payload.batt != null)\n{\n    msg.url = msg.url + \"&batt=\" + msg.payload.batt\n}\n\nif (msg.payload.hdop != null)\n{\n    msg.url = msg.url + \"&hdop=\" + msg.payload.hdop\n}\n\nreturn msg;",
        "outputs": 1,
        "timeout": "",
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 260,
        "y": 920,
        "wires": [
            [
                "4e134a97278a50c7"
            ]
        ]
    },
    {
        "id": "35fd3a139e459f2a",
		"type": "mqtt-broker",
		"name": "YOUR_MQTT_SERVER_NAME",
		"broker": "1YOUR_MQTT_SERVER_URL",
		"port": "YOUR_MQTT_SERVER_PORT",
		"clientid": "",
		"autoConnect": true,
		"usetls": false,
		"protocolVersion": "4",
		"keepalive": "60",
		"cleansession": true,
		"birthTopic": "",
		"birthQos": "0",
		"birthPayload": "",
		"birthMsg": {},
		"closeTopic": "",
		"closeQos": "0",
		"closePayload": "",
		"closeMsg": {},
		"willTopic": "",
		"willQos": "0",
		"willPayload": "",
		"willMsg": {},
		"sessionExpiry": ""
    }
]

I tried split, join, change nodes but as I don't understand what I do, I could try for few years more without succeeding and I'm loosing faith. so I'm asking help here because all this is way beyond my pay grade.

Thanks for your help.

Arnaud

2

Welcome to the forum @Bigsausage64.

I don't know Home Assistant so no idea what an ingest node is but can't you copy his code restructuring the input?

Well done for including a sample of your data.
If you connect this input data to a debug node you can explore it's structure by clicking on the little arrows to expand each property.
When you hover the mouse over a property there is a little button which lets you copy the element's path. For example, this element is msg.payload.deviceInfo.devEui

image
image627×733 28.9 KB

You will need to identify each element you need to pass forwards.

Since your json data has quite a complicated structure, it's probably easier to use a function node than core Node-red nodes to parse it.

Precisely how to do so depends on your required output data structure, which I don't think you have shared.

3

Hi and thanks for reading and welcoming me.

The ingest node I'm talking about is part of the first flow the guy shared. It only allows to input test data in the flow to check how it works.

I've already connected my MQTT input with the guy's flow so I see the structure as you said. That's the code I shared. I used the exact button you mentionned to copy the following code:

{
    "deduplicationId": "58d3b1f7-5700-47fb-8629-a25cc0092476",
    "time": "2025-03-24T14:01:20.019564078+00:00",
    "deviceInfo": {
        "tenantId": "52f14cd4-c6f1-4fbd-8f87-4025e1d49242",
        "tenantName": "SenseCAP",
        "applicationId": "053edc9b-4c47-4a2f-b0e8-414dacbada7a",
        "applicationName": "Home Assistant",
        "deviceProfileId": "3c6aa0b4-11cd-4a2d-abaa-6b3eb3e7edbf",
        "deviceProfileName": "Tracker",
        "deviceName": "Tracker Anaïs",
        "devEui": "2cf7f1c064900884",
        "tags": {}
    },
    "devAddr": "01235505",
    "adr": true,
    "dr": 5,
    "fCnt": 305,
    "fPort": 5,
    "confirmed": true,
    "data": "BwAAAABn4WWvaD99yQhAvZglSi2yUNgYYiyj08Wl/////////wEAABVU",
    "object": {
        "messages": [
            [
                {
                    "motionId": 0,
                    "type": "Event Status",
                    "measurementValue": [],
                    "measurementId": "4200",
                    "timestamp": 1742824879000
                },
                {
                    "type": "Wi-Fi Scan",
                    "measurementId": "5001",
                    "measurementValue": [
                        {
                            "rssi": -67,
                            "mac": "68:3F:7D:C9:08:40"
                        },
                        {
                            "rssi": -40,
                            "mac": "98:25:4A:2D:B2:50"
                        },
                        {
                            "mac": "18:62:2C:A3:D3:C5",
                            "rssi": -91
                        }
                    ],
                    "timestamp": 1742824879000,
                    "motionId": 0
                },
                {
                    "measurementId": "4097",
                    "measurementValue": 25.6,
                    "type": "Air Temperature",
                    "motionId": 0,
                    "timestamp": 1742824879000
                },
                {
                    "measurementValue": 21,
                    "motionId": 0,
                    "type": "Light",
                    "timestamp": 1742824879000,
                    "measurementId": "4199"
                },
                {
                    "motionId": 0,
                    "timestamp": 1742824879000,
                    "type": "Battery",
                    "measurementValue": 84,
                    "measurementId": "3000"
                }
            ]
        ],
        "valid": true,
        "err": 0,
        "payload": "070000000067e165af683f7dc90840bd98254a2db250d818622ca3d3c5a5ffffffffffffff0100001554"
    },
    "rxInfo": [
        {
            "gatewayId": "2cf7f110626000a2",
            "uplinkId": 1227594873,
            "rssi": -38,
            "snr": 14.8,
            "channel": 3,
            "location": {
                "latitude": 48.89110415553951,
                "longitude": 2.113258838653565
            },
            "context": "2ZrkFA==",
            "metadata": {
                "region_common_name": "EU868",
                "region_config_id": "EU_863_870_TTN"
            },
            "crcStatus": "CRC_OK"
        }
    ],
    "txInfo": {
        "frequency": 867100000,
        "modulation": {
            "lora": {
                "bandwidth": 125000,
                "spreadingFactor": 7,
                "codeRate": "CR_4_5"
            }
        }
    }
}

That is what I have as an input. I also succeeded in identifying the data I need. What I struggle with is how to "manipulate" these data.

Reading your last sentence, I understand my explanation was not clear enough. I think I would need to manipulate date I receive as in input, so it becomes like the code the guy put in his ingest node:

{
    "value": [
        {
            "mac": "1C:57:3E:39:47:50",
            "rssi": "-71"
        },
        {
            "mac": "D4:F8:29:B3:2B:30",
            "rssi": "-90"
        },
        {
            "mac": "98:25:4A:2D:B2:50",
            "rssi": "-52"
        },
        {
            "mac": "68:3F:7D:C9:08:40",
            "rssi": "-70"
        }
    ],
    "timestamp": 1742821277000,
    "measureId": "5001",
    "deviceId": "2cg7f1c064900884"
}

I already identified where to retrieve the deviceId and timestamp from the payload, but I can't have a result that the flow is accepting.

I put a debug node after each step of the guy, trying to understand what happens and what is expected, but as soon as I try something I loose data or have too much data, which fails obviously.

Let me know if it's still not clear.

Arnaud

4

Something like this perhaps?

image

[{"id":"9b5cded30716297c","type":"inject","z":"7cbab40455148818","name":"Sample data","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"deduplicationId\":\"58d3b1f7-5700-47fb-8629-a25cc0092476\",\"time\":\"2025-03-24T14:01:20.019564078+00:00\",\"deviceInfo\":{\"tenantId\":\"52f14cd4-c6f1-4fbd-8f87-4025e1d49242\",\"tenantName\":\"SenseCAP\",\"applicationId\":\"053edc9b-4c47-4a2f-b0e8-414dacbada7a\",\"applicationName\":\"Home Assistant\",\"deviceProfileId\":\"3c6aa0b4-11cd-4a2d-abaa-6b3eb3e7edbf\",\"deviceProfileName\":\"Tracker\",\"deviceName\":\"Tracker Anaïs\",\"devEui\":\"2cf7f1c064900884\",\"tags\":{}},\"devAddr\":\"01235505\",\"adr\":true,\"dr\":5,\"fCnt\":305,\"fPort\":5,\"confirmed\":true,\"data\":\"BwAAAABn4WWvaD99yQhAvZglSi2yUNgYYiyj08Wl/////////wEAABVU\",\"object\":{\"messages\":[[{\"motionId\":0,\"type\":\"Event Status\",\"measurementValue\":[],\"measurementId\":\"4200\",\"timestamp\":1742824879000},{\"type\":\"Wi-Fi Scan\",\"measurementId\":\"5001\",\"measurementValue\":[{\"rssi\":-67,\"mac\":\"68:3F:7D:C9:08:40\"},{\"rssi\":-40,\"mac\":\"98:25:4A:2D:B2:50\"},{\"mac\":\"18:62:2C:A3:D3:C5\",\"rssi\":-91}],\"timestamp\":1742824879000,\"motionId\":0},{\"measurementId\":\"4097\",\"measurementValue\":25.6,\"type\":\"Air Temperature\",\"motionId\":0,\"timestamp\":1742824879000},{\"measurementValue\":21,\"motionId\":0,\"type\":\"Light\",\"timestamp\":1742824879000,\"measurementId\":\"4199\"},{\"motionId\":0,\"timestamp\":1742824879000,\"type\":\"Battery\",\"measurementValue\":84,\"measurementId\":\"3000\"}]],\"valid\":true,\"err\":0,\"payload\":\"070000000067e165af683f7dc90840bd98254a2db250d818622ca3d3c5a5ffffffffffffff0100001554\"},\"rxInfo\":[{\"gatewayId\":\"2cf7f110626000a2\",\"uplinkId\":1227594873,\"rssi\":-38,\"snr\":14.8,\"channel\":3,\"location\":{\"latitude\":48.89110415553951,\"longitude\":2.113258838653565},\"context\":\"2ZrkFA==\",\"metadata\":{\"region_common_name\":\"EU868\",\"region_config_id\":\"EU_863_870_TTN\"},\"crcStatus\":\"CRC_OK\"}],\"txInfo\":{\"frequency\":867100000,\"modulation\":{\"lora\":{\"bandwidth\":125000,\"spreadingFactor\":7,\"codeRate\":\"CR_4_5\"}}}}","payloadType":"json","x":290,"y":80,"wires":[["236ca87df209b72d"]]},{"id":"38c5276b2f6daa97","type":"debug","z":"7cbab40455148818","name":"Result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":630,"y":80,"wires":[]},{"id":"236ca87df209b72d","type":"function","z":"7cbab40455148818","name":"Relevant bits","func":"msg.payload = {\n    value: msg.payload.object.messages[0][1].measurementValue,\n    timestamp: msg.payload.object.messages[0][1].timestamp,\n    measureId:\"something\",\n    deviceID:\"something else\",\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":470,"y":80,"wires":[["38c5276b2f6daa97"]]}]

Which gives me

image
image358×538 9.3 KB

5

Nice One, we're getting close !
With your "relevant bits" the result is:

image
image231×189 3.98 KB

And here, the result when I test with the guy's ingest node:

image
image228×210 5.32 KB

6

The guy's putting the deviceId into the data.id in this node:

image
image526×614 18 KB

I put a debug node just after, I can still see the deviceID, so data is not lost, but in the second flow, he builds a URL:

msg.url=env.get("host")+":"+env.get("portOsmand")+"/?id="+msg.payload.id+"&lat="+msg.payload.lat+"&lon="+msg.payload.lon+"&timestamp="+msg.payload.timestamp

if (msg.payload.speed != null)
{
    msg.url = msg.url + "&speed=" + msg.payload.speed
}

if (msg.payload.altitude != null)
{
    msg.url = msg.url + "&altitude=" + msg.payload.altitude
}

if (msg.payload.batt != null)
{
    msg.url = msg.url + "&batt=" + msg.payload.batt
}

if (msg.payload.hdop != null)
{
    msg.url = msg.url + "&hdop=" + msg.payload.hdop
}

return msg;

and this is where the id gets lost, I don't know why. it seems msg.payload.id is nowhere to be found. Every other information gets properly transmitted

8

déplacer ... vers means Move ... to?

image
image684×183 17.6 KB

Attention to detail! msg.payload.id does not exist does it? :grinning_face:

There is an option in the debug node to display the entire message object.

9

déplacer ... vers means Move ... to?
=> CORRECT

When you say msg.data.id doesn't exist, I agree, it's the first tim in his code it appears..........but it's the same for msg.data.timestamp, and this one is working fine. How can it work with his ingest data and not with my data. I'm sorry, I'm lost.

10

I imagine we are both getting confused.

Let's ignore your colleague's data and his code for now.

Does the device id appear in your input data? Where?
I see 2cf7f1c064900884 in msg.payload.deviceInfo.devEui but perhaps I'm looking at his data not yours, perhaps this is not the ID you need.

Where do you need to put this value in your output data? msg.payload.id or msg.data.id?

If it seems to you that I am still misunderstanding, please post again:

  • The input data that you are working with.
  • The message structure that you need to construct from it.
11

Does the device id appear in your input data?
=> Yes
2cf7f1c064900884
=>Yes, this is the correct id
Where ?
=> correct, msg.payload.deviceInfo.devEui

As far as I can understand, you perfeclty understood this part.

Now, your next question is confusing me lol. It's not really a matter of where I need the data, it's more a question of what the flow is expecting to work (sorry to get back to this code, but that's what is driving me here).

In the end, I believe that's what I should see for the thing to work:

image
image228×210 5.32 KB

I get this result when ingesting the fake data the guy included. As soon as I use the "sample data" you gave me with my own data, this msg.data.id disappears.........that's why I'm not getting it.

Arnaud

12

It works !!!!!!!!!!!!

13

In the picture you just posted of what you believe the data should look like, you have msg.payload.id.

In an earlier picture of a change node, you seem to be setting msg.data.id.

Msg.data and it's property msg.data.id are not part of msg.payload.

I have tried to hint that you might get a clue from showing the entire message object.
Perhaps you will, perhaps mot. I really don't know.

14

In an earlier picture of a change node, you seem to be setting msg.data.id
=> yes, the guy's flow is set this way. I believe he stores data in msg.data to use them later in his flow. I'm not doing this, but I'm trying to adapt my input to fit in the flow.

I found the "entire message object" option in debug mode. Which message do you want to see in detail ?

FYI, the global picture here : the device is a gps tracker on LoRa Network, it periodically send messages with its position to my gateway which publishes data on MQTT.

The flows are identifying which data will be received (measureId=5001, wifi ssid are provided, measureId=4197+measureId=4198 are respectively latitude and longitude), and applies different treatment depending on which data was received.

The issue I was having was a typo in deviceID, when deviceId (NO CAPITAL "D") was expected. The Wifi flow is Ok.

Now you helped me fix this (thank you very much) I need to find how I can treat the data for the gps coordinates.

I know how to differentiate the measureId no problem.

In the function node you gave me:

msg.payload = {
    value: msg.payload.object.messages[0][1].measurementValue,
    timestamp: msg.payload.object.messages[0][1].timestamp,
    measureId:msg.payload.object.messages[0][1].measurementId,
    deviceId:msg.payload.deviceInfo.devEui.toUpperCase()
}
return msg;

this works because all ssid are grouped in msg.payload.object.messages[0][1].measurementValue

but for GPS, I would need to take:
msg.payload.object.messages[0][1].measurementValue AND msg.payload.object.messages[0][2].measurementValue

Do you know how I would do that ?

The flow is using a split node depending on the measureId. Each case has to receive different payloads, that's where I'm stuck now because with the code you gave me dismisses other arrays as far as I can tell. I hope I make myself clear.

15

Here is the input message I receive with GPS data:

{
    "deduplicationId": "697a9b25-4a10-4a1e-8b6b-06518307dcc5",
    "time": "2025-03-24T19:07:46.783937637+00:00",
    "deviceInfo": {
        "tenantId": "52f14cd4-c6f1-4fbd-8f87-4025e1d49242",
        "tenantName": "SenseCAP",
        "applicationId": "053edc9b-4c47-4a2f-b0e8-414dacbada7a",
        "applicationName": "Home Assistant",
        "deviceProfileId": "3c6aa0b4-11cd-4a2d-abaa-6b3eb3e7edbf",
        "deviceProfileName": "Tracker",
        "deviceName": "Tracker Anaïs",
        "devEui": "2cf7f1c064900884",
        "tags": {}
    },
    "devAddr": "01235505",
    "adr": true,
    "dr": 5,
    "fCnt": 372,
    "fPort": 5,
    "confirmed": true,
    "data": "BgAAAABn4a1/ACA/ggLqBWAAxwAAVA==",
    "object": {
        "messages": [
            [
                {
                    "motionId": 0,
                    "measurementValue": [],
                    "type": "Event Status",
                    "timestamp": 1742843263000,
                    "measurementId": "4200"
                },
                {
                    "measurementId": "4197",
                    "motionId": 0,
                    "type": "Longitude",
                    "measurementValue": 2.11341,
                    "timestamp": 1742843263000
                },
                {
                    "timestamp": 1742843263000,
                    "motionId": 0,
                    "measurementId": "4198",
                    "type": "Latitude",
                    "measurementValue": 48.891232
                },
                {
                    "measurementValue": 19.9,
                    "type": "Air Temperature",
                    "motionId": 0,
                    "timestamp": 1742843263000,
                    "measurementId": "4097"
                },
                {
                    "measurementValue": 0,
                    "motionId": 0,
                    "timestamp": 1742843263000,
                    "type": "Light",
                    "measurementId": "4199"
                },
                {
                    "motionId": 0,
                    "timestamp": 1742843263000,
                    "type": "Battery",
                    "measurementId": "3000",
                    "measurementValue": 84
                }
            ]
        ],
        "err": 0,
        "valid": true,
        "payload": "060000000067e1ad7f00203f8202ea056000c7000054"
    },
    "rxInfo": [
        {
            "gatewayId": "2cf7f110626000a2",
            "uplinkId": 2114729393,
            "rssi": -46,
            "snr": 14.8,
            "channel": 3,
            "location": {
                "latitude": 48.89110415553951,
                "longitude": 2.113258838653565
            },
            "context": "IYqICA==",
            "metadata": {
                "region_common_name": "EU868",
                "region_config_id": "EU_863_870_TTN"
            },
            "crcStatus": "CRC_OK"
        }
    ],
    "txInfo": {
        "frequency": 867100000,
        "modulation": {
            "lora": {
                "bandwidth": 125000,
                "spreadingFactor": 7,
                "codeRate": "CR_4_5"
            }
        }
    }
}
16

Indeed the function code I offered looks only at msg.payload.object.messages[0][1].
This may have worked to some extent but it does depend on the value and timestamp you require always being in the same element of the array.
It is bad programming style to make this kind of assumption and you were probably right not to adopt this approach :upside_down_face:.

Now we see that you need data which is [may be] in other array elements too.

There are various possibilities.

  1. Use Jsonata code in a single change node to extract the data from array elements with type "longitude", "latitude" or "Event Status" This is certainly beyond my skill level with Jsonata.
  2. Split the message object, split again the data.messages part, extract the bits you require and reassemble it into a new payload. I think this approach is likely to end up with a lot of split, join, change and switch nodes.
  3. Use a function node which loops through msg.payload.data.messages testing each one's type property and constructing [probably a new] msg.payload as it goes.
  4. Perhaps a template node would be applicable, though I'm not sure how it would handle the array elements.
  5. If you have control over the code which sends this data over Lora, make it send only the data you actually require, and in a flatter json format. Sending the latitude, longitude and timestamp as key: value pairs in an object rather than as anonymous array elements would simplify things considerably. This has the advantage of using the slow Lora link more efficiently.
  6. Other people might be able to suggest better options.

Option 5 looks best to me, but if it's not possible I'd go for option 3.

17

The data in measurementValue where measurementId == "5001" is an array of rsssi and mac measurements.
I don't see any measureId 4197 or 4198 in your data, but I do see latitude and longitude at msg.payload.rxInfo[0].location.

Whoever does this work for you is going to need an accurate and definitive description of the input data nad the required output to generate from it.

18

You can see it in my last message:

    "data": "BgAAAABn4a1/ACA/ggLqBWAAxwAAVA==",
    "object": {
        "messages": [
            [
                {
                    "motionId": 0,
                    "measurementValue": [],
                    "type": "Event Status",
                    "timestamp": 1742843263000,
                    "measurementId": "4200"
                },
                {
                    "measurementId": "4197",
                    "motionId": 0,
                    "type": "Longitude",
                    "measurementValue": 2.11341,
                    "timestamp": 1742843263000
                },
                {
                    "timestamp": 1742843263000,
                    "motionId": 0,
                    "measurementId": "4198",
                    "type": "Latitude",
                    "measurementValue": 48.891232
                },
19

Ah yes. My mistake.

20

For those who are interested, I posted the solution on the original website where I found the flows that worked with SenseCAP portal and I adapted to be used with SenseCAP gateway.

You can find working solution here: https://www.traccar.org/forums/topic/sensecap-t1000-support/page/3/#post-109452

And thanks a lot to jbudd who also helped me a lot !!!

Arnaud

2 Likes