Getting raw data from TTN Uplink node

use copy value

image

then

```
paste code between backticks like this
```

PS - is your Raspberry Pi on the same network as your Win7 PC?

EDIT - try pasting images now in new post below - should work now

Hi Steve,

There's no _msgid line in the debug window. I copied the screenshot from the debug window, just warning messages:
'''


'''
Yes, RPi is in the same network as Win7. I did some changes to RPi network config files, so it has fixed IP now (it makes easier accessing NODE-RED via browser).

I am not quite sure what you mean "flow" - isn't it the same as the picture with nodes connected? The Function node consists of code copied from your message, and that's all.

My screen shot was just an example of how to copy from debug window.

No. That's is a picture. I can't see inside the function node from a picture. When someone asks you to post your flow it means export the node-red flow as JSON from the hamburger menu in node-red menu-->export

Never mind.

As I suspected, your data is NOT a buffer but an ArrayBuffer.

Do the following 2 things....

  1. Select the 4 nodes on your flow then press CTRL+E, the copy the flow and paste it between 3 backticks
  2. Copy the value from the debug window for the bottom debug node (the one names msg) (using the copy button as I demonstrated) and paste between 3 backticks

Steve,

I skipped the Function node and connected the Uplink node directly to Debug node confu=igured to "full message". Below is the result copied from the Debug window (as suggested):

{"app_id":"agricheck-test-jas","dev_id":"lora-agri-evl","hardware_serial":"60C5A8FFFE766873","port":1,"counter":13519,"payload_raw":[49,57,46,49,67,47,54,54,46,51,70],"metadata":{"time":"2020-05-10T13:34:33.905979911Z","frequency":868.5,"modulation":"LORA","data_rate":"SF9BW125","airtime":205824000,"coding_rate":"4/5","gateways":[{"gtw_id":"eui-58a0cbfffe8010ac","timestamp":2343304300,"time":"2020-05-10T13:34:33.436207056Z","channel":0,"rssi":-51,"snr":9.75,"rf_chain":0}]},"payload":[49,57,46,49,67,47,54,54,46,51,70],"_msgid":"46ae7100.d6206"}

''' are NOT backticks. These are ```. Code is no good. Please edit your post and do it like I described.

Never mind - I'll do it!

And do this ^

Steve,
Below are results of both 1. and 2.

[{"id":"833a11ea.79d57","type":"tab","label":"Flow 2","disabled":false,"info":""},{"id":"a58dd190.a79f7","type":"ttn uplink","z":"833a11ea.79d57","name":"RAK II","app":"a1b2d64e.20ce98","dev_id":"lora-agri-evl","field":"","x":130,"y":100,"wires":[["475d5624.c84e18","d6547c.be620b88"]]},{"id":"475d5624.c84e18","type":"function","z":"833a11ea.79d57","name":"","func":"\nnode.warn([\"typeof msg.payload.buffer\", typeof msg.payload.buffer]);\nnode.warn([\"msg.payload.buffer.constructor.name\", msg.payload.buffer.constructor.name]);\n\nif(Buffer.isBuffer(msg.payload.buffer) == false){\n  msg.oldPayloadBuffer = msg.payload.buffer; //keep a copy - for later debug analysis\n  node.warn(\"msg.payload.buffer is not a Buffer\");\n  msg.payload.buffer = Buffer.from(msg.payload.buffer);\n}\n\nnode.warn([\"typeof msg.payload.buffer\", typeof msg.payload.buffer]);\n\nif(msg.payload.buffer.length < 1){\n  node.warn(\"msg.payload.buffer does not have enough bytes\");\n  return null; //halt flow\n}\nmsg.buffer = msg.payload.buffer; //first, keep a ref to buffer - for later debug analysis\nnode.warn([\"This is msg.payload.buffer\", msg.payload.buffer]); //check debug window - is byte zero correct? \n\nvar newp = {};//create new empty object\nnewp.tempC = msg.payload.buffer[0]; //get first byte\nmsg.payload = newp;//now replace payload with new payload\nreturn msg;","outputs":1,"noerr":0,"x":310,"y":100,"wires":[["ced02a66.4c8608"]]},{"id":"ced02a66.4c8608","type":"debug","z":"833a11ea.79d57","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":530,"y":100,"wires":[]},{"id":"d6547c.be620b88","type":"debug","z":"833a11ea.79d57","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":510,"y":200,"wires":[]},{"id":"a1b2d64e.20ce98","type":"ttn app","z":"","appId":"agricheck-test-jas","accessKey":"ttn-account-v2.MegQTVjXqaAFDfaDbfSzV3ihbvdPBhDjXq_ddgqKRtQ","discovery":"discovery.thethingsnetwork.org:1900"}]
{"app_id":"agricheck-test-jas","dev_id":"lora-agri-evl","hardware_serial":"60C5A8FFFE766873","port":1,"counter":13546,"payload_raw":[49,57,46,55,67,47,54,55,46,52,70],"metadata":{"time":"2020-05-10T13:58:06.70509844Z","frequency":867.9,"modulation":"LORA","data_rate":"SF9BW125","airtime":205824000,"coding_rate":"4/5","gateways":[{"gtw_id":"eui-58a0cbfffe8010ac","timestamp":3756111212,"time":"2020-05-10T13:58:06.22846198Z","channel":0,"rssi":-55,"snr":12.75,"rf_chain":0}]},"payload":[49,57,46,55,67,47,54,55,46,52,70],"_msgid":"256ff7a7.5743c8"}

The reason for the misunderstanding was the use of the word "backtick" - I did not know the term. It would be more clear to me to use a "Grave accent" or its hex code 0x60h. . I apologize for causing you a problem.

Sure no problem. However I did put this in the message too ...

No harm, no foul.

Ps, not too many folk know what the grave accent is and even less know the hex code haha.

But it would be easier (for me) to find the right character in the ASCII table. The same with hexadecimal code - I worked with ASCII hex codes many years ago :wink:
BTW. Any suggestions after looking at the flow and value codes?

looking now.

I need you to try something - 2 seconds

So i simulated your data but objiously the Array/Buffer whatever it is, is represented as a simple array in JSON (so my simulation is not exact)...

Try the function code from this

[{"id":"db8a6e9d.c8ec6","type":"function","z":"dadb596f.70e948","name":"inspect msg & get 1st byte","func":"\nmsg.isPayloadABuffer = Buffer.isBuffer(msg.payload)\nmsg.isPayloadRawABuffer = Buffer.isBuffer(msg.payload_raw);\nvar buf;\nif(typeof msg.payload == \"object\"){\n    node.warn([\"Object.keys(msg.payload)\", Object.keys(msg.payload)]); \n    node.warn([\"This is msg.payload\", msg.payload]); \n    if(typeof msg.payload == \"object\"){\n        if(msg.payload.buffer) {\n            node.warn([\"This is msg.payload.buffer\", msg.payload.buffer]); \n            node.warn([\"Object.keys(msg.payload.buffer)\", Object.keys(msg.payload.buffer)]); \n        }\n    }\n}\n\nif(msg.payload_raw && typeof msg.payload_raw == \"object\"){\n    node.warn([\"Object.keys(msg.payload_raw)\", Object.keys(msg.payload_raw)]); \n    node.warn([\"This is msg.payload_raw\", msg.payload_raw]); \n}\n\nif(Buffer.isBuffer(msg.payload)){\n    msg.info = \"msg.payload is a buffer\";\n    buf = msg.payload;\n} else if(typeof msg.payload == \"object\" && msg.payload.buffer && Buffer.isBuffer(msg.payload.buffer)){\n    msg.info = \"msg.payload.buffer is a buffer\";\n    buf = msg.payload.buffer;\n} else if(Buffer.isBuffer(msg.payload_raw)){\n    msg.info = \"msg.payload_raw is a buffer\";\n    buf = msg.payload_raw;\n} else {\n    try {\n        node.warn(\"payload & payload_raw are not buffers! Attempting to create a buffer out of msg.payload_raw\")\n        buf = Buffer.from(msg.payload_raw);\n    } catch(e) {\n        node.warn(\"Failed to create a buffer out of msg.payload_raw - gonna try using that by itself\");\n        buf = msg.payload_raw;\n    }\n}\n\nmsg.buffer = buf; //first, keep a ref to buffer - for later debug analysis\n\nif(buf.length < 1){\n  node.warn(\"buf does not have enough bytes\");\n  msg.payload = null;\n  return msg; \n}\n\nmsg.payload  = {\n    tempC:  buf[0] //get first byte\n};\n\nreturn msg;","outputs":1,"noerr":0,"x":1160,"y":300,"wires":[["9e453a14.fa6bf8"]]},{"id":"9e453a14.fa6bf8","type":"debug","z":"dadb596f.70e948","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":1230,"y":340,"wires":[]},{"id":"19fd60b9.4a307f","type":"debug","z":"dadb596f.70e948","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":870,"y":340,"wires":[]},{"id":"457e9ee6.54654","type":"inject","z":"dadb596f.70e948","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":860,"y":260,"wires":[["e496491a.da0af8"]]},{"id":"e496491a.da0af8","type":"function","z":"dadb596f.70e948","name":"fake ttn","func":"msg = {\n   \"app_id\":\"agricheck-test-jas\",\n   \"dev_id\":\"lora-agri-evl\",\n   \"hardware_serial\":\"60C5A8FFFE766873\",\n   \"port\":1,\n   \"counter\":13546,\n   \"payload_raw\":[\n      49,\n      57,\n      46,\n      55,\n      67,\n      47,\n      54,\n      55,\n      46,\n      52,\n      70\n   ],\n   \"metadata\":{\n      \"time\":\"2020-05-10T13:58:06.70509844Z\",\n      \"frequency\":867.9,\n      \"modulation\":\"LORA\",\n      \"data_rate\":\"SF9BW125\",\n      \"airtime\":205824000,\n      \"coding_rate\":\"4/5\",\n      \"gateways\":[\n         {\n            \"gtw_id\":\"eui-58a0cbfffe8010ac\",\n            \"timestamp\":3756111212,\n            \"time\":\"2020-05-10T13:58:06.22846198Z\",\n            \"channel\":0,\n            \"rssi\":-55,\n            \"snr\":12.75,\n            \"rf_chain\":0\n         }\n      ]\n   },\n   \"payload\":[\n      49,\n      57,\n      46,\n      55,\n      67,\n      47,\n      54,\n      55,\n      46,\n      52,\n      70\n   ],\n   \"_msgid\":\"256ff7a7.5743c8\"\n}\n\n\n\nreturn msg;","outputs":1,"noerr":0,"x":860,"y":300,"wires":[["db8a6e9d.c8ec6","19fd60b9.4a307f"]]}]

OK, I pasted the code into a new Function node and connected it to the Debug node. Here is screen dump with the extended msg:

Tell me please if you need something more.

no - not enough unfortunately.

I have one last idea - I've modified the function to capture more detail IN THE msg itself

So...

  1. modify the "inspect msg & get 1st byte" function & copy/paste in the code below
  2. operate your RAK ||
  3. capture the message by the copy object button & paste in a reply (between ``` backticks (grave accents :wink: )

new function code (to replace the old)...


msg.isPayloadABuffer = Buffer.isBuffer(msg.payload)
msg.isPayloadRawABuffer = Buffer.isBuffer(msg.payload_raw);

var buf;

try {
    msg.payload_raw_info = "msg.payload_raw.constructor.name="
    msg.payload_raw_info += msg.payload_raw.constructor.name
} catch(e) {}


if(typeof msg.payload == "object"){

    try {
        msg.payload_info = "msg.payload.constructor.name="
        msg.payload_info += msg.payload.constructor.name
        msg.payload_keys = Object.keys(msg.payload);
    } catch(e) {}
    
    if(typeof msg.payload == "object"){
        if(msg.payload.buffer) {
            try {
                msg.payload_buffer_info = "msg.payload.buffer.constructor.name="
                msg.payload_buffer_info += msg.payload.buffer.constructor.name
                msg.payload_buffer_keys = Object.keys(msg.payload.buffer);
            } catch(e) {}
        }
    }
}

msg.payload_isBuffer = false;
msg.payload_raw_isBuffer = false;
msg.payload_buffer_isBuffer = false;

if(Buffer.isBuffer(msg.payload)){
    msg.payload_isBuffer = true;
    msg.info = "msg.payload is a buffer";
    buf = msg.payload;
} else if(typeof msg.payload == "object" && msg.payload.buffer && Buffer.isBuffer(msg.payload.buffer)){
    msg.payload_buffer_isBuffer = true;
    msg.info = "msg.payload.buffer is a buffer";
    buf = msg.payload.buffer;
} else if(Buffer.isBuffer(msg.payload_raw)){
    msg.payload_raw_isBuffer = true;
    msg.info = "msg.payload_raw is a buffer";
    buf = msg.payload_raw;
} else {
    try {
        msg.payload_buffer_isBuffer = false;
        msg.info1 = "payload & payload_raw are not buffers! Attempting to create a buffer out of msg.payload_raw"
        buf = Buffer.from(msg.payload_raw);
    } catch(e) {
        msg.info1Error = "Failed to create a buffer out of msg.payload_raw " + err.message;
        buf = msg.payload_raw;
    }
}

msg.buffer = buf; //first, keep a ref to buffer - for later debug analysis

if(buf.length < 1){
    msg.length_issue = "buf.length is < 1"
    node.warn("buf does not have enough bytes");
    msg.payload = null;
    return msg; 
}

msg.payload  = {
    tempC:  buf[0] //get first byte
};

return msg;

What to copy

What exactly you mean? Do I need to do something special with the device (hardware)?

Here's the value copied after modifying the Function node:

{"app_id":"agricheck-test-jas","dev_id":"lora-agri-evl","hardware_serial":"60C5A8FFFE766873","port":1,"counter":15511,"payload_raw":[49,57,46,49,67,47,54,54,46,51,70],"metadata":{"time":"2020-05-11T18:31:47.734612754Z","frequency":867.9,"modulation":"LORA","data_rate":"SF9BW125","airtime":205824000,"coding_rate":"4/5","gateways":[{"gtw_id":"eui-58a0cbfffe8010ac","timestamp":2480619420,"time":"2020-05-11T18:31:47.603984117Z","channel":0,"rssi":-51,"snr":14,"rf_chain":0}]},"payload":{"tempC":49},"_msgid":"3bbfa6ef.784eea","isPayloadABuffer":true,"isPayloadRawABuffer":true,"payload_raw_info":"msg.payload_raw.constructor.name=Buffer","payload_info":"msg.payload.constructor.name=Buffer","payload_keys":["0","1","2","3","4","5","6","7","8","9","10"],"payload_buffer_info":"msg.payload.buffer.constructor.name=ArrayBuffer","payload_buffer_keys":[],"payload_isBuffer":true,"payload_raw_isBuffer":false,"payload_buffer_isBuffer":false,"info":"msg.payload is a buffer","buffer":[49,57,46,49,67,47,54,54,46,51,70]}

Ok well now I'm stumped. It seems msg.payload IS actually a buffer. After all that nonsense.

Look at payload - it IS what you wanted - tempC = 49

You can pretty much delete the majority of the function node code and get values directly from msg.payload[0].

It seems you're absolutely right. I copied your previous code into Function node:

var newp = {};//create new empty object
newp.tempC = msg.payload[0]; //get first byte
msg.payload = newp;//now replace payload with new payload
return msg;

And the result is as expected:
image

So I am finally "at home" :slight_smile:
I can access data in this "strange" buffer and manipulate with its data.
Thank you very much Steve, I owe you a big beer. :slight_smile:

2 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.