Extracting my valuable data from a msg.payload?

General
21

Hi bakman2, Thank you. I have:
3 rows retrieved, max rows: 10; local time: Tue Mar 03 18:59:09 UTC 2020
execution start: 2020-03-03 18:30:22.153 UTC
{"compile-ms":"9","exec-ms":"94","server-rec'd":"1583260222134","statusDetail":"","status":"DEFAULT","host":"python-85fb5dbf54-t8tbs","total-ms":"107"}

2020-03-03 18:30:22.153 INFO 451: starting: api replaySession, level: TRACE
2020-03-03 18:30:22.165 DEBUG starting execution
2020-03-03 18:30:22.165 DEBUG replaySession
2020-03-03 18:30:22.166 DEBUG connect-id : r6GJIPvyVflqXpJd
2020-03-03 18:30:22.166 DEBUG sco-id : 11692
2020-03-03 18:30:22.166 DEBUG asset-id : 58754
2020-03-03 18:30:22.235 TRACE PS: SELECT date_start, date_end, attendees_list, transcripts, sco_info, meeting_permissions_list, server FROM session_log WHERE connect_id = ? AND sco_id = ? AND asset_id = ? /* ec81aa30-5d7c-11ea-8ee0-53b4772b249d replaySession */
  -> args: ('r6GJIPvyVflqXpJd', 11692, 58754)
  -> consistency: ONE
2020-03-03 18:30:22.258 DEBUG Failed to find meeting [r6GJIPvyVflqXpJd / 11692 / 58754] in session log
2020-03-03 18:30:22.259 DEBUG 1 datalake keyspace queries
2020-03-03 18:30:22.259 DEBUG executed without exceptions

I want to extract the values for connect-id, sco-id and asset-id

22

Please provide your flow so we can see what you are doing

23

Not sure how you end up with an array if you use a split with tab with the file output you provided.
but you can try this flow (change the filename ofcourse).

Next time you post output/code, please use the correct formatting for the forum else it is unreadable (as you can see).

[{"id":"33dc6e4d.850672","type":"inject","z":"8e22637.94c9ca","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":108,"y":360,"wires":[["32f3e9a7.b92946"]]},{"id":"32f3e9a7.b92946","type":"file in","z":"8e22637.94c9ca","name":"","filename":"/home/administrator/.node-red/test.txt","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":270,"y":408,"wires":[["3a7f5cb9.f176f4"]]},{"id":"3a7f5cb9.f176f4","type":"split","z":"8e22637.94c9ca","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":506,"y":408,"wires":[["dde4ad4e.5f2598"]]},{"id":"dde4ad4e.5f2598","type":"switch","z":"8e22637.94c9ca","name":"","property":"payload","propertyType":"msg","rules":[{"t":"regex","v":"(connect-id|asset-id|sco-id)","vt":"str","case":false}],"checkall":"true","repair":false,"outputs":1,"x":626,"y":408,"wires":[["d1ea3a97.672278"]]},{"id":"2f1ceb90.dd9574","type":"debug","z":"8e22637.94c9ca","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":866,"y":408,"wires":[]},{"id":"d1ea3a97.672278","type":"function","z":"8e22637.94c9ca","name":"","func":"m = msg.payload\n\ns = m.match(/(connect-id|sco-id|asset-id) : (.+)/); \nkey = s[1]\nval = s[2]\nnode.send({payload:{[key]:val}})","outputs":1,"noerr":0,"x":746,"y":408,"wires":[["2f1ceb90.dd9574"]]}]
24

Ok bakman2, Thank you. I know receive the debug output of:
msg: Object
{ payload: object,_msgid: "7667bee4.6c22a"}
{ payload: object,_msgid: "e136a3b.a1c366"}
:
repeats

25

Because you did not provide complete information.

26

(Hi bakman2, I am attempting to track down the problem. At the output of the switch I received the connect-id, co-id and asset-id as shown below: " 2020-03-03 18:30:22.166 DEBUG connect-id : r6GJIPvyVflqXpJd " At the output of the Function I get: 3/11/2020, 3:40:09 PMnode: 70633e6d.3dccemsg : Object { payload: object, _msgid: "48e30cef.3bf5e4" } 0:09 PMnode: c2645ecc.e9233msg : string[21] "No filename specified")

27

Format your output as i suggested aboved.

And as the output indicates: no filename specified.

28 
[{"id":"7c21d922.203238","type":"inject","z":"dcd1d133.03d3e","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":120,"y":160,"wires":[["5e4025ce.f0019c"]]},{"id":"beaabc21.43a51","type":"file in","z":"dcd1d133.03d3e","name":"","filename":"","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":310,"y":220,"wires":[["90e744a5.479a48","c81e5af4.b68268"]],"info":"msg.filename=\"meeting.log\";\nreturn msg;"},{"id":"90e744a5.479a48","type":"split","z":"dcd1d133.03d3e","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":510,"y":220,"wires":[["d6d341f1.6689a","7daaaca7.56f904"]]},{"id":"d6d341f1.6689a","type":"switch","z":"dcd1d133.03d3e","name":"","property":"payload","propertyType":"msg","rules":[{"t":"regex","v":"(connect-id|asset-id|sco-id)","vt":"str","case":false}],"checkall":"true","repair":false,"outputs":1,"x":650,"y":220,"wires":[["2f6f0b14.8ee084","e72bdb94.6f8998"]]},{"id":"70633e6d.3dcce","type":"debug","z":"dcd1d133.03d3e","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":690,"y":340,"wires":[]},{"id":"2f6f0b14.8ee084","type":"function","z":"dcd1d133.03d3e","name":"","func":"m = msg.payload\n\ns = m.match(/(connect-id|sco-id|asset-id) : (.+)/); \nkey = s[1]\nval = s[2]\nnode.send({payload:{[key]:val}})\n","outputs":1,"noerr":0,"x":770,"y":260,"wires":[["70633e6d.3dcce","c2645ecc.e9233"]]},{"id":"c81e5af4.b68268","type":"debug","z":"dcd1d133.03d3e","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":430,"y":280,"wires":[]},{"id":"5e4025ce.f0019c","type":"function","z":"dcd1d133.03d3e","name":"Gets Data From Meeting Log","func":"msg.filename=\"meeting.log\";\nreturn msg;","outputs":1,"noerr":0,"x":320,"y":60,"wires":[["beaabc21.43a51"]]},{"id":"7daaaca7.56f904","type":"debug","z":"dcd1d133.03d3e","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":560,"y":140,"wires":[]},{"id":"e72bdb94.6f8998","type":"debug","z":"dcd1d133.03d3e","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":740,"y":80,"wires":[]}]
29

Hi zenofmud,
Thanks to bakman2. I was able to progress to get the proper output of the switch with the values for connect-id, sco-id and asset-id. The debug shows that I get two sets of these values and then I get "No Filename specified".

30

Sorry bakman2, i am at a loss for what can be wrong. Are you saying the log I sent you was not formatted properly? What I received from you was a straight drag and copy to node-red.

31

@GilHub1 your flow isn't exactly what I had suggested. try this, feed your file-out node into this. I've done the bottom flow and left the others for you and you meed to format the end result as you need it.

[{"id":"2a71e005.30ecc","type":"debug","z":"90560af0.98518","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":470,"y":200,"wires":[]},{"id":"b4b7cbdc.45d48","type":"split","z":"90560af0.98518","name":"","splt":" : ","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":450,"y":320,"wires":[["bd348b2e.ff39d"]]},{"id":"ca19743e.7cce28","type":"switch","z":"90560af0.98518","name":"","property":"payload","propertyType":"msg","rules":[{"t":"cont","v":"connect-id","vt":"str"},{"t":"cont","v":"sco-id","vt":"str"},{"t":"cont","v":"asset-id","vt":"str"}],"checkall":"false","repair":false,"outputs":3,"x":270,"y":280,"wires":[["2a71e005.30ecc","ad16c3fc.5bc348"],["c279e347.d2bc48"],["b4b7cbdc.45d48"]]},{"id":"8f4a7dc7.d1fc28","type":"debug","z":"90560af0.98518","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":790,"y":280,"wires":[]},{"id":"ab22e750.30b708","type":"debug","z":"90560af0.98518","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":710,"y":400,"wires":[]},{"id":"c279e347.d2bc48","type":"split","z":"90560af0.98518","name":"","splt":" : ","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":450,"y":280,"wires":[["8f4a7dc7.d1fc28"]]},{"id":"bd348b2e.ff39d","type":"switch","z":"90560af0.98518","name":"","property":"payload","propertyType":"msg","rules":[{"t":"cont","v":"asset-id","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":570,"y":320,"wires":[[],["f60a6251.4b4d98","ab22e750.30b708"]]},{"id":"f60a6251.4b4d98","type":"change","z":"90560af0.98518","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"asset-id = \"&payload","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":740,"y":320,"wires":[["ab22e750.30b708"]]},{"id":"ad16c3fc.5bc348","type":"split","z":"90560af0.98518","name":"","splt":" : ","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":450,"y":240,"wires":[["7cad026d.bdcbec"]]},{"id":"7cad026d.bdcbec","type":"debug","z":"90560af0.98518","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":790,"y":240,"wires":[]}]
32

Thank you very much zenofmud. I did feed this after the split bakman2 gave me. This did isolate the outputs separately down for connect-id, sco-id and asset-id. Very good. Is there a way to just the display the value 1 time only? It seems
to repeat 2 or 3 times.

I will take this to an output file and to a dashboard for viewing. I think I know how to display that. You have been a great help zenofmud. Thanks again.

33

Thank you Paul. You have been ā€œtremendous helpā€. I thank you very much!!!

1 Like
34

Thank you bakmans2. You were tremendously helpful!!