How to combine specific payload that didn't go to join node at the same time?

I create a Raspberry Pi status report on twitter. Such as Temperature, CPU, RAM and Disk. I trigger to get data from raspberry pi specific period for each value. Temperature,CPU Load,Memory usage trigger value every 10 seconds. But Disk usage trigger every 10 minutes. I need to join all of the value and make a sentece for report status on twitter. But when I join them. Because of different time of trigger action. Value of Temperature CPU&RAM go to join node 2 times and make the report on twitter without Disk value in sentence. How to make join node waiting for specific payload to prevent join node get same payload and combine them. Thanks!

[{"id":"5833bee0.58dfd","type":"function","z":"8d8ea68a.944cc8","name":"Index setting (Disk)","func":"msg.payload = {\n    \"disk\": msg.payload\n}\nreturn msg;","outputs":1,"noerr":0,"x":1250,"y":800,"wires":[["8b1cd29d.d34ab"]]},{"id":"e67b186.a181968","type":"inject","z":"8d8ea68a.944cc8","name":"Trigger every 10 sec","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":"","x":524.5,"y":548.25,"wires":[["863c6bee.e7758","e87ed1cb.21e148","ecc0eb24.c8724"]]},{"id":"28415a81.538026","type":"inject","z":"8d8ea68a.944cc8","name":"Trigger every 10 min","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":"","x":510,"y":780,"wires":[["d3ad4fa1.b4f79"]]},{"id":"d3ad4fa1.b4f79","type":"exec","z":"8d8ea68a.944cc8","command":"df -h","addpay":false,"append":"","useSpawn":"","timer":"","oldrc":false,"name":"Disk Usage","x":710,"y":780,"wires":[["e216e72a.da5db"],[],[]]},{"id":"ecc0eb24.c8724","type":"exec","z":"8d8ea68a.944cc8","command":"free | grep Mem | awk '{printf \"%2.1f\",$4/($2 /100.0)}'","addpay":false,"append":"","useSpawn":"","timer":"","oldrc":false,"name":"Memory Usage","x":710,"y":700,"wires":[["7ffa53be.b43374"],[],[]]},{"id":"e87ed1cb.21e148","type":"exec","z":"8d8ea68a.944cc8","command":"top -d 0.5 -b -n2 | grep \"Cpu(s)\"|tail -n 1 | awk '{print $2 + $4}'","addpay":false,"append":"","useSpawn":"","timer":"","oldrc":false,"name":"CPU Load","x":710,"y":620,"wires":[["bf725c4.50728a"],[],[]]},{"id":"863c6bee.e7758","type":"exec","z":"8d8ea68a.944cc8","command":"vcgencmd measure_temp","addpay":false,"append":"","useSpawn":"","timer":"","oldrc":false,"name":"RPi Temp.","x":710,"y":540,"wires":[["c85e2120.1b0788"],[],[]]},{"id":"c85e2120.1b0788","type":"function","z":"8d8ea68a.944cc8","name":"","func":"str = msg.payload\nmsg.payload = str.substring(5,9);\nreturn msg;","outputs":1,"noerr":0,"x":890,"y":540,"wires":[["3a46b4cf.5e5a6c"]]},{"id":"3a46b4cf.5e5a6c","type":"function","z":"8d8ea68a.944cc8","name":"Index setting (Temp)","func":"var i = msg.payload\nmsg.payload = {\n    \"temp\": i\n}\nreturn msg;","outputs":1,"noerr":0,"x":1240,"y":580,"wires":[["794bc4ac.dd0bd4"]]},{"id":"e216e72a.da5db","type":"function","z":"8d8ea68a.944cc8","name":"","func":"var re = /([0-9]{2})%/\nvar idx = msg.payload.search(re);\nvar str = msg.payload;\nif (idx >=0) {\n    str = msg.payload.substring(idx, idx + 2);\n}\nmsg.payload = str;\nreturn msg;","outputs":1,"noerr":0,"x":890,"y":780,"wires":[["5833bee0.58dfd"]]},{"id":"7ffa53be.b43374","type":"function","z":"8d8ea68a.944cc8","name":"Index setting (RAM)","func":"var i = msg.payload\nmsg.payload = {\n    \"ram\": i\n}\nreturn msg;","outputs":1,"noerr":0,"x":1240,"y":700,"wires":[["794bc4ac.dd0bd4"]]},{"id":"bf725c4.50728a","type":"function","z":"8d8ea68a.944cc8","name":"Index setting (CPU)","func":"var i = msg.payload\nmsg.payload = {\n    \"cpu\": i\n}\nreturn msg;","outputs":1,"noerr":0,"x":1240,"y":620,"wires":[["794bc4ac.dd0bd4"]]},{"id":"794bc4ac.dd0bd4","type":"join","z":"8d8ea68a.944cc8","name":"","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"3","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":1460,"y":620,"wires":[["8b1cd29d.d34ab"]]},{"id":"8b1cd29d.d34ab","type":"join","z":"8d8ea68a.944cc8","name":"","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":1630,"y":680,"wires":[["c619f4ba.e4bf5"]]},{"id":"c619f4ba.e4bf5","type":"function","z":"8d8ea68a.944cc8","name":"","func":"var t = msg.payload[0][0].temp\nvar c = msg.payload[0][2].cpu\nvar r = msg.payload[0][1].ram\nvar d = msg.payload[1].disk\nvar dt = Date(Date.now()); \nmsg.payload = \"My Home Pi temperature is \" + t +\"°C. CPU usage is \" + c + \" %. RAM usage is \" + r + \" %. Disk usage \" + d + \" % at \" + dt;\nreturn msg;","outputs":1,"noerr":0,"x":1790,"y":680,"wires":[["6f98fe78.c7c548"]]},{"id":"6f98fe78.c7c548","type":"delay","z":"8d8ea68a.944cc8","name":"Delay for TWEET!!","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"hour","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":2030,"y":680,"wires":[[]]}]

So you are sending a status tweet every 10 seconds to twitter ? Why? (I wonder why twitter even allows this, but ok...) does not sound like the ultimate platform for a system status (suggestion: look into telegram bots for example).

You could solve this by using flow context, actually you could use this for all the items listed.

1 Like

See example:

[{"id":"6dd6c73d.71c72","type":"inject","z":"c6151fb4.afb1","name":"Trigger every 10 sec","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":"","x":156,"y":176,"wires":[["499e25e9.3dd614","bb724b6b.3c5318","db37b727.f0aa6"]]},{"id":"973d38db.e8a2b","type":"inject","z":"c6151fb4.afb1","name":"Trigger every 10 min","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":"","x":155.5,"y":383.75,"wires":[["87f2ead6.d47cd"]]},{"id":"87f2ead6.d47cd","type":"exec","z":"c6151fb4.afb1","command":"df -h","addpay":false,"append":"","useSpawn":"","timer":"","oldrc":false,"name":"Disk Usage","x":355.5,"y":383.75,"wires":[["caefdf61.6cf06"],[],[]]},{"id":"db37b727.f0aa6","type":"exec","z":"c6151fb4.afb1","command":"free | grep Mem | awk '{printf \"%2.1f\",$4/($2 /100.0)}'","addpay":false,"append":"","useSpawn":"","timer":"","oldrc":false,"name":"Memory Usage","x":365.5,"y":295.75,"wires":[["9046a4ce.7f1298"],[],[]]},{"id":"bb724b6b.3c5318","type":"exec","z":"c6151fb4.afb1","command":"top -d 0.5 -b -n2 | grep \"Cpu(s)\"|tail -n 1 | awk '{print $2 + $4}'","addpay":false,"append":"","useSpawn":"","timer":"","oldrc":false,"name":"CPU Load","x":355.5,"y":229.75,"wires":[["37ecbfef.a5d16"],[],[]]},{"id":"499e25e9.3dd614","type":"exec","z":"c6151fb4.afb1","command":"vcgencmd measure_temp","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"RPi Temp.","x":355.5,"y":163.75,"wires":[["d90519af.1e8ab8"],[],[]]},{"id":"d90519af.1e8ab8","type":"function","z":"c6151fb4.afb1","name":"temp","func":"str = msg.payload\nflow.set(\"temp\",str.substring(5,9))","outputs":1,"noerr":0,"x":512,"y":154,"wires":[[]]},{"id":"caefdf61.6cf06","type":"function","z":"c6151fb4.afb1","name":"disk","func":"var re = /([0-9]{2})%/\nvar idx = msg.payload.search(re);\nvar str = msg.payload;\nif (idx >=0) {\n    str = msg.payload.substring(idx, idx + 2);\n}\nmsg.payload = str;\nflow.set(\"disk\",str)\n","outputs":1,"noerr":0,"x":512,"y":374,"wires":[[]]},{"id":"9046a4ce.7f1298","type":"function","z":"c6151fb4.afb1","name":"ram","func":"flow.set(\"ram\",msg.payload)","outputs":1,"noerr":0,"x":512,"y":286,"wires":[[]]},{"id":"37ecbfef.a5d16","type":"function","z":"c6151fb4.afb1","name":"cpu","func":"flow.set(\"cpu\",msg.payload)","outputs":1,"noerr":0,"x":512,"y":220,"wires":[[]]},{"id":"b8017fe7.bb1c8","type":"function","z":"c6151fb4.afb1","name":"send tweet","func":"var t = flow.get(\"temp\")\nvar c = flow.get(\"cpu\")\nvar r = flow.get(\"ram\")\nvar d = flow.get(\"disk\")\nvar dt = Date(Date.now()); \nmsg.payload = \"My Home Pi temperature is \" + t +\"°C. CPU usage is \" + c + \" %. RAM usage is \" + r + \" %. Disk usage \" + d + \" % at \" + dt;\nreturn msg;","outputs":1,"noerr":0,"x":356,"y":462,"wires":[[]]},{"id":"f7d1bb30.ef5bd8","type":"inject","z":"c6151fb4.afb1","name":"","topic":"","payload":"","payloadType":"date","repeat":"300","crontab":"","once":false,"onceDelay":0.1,"x":158,"y":462,"wires":[["b8017fe7.bb1c8"]]}]
1 Like

Twitter doesn’t allow it. You would hit their API limit within an hour

I forget to say that. Inside delay for TWEET! node. I use rate limit 1msg/hour. But now flow context help me a lot. I don't need that node anymore.Thanks :slight_smile:

But if you send once an hour, why run the exec nodes every 10 seconds ?

I need to because I use exec node together with dashboard that need to show real-time status

1 Like

Twitter has several kinds of rate limits, which for user levels depend on the endpoint you're using. For example, for lookups on user search I'm able to utilise endpoints (in parallel, for improved indexing) where some allow 15, others 180, yet another endpoint up to 900 per 15 minute window. For posting tweets/retweeting there's a 3 hour window with at most 300 tweets. That gives you about 1 tweet every 2 minutes, if you use that full block evenly.

Sounds like you could have a unique msg.topic for each input source msg -- connect those to a join node set to merge all inputs into a single object, keyed on the topic, and set to emit the updated object everytime one of the values is updated:

4 Likes

Never used msg.topic before. I will try that. Thanks :slight_smile: