Help with formatting Moment Node Output

General
1

Guys,

Taking a reading in through MQTT and trying to reformat the time into Sydney Australia format with date and time

DD/MM/YY HH:MM (AM/PM Format)

Here is the debug output of the MQTT in

image

The first one is with no formatting - just msg.payload, the 2nd one i am trying various combinations of moment and do not seem to be getting anywhere - any knowledgable people able to take a look please ?

[{"id":"17c3e857.ecfd48","type":"mqtt in","z":"65c8a789.964be8","name":"Last Update Time","topic":"blueriiot/pool_b6744e50-f056-44d7-85d6-e085cacab537/last_measurement/state","qos":"2","datatype":"utf8","broker":"8c5be0a1.64904","x":280,"y":180,"wires":[["7b716ec.710949","a2f8f465.3bc6d8"]]},{"id":"7b716ec.710949","type":"moment","z":"65c8a789.964be8","name":"Format Time","topic":"","input":"","inputType":"msg","inTz":"Australia/Sydney","adjAmount":0,"adjType":"days","adjDir":"add","format":"","locale":"en_AU","output":"","outputType":"msg","outTz":"Australia/Sydney","x":530,"y":200,"wires":[["9720f50f.10e618"]]},{"id":"9720f50f.10e618","type":"debug","z":"65c8a789.964be8","name":"Last Update Time","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":750,"y":180,"wires":[]},{"id":"a2f8f465.3bc6d8","type":"debug","z":"65c8a789.964be8","name":"Last Update Time (no moment)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":790,"y":140,"wires":[]},{"id":"8c5be0a1.64904","type":"mqtt-broker","z":"","name":"MQTT-On-Dev","broker":"172.16.100.36","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]

Thanks

regards

Craig

2

This is the closest i have gotten - BUT - it is not returning the right year - is this me or could this be a bug ??

[{"id":"17c3e857.ecfd48","type":"mqtt in","z":"65c8a789.964be8","name":"Last Update Time","topic":"blueriiot/pool_b6744e50-f056-44d7-85d6-e085cacab537/last_measurement/state","qos":"2","datatype":"utf8","broker":"8c5be0a1.64904","x":280,"y":180,"wires":[["7b716ec.710949","a2f8f465.3bc6d8"]]},{"id":"7b716ec.710949","type":"moment","z":"65c8a789.964be8","name":"Format Time","topic":"","input":"","inputType":"msg","inTz":"ETC/GMT","adjAmount":0,"adjType":"days","adjDir":"add","format":"DD-MM-YY hh:mm","locale":"en_AU","output":"","outputType":"msg","outTz":"Australia/Sydney","x":530,"y":200,"wires":[["9720f50f.10e618"]]},{"id":"9720f50f.10e618","type":"debug","z":"65c8a789.964be8","name":"Last Update Time","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":750,"y":180,"wires":[]},{"id":"a2f8f465.3bc6d8","type":"debug","z":"65c8a789.964be8","name":"Last Update Time (no moment)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":790,"y":140,"wires":[]},{"id":"8c5be0a1.64904","type":"mqtt-broker","z":"","name":"MQTT-On-Dev","broker":"172.16.100.36","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]

image

thanks

Craig

3

Hmm - the plot thickens !!

I wonder if this node needs an update to incorporate a new range of years ?

I will attempt to update now

@TotallyInformation - Julian ?

Craig

4

Yep that did it - update to V4 of Julians moment node and all good

regards

Craig

1 Like
5

I have to say that the days are numbered for this node. MomentJS is no longer being developed and so will need to be deprecated at some point. We need to find a new library to use. Currently, I'm not sure that it is worth redoing this node or whether it would be better to create something new and then eventually remove this one.

6

Hi @TotallyInformation.

Just my 2p worth. I have found the need for small date-time manipulation functions to be more useful than this type of all encompassing library but I suppose those with coding skills just write their own date manipulation library so it just depends what audience you are addressing.

I am not dissing MomentJS at all, just too big for Pi zero projects that do not need to know the angle of the sun at any place and point in time.

7

I agree that MomentJS is far too big and bulky which is one of the reasons it has come to the end of its life.

However, writing date/time functions is fraught with edge-cases - I know this all too well, my first experience with writing professional date/time manipulations was back in the 1980's!

I would never recommend anyone take that on even if they are experienced. Of course, really simple things like a single fixed format of a date or a time is OK. But as soon as you need to think about DST and timezones and date/time calculations, you need a robust library and these are rarely light weight because of the sheer complexity of the calculations.

The best approach for all systems is to maintain date/times in UTC either as a JS Date object or ISO8201 format. Only convert to local format at the point of entry/visibility to humans. That at least minimises complexity.

Oh, and by the way, if you have Node-RED running on your Pi-Zero, you already have the MomentJS module loaded as it is now included in JSONata. One of the advantages of using Node.js is that the module will only be loaded once. So adding the contrib-moment node to a flow should no longer make much difference. :wink:

8

NR on Pi-Zero is tight and red-lines cpu for ages before it catches up.

Is there a way of removing it ?

A debate on SO about npm prune is not clear to me.
https://stackoverflow.com/questions/21417014/npm-command-to-uninstall-or-prune-unused-packages-in-node-js

Possibly getting off topic.

9

If you don't use the moment node (or it's equivalent in JSONata) then it won't make any difference to cpu usage once node-red is started. When exactly are you seeing it red lining the cpu? I have a pi Z running node-red 1.2.6 and it takes just 20 seconds to restart node-red and for the cpu to stabilise at about 6% for node-red.

10

Hi @Colin,
Thank you I did not know that.

I suppose it depends on how you measure 'red-lining'.
image

[{"id":"b8f4b5e7.03f338","type":"exec","z":"d603eefa.f994d","command":"top -d 0.5 -b -n1","addpay":false,"append":"","useSpawn":"","timer":"","name":"CPU Load","x":350,"y":120,"wires":[["8281f436.dbb8d8"],[],[]]},{"id":"8281f436.dbb8d8","type":"function","z":"d603eefa.f994d","name":"top","func":"var msg1 = RED.util.cloneMessage(msg);\nvar msg2 = RED.util.cloneMessage(msg);\nvar msg3 = RED.util.cloneMessage(msg);\nvar msg4 = RED.util.cloneMessage(msg);\n//var msg5 = RED.util.cloneMessage(msg);\nvar str = msg.payload;\nvar a = str.split('\\n');\n//if (idx >=0) {\n//    str = msg.payload.substring(idx, idx + 2);\n//}\nvar s = (a[0]).split('average:');\nvar ut = (s[0]).split(',');\nvar la = (s[1]).split(',');\n//msg1.payload = (ut[0]).split('top - ')[1];\nmsg2.payload = parseInt(100*parseFloat(la[0]),10);\nmsg3.payload = parseFloat(la[1]);\nmsg4.payload = parseFloat(la[2]);\nmsg2.topic = \"load-average-1m\";\nmsg3.topic = \"load-average-5m\";\nmsg4.topic = \"load-average-15m\";\nreturn [msg, msg1, msg2, msg3, msg4];","outputs":5,"noerr":0,"x":530,"y":140,"wires":[["bbb8d03f.96001"],["c859b06d.34894"],["c859b06d.34894","11ca7103.39bd9f"],["c859b06d.34894"],["c859b06d.34894"]]},{"id":"bf8dacb5.81164","type":"inject","z":"d603eefa.f994d","name":"","topic":"","payload":"","payloadType":"date","repeat":"5","crontab":"","once":false,"onceDelay":0.1,"x":210,"y":120,"wires":[["b8f4b5e7.03f338"]]},{"id":"bbb8d03f.96001","type":"ui_chart","z":"d603eefa.f994d","name":"usage","group":"e4e233a8.1f2d","order":2,"width":"4","height":"3","label":"","chartType":"line","legend":"false","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"0","ymax":"100","removeOlder":"24","removeOlderPoints":"1440","removeOlderUnit":"3600","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"x":690,"y":180,"wires":[[]]},{"id":"c859b06d.34894","type":"debug","z":"d603eefa.f994d","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":540,"y":400,"wires":[]},{"id":"11ca7103.39bd9f","type":"ui_gauge","z":"d603eefa.f994d","name":"","group":"e4e233a8.1f2d","order":1,"width":"4","height":"3","gtype":"gage","title":"CPU Usage","label":"CPU","format":"{{value}} %","min":0,"max":"100","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":710,"y":140,"wires":[]},{"id":"e4e233a8.1f2d","type":"ui_group","z":"","name":"System","tab":"fb63826f.aa9b7","order":4,"disp":true,"width":"8","collapse":true},{"id":"fb63826f.aa9b7","type":"ui_tab","z":"","name":"Hydroponics","icon":"dashboard","order":1,"disabled":false,"hidden":false}]

and of course what is in your flows. One device eventually rests at 60% but it is controlling and monitoring water quality.

A deploy keeps the cpu backlog well over 120% for a minute or so.

Never profiled the execution (if that can be done ?) though.

P.S. I found that exec nodes calling python script to be the worst offenders when doing instrumentation stuff.

11

indeed - calling out to a standalone runtime means that environment has to get stood up, executed and shut down. If doing that regularly then it would be better to either make the process a daemon that you can interrogate, or to make it standalone process that reports via something like MQTT, as that way it can use a separate processor thread more efficiently.

1 Like
12

Thank you. I do exactly that with pigpiod and scripted real-time hardware timer sampling/output.

I just commented for anyone else who had not realised the massive overhead of that approach.

I think it is amazing that NR runs so smoothly on a Pi Z.

P.S. At the risk of going off topic, is there an established way of profiling a flow for performance bottlenecks ?

13

Nothing sensible.

The issue is that it isn't an unused module, the module is in use by Node-RED. In fact it is in use in a core node so certainly not easily removed.

You really are getting a double hit there. Since Node.js has to stand up a BASH environment and then Python kicks in and that is itself a bit of a monster.

You are actually probably better setting up a Python server that will listen for commands and return data to Node-RED.

14

Oh wow - i use this pretty extensively - i suppose i better start looking around for an alternative.

If moment.js is no longer being developed is there a viable replacement ?

Craig

15

Well they recommend something on their website but I've not had any time to investigate options.

16

So you would not be rolling out a new NR wrapper for it ?

regards

Craig

17

Only if I get some time to do something. Its rather in short supply at the moment I'm afraid.

1 Like
18

Totally understand - will do some research on the options.

regards

Craig

19

As I see it, the issue with moving on from moment.js is the risk of loosing some of the corporate knowledge of the development team. They recognize that its technology is obsolete, but migrating all
the library's capabilities to a new one won't be trivial.

I have no experience with any of the databases usually discussed in the forum, so I don't know how they create or use timestamps. However, I do have lots of experience planning and executing government and military operations world-wide, so here is a non-software perspective.

ISO 8601 format (YYYY-MM-DD), absolutely. UTC, not so much. Local time with offset can be preferable, since it converts instantly into UTC and contains additional information about the source of the data.

An important distinction is that for past events conversion from UTC to local time is perfectly well-defined, provided one has an accurate timezone database with complete historical information.

Scheduling future events is a completely different matter. For an action to be taken simultaneously at multiple locations, use UTC. For an action synchronized with human activities, use local time. For astronomical events, name the event. I can discuss the reasons for this advice, but they are mostly obvious.

20

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.