Playing with timestamps

#1

I'm working on a flow that is chopping up a data feed and pushing it into influxDB to use for visual monitoring and alerts. For my previous uses, I had simply been generating a timestamp at the time the message is sent out using timestamp: new Date(). I think this could be done better, though, since the data feed I'm getting is just chock full of time stamp information. Example below:

MSH|^~&|REDACTED|REDACTED|||20180912102253||ORU^R01|20180912102253373722288|P|2.3|||AL|NE|||||
PID|||||||||||||||||||||||||||
PV1|||DEVICEREDACTED|||||||||||||||||||||||||||||||||||||||||||||||
OBR|1||||||20180912102253|||DEVICEREDACTED||||||||||||20180912102253|||F|||||||||||
OBX|1|NM|HR|DEVICE|61|bpm|||||F|||20180912102253|||||
OBX|2|ST|0100-1006|DEVICE|0.0|mm|||||F|||20180912102253|||||
OBX|3|ST|0100-1008|DEVICE|0.0|mm|||||F|||20180912102253|||||
OBX|4|ST|TEMPC|DEVICE|35.6|C|||||F|||20180912102253|||||
OBX|5|NM|SPO2|DEVICE|99|%|||||F|||20180912102253|||||
OBX|6|NM|PR|DEVICE|61|bpm|||||F|||20180912102253|||||
OBX|7|NM|CVP|DEVICE|6|mmHg|||||F|||20180912102253|||||
OBX|8|NM|NBPM|DEVICE|85|mmHg|||||F|||20180912102253|||||
OBX|9|NM|ARTSBP|DEVICE|135|mmHg|||||F|||20180912102253|||||
OBX|10|NM|ARTDBP|DEVICE|56|mmHg|||||F|||20180912102253|||||

I was considering just grabbing the first timestamp (20180912102253) as a string, inserting the symbols necessary to make that into rfc3339 format, running the result through a function to subtract the time differential (because these are UTC-5, not zulu) and calling it good but honestly that feels kind of like a kludge.

Is there a more elegant way to grab that timestamp that anyone would recommend?

0 Likes

#2

Keep the timestamps always as they are for calculations. Format them to be human readable only for display purpose.

0 Likes

#3

Jay, InfluxDB generates it's own timestamp when you insert data.

0 Likes

#4

Well if I'm using them for real time alerting they need to be in zulu time, not UTC-5. That's half my problem: the system that sends me the data is using local time in near-human-readable format as a convention. If I can get it into accurate Z time, the database will automatically convert it to epoch anyway.

0 Likes

#5

InfluxDB generates it's own timestamp when you insert data.

Unless you specify a time for your measurement. One of the upstream systems sometimes hangs (in a store and forward failure mode.) I'm trying to correct for that for historical monitoring purposes (so if they send me 16000 messages in 30 seconds, they're not all saved in the DB as being measured in that 30 second window, but rather as their actual time of measurement.)

0 Likes

#6

Fair enough, I'd forgotten that you can specify a Time column!

0 Likes

#7

The automatic stamping is one of my favorite features for quick data gathering expeditions (hey intern! carry this laptop around the hospital, walk by every room, and put the room number in this text entry field for each one you visit! [meanwhile the flow is constant pinging the piece of medical equipment on the cart with their laptop along with the laptop itself so we can validate the new gear isn't some sort of special wifi snowflake.])

1 Like

#8

You'll want those local datetime "strings" to be real Utc JS dates, so you don't lose any accuracy when they get inserted into InfluxDb. One way would be to use a regex in a function node to split the string into the date parts, like so:

> var arr = "20180912102253".match(/(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})/).map((d,i) => +d)
[ 20180912102253, 2018, 9, 12, 10, 22, 53 ]

Then you can use this integer array to create a UTC date using the six-arg constructor:

> new Date(arr[1], arr[2]-1, arr[3], arr[4], arr[5], arr[6])
2018-09-12T14:22:53.000Z

Of course, the date will look different if you are not in the EDT timezone like I am... but it should be correct. If not, you may be better off using node-red-contrib-moment to adjust for the local timezone and any DST calculations -- which are notoriously hard to code by hand and still cover all the edge cases.

0 Likes

#9

Just a followup (which I forgot to do.) Wound up resorting to using the substring function to chop apart the date string and inserted characters to convert it into rfc3339 format and then used the moment node (friggin' love that thing) to do the timezone correction to get it into zulu time. Function node code as follows:

var month =  msg.payload.toString('month').substr(4, 2);
var day =  msg.payload.toString('day').substr(6, 2);
var hour =  msg.payload.toString('hour').substr(8, 2);
var minute =  msg.payload.toString('minute').substr(10, 2);
var second =  msg.payload.toString('second').substr(12, 2);
msg.payload = (year + "-" + month + "-" + day + "T" + hour + ":" + minute + ":" + second);
return msg;```
0 Likes

#10

I'm not sure what that toString(...) method is trying to do, since the 'month' argument is being ignored. That being said (and just to provide another alternative for others later), you could also use the string replace method with a regex to convert your particular input to rfc3339 format using this js code:

var fmt = /(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2}).*/;
msg.payload = msg.payload.replace(fmt, "$1-$2-$3T$4:$5:$6Z");
return msg;
0 Likes