Trouble Converting JS Object

General
1

Hi, I am having trouble in Node Red converting a JavaScript object to an array or format where I can manipulate the data. Below is what the object looks like, basically a single object "array" with data, where the keys (left hand side) are timestamps that continually change. I am reading these from a JSON file and converting all the info from the file to JS objects. How do I convert this "table" to an array or other format, so I can use the data (right hand side)? I am actually only interested in the last value in the table.

Thanks in advance!
jstou

10‎/‎14‎/‎2019‎ ‎11‎:‎05‎:‎10‎ ‎AMnode: 803a9dad.98efb
msg.payload : Object

object
2019-10-04T10:36:34.4726616-04:00: 16
2019-10-04T10:37:34.4726616-04:00: 17
2019-10-04T10:38:34.4726616-04:00: 16
2019-10-04T10:39:34.4726616-04:00: 24
2019-10-04T10:40:34.4726616-04:00: 7
2019-10-04T10:41:34.4726616-04:00: 14
2019-10-04T10:42:34.4716544-04:00: 27
2019-10-04T10:43:34.4706866-04:00: 27
2019-10-04T10:44:34.4686451-04:00: 18
2019-10-04T10:47:59.9528671-04:00: 29.053
2019-10-04T10:53:05.2162999-04:00: 29.053
2019-10-04T10:54:56.0889368-04:00: 29.053
2019-10-04T10:56:20.0673513-04:00: 29.053
2019-10-04T10:56:46.6303011-04:00: 29.053
2019-10-04T10:57:12.9376103-04:00: 29.053

2

Can you share the bit of the flow where you are doing this part? If you are only interested in the last value, then it would be far more efficient to do that as part of the flow reading the file. This is because a JavaScript Object has no order. To get the 'last' entry from the object, you'd have to get all of its keys (Object.keys(msg.payload)) then sort them to get them into date order and then use that to access the most recent.

It would be far better if you have the data in an array earlier in the flow, straight from the file, to just get the last line of the file.

3

Knolleary, what I have so far is the file input node reading in the JSON file contents, a JSON node converting all the data to JS objects, then I have individual function nodes to manipulate the different data in which I'm interested. In the function node, I tried to read in the last value of the table, but Node Red did not like this, so I have to read in the entire object, which gives me the entire table.

When you say "better if you have the data in an array earlier in the flow", are you saying before converting the JSON file contents into JS objects?

Thanks,
jstou

4

It depends. I asked if you could share the flow so we can see exactly what you are doing. Without knowing what format your data is in, or what you are doing with it, it's hard to give you specific advice.

So, please can you share:

  1. an example of the file in which you are storing the data so we can see what format it is in
  2. an export of the bit of your flow that reads the file and converts it to objects.
5

Ok I have attached the image of the part of the flow where I am reading in the JSON file. Below is the snippet from the JSON showing the "table" that I trying to work with:

{
"Sites": [
{
"PastPowerMeasurements":{"2019-10-04T10:36:34.4726616-04:00":16,"2019-10- 04T10:37:34.4726616-04:00":17,"2019-10-04T10:38:34.4726616-04:00":16,"2019-10-04T10:39:34.4726616-04:00":24,"2019-10-04T10:40:34.4726616-04:00":7,"2019-10-04T10:41:34.4726616-04:00":14,"2019-10-04T10:42:34.4716544-04:00":27,"2019-10-04T10:43:34.4706866-04:00":27,"2019-10-04T10:44:34.4686451-04:00":18,"2019-10-04T10:47:59.9528671-04:00":29.053,"2019-10-04T10:53:05.2162999-04:00":29.053,"2019-10-04T10:54:56.0889368-04:00":29.053,"2019-10-04T10:56:20.0673513-04:00":29.053,"2019-10-04T10:56:46.6303011-04:00":29.053,"2019-10-04T10:57:12.9376103-04:00":29.053}
}
]
}

image

6

I am actually only interested in the last value in the table.

In a function node you could use:

m = msg.payload.Sites[0].PastPowerMeasurements
last = Object.values(m)[Object.values(m).length - 1];
return {payload:last};
7

Technically you shouldn't. Object.values and Object.keys do not make any guarantee as to the order of their results. You should not assume the last entry in their results is the 'last' object property. This is why i said you'd have to use Object.keys to get all the timestamps, then sort it to ensure the right order and only then get the last entry.

1 Like
8

Thanks for your inputs. Let me try this and get back to you.

9

I should read. Learn something every day.

3 Likes
10

Just for reference, if you need a key-value mapping with guaranteed insertion order, there is a Map implementation since ES6. I try to avoid using "plain old JS objects" for dictionary purposes.

1 Like
11

There is a discussion of a (somehow) similar use case in this topic. The final solution for that case was like Nick suggested you now: generate an array from the object, sort the array by timestamp and finally take the last element of the array.

1 Like
12

Is the data you want always in the last line of the file? If so then pick up the complete file as a string, split it into an array by splitting at the end of line character, and take the last line of the array.

13

the file in node can produce multiple messages (one per line) - and they will be tagged with a msg.parts property... the last one should have a msg.parts.count i think (need to check) - but does mark the last one of the file somehow.

14

A switch node configured with the tail option is an elegant solution to filter the last spitted element of an array.

tail-switch-node

[{"id":"7c5f78ae.9c8718","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"13d4cec8.086321","type":"inject","z":"7c5f78ae.9c8718","name":"","topic":"","payload":"[1,2,3,4,5]","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":180,"wires":[["a6d216cd.5a6cf8"]]},{"id":"a6d216cd.5a6cf8","type":"split","z":"7c5f78ae.9c8718","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":340,"y":180,"wires":[["77b37282.c1849c"]]},{"id":"58850c0a.a3ae64","type":"debug","z":"7c5f78ae.9c8718","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":710,"y":180,"wires":[]},{"id":"77b37282.c1849c","type":"switch","z":"7c5f78ae.9c8718","name":"","property":"payload","propertyType":"msg","rules":[{"t":"tail","v":"1","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":510,"y":180,"wires":[["58850c0a.a3ae64"]]}]
1 Like
15

So I found out that the data table/object is already pre-sorted when inserted into the JSON file. So I tried bakman's code and it worked! Thanks for the tip!

Thanks for the other suggestions, I will keep them in mind based on future implementations.

16

I completely agree with Nick -- just because your test of this small dataset seems to work, I would still sort by timestamp and return the latest value...

Here is how I would do it with a JSONata expression:

payload.Sites.PastPowerMeasurements.(
    $latest := $keys()^(>$)[0];
    $lookup($, $latest)
)

And if i paste your data into the Expression tester inside my change node, here's what I get:

image
image873×642 28.5 KB

The key bit is to sort the timestamps descending, and take the first one in the list:
$latest := $keys()^(>$)[0];

Then you can lookup the value that matches that key -- and viola!

4 Likes