How do you group by a certain key value

General
1

I'm working on a flow that pulls down weather alerts from the NWS API and i'm trying to figure out how to group alerts. So, for instance, if there has been 3 severe weather alerts (initial alert and 2 alerts extending the expiration date), i want to group those, sort them, and pull the most recent by expiration date. I can do this in an exec node with powershell and pass the JSON back to Node-Red. But was wondering if there is a way to do it natively in Node-Red with either a function or a module.

Here is one that has multiple alerts of the same event type ("Winter Weather Advisory") but with different "expires" dates.

https://api.weather.gov/alerts/active?zone=AKC068

2

What have you tried so far? Can you show us your flow?
Also what version of NR are you running?

This sounds like a useful project!

3

When I need to restructure some JSON data, I reach for a change node using a JSONata expression -- here is one that returns a single object with the latest timestamp for each event type:

image
image1311Ă—813 102 KB

The expression syntax is complex but powerful (it's like XSLT's xpath syntax, but for JSON data):

payload.features.properties^(>expires)  {
    event: [expires][0]
}

In a nutshell, it says:

  • Get all the payload.features.properties objects,
  • sort them descending by expires
  • build a single object with each event name as the field...
  • and the first event expires as the value
1 Like
4

Here is the flow i'm working with that has @shrickus solution he provided.

[{"id":"5076ebd24b9b3623","type":"tab","label":"Flow 1","disabled":false,"info":"","env":[]},{"id":"f9d47c5d2adf7166","type":"http request","z":"5076ebd24b9b3623","name":"Query Weather Service","method":"GET","ret":"obj","paytoqs":"ignore","url":"https://api.weather.gov/alerts/active?zone=AKC090","tls":"","persist":false,"proxy":"","authType":"","senderr":false,"x":350,"y":200,"wires":[["bd2418ee33282d0e"]]},{"id":"6968d276d07c1eac","type":"inject","z":"5076ebd24b9b3623","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":150,"y":200,"wires":[["f9d47c5d2adf7166"]]},{"id":"e549e8f7651e34d6","type":"comment","z":"5076ebd24b9b3623","name":"NWS Weather Alert Testing","info":"","x":190,"y":140,"wires":[]},{"id":"9be345f2b7a5cddc","type":"debug","z":"5076ebd24b9b3623","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":810,"y":200,"wires":[]},{"id":"bd2418ee33282d0e","type":"change","z":"5076ebd24b9b3623","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.features^(>expires)  {\t    event: [expires][0]\t}\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":200,"wires":[["9be345f2b7a5cddc"]]}]

Running v2.1.4 of Node-Red

5

How would I get the entire JSON branch for that? I'm not as well versed in this syntax as i would like to be.

6

also, just for some reference, here is my PowerShell code i call in an exec node:

$alerts = $null

$alerts = (Invoke-WebRequest -Uri "https://api.weather.gov/alerts/active?zone=AKC090" | ConvertFrom-Json).features.properties | Group-Object Event

if ($alerts -eq $null) 
{
    #do nothing, there are not alerts    
}



foreach ($alert in $alerts)
{
    if ($alert.Count -gt 1)
    {
        ($alert.Group | Sort-Object Expires)[-1] | ConvertTo-Json
    }
    else
    {
       $alert.Group | ConvertTo-Jsone
    }
}

my goal is to produce a string from the JSON like this that i can send to TTS and play out of my sonos speakers and send to Pushover.

The National Weather Service out of Fairbanks has issued a Moderate Winter Weather Advisory for Deltana and Tanana Flats effective until Thursday, December 23rd, 2021 09:00:00 AM

7

Two things:

  1. in your flow the change node uses
payload.features^(>expires)  {
    event: [expires][0]
}

while @shrickus example's is:

payload.features.properties^(>expires)  {
    event: [expires][0]
}

(note the missing'property')
2) you need to check if msg.payload.features is empty or the change node will get this error:

"Invalid JSONata expression: Key in object structure must evaluate to a string; got: undefined"

So add a switch node after the http request node to check that there is data to process. Here is the flow with the switch node added:

[{"id":"fef5cc626098b342","type":"http request","z":"d94067cbd6def60f","name":"Query Weather Service","method":"GET","ret":"obj","paytoqs":"ignore","url":"https://api.weather.gov/alerts/active?zone=AKC090","tls":"","persist":false,"proxy":"","authType":"","senderr":false,"x":200,"y":200,"wires":[["0e8bc8b3af167e81"]]},{"id":"68b324e2ec702aeb","type":"inject","z":"d94067cbd6def60f","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":160,"y":140,"wires":[["fef5cc626098b342"]]},{"id":"c31c2e258eeca7ad","type":"comment","z":"d94067cbd6def60f","name":"NWS Weather Alert Testing","info":"","x":190,"y":80,"wires":[]},{"id":"b17e4b9c323eccae","type":"debug","z":"d94067cbd6def60f","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":750,"y":200,"wires":[]},{"id":"ea48d5d589bcb58e","type":"change","z":"d94067cbd6def60f","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.features","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":560,"y":180,"wires":[["b17e4b9c323eccae"]]},{"id":"0e8bc8b3af167e81","type":"switch","z":"d94067cbd6def60f","name":"","property":"payload.features","propertyType":"msg","rules":[{"t":"nempty"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":390,"y":200,"wires":[["ea48d5d589bcb58e"],["00d92f5cc9208133"]]},{"id":"00d92f5cc9208133","type":"change","z":"d94067cbd6def60f","name":"no alerts","rules":[{"t":"set","p":"payload","pt":"msg","to":"No current alerts","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":540,"y":220,"wires":[["b17e4b9c323eccae"]]}]
8

Yeah, I was tinkering around with the code trying to get it to display the full announcements and forgot to add it back. Sorry about that.

1 Like
9

I was wondering if that was what you really wanted ;*)

It looks like the powershell script sorts the list of objects, and just returns the latest one -- is that right? If so, the expression would change to be:

(payload.features^(>properties.expires) {
    properties.event: [$][0]
}).*

Conceptually, it finds all features, sorts them desc by expires, groups by event, and takes the first one from each group. Is this closer to what you need?

1 Like
10

Yes sir! That's exactly what I'm looking for. Now that i see the code it makes total sense.

Thank you for taking the time to help out with this!! I do appreciate it!

1 Like
11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.