HTTP Get for each item in array

General
1

So I have an API that I am requesting JSON data from, and it returns an array or 60 or so site names with descriptions. I can access a single site name like so: msg.payload.data[3].name but what I need to do is the following (written in pseudo code)

for each item in array
msg.url = "http: //apiproviders url/ + msg.payload.data[3].name + "/authcode"

Then when I have the URL built, I send that on with headers to the HTTP Request node and the last step is storing the msg.payload.data[].name with the returned json data and push it into Influxdb.

Do I need to write a function that sets a variable if there is none, gets first array item, constructs the url, increments the variable, returns the msg.url and msg.headers, pipes that into a http request node and then loops back to said function to run the next entry?

Should I be writing each result into influx after the http request and before looping or wait until I build up another array?

I tried handling the data even in pure JSON form from an exec node using my script below but it would split it into two arrays. I'm getting all the data I need back now - just up to the loop part.

Here is what I have so far:

[{"id":"81865ef2.450ea","type":"json","z":"523f91ce.e2d68","name":"","property":"payload","action":"obj","pretty":false,"x":730,"y":100,"wires":[["af28558c.02a278"]]},{"id":"273bc210.ad209e","type":"debug","z":"523f91ce.e2d68","name":"JSON","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":690,"y":240,"wires":[]},{"id":"32296b12.d1ff34","type":"inject","z":"523f91ce.e2d68","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":160,"y":100,"wires":[["8e56f979.9e3248"]]},{"id":"867a1300.70abb","type":"http request","z":"523f91ce.e2d68","name":"Request Xirrus AuthKey","method":"GET","ret":"obj","url":"https://api.cloud.xirrus.com/v1/oauth/token?password=password&grant_type=password&username=username","tls":"","x":530,"y":100,"wires":[["81865ef2.450ea"]]},{"id":"8e56f979.9e3248","type":"function","z":"523f91ce.e2d68","name":"set headers","func":"msg.headers = {};\nmsg.headers['Content-type'] = 'application/json';\nmsg.headers['x-api-key'] = 'somekey';\nreturn msg;","outputs":1,"noerr":0,"x":310,"y":100,"wires":[["867a1300.70abb"]]},{"id":"f6db19de.a820f8","type":"http request","z":"523f91ce.e2d68","name":"Request Profiles","method":"GET","ret":"obj","url":"http s://api.cloud.xirrus.com/v1/rest/api/profiles.json","tls":"","x":500,"y":180,"wires":[["7780aa81.cb24d4"]]},{"id":"af28558c.02a278","type":"function","z":"523f91ce.e2d68","name":"set headers","func":"global.set(\"auth_key\",msg.payload.token_type + \" \" + msg.payload.access_token);\nmsg.headers = {};\nmsg.headers['Content-type'] = 'application/json';\nmsg.headers['x-api-key'] = 'somekey';\nmsg.headers['Authorization'] = global.get(\"auth_key\");\nreturn msg;","outputs":1,"noerr":0,"x":310,"y":180,"wires":[["f6db19de.a820f8"]]},{"id":"7780aa81.cb24d4","type":"json","z":"523f91ce.e2d68","name":"","property":"payload","action":"obj","pretty":false,"x":650,"y":180,"wires":[["bd3a5c9b.f293a"]]},{"id":"7e6e536e.54792c","type":"http request","z":"523f91ce.e2d68","name":"","method":"GET","ret":"txt","url":"","tls":"","x":490,"y":240,"wires":[["273bc210.ad209e"]]},{"id":"bd3a5c9b.f293a","type":"function","z":"523f91ce.e2d68","name":"Get 1 site APs","func":"\nmsg.url = {};\nmsg.url = \"http s://api.cloud.xirrus.com/v1/rest/api/profiles.json/\" + msg.payload.data[3].name + \"/accesspoints/statuses\";\nmsg.headers = {};\nmsg.headers['Content-type'] = 'application/json';\nmsg.headers['x-api-key'] = 'somekey';\nmsg.headers['Authorization'] = global.get(\"auth_key\");\nreturn msg;","outputs":1,"noerr":0,"x":320,"y":240,"wires":[["7e6e536e.54792c"]]}]

Here is my python script to give you an idea of how im doing it in python:

import requests
import json
header ={
        "content-Type": "application/json",
        "x-api-key": "somekey"
        }
api_response = requests.get("htt ps://api.cloud.xirrus.com/v1/oauth/token?password=xirruspwd&grant_type=password&username=ourusername", headers = header)
api_set = json.dumps(api_response.json())
auth_key = (json.loads(api_set)['access_token'])
xms_url = "ht tps://api.cloud.xirrus.com/v1/rest/api/profiles.json"
header ={
        "content-Type": "application/json",
        "x-api-key": "somekey",
        "Authorization": "bearer "+auth_key
        }
response = requests.get(xms_url, headers = header)
#print(response.json())

profiles = json.dumps(response.json()) 
profile_list = []
try:
    decoded = json.loads(profiles)
    # Access data
    for x in decoded['data']:
       profile_list.append(x['name'])
 
except (ValueError, KeyError, TypeError):
    print "JSON format error"

for site in profile_list:
    xms_status_url = "ht tps://api.cloud.xirrus.com/v1/rest/api/profiles.json/"+site+"/accesspoints/statuses"
    header ={
        "Content-Type": "application/json",
        "x-api-key": "somekey",
        "Authorization": "bearer "+auth_key
        }
    ap_count = requests.get(xms_status_url, headers = header)
    ap_status = json.dumps(ap_count.json())
    ap_array = json.loads(ap_status)
    #print site, ":", ap_array["upCount"], "/", ap_array["totalCount"], "Online"
    print site, ":", ap_array["downCount"], "down." , ap_array["upCount"], "/" , ap_array["totalCount"], "Online" , ap_array["adminOfflineCount"], " Under maintenance."
2

I would start by passing the message through a Change node configured to Move msg.payload.data to msg.payload so the array is in the payload, then you can use a Split node to split the array into a sequence of messages, one for each element of the array. Then you can do what you want with each one.

3

I get two errors with that approach:

"TypeError: this is not a typed array."

"TypeError: Cannot read property '0' of undefined"

4

Once you pass your array through the split node, you don't need the 'msg.payload[0]' array syntax since each msg will have a single payload value. If you put a debug node after the split node you should see each one in turn -- hovering the mouse over any value in the debug sidebar show two little icons that you can use to copy either the value itself or the "path" to the value.

If you can post the flow as JSON (use the Menu -> Export -> Clipboard -> Selected Nodes) surrounded by a line with 3-backtics ``` then we can try to import it and give you some better feedback.

5

ok, the Move topic seems to have worked, I now get the names as payload[1].name

How do I split the data, im not getting any output from the split node?

6

So I found a way to split the array out to separate outputs from a function node - But I'm not a fan of this as the function outputs don't scale to match the payload array length.

    return {payload: p};
});
return splitsites; ```

I'd much rather run a loop and and for each loop it runs it spits out the payload[i].name and I can rate limit the output. Then I can run the rest of the query and log the result to the database before starting on the next payload[i].name.
7

This was my solution with a rate limiter on the output.

for (i = 1; i < msg.payload.length; i=i+1) {
    var sitename = RED.util.cloneMessage(msg);
    sitename.payload = msg.payload[i].name;
    node.send(sitename);
}
return null; ```
8

What was wrong with using the split node. Does that not give you individual messages for each item in the array?

9

Didn't work for me sorry.

10

What, you mean the Split node did not split the array into individual messages? If you passed it an array and it did not split it (assuming it was configured appropriately) then you have found a serious bug that needs fixing.

11

It looked ok as far as what I was passing it, but I may not have known how to configure the split node. I'll have a closer look tomorrow.

The code above worked, but skips the first entry.. which in our case is a template entry so that's fine.

12

if i starts at 1 then it will indeed skip the first entry (as arrays are indexed from 0) - but as that is what you want then what you have is probably the best way to do it anyway - as split would not skip it :slight_smile: