NodeRed Json to Influxdb

General
1

hi everyone,
first post first question :slightly_smiling_face:
i have a little flow witch grab data from a website and should push it to an influx db.
in my debug i get a nice arry with all objects.
now i try to oush it to the influx db and the frist array item [0] write the way i want it to do .
But for all the rest i get the following error:

Error: A 400 Bad Request error occurred: {"error":"partial write: unable to parse '"JSON Informations ARRAY[1]': missing tag value dropped=0"}

Hope someone can help me with it.
Thanks and kind Regards
Matthias

2

Perhaps if you post the flow itself would be easier to help.

3

Also add a debug node showing what you are sending to the influx node and post a screenshot here.

4

ok you are right here we go:

[{"id":"e029be7b.c526c","type":"inject","z":"8c89de7c.7bfdf","d":true,"name":"","topic":"","payload":"connected","payloadType":"str","repeat":"43200","crontab":"","once":false,"onceDelay":0.1,"x":150,"y":460,"wires":[["c94d88eb.db1c58"]]},{"id":"9a90be2b.616c2","type":"influxdb out","z":"8c89de7c.7bfdf","influxdb":"623085fd.8e411c","name":"corona","measurement":"corona","precision":"n","retentionPolicy":"","x":1030,"y":480,"wires":[]},{"id":"2816a775.689038","type":"debug","z":"8c89de7c.7bfdf","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":1010,"y":600,"wires":[]},{"id":"c94d88eb.db1c58","type":"http request","z":"8c89de7c.7bfdf","name":"","method":"GET","ret":"txt","paytoqs":false,"url":"https://coronavirus-19-api.herokuapp.com/countries","tls":"","persist":false,"proxy":"","authType":"","x":350,"y":580,"wires":[["b8a45152.12aef"]]},{"id":"7b467edd.061eb","type":"json","z":"8c89de7c.7bfdf","name":"","property":"payload","action":"","pretty":false,"x":730,"y":580,"wires":[["2816a775.689038","9a90be2b.616c2"]]},{"id":"cf1dfabe.9d1f58","type":"inject","z":"8c89de7c.7bfdf","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":600,"wires":[["c94d88eb.db1c58"]]},{"id":"b8a45152.12aef","type":"function","z":"8c89de7c.7bfdf","name":"decode","func":"msg.payload = msg.payload.toString('utf8');\nreturn msg;","outputs":1,"noerr":0,"x":550,"y":580,"wires":[["7b467edd.061eb"]]},{"id":"623085fd.8e411c","type":"influxdb","z":"","hostname":"influxdb","port":"8086","protocol":"http","database":"corona2","name":"influx_corona","usetls":false,"tls":""}]

And the screenshot:
image
the flow:

image
image1198×297 17.1 KB

hope this helps

5

We need to see what the debug node is showing, if it is showing objects then expand them enough to see what is there. Once you have that then check the info panel for the influx node and check that the data matches what the node requires. Looking at the error message I suspect it is not.

6

here it is.

image
image338×995 19.4 KB

where i found the information about the data format for influx?

7

Click the node and look in the Info pane on the right hand side (click the 'i' at the top).

8

ok found it. but i guess i´m too stupid. i don´t get it. to less programming background.
sorry

9

I can see basically the information you are trying to write, but I don't know how you have designed your database of course. A couple of questions:

  1. Are all those values to be associated with a timestamp of 'now'?
  2. Which of those values are fields and which are tags? I could guess that country is a tag and the rest are fields but I don't know. FirstCase isn't associated with 'now' so I am not sure what you want to do with that.
10

hi i don´t design it i just want to use it.
i like to use the soruce:
https://coronavirus-19-api.herokuapp.com/countries
as a json source i use the http request to grab thes json and i like to put it to influxdb.

11

You can't just throw data at a database and hope it will do something useful, you need to know how you want to organise the data in the database and then pass it the data in the correct way.

12

if i just use one country it worked only if i want to insert all i get the issue.
i don´t know how i should explain it. put the first insert worked (first part of arry object 0) but all other failed.

13

If you just insert one object then it writes data (though not necessarily the way you want it) to the database because, as it says in the node Info
" If msg.payload is an object containing multiple properties, the fields will be written to the measurement."
So it has written each item in the object as a field in the database with the current timestamp. So, using the info from the debug you posted it will have recorded that at that time the field country had the value "USA" and the field cases has the value 215300 and so on. If, for example, you were then to post the next object then it would record that the field country now has the value "Italy" and cases now has the value 110574. The result is that in the database the country has changed from USA to Italy and also the value cases has changed from 215300 to 110574. Note that it assumes that all these are independent. There is no linking between the country and cases at each time. If you want that linkage you will have to understand how influx databases work and in particular understand the difference between fields and tags.
The reason you get the error is because, again from the info tab, the only other ways of passing data are as an array of two objects, which yours isn't , or an array of arrays, which again it isn't.
So, as I indicated before, if you want to use an Influx database then you need to do a bit of research and at least understand the basics of how to use it.

14

ok thanks, i thougt it will be easyer because if i only write one country it writes the country name in a own table as you can see here in the table:

image
image1510×324 6.96 KB

and my thinking was if there are user countrys it only changes the contry and than all the rest values. like i did it in another way (but the data source is broken):
image
image1803×528 43.8 KB

15

There are many sources for the covid-19 data. I use a different source, example flow:

[{"id":"1a337dbc.75e662","type":"inject","z":"8866261d.bd07b","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":168,"y":336,"wires":[["66b84d5.35bbab4"]]},{"id":"66b84d5.35bbab4","type":"http request","z":"8866261d.bd07b","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://pomber.github.io/covid19/timeseries.json","tls":"","persist":false,"proxy":"","authType":"","x":255,"y":264,"wires":[["8f6fa79f.7d74a"]],"l":false},{"id":"8f6fa79f.7d74a","type":"function","z":"8866261d.bd07b","name":"","func":"m = msg.payload\n\narr = []\ndata = Object.entries(m);\ndata.forEach(([key, value]) => {\n    \n    country = key\n   \n    for (x = 0; x < value.length; x++) {\n        udate = m[key][x].date.split(\"-\")\n        if (udate[1] < 10) {\n            udate[1] = \"0\" + udate[1]\n        }\n        if (udate[2] < 10) {\n            udate[2] = \"0\" + udate[2]\n        }\n        ndate = udate[0] + \"-\" + udate[1] + \"-\" + udate[2]\n        timestamp = new Date(ndate)\n        confirmed = m[key][x].confirmed\n        deaths = m[key][x].deaths\n        recovered = m[key][x].recovered\n        arr.push({\n            measurement: \"corona\",\n            fields: {\n                location: country,\n                confirmed: confirmed,\n                deaths: deaths,\n                recovered: recovered\n            },\n            tags: {\n                country: country\n            },\n            timestamp: timestamp\n        });\n    }\n})\nreturn {\n    payload: arr\n}","outputs":1,"noerr":0,"x":303,"y":264,"wires":[["1d958c90.4853eb","8c6dff58.bc8f58"]],"l":false},{"id":"1d958c90.4853eb","type":"debug","z":"8866261d.bd07b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":410,"y":240,"wires":[]}]

Look at the function node, the part in arr.push, it creates a suitable array for influx (as influx needs fields, tags and a timestamp), connect that to a influx-batch node and it will be inserted.

1 Like
16

Influx db doesn't have tables. It has measurements which are rather similar, but they aren't the same as tables.

17

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.