JSON objects not importing into influxDB

Hi All,

Super new here and new to node red (and influxDB and all coding as well) so sorry if this issue is simple.

firstly I'm using node red and influx DB set up on a raspberry pi zero 2 in Docker containers.

My data is coming from the new Pimoroni indoor, it packages up sensor data and other board data in JSON objects like this.

17/11/2022, 15:31:11node: debug 3
enviro/indoor01 : msg.payload : Object
object
readings: object
pressure: 981.38
temperature: 24.02
voltage: 4.975
color_temperature: 5990
gas_resistance: 88215
aqi: 13.5
humidity: 52.48
luminance: 53
nickname: "indoor01"
model: "indoor"
uid: "e6614c311b37ab37"
timestamp: "2022-11-17T15:30:19Z"

From what I read about influx, it should be able to accept this type of data, but it's not going anywhere.

my flow at the moment looks like this. a simple in and out

apologies if there's anything missing.

Thanks

James

Welcome to the forum @Jimdowding

If you look at the help text you will see that this is not a valid structure for the data. You are trying to save an object called readings to the database, but it can only store simple data, not objects. If you added a Change node to Move msg.payload.readings To msg.payload then it should work. Note though, that if you the data is for the current time then there is no need to save the time, so I suggest removing that (which you can also do in the change node). In addition I suspect that one or more of the string values should go to influx tags rather than fields (so that they are indexed and can be used for selecting data efficiently).

Colin,

Thank you so very much for your quick and knowledgeable response. it appears to have worked!! hurray! my data is getting through to influx DB and now showing some of the data on grafana.

I thought multiple objects were allowed in influx DB because of this

If msg.payload is an array containing two objects, the first object will be written as the set of named fields, the second is the set of named tags.

as for the data. this is being logged every 15 mins on the board and then every 5 logs are going to be uploaded so the time stamp is going to be necessary.

is there a way to include that?

do I need to include some other function.

Thank you again.

James

Possibly that could have been better phrased. It means that the members of the first object are written as the fields of the record, and the members of the second object are written as the tags of the record. Probably that is what you should be doing, with nickname and model as tags. Possibly also the uid, it depends what it is. If it changes each time then it should be a field, if it is fixed with the sensor then probably it should be a tag.

To include the timestamp it should be written a field called time. I think it will cope with an ISO timestamp so it may just be a matter of changing it from timestamp to time.

Can't you send it immediately rather than cacheing it and sending it later?

The most efficient way to write multiple records is to use the influx batch node, which will do all five at once. However it is a bit fiddly to setup and since you are dealing with such a small amount of data the efficiency is irrelevant, so I suggest carrying on the way you are, just moving the timestamp value to time.

Hi @Colin,

Thanks very much for your response.

And apologies, for some of the info you mention I don't quite understand.

I've moved the reading object to the payload, but the items in the 'outside' object like time, how do I move that to be included within the reading in the payload.

The reason for logged and delayed uploads is because it's a simple board and is being powered by some AAA batteries, so fewer uploads means longer battery life.

Thanks and once again, apologies for not knowing anything.

James

You have to be careful not to overwrite the existing data in the payload before you have go all the data. One way is, move the data to a temporary property, msg.data, before overwriting the payload, so in the Change node do something like:

Move msg.payload.readings To msg.data
Set msg.data.time To msg.payload.timestamp
Move msg.data To msg.payload

Feed that into a debug node to make sure it looks right before connection to the database node.