Max number of elements in array

[hazymat]

I have a sensor (Shelly EM) which returns various readings over MQTT about power usage every 30 seconds, and I'm sending this to InfluxDB through Node-RED. I have Grafana hooked up and I'm able to visualise the data nicely:

Despite the above, I would still like to be able to view the raw data in a CSV file.

So in Node-RED I have an InfluxDB input node with the following query: select * from hallway_power; connected to a function node which takes the UNIX timestamp and adds a readable time into each array element in the message. Here's a screenshot of that message, to give you an idea:

As you can see, I'm writing the file to power.csv, and the CSV node is converting from an array of objects into a CSV file, then the Dropbox node allows me to open that up immediately on my laptop.

The data in Influx has 13730 rows, but the CSV file is showing only 755 rows.

It gets a bit difficult trying to troubleshoot this stuff, because the debug node only shows the first 999 elements in the array, and the output of the CSV node truncates after a short amount of text (although in debug it shows as msg.payload : string[1439082], so maybe that's a clue?)

What limit am I hitting up against here?

[hazymat]

Being a bit dim, I didn't think to check the data that was saved to the file on my Node-RED server... and it does indeed have all the rows of data.

So it looks like the Dropbox node is failing to upload the whole file somehow. I have tested this a couple of times, and each time it is just uploading the first 755 rows. Strange!

Wonder if anyone else has come across a similar issue with Dropbox node?

[bakman2]

The debug output shows max 999 elements in an array. The data is there.
When uploading the file, has it been fully written/closed before it is uploading ? You could try a delay node and wait a couple of seconds before uploading.