Storing JSON formatted values in a file and retrieving them

[beetlezap]

I am getting stuck with an extremely simple use-case and I really do not understand why it is failing.

This is where i store the data:

This is the output stored in the file:

}{
    "name": "surf-John Doe",
    "fname": "John",
    "lname": "Doe",
    "replyto": 1990,
    "chatId": -402899591,
    "messageId": 1994,
    "spot": "kvsk",
    "start": "2021-03-10T09:57:59.513Z",
    "stop": "2021-03-10T09:58:07.463Z",
    "sail": "1",
    "rating": "7"
}{
    "name": "surf-JaneDoe",
    "fname": "Jane",
    "lname": "Doe",
    "replyto": 1996,
    "chatId": -402899591,
    "messageId": 2000,
    "spot": "mossby",
    "start": "2021-03-10T09:58:11.605Z",
    "stop": "2021-03-10T09:58:20.559Z",
    "sail": "3",
    "rating": "8"
}

When i then try to read this file (here is the flow):

It doesn't matter how i try to fix formatting (adding comma between entries, removing spaces, EOL, NL characters) the JSON parser outputs an error (on a position in the file) and fails. I suspect that the JSON parser may not be able to manage nested objects and need one to one between entries in file and object created.

Anyone got any insights since I am stuck

//Beetle

[Steve-Mcl]

@beetlezap

This is the output stored in the file

Is that the complete file? It starts with a } that is not valid JSON.

As an array, it should be [{},{},{}]

Ps, there are loads of online json validator web pages (Google is your friend)

[beetlezap]

Thanks Steve ! I just realized that after I read some more posts here. I do not really like to pre-process a file and would like it to be "JSON" formatted on write..problem is that the file-out node only can do append omitting both [ and ] and the JSON node formats output for one object (omitting the comma that is needed to separate objects). So i think my question was the wrong one...how can i easily store a series of objects in JSON format in a file which i easily can read and convert to an object array for further processing ?

[dceejay]

as you note they are just written to the file one line at a time... so to handle that - set the file in node to also read one line at a time - then pass to the json node to retrieve each "line" of information

[beetlezap]

Thanks...but sort of removes the need for JSON. Then it is likely easier to go with a Database-solution using SQLITE (where I can query the db for the records I am interested in)...

[beetlezap]

I have this working using SQLITE but think it is a clumsy and ugly solution

[dceejay]

Well we don't know what your actual overall goal is - so can't really comment if there is a better way.

[beetlezap]

Thanks ! My overall goal is to have a "logbook" stored that contains name, place, start time and stop time. In parallell i have InfluxDB storing a set of weather parameters connected to "place". I want (on user input) collect all entries related to that user from "logbook" and for each entry lookup weather parameters and report on that back to user (either UI or short Telegram message). I hope that made some sense at least

[Colin]

The place should be a tag in the influx weather records, so then you can query the influx data for records for a particular place. Possibly that is what you have already.
Why are you not storing the logbook data in influx too?

[beetlezap]

Maybe i had a flaw in my thinking since I only considered Influx for measurements and not key/value store. I looked to influx due to the ease of getting measurements in/out based on time buckets. Maybe it can store the "logbook" as well..

[beetlezap]

And yes, you are right. The measurements in influx are tagged to "station-id" and "station-name"

[beetlezap]

SOLVED: Store object directly in MongoDB. Retrieve objects without JSON translation.