Callback function to handle http request

Developing Nodes
1

Hi,

I am writing a node, which purpose is to forward its input via a HTTP POST request.

I use the node module 'better-sqlite3' to sample the input data, with the following columns:

ID, timestamp, feedname, value, uploadStatus

The uploadStatus column is updated, with a value indicating that the row has been uploaded correctly via the HTTP POST request.

Here I use the 'request module' for making the HTTP POST request.

My problem is, when I inject my node with data, everything is fine, except for if I inject some more data rapidly after each other. This results in that the last HTTP POST request callback function was not finished.

        node.on('input', function(msg) {    
        
        var time = new Date();
        value = msg.payload;
        timestamp = `${time.getUTCFullYear()}-${time.getUTCMonth()+1}-${time.getUTCDate()} ${time.getUTCHours()}:${time.getUTCMinutes()}:${time.getUTCSeconds()}`;
        uploadStatus = 0;
        uploadCount++;

        //Insert data into sql database, ROW_ID will increment by 1 for each datainput.
        sqliteInsertValue(tableName, timestamp, feedname, value, uploadStatus);
        
        postData();
        });

The scenario is that, when the HTTP POST request is ongoing, it is possible to inject some more data, which means the SQL database is newer, than the current one. I want that data to be sent as well. Does anybody have an idea, how to handle that?

2

Not all that clear how you want things to work. But a couple of ideas spring to mind.

  • Change the db so it isn't relying on the autoincrement row_id but uses a fine-grained timestamp as its index. So that you no longer have to worry about clashing rows.
  • Implement your db writes as an API with caching so that the relatively slow db write no longer matters.
1 Like