[ASK] Read Datalog using Node red

General
1

Hii Guys

I would like to read the datalog of my machine. The datalog is using CSV and I just want to read the last line.
Im facing some problem to do that.

  1. My datalog filename is changing over weeks. By example it now is week 23 2018 the filename is “Log1823” file name is changing over time.

  2. I would like to check my latest line using fastCSV. and i still try to explore it.

My code as follows :

[{"id":"9e324255.c4cc3","type":"fastcsv","z":"9e7f522.6e5d5b","name":"fastcsv","headers":true,"headerstr":"","ignoreEmpty":true,"discardUnmappedColumns":true,"strictColumnHandling":false,"delimiter":",","quote":"\"","escape":"\"","comment":"#","ltrim":false,"rtrim":false,"rowDelimiter":"\\n","includeEndRowDelimiter":false,"quoteHeaders":false,"quoteColumns":false,"x":780,"y":240,"wires":[[]]},{"id":"4ff34b14.fbd8a4","type":"inject","z":"9e7f522.6e5d5b","name":"","topic":"","payload":"true","payloadType":"bool","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":220,"wires":[["a9e0d871.9c4be8"]]},{"id":"6f6ec532.c1b0ec","type":"file in","z":"9e7f522.6e5d5b","name":"","filename":"","format":"utf8","chunk":false,"sendError":false,"x":510,"y":280,"wires":[["9e324255.c4cc3","d330cc78.0c9b"]]},{"id":"a9e0d871.9c4be8","type":"function","z":"9e7f522.6e5d5b","name":"FilenameCoordinator","func":"msg.payload = \"D:\\log_rotary_common_tester_1743.csv\"\nreturn msg;","outputs":1,"noerr":0,"x":330,"y":140,"wires":[["cb25d4d1.cbbe98"]]},{"id":"d330cc78.0c9b","type":"debug","z":"9e7f522.6e5d5b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":740,"y":140,"wires":[]},{"id":"cb25d4d1.cbbe98","type":"change","z":"9e7f522.6e5d5b","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"filename","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":300,"y":300,"wires":[["6f6ec532.c1b0ec"]]}]

Thanks in advance

Really appreciate your kind help to fix my problem. :smiley:

2

Here is some code that you can use to determine the week of the year:


Date.prototype.getWeek = function() {
  var onejan = new Date(this.getFullYear(),0,1);
  return Math.ceil((((this - onejan) / 86400000) + onejan.getDay()+1)/7);
}
var today = new Date();
var week_of_year = today.getWeek();
msg.payload = week_of_year;
return msg;

So you should be able to build the file name getting the last two digits of the year and concatinating that with "Log " and the year and week.

3

FYI, you can also generate the filename using the moment node – inject the current timestamp, using the format string \LogYYww, and voila…

[{
    "id": "975e4454.2e6378",
    "type": "moment",
    "z": "f05a42da.00364",
    "name": "LogYYww",
    "topic": "",
    "input": "",
    "inputType": "msg",
    "inTz": "America/New_York",
    "adjAmount": 0,
    "adjType": "days",
    "adjDir": "add",
    "format": "\\LogYYww",
    "locale": "en_US",
    "output": "",
    "outputType": "msg",
    "outTz": "America/New_York",
    "x": 340,
    "y": 380,
    "wires": [
        [
            "6f64e755.1ce9e8"
        ]
    ]
},
{
    "id": "38599fdc.793ab",
    "type": "inject",
    "z": "f05a42da.00364",
    "name": "",
    "topic": "",
    "payload": "",
    "payloadType": "date",
    "repeat": "",
    "crontab": "",
    "once": false,
    "onceDelay": 0.1,
    "x": 160,
    "y": 380,
    "wires": [
        [
            "975e4454.2e6378"
        ]
    ]
},
{
    "id": "6f64e755.1ce9e8",
    "type": "debug",
    "z": "f05a42da.00364",
    "name": "",
    "active": true,
    "tosidebar": true,
    "console": false,
    "tostatus": false,
    "complete": "false",
    "x": 530,
    "y": 380,
    "wires": []
}]
1 Like
4

@shrickus - man I hate it when I forget to look for a node that already exists that will solve the problem :roll_eyes:

5

Hey, there are always multiple solutions, and yours does not require any 3rd party nodes to be installed. Is that the formula used to calculate the ISO week, or something else? Seems like that should be part of the Date object itself…

But whenever I need to fiddle with date arithmetic, calculate timespans, or output human readable data/time formats, my go-to library is “moment”. It’s the only way I know of to reliably handle the vagaries of leap years, time zones, daylight savings time, and random leap second adjustments.

1 Like
6

Once you have determined the log file name (using some of the suggestions above), the simplest way I can think of to get the last line is to use the unix tail -1 LogYYww command.

Put this into an exec node (unless you are on windows) and it will instantly retrieve just the last line of text -- no csv parsing required (fast, or otherwise):

image

Then just send the complete path to the log file in your msg.payload -- the output payload should be the text string you need.

Alternately, instead of building the log file name and polling for the latest data, take a look at using the watch node (wfwatch on windows) to look for changes to the log directory -- whenever a new file arrives, you will get a msg with the filename already in it.