Reading a FIT file, and FitSDK

Good afternoon,
another idea, another issue. I would like to explore some possibility using *.fit files.
The actual fit files cannot be read by a human being. Fit files uses XML structure, so I intend to uses the both nodes: "read file" and "XML"

However, using notepad++, and despite using encoding UTF-8, i get:

"ZX .FIT`3@ Œ†„ „ À†$ º÷d6 W @ ý† „Œ„„ º÷d6 W À†$ „ @ ý† † º÷d6 @ ý† ……„†† ¼÷d6¤‡$Î@Èÿ¸À À ½÷d6K‡$Û?Èÿd ¤6 Æ÷d6‚$§0Èÿ†Ø t, É÷d6à$30ÈÿL$ L- Ï÷d6$$Ü)Èÿæ "

Through my research I have found that I would need to work with the FitSDK. It seems quite difficult for me to understand how to use it. So I wonder if someone has already digged into it with node-red

Sylvano
note: my fit file works pretty fine, and I could upload it on strava.

If you know how to require a library you could import this and use a function node

Any tutorial to recommend ? :smiley:
(in both linux and windows)

You dont really need a tutorial

also, this lib might be better - https://www.npmjs.com/package/fit-file-parser

  • install the lib to you node-red folder

    • cd ~/.node-red
    • npm install fit-file-parser
  • add a reference to it in your settings.js file

      functionGlobalContext: { // enables and pre-populates the context.global variable
          // -- Pass in Libraries for convenience in function nodes -- //
          'fit-file-parser' : require('fit-file-parser'),
          'fs' : require('fs')
      },
    
  • access it in your function...


// Require the modules
var FitParser = global.get("fit-file-parser").default;
var fs = global.get("fs");
 
// Read a .FIT file
var fs = require('fs');
fs.readFile('./example.fit', function (err, content) {
 
  // Create a FitParser instance (options argument is optional)
  var fitParser = new FitParser({
    force: true,
    speedUnit: 'km/h',
    lengthUnit: 'km',
    temperatureUnit: 'kelvin',
    elapsedRecordField: true,
    mode: 'cascade',
  });
  
  // Parse your file
  fitParser.parse(content, function (error, data) {
    // Handle result of parse method
    if (error) {
      node.error(error, msg);
    } else {
      //send the data out in the payload
      node.send( { payload: data } );
    }
  });
  
});

NOTE: this is untested - but should get you close or push you in the right direction.

Thanks Steve, i am trying hard but without success for now.
I have an issue with the readFile property.
to be continued

So I have followed as much as possible all your recommendations, and I am getting the error when deploying: "TypeError: Cannot read property 'on' of null"

I have minimized the node function to a single line, but got the same error:
var fs = global.get('fs')
Not that my setting files is:

functionGlobalContext: {
		// -- Pass in Libraries for convenience in function nodes -- // 
		'fs' : require('fs'),
		'path' : require('path'),
		'fit-file-parser' : require('fit-file-parser'),
    },

Reading through the forum, some people mentionned that fs is part of nodeJS so we do not need to install it. any idea about the rigin of this error?

Sylvano

Correct, fs is part of nodejs so no need to install that.

This is strange.

Can you post your startup log please and the path to your settings file? (Sometimes the wrong settings file is edited)

The path to my settings.js file is:
C:\Users\Sylvain\.node-red

Here is the copy-paste of the Cmd window (Windows system)

> Microsoft Windows [Version 6.3.9600]
> (c) 2013 Microsoft Corporation. All rights reserved.
> 
> C:\Windows\System32>node-red
> 5 Jul 23:20:58 - [info]
> 
> Welcome to Node-RED
> ===================
> 
> 5 Jul 23:20:58 - [info] Node-RED version: v1.1.0
> 5 Jul 23:20:58 - [info] Node.js  version: v12.13.1
> 5 Jul 23:20:58 - [info] Windows_NT 6.3.9600 x64 LE
> 5 Jul 23:20:59 - [info] Loading palette nodes
> 5 Jul 23:21:00 - [info] Dashboard version 2.19.3 started at /ui
> 5 Jul 23:21:00 - [info] Settings file  : \Users\Sylvain\.node-red\settings.js
> 5 Jul 23:21:00 - [info] Context store  : 'default' [module=memory]
> 5 Jul 23:21:00 - [info] User directory : \Users\Sylvain\.node-red
> 5 Jul 23:21:00 - [warn] Projects disabled : editorTheme.projects.enabled=false
> 5 Jul 23:21:00 - [info] Flows file     : \Users\Sylvain\.node-red\flows_Desktop.json
> 5 Jul 23:21:00 - [info] Server now running at http://127.0.0.1:1880/
> 5 Jul 23:21:00 - [warn]
> 
> ---------------------------------------------------------------------
> Your flow credentials file is encrypted using a system-generated key.
> 
> If the system-generated key is lost for any reason, your credentials
> file will not be recoverable, you will have to delete it and re-enter
> your credentials.
> 
> You should set your own key using the 'credentialSecret' option in
> your settings file. Node-RED will then re-encrypt your credentials
> file using your chosen key the next time you deploy a change.
> ---------------------------------------------------------------------
> 
> 5 Jul 23:21:00 - [info] Starting flows
> 5 Jul 23:21:00 - [error] [arduino-board:4837b82.05ffd48] TypeError: Cannot read
> property 'on' of null
> 5 Jul 23:21:00 - [info] Started flows
> 5 Jul 23:21:08 - [info] Stopping flows
> 5 Jul 23:21:08 - [info] Stopped flows
> 5 Jul 23:21:08 - [info] Starting flows
> 5 Jul 23:21:08 - [error] [arduino-board:4837b82.05ffd48] TypeError: Cannot read
> property 'on' of null
> 5 Jul 23:21:08 - [info] Started flows
> 5 Jul 23:32:59 - [info] Stopping flows
> 5 Jul 23:32:59 - [info] Stopped flows
> 5 Jul 23:32:59 - [info] Starting flows
> 5 Jul 23:32:59 - [error] [arduino-board:4837b82.05ffd48] TypeError: Cannot read
> property 'on' of null
> 5 Jul 23:32:59 - [info] Started flows

I tried the same with my RPi3, and the error is not showing up while deploying.

What is this node that is causing the error? Is that the function node with the global.get("fs") inside?

PS, you can use node-red search (CTRL+F) to find 4837b82.05ffd48 or arduino-board

Thank you Steve, it seemed not to work at the beginning, but the error was cleared after a full restart of my PC. Now back to the fit-file-parser. :slight_smile:
Sylvain

Here is what I have achieved so far, and it works quite well.
I have added a callback function to use the "try/catch" as per the tutorial.

However, I am missing much of the output data.
I believe this is due to the fit-file-parser module. what do you think?

[{"id":"93f01531.a7adf8","type":"tab","label":"Flow 7","disabled":false,"info":""},{"id":"49c02ec2.bed66","type":"fs-ops-access","z":"93f01531.a7adf8","name":"","path":"path","pathType":"msg","filename":"file","filenameType":"msg","read":true,"write":true,"throwerror":true,"x":490,"y":100,"wires":[["755cf720.9aafa8"],[]]},{"id":"94916ef8.2bb0d","type":"inject","z":"93f01531.a7adf8","name":"file (right path)","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"true","payloadType":"bool","x":180,"y":100,"wires":[["aa5f27ce.152ed8"]]},{"id":"aa5f27ce.152ed8","type":"function","z":"93f01531.a7adf8","name":"right path","func":"\nmsg.path = 'C:\\\\Users\\\\Sylvain\\\\Documents\\\\LiNK for Windows\\\\Exported Files\\\\20200607-4x500m\\\\'\nmsg.file = 'Sylvain Boyer 20200607 1030am.fit'\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":340,"y":100,"wires":[["49c02ec2.bed66"]]},{"id":"755cf720.9aafa8","type":"function","z":"93f01531.a7adf8","name":"Fit File Parser (NK file)","func":"// import modules\nvar FitParser = global.get(\"fit-file-parser\").default;\nvar fs = global.get('fs')\n\n// check for settings\nif ( (typeof fs === 'undefined')  ) {\n    node.error('fs module could not be found in global context')\n    return    \n}\nif ( (typeof FitParser === 'undefined') ) {\n    node.error('FitParser module could not be found in global context')\n    return    \n}\n// And make sure they are functions\nif ( (typeof fs !== 'object') ) {\n    node.error('fs from global context is not an object & it must be')\n    return    \n}\n\n\n//import the filepath + filename\nvar file = msg.path.concat(msg.file);\nnode.status(file);\n\n// No calback to handle error: risk of nod-red crashing\n/*fs.readFile(file, function (err, content) {\n  // Create a FitParser instance (options argument is optional)\n  var fitParser = new FitParser({\n    force: true,\n    speedUnit: 'km/h',\n    lengthUnit: 'km',\n    temperatureUnit: 'kelvin',\n    elapsedRecordField: true,\n    mode: 'cascade',\n    //mode:'list',\n  });\n  // Parse your file\n  fitParser.parse(content, function (error, data) {\n    // Handle result of parse method\n    if (error) {\n      console.log(error);\n    } else {\n      console.log(JSON.stringify(data));\n      msg.payload = JSON.stringify(data);\n      node.send( msg );\n    }\n  });\n});*/\n\n// Testing a callback fucntion to handle errors and prevent node-red crash\nfs.readFile(file,callback)\nfunction callback(err, data) {\n    if (err) {\n        node.error('Oops! there is no data: ' + folder, err)\n        return\n    }\n    try {\n        data\n            var fitParser = new FitParser({\n                //put the options here \n            });\n            fitParser.parse(data, function (error, data) {\n                // Handle result of parse method\n                if (error) {\n                  console.log(error);\n                } else {\n                  console.log(JSON.stringify(data));    //by defaukt in the console\n                  msg.payload = JSON.stringify(data);   //build the msg payload + output from the node\n                  node.send( msg );\n                }\n              });\n    } \n    catch (err) {\n        node.error('Ouch! Something went badly wrong processing the data', err)\n        return\n    }\n}","outputs":1,"noerr":0,"initialize":"","finalize":"","x":680,"y":80,"wires":[["d9824cd.203d5b"]]},{"id":"f182bb72.76931","type":"debug","z":"93f01531.a7adf8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":990,"y":80,"wires":[]},{"id":"d9824cd.203d5b","type":"json","z":"93f01531.a7adf8","name":"","property":"payload","action":"","pretty":false,"x":850,"y":80,"wires":[["f182bb72.76931"]]}]

Potentially yes. I don't personally know the file format, nor do I have a file to test or what to expect out of said file.

I was pushing you towards a potential solution (i.e. using global context and require modules).

I would suggest if you are certain the issue is with the parser, I hope the author would appreciate you raising an issue on the GitHub repo - providing the Dev a test file and details of the device that created the file.

Then hopefully it'd get fixed for all to benefit from. Win win.


If anyone else on the forum is familiar with fit file format and has the means to debug the issue, please chip in.

i will resubmit the flow with 2 fit files: the example.fit and my own fit file (with its translation in CSV).
i had a look at the fit.js file and it seems that the names are fixer. So as an example you won't see the Power.
i will raise the issue on GitHub.

The other thing is that "JS is not on the roadmap for the SDK" as per a busy thread on thisisant.com .

There are alternatives such as build a C executable and call it from Node-Red (I would need to learn all of this).
(The SDK provides some C++ code as well as Java, from my readings C++ was found to be slow while C u was found robust).

As promise, here is the flow + files.

(the forum does not let me to upload .fit file)

A quick feedback,

Fit-File-parser is doing the job. The manufacturer of my device confimed there were only exporting some data to the FIT file, while exporting all data to the CSV file.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.