How to use node-red-contrib-machine-learning-v2 (NEED HELP AND GUIDANCE FOR UNI PROJECTS)

puteramelaka · 15 October 2021 20:31

Hello dear friends, im currently collecting accelerometer data for vibration analysis for rotating machinery...the data is send from MQTT to node-red, after asking in this forum i found out there is machine learning in node red but not sure either its still working or not...since im trying the ftt nodes but its already unuseable nodes(due to error and no update from the owner).

the machine learning nodes(v2) -- node-red-contrib-machine-learning-v2 (node) - Node-RED

there are many options but it doesnt have example ..i need help to build one machine-learning for my data

the flow of example given

the debug shows error ,but theN i change the MQTT server and this output obtained

what im trying to do is the data i obtained will be pass through the machine learning nodes( not sure how),, im using the data for vibration on 1k rpm machine ..the only thing i want to detect is either the machine is healthy or defects

i notice there is training module,,maybe for training the machine learning and make it more accurate,but not sure how to use it..

for example, the machine is 10hz and suddenly drop to 6hz,,so it will send email/sms or alert about abnormalities

does anybody have use this machine-learning nodes or have experience/knowledge please help out im really clueless

Hopperpop · 15 October 2021 21:58

Do you know how any of these classifiers work? Which one of these do you want to use to solve your problem?

If you can provide your plan (pseudo code) on how you will solving the problem, you will find more help here on how to do it in node-red. But if you need help on how and which model to use, you start better of at the website of the underlying libraries that are used in the node (ex. scikit).

puteramelaka · 16 October 2021 12:26

well hello there sir hopperpop,
i not really sure but, what im thinking is
decision tree classifier trainer -- use to train and collect data from dataset provided
decision tree classifier predictor --use the data from decision tree classifier trainer to make prediction

i will try to find there thanks

Hopperpop · 16 October 2021 15:11

What are the features from your dataset you want to use to do the classification? And do you have a collection of labeled datasets that you can use to train the model?

Note: I haven't used any machine learning methods myself. But if you can prove that a decision tree classifier can work for this problem. Than I'm all ears.

puteramelaka · 16 October 2021 15:43

the thing that i want to classify is healthy or not healthy..i do have data set and can create more datasets,and probably obtain more datasets,thats not a problem i guess

im not sure its work or not ,since im also asking for help in this forum,since the last node-red update ,many nodes are not compatible/error...im trying to create one..if u are good with node-red i guess u can try it out also..

here are example flows,,do update the result when u try it out

[
    {
        "id": "da8ca300.2dfe6",
        "type": "create dataset",
        "z": "21ce826.2ff977e",
        "name": "",
        "path": "test/iris.data",
        "saveFolder": "test/datasets",
        "saveName": "iris",
        "input": "0,1,2,3",
        "output": "4",
        "trainingPartition": "",
        "shuffle": true,
        "seed": "",
        "x": 340,
        "y": 80,
        "wires": [
            [
                "4fb0a8dc.f6baf8"
            ]
        ]
    },
    {
        "id": "44b6f4b0.34d7dc",
        "type": "load dataset",
        "z": "21ce826.2ff977e",
        "name": "",
        "datasetFolder": "test/datasets",
        "datasetName": "iris",
        "partition": "train.csv",
        "input": true,
        "output": true,
        "x": 290,
        "y": 200,
        "wires": [
            [
                "26110acb.cbf526"
            ],
            [
                "86385870.9f6b88"
            ]
        ]
    },
    {
        "id": "4f7cc53d.87a22c",
        "type": "inject",
        "z": "21ce826.2ff977e",
        "name": "start",
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 110,
        "y": 80,
        "wires": [
            [
                "da8ca300.2dfe6"
            ]
        ]
    },
    {
        "id": "d3e9e7ab.a06d68",
        "type": "inject",
        "z": "21ce826.2ff977e",
        "name": "start",
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 110,
        "y": 200,
        "wires": [
            [
                "44b6f4b0.34d7dc"
            ]
        ]
    },
    {
        "id": "b21982e2.99cf1",
        "type": "inject",
        "z": "21ce826.2ff977e",
        "name": "start",
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 110,
        "y": 440,
        "wires": [
            [
                "f1b47338.aab82",
                "1ea9f445.89d0bc"
            ]
        ]
    },
    {
        "id": "4fb0a8dc.f6baf8",
        "type": "debug",
        "z": "21ce826.2ff977e",
        "name": "print",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "x": 570,
        "y": 80,
        "wires": []
    },
    {
        "id": "86385870.9f6b88",
        "type": "debug",
        "z": "21ce826.2ff977e",
        "name": "error",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "x": 770,
        "y": 240,
        "wires": []
    },
    {
        "id": "2270c854.c34e08",
        "type": "debug",
        "z": "21ce826.2ff977e",
        "name": "print",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "x": 750,
        "y": 160,
        "wires": []
    },
    {
        "id": "e69a3271.c7cab",
        "type": "predictor",
        "z": "21ce826.2ff977e",
        "name": "decision tree classifier predictor",
        "modelPath": "test/models",
        "modelName": "dtc.b",
        "x": 550,
        "y": 420,
        "wires": [
            [
                "b8f2ab19.e693a8"
            ],
            [
                "f7c59de2.be773"
            ]
        ]
    },
    {
        "id": "26110acb.cbf526",
        "type": "decision tree classifier",
        "z": "21ce826.2ff977e",
        "name": "decision tree classifier trainer",
        "savePath": "test/models",
        "saveName": "dtc.b",
        "maxDepth": "",
        "criterion": "gini",
        "splitter": "best",
        "x": 540,
        "y": 200,
        "wires": [
            [
                "2270c854.c34e08"
            ],
            [
                "86385870.9f6b88"
            ]
        ]
    },
    {
        "id": "b8f2ab19.e693a8",
        "type": "assessment",
        "z": "21ce826.2ff977e",
        "name": "",
        "score": "accuracy_score",
        "x": 590,
        "y": 360,
        "wires": [
            [
                "808a0c93.8ee38"
            ],
            [
                "f7c59de2.be773"
            ]
        ]
    },
    {
        "id": "f1b47338.aab82",
        "type": "load dataset",
        "z": "21ce826.2ff977e",
        "name": "",
        "datasetFolder": "test/datasets",
        "datasetName": "iris",
        "partition": "test.csv",
        "input": false,
        "output": true,
        "x": 290,
        "y": 360,
        "wires": [
            [
                "b8f2ab19.e693a8"
            ],
            [
                "f7c59de2.be773"
            ]
        ]
    },
    {
        "id": "1ea9f445.89d0bc",
        "type": "load dataset",
        "z": "21ce826.2ff977e",
        "name": "",
        "datasetFolder": "test/datasets",
        "datasetName": "iris",
        "partition": "test.csv",
        "input": true,
        "output": false,
        "x": 290,
        "y": 480,
        "wires": [
            [
                "e69a3271.c7cab"
            ],
            [
                "f7c59de2.be773"
            ]
        ]
    },
    {
        "id": "f7c59de2.be773",
        "type": "debug",
        "z": "21ce826.2ff977e",
        "name": "error",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "x": 790,
        "y": 480,
        "wires": []
    },
    {
        "id": "808a0c93.8ee38",
        "type": "debug",
        "z": "21ce826.2ff977e",
        "name": "print",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "x": 790,
        "y": 360,
        "wires": []
    },
    {
        "id": "8a4ea95c.f860b8",
        "type": "predictor",
        "z": "21ce826.2ff977e",
        "name": "decision tree classifier predictor",
        "modelPath": "test/models",
        "modelName": "dtc.b",
        "x": 450,
        "y": 580,
        "wires": [
            [
                "e967043f.480868"
            ],
            [
                "e66df10b.40ba8"
            ]
        ]
    },
    {
        "id": "e967043f.480868",
        "type": "mqtt out",
        "z": "21ce826.2ff977e",
        "name": "",
        "topic": "predictions",
        "qos": "",
        "retain": "",
        "broker": "cb216faf.d9136",
        "x": 730,
        "y": 540,
        "wires": []
    },
    {
        "id": "e66df10b.40ba8",
        "type": "debug",
        "z": "21ce826.2ff977e",
        "name": "error",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "x": 710,
        "y": 620,
        "wires": []
    },
    {
        "id": "3cd1a442.2bc73c",
        "type": "mqtt in",
        "z": "21ce826.2ff977e",
        "name": "",
        "topic": "classification",
        "qos": "2",
        "broker": "cb216faf.d9136",
        "x": 140,
        "y": 580,
        "wires": [
            [
                "8a4ea95c.f860b8"
            ]
        ]
    },
    {
        "id": "cb216faf.d9136",
        "type": "mqtt-broker",
        "z": "",
        "name": "",
        "broker": "iot.eclipse.org",
        "port": "1883",
        "clientid": "",
        "usetls": false,
        "compatmode": true,
        "keepalive": "60",
        "cleansession": true,
        "willTopic": "",
        "willQos": "0",
        "willPayload": "",
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": ""
    }
]

Hopperpop · 16 October 2021 21:25

For me it seems to work. Only the assessment node, needed a small fix to get it running.
In assessment.py, I changed getargspec to getfullargspec. Could probably depend on what python version your using (I'm on 3.8.10).

But still even if you have it running, I don't see how it will solve your problem. It not that you just can dump your accelerometer data into it, and it will say healthy or not healthy. You still need to extract features from your dataset that can be used in a decision tree classifier. As long as you can't do that, machine learning wont help you anything. First try to build something that works with a simple threshold and than switch to machine learning. If you have something in mind to extract features, please share. If you don't, I think you are wasting your time finding a magic solution that will work out of the box.

import sklearn.metrics as m
import json
import sys
import numpy
from inspect import getfullargspec

#read configurations
config = json.loads(input())

while True:
	#read request
	data = json.loads(input())
	
	get_score = getattr(m, config['score'])
	kwargs = {}
	if 'average' in getfullargspec(get_score).args:
		kwargs['average'] = 'micro'
	if 'beta' in getfullargspec(get_score).args:
		kwargs['beta'] = 1
	score = get_score(data['real'], data['predicted'], **kwargs)
	if type(score) is numpy.ndarray:
		score = json.dumps(score.tolist())
	print(score)

puteramelaka · 17 October 2021 23:34

well im noted with that,,

well actually there is fft node , but it doesnt work anymore..its kinda sad

well actually that is the sample flow, there is other nodes in the package ,but at this point i might just move to phyton to make machine learning works, im not getting solution via node-red

Hopperpop · 18 October 2021 10:19

I think you are indeed better of to do the heavy processing directly in a language like python.

Node-red is descriped as: "Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways." lt's not really designed for signal processing tasks like yours, that's why the existing nodes like fft & machine learning aren't the highest quality. Python has many libraries, and it wouldn't make any sense that all of these have been wrapped into a node. Use the tools for what they are designed for.

Nothing stops you from collection your data in node-red, process it in a python script and get it back into node-red for the final notification.

puteramelaka · 18 October 2021 13:50

thats a great explaination!

it just im trying to make all process in real-time manner, but yeah i guess thats it...i will make near-real time process

krambriw · 19 October 2021 06:59

Why you think doing it in a separate Python process will make it less near-real-time than doing it all in NR?
Remember NR is single threaded whereas Python is mullti-threaded, multi-processing. So in the end, it could be you get the results faster using an external Python script and returning it back to NR

puteramelaka · 20 October 2021 23:25

because in-order to save the data in csv file i need to stop the flow..thus i need to stop the flow first ,then only the .csv file appear ..only then i will be able to accesss it via phyton..making it near-real time but not real-time manner

system · 3 November 2021 23:25

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Node-red machine learning(I need help) General	9	1174	12 October 2021
Neural network - Internal Node or Watson General	24	3218	28 October 2018
Machine learning with NR General	5	386	24 March 2021
Node-red in workplace. What are you using it for? General	36	5970	19 September 2019
New to Node Red - trying to decide which route? General	41	2093	2 June 2020

How to use node-red-contrib-machine-learning-v2 (NEED HELP AND GUIDANCE FOR UNI PROJECTS)

Related topics