How to reduce 5000 lines of code to just a few using a variable?

Gino_Schreuders · 29 March 2021 07:56

Dear reader,

I'm a intern at a company that distrubits camera. Together with my internship supervisor we are making software for Hikvision camera's using Node-Red.
We got our program working, however we have a function that consists of more than 5000 lines of code. We're sure this can be reduced to a few lines of code. I know we should make some values variable but we cannot find out how to make this working.
So I hope someone on this forum is able to help us?
Here are the flows and code:
flows (5).json (981.2 KB)
I think it should look something like this:

msg.headers = {};
msg.headers['content-type'] = 'application/xml';

if (msg.payload.length == 99)
{
    msg.payload = '<?xml version="1.0"?><LPListAuditSearchResult xmlns="http://www.std-cgi.org/ver20/XMLSchema" version="2.0"><LicensePlateInfoList><LicensePlateInfo><id>1<id><LicensePlate>'+msg.payload[0].Kenteken+'</LicensePlate><type>'+msg.payload[0].Type+'</type><effectiveTime>'+msg.payload[0].Geldig+'</effectiveTime></LicensePlateInfo>

Sincerly,

Gino

zenofmud · 29 March 2021 08:19

First off, what is the function suppose to do? i.e. explain what output you need based on the msg.payload.length.

Colin · 29 March 2021 08:30

As suggested, you need a specification. Write a detailed specification of what goes into the function and what should come out. Once you have a spec then a solution can be found.

Steve-Mcl · 29 March 2021 08:33

Hi, forgive me for saying so but - WOW.

↑ that is not how you deal with variable sized data.

First and easy fix is to loop through the array to build your XML - but this is a botch job & not recommended.

The better solution is to work with native JS objects & build / transform the SQLite data into the correct shape Js object then pass it through an XML node to convert the JS object into XML.

If you can provide 2 things...

a sample of 2 SQLlite rows of data by adding a debug node after the SQLite node, query the DB for 2 rows (limit 2) then use the copy value button that appears under your mouse cursor when you hover over the debug output.
the final XML that you expect (with the same 2 data rows converted to XML)

I can show you how to do this in a more "node-red" way.

molesworth · 29 March 2021 09:15

I'll reiterate @Steve-Mcl's "Wow!". That's very much not the way to do this kind of task.

I'm not trying to put down your efforts at getting something working, but if you're doing this as a commercial product you really should find an experienced software developer, either a contractor for a few months, or a permanent employee if you're going to be doing more, similar things in the future.

It may seem like unnecessary cost, even for a couple of months of a contractor, but it will pay off long term with much more maintainable and upgradeable code, fewer bugs, and fewer customer problems.

Gino_Schreuders · 29 March 2021 09:35

I'm sorry but the code is from the previous intern.
We are trying to get this data:

Steve-Mcl · 29 March 2021 09:48

I meant no offence - only to highlight it was not a great way to handle this type of task (completely unmaintainable and lots of error prone copy+pasting). I am sorry if offence was taken.

To help you, as I said before, 2 things are needed...

I will spell them out a little clearer..

Add a debug AFTER sqlite node
- capture the output of the payload by clicking the "copy value" button that appears under your mouse in the debug window
- I only need a few rows
Provide a "finished" XML string for these data rows (as a code block, not a screenshot)

The reason why anyone would need this ↑ - I dont have access to your data to try something, I would need a final (good) XML to compare what my demo flow produces is correct for you.

If you dont want to do that, then I suggest using the following approach...

sqlite --> function (transform the rows into a JS object that represents the final XML) --> XML node --> file node.

krambriw · 29 March 2021 10:18

My personal view, yes, agree, a serious company should bear the costs for their work to be done. I think we see this more often, companies seeking "free" advice and help. Maybe room for a new category, "Seek & Sell" ?

Gino_Schreuders · 29 March 2021 10:42

[{"id":1,"Kenteken":"34jlrd","Type":"blackList","Geldig":"2020-12-12","AantalKeren":"44"},{"id":2,"Kenteken":"68-bg-po","Type":"whiteList","Geldig":"2020-12-12","AantalKeren":"48"},{"id":3,"Kenteken":"01aa01","Type":"blackList","Geldig":"2020-12-12","AantalKeren":"48"},{"id":4,"Kenteken":"02aa01","Type":"whiteList","Geldig":"2020-12-12","AantalKeren":"48"},{"id":6,"Kenteken":"04aa01","Type":"whiteList","Geldig":"2020-12-12","AantalKeren":"48"}]

{"_msgid":"615ef601.067c28","payload":"<?xml version=\"1.0\"?><LPListAuditSearchResult xmlns=\"http://www.std-cgi.org/ver20/XMLSchema\" version=\"2.0\"><LicensePlateInfoList><LicensePlateInfo><id>1<id><LicensePlate>34jlrd</LicensePlate><type>blackList</type><effectiveTime>2020-12-12</effectiveTime></LicensePlateInfo><LicensePlateInfo><id>2<id><LicensePlate>68-bg-po</LicensePlate><type>whiteList</type><effectiveTime>2020-12-12</effectiveTime></LicensePlateInfo><LicensePlateInfo><id>3</id><LicensePlate>01aa01</LicensePlate><type>blackList</type><effectiveTime>2020-12-12</effectiveTime></LicensePlateInfo><LicensePlateInfo><id>4</id><LicensePlate>02aa01</LicensePlate><type>whiteList</type><effectiveTime>2020-12-12</effectiveTime></LicensePlateInfo><LicensePlateInfo><id>5<id><LicensePlate>04aa01</LicensePlate><type>whiteList</type><effectiveTime>2020-12-12</effectiveTime></LicensePlateInfo><LicensePlateInfo><id>6<id><LicensePlate>05aa01</LicensePlate><type>blackList</type><effectiveTime>2020-12-12</effectiveTime></LicensePlateInfo><Li...","headers":{"content-type":"application/xml"}}

Steve-Mcl · 29 March 2021 10:47

Your XML in the 2nd message is incomplete...

Try reducing the query to 2 rows and try again.

Gino_Schreuders · 29 March 2021 10:57

KENTEKENS.txt (107 Bytes)

Steve-Mcl · 29 March 2021 11:11

Here you go...

Your 5000 line hard coded function becomes 100% dynamic (1 row or 10000 rows - its the same)...

//make the base JS object
var dataStructure ={
    "LPListAuditSearchResult":{
        "$": {
            "xmlns":"http://www.std-cgi.org/ver20/XMLSchema","version":"2.0"
        },
        "LicensePlateInfoList": [
            {
                "LicensePlateInfo":[ ]
            }
        ]
    }
}

//make a data item from a DB row
function makeDataRow(row) {
    return  {
        "id":[row.id],
        "LicensePlate":[row.Kenteken],
        "type":[row.Type],
        "effectiveTime":[row.Geldig]
    }
}

for (let i = 0; i < msg.payload.length; i++) {
    const row = msg.payload[i];
    //add the converted DB row to the JS dataStructure object
    dataStructure.LPListAuditSearchResult.LicensePlateInfoList[0].LicensePlateInfo.push(makeDataRow(row));
}
msg.payload = dataStructure
return msg;

Demo flow...

[{"id":"2ebcaf60.912b7","type":"inject","z":"4656711a.6b16a","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":330,"y":280,"wires":[["d818ecec.c4de"]]},{"id":"d818ecec.c4de","type":"function","z":"4656711a.6b16a","name":"Database data (fake query result)","func":"msg.payload = [{\"id\":1,\"Kenteken\":\"34jlrd\",\"Type\":\"blackList\",\"Geldig\":\"2020-12-12\",\"AantalKeren\":\"44\"},{\"id\":2,\"Kenteken\":\"68-bg-po\",\"Type\":\"whiteList\",\"Geldig\":\"2020-12-12\",\"AantalKeren\":\"48\"},{\"id\":3,\"Kenteken\":\"01aa01\",\"Type\":\"blackList\",\"Geldig\":\"2020-12-12\",\"AantalKeren\":\"48\"},{\"id\":4,\"Kenteken\":\"02aa01\",\"Type\":\"whiteList\",\"Geldig\":\"2020-12-12\",\"AantalKeren\":\"48\"},{\"id\":6,\"Kenteken\":\"04aa01\",\"Type\":\"whiteList\",\"Geldig\":\"2020-12-12\",\"AantalKeren\":\"48\"}]\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":580,"y":280,"wires":[["5f772990.9cb2b8","b2fc9b8f.7fef28"]]},{"id":"5f772990.9cb2b8","type":"debug","z":"4656711a.6b16a","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":930,"y":280,"wires":[]},{"id":"b2fc9b8f.7fef28","type":"function","z":"4656711a.6b16a","name":"transform to nice JS object","func":"//make the base JS object\nvar dataStructure ={\n    \"LPListAuditSearchResult\":{\n        \"$\": {\n            \"xmlns\":\"http://www.std-cgi.org/ver20/XMLSchema\",\"version\":\"2.0\"\n        },\n        \"LicensePlateInfoList\": [\n            {\n                \"LicensePlateInfo\":[ ]\n            }\n        ]\n    }\n}\n\n//make a data item from a DB row\nfunction makeDataRow(row) {\n    return  {\n        \"id\":[row.id],\n        \"LicensePlate\":[row.Kenteken],\n        \"type\":[row.Type],\n        \"effectiveTime\":[row.Geldig]\n    }\n}\n\nfor (let i = 0; i < msg.payload.length; i++) {\n    const row = msg.payload[i];\n    //add the converted DB row to the JS dataStructure object\n    dataStructure.LPListAuditSearchResult.LicensePlateInfoList[0].LicensePlateInfo.push(makeDataRow(row));\n}\nmsg.payload = dataStructure\nreturn msg;\n\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":560,"y":340,"wires":[["ecb6b334.0d41","61af7ae1.0ee784"]]},{"id":"ecb6b334.0d41","type":"xml","z":"4656711a.6b16a","name":"","property":"payload","attr":"","chr":"","x":770,"y":340,"wires":[["497a134c.f828fc"]]},{"id":"497a134c.f828fc","type":"debug","z":"4656711a.6b16a","name":"Finished XML","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":940,"y":340,"wires":[]},{"id":"61af7ae1.0ee784","type":"debug","z":"4656711a.6b16a","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":790,"y":400,"wires":[]}]

PS - your original XML is still invalid (and the exact reason I said you should work with Js objects instead of a hand coded string concatenation)

system · 28 May 2021 11:12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Sort msg.payload Arrays into sections General	62	6482	2 November 2019
Problem while depending of object length General function-node	6	535	23 April 2022
Payload[variable] General	3	344	3 August 2021
Problem with flow-variables General	9	1105	9 August 2020
Msg in a file, thanks :) General	2	290	14 June 2020

How to reduce 5000 lines of code to just a few using a variable?

Related Topics