Node-red crashes consistently (MySQL)

[reynardlee]

I was working just fine, yesterday and now there is a problem. It starts normally, but when I select a Date Picker along with two Dropdowns, it seems to stop working. It seems like there's a problem in the MySQL node (it was connected fine). Thanks in advance, a newb here using node-red.

the Date pickers and Dropdowns I mentioned:

node-red logs:

node-red mysql - Pastebin.com

[bakman2]

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

When you render the chart, how much data are you pushing to it ?

[reynardlee]

around 1500 at least .

thinking about it, it is quite alot. Is that too much?

[Steve-Mcl]

Unless your chart is 1500 pixels wide, yes

[Colin]

@reynardlee

Is that too much?

Probably, but it should not be enough to crash it. Could it be that the Join node is outputting intermediate results before you have configured all the inputs? You could add a debug node on the output of the select query function node and then add a two second delay before that is fed to the sql node (to ensure the debug is displayed before the sql node is run, possibly crashing node-red). Then you will be able to see what query is crashing it.

[dceejay]

Well something is blowing memory so either too big a query - or too many queries too fast. but a result set of 1500 "small" objects should be fine. Or maybe something else is filling memory leaving not much for Node-RED

[reynardlee]

The Join node will only output results when it receives 3 inputs, so that shouldn't be the problem.

And I was able to display the debug on the output of the Select Query function.
But nothing displayed at the MySQL debug node until I disconnect it from the Function and Dashboard Chart nodes, it works/debugs just fine.

Seems like it is the function node (which reformats array into an appropriate array for the dashboard chart) right after that caused the crash. As @dceejay pointed out, I think there is too big of a query feeding into the Function node, which caused it to crash.

I went for the approach below but I think, the output from MySQL node is still consider as one msg (a big array), therefore it still doesnt work.

I will try to work on reducing the amount of data inputting into the function/chart node, perhaps with a smooth node or rbe node. Do recommend me any other methods, I will try them out!

Thank yall so much for your help! Have a good day ahead.

[dceejay]

yes the mysql node will return a single array. So either you can...
change the query to be more specific to return a more 'focussed" set of results with just the required data in, and then reformat that smaller amount.
or
use a split node to split the array and then filter/reformat each individual result as appropriate.

It seem like you are trying to feed a chart so you can either send it all the data at once (as an array) - or as a timeseries one point at a time - so you can send data with both a payload and timestamp (and maybe topic) properties if you have existing timestamp data in your array. (IE it can handle using the array, or splitting the data if you want to try that approach)

[reynardlee]

A funny thing that I discovered/realised today is that, in the For loop of my function node, I wrongly used i+2 instead of i++. I changed nothing else but the i condition for the For loop and it works fine now. There could possibly some contradicting error there when i+2 that caused it to crash.

It works even when the input array has over 10000 values.

[Colin]

Were you going off the end of the array? Anything can happen in that case. What did the for loop look like?

[Steve-Mcl]

@reynardlee

in the For loop of my function node, I wrongly used i+2 instead of i++

Was that in the 3rd part like this....

for(let i = 0; i < someArray.length; i+2) {
  //do a bunch of stuff
}

... if so, then I'd go with that being the cause - as i will remain 0 for ever & i+2 was probably less than the for break out condition - this results in an infinite loop / eventual crash.

[reynardlee]

yes. exactly! The most basic mistake ever.
i++ means it will be an increment of +1 every time. Is there a way to have an increment of +2 every time?

edit: I totally forgotten about i+=2. That would work perfectly if I am not wrong.

[Steve-Mcl]

@reynardlee

I totally forgotten about i+=2 . That would work perfectly if I am not wrong

It would work yes, it would step by 2 (skip odd elements)

[Colin]

If you are incrementing by 2 does that mean that at some point in the loop you us someArray[i+1]? If you do then in the for statement it should be
for(let i = 0; i < someArray.length-1; i+2) {
otherwise if the length should somehow be an odd number then you would be indexing off the end of the array.

[reynardlee]

Yes, that is right.
ooh that's true. Thanks for help! much appreciated, loving this friendly community.

[Steve-Mcl]

@Colin

otherwise if the length should somehow be an odd number then you would be indexing off the end of the array.

Speaking from experience I bet (I've done it).

PS, also might be worth mentioning that due to sparse nature of JS arrays, indexing outside the array length will do pretty much nothing except return undefined (e.g. doesnt blow in ya face up like c does)

for example i is clearly going out of bounds...

for (let i = 0; i < 10 ; i++) {
  console.log( [1,2,3,4,5][i] );
}

but it keep on rollin' ...

1
2
3
4
5
undefined
undefined
undefined
undefined
undefined
undefined