UI Tabular Table Sorting

Dashboard
127

Ok. This could be fun and interesting. The first thing you're going to want to figure out is if you can somehow index the data you're getting so you have a unique reference for each item. If you can assign a unique ID to each dataset, it will make it a lot easier to make pulls from your API and add them to the persistent data as you pull them.

So as a starting point, is there anything in your dataset that you can use that is unique? Or is there a combination of datapoints in a dataset that when combined could create a unique key that no other dataset has?

128

Got the perfect thing: obj.trade_id ... unique transaction id :slight_smile:

129

Perfect. That will make this so easy to work with. I'll give you a code block that lays out what you'll do and then you can apply either that code block or that principle to your code itselt.

var temp = context.get("recordKeeper");
if(temp.hasOwnProperty(msg.trade_id){
    temp[msg.trade_id] = msg.payload;
}
context.set('recordKeeper', temp);

Here's what's happening in there. I'll assume that msg.payload is a single object containing the valuable stuff along side a unique identifier set to the property "trade_id". This means those are the only two things you care about. If the given trade_id exists in the dataset recordKeeper, nothing happens. If that unique ID doesn't exist, it adds it under the property named for the individual trade_id. This is the easiest way I can think of to keep track of unique records.

To reference the full dataset, you can do this:

var records = Object.keys(context.get('recordKeeper'));

To get a list of all the trade_id properties you have so far. Once you have that, you can iterate through it as a reference list and use that to send different data to the new table.

Play around with that a little and see if you can build a stable dataset that adds just the unique stuff into it. Once you have that working, we'll import it into the table. I just don't want to jump to the table yet and think that there could be something wrong with the dataset.

130

Thank you - I'll give it a try. Here a sample of the records.

[{"id":21603253,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"NEBL","instant":false,"status":"SL","leverage":{"enabled":false},"position":"9972","take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"2.1282","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_0.61_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","note_raw":"5m---_0.61_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"3.92","bid":"2.174","ask":"2.177","last":"2.174","quote_volume":"1265378.9122"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"2.133","average_close_price":"2.12","average_enter_price_without_commission":"2.131","average_close_price_without_commission":"2.123","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"4675.0","entered_total":"9972.387425","closed_amount":"4675.0","closed_total":"9915.099975","created_at":"2022-11-29T22:30:28.831Z","updated_at":"2022-11-29T22:30:28.831Z","closed_at":"2022-11-30T00:32:19.751Z","type":"smart_trade"},"profit":{"volume":"-57.28745","usd":"-57.28745","percent":"-0.57","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false,"trade_id":21603253,"profit_usd":"-57.29","profit_percent":"-0.57","to_sl":"0.23","volume":"1265378.9122","close_price":"2.12","closed":"2022-11-30T00:32:19.751Z","S_L":"0.23","T_P":"","in":["5m---","0.61","0.5","1","S.BUY","2.5","1","2","๐Ÿก‡","1"],"frames":"5m---","gain":"0.61","gain_st":"0.5","freq":"1","analysis":"S.BUY","tp_a":"2.5","tp_trl_a":"1","sl_a":"2","sl_trl_a":"๐Ÿก‡","tm_a":"1"},{"id":21603293,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"FARM","instant":false,"status":"SL","leverage":{"enabled":false},"position":"9932","take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"38.428","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_0.78_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","note_raw":"5m---_0.78_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"11.143","bid":"38.8","ask":"38.9","last":"38.9","quote_volume":"9694195.0607"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"38.3","average_close_price":"38.3","average_enter_price_without_commission":"38.3","average_close_price_without_commission":"38.4","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"259.067","entered_total":"9932.1883661","closed_amount":"259.067","closed_total":"9938.2246272","created_at":"2022-11-29T22:39:19.897Z","updated_at":"2022-11-29T22:39:19.897Z","closed_at":"2022-11-29T22:45:48.618Z","type":"smart_trade"},"profit":{"volume":"6.0362611","usd":"6.0362611","percent":"0.06","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false,"trade_id":21603293,"profit_usd":"6.04","profit_percent":"0.06","to_sl":"-0.33","volume":"9694195.0607","close_price":"38.3","closed":"2022-11-29T22:45:48.618Z","S_L":"-0.33","T_P":"","in":["5m---","0.78","0.5","1","S.BUY","2.5","1","2","๐Ÿก‡","1"],"frames":"5m---","gain":"0.78","gain_st":"0.5","freq":"1","analysis":"S.BUY","tp_a":"2.5","tp_trl_a":"1","sl_a":"2","sl_trl_a":"๐Ÿก‡","tm_a":"1"},{"id":21603291,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"MOB","instant":false,"status":"TP","leverage":{"enabled":false},"position":"9897","take_profit":{"enabled":true,"steps":[{"id":90533897,"order_type":"market","editable":false,"units":{"value":"7037.29"},"price":{"value":"1.46544","type":"last","percent":null},"volume":"100.0","total":"10312.7262576","trailing":{"enabled":true,"percent":"-1.0"},"status":{"type":"finished","basic_type":"finished","title":"Finished"},"data":{"cancelable":false,"panic_sell_available":false},"position":1}]},"stop_loss":{"enabled":false},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_0.78_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","note_raw":"5m---_0.78_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"0.623","bid":"1.454","ask":"1.458","last":"1.454","quote_volume":"6428167.63679"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"1.406","average_close_price":"1.46","average_enter_price_without_commission":"1.405","average_close_price_without_commission":"1.462","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"7037.29","entered_total":"9897.27984245","closed_amount":"7037.29","closed_total":"10278.22946202","created_at":"2022-11-29T22:38:32.235Z","updated_at":"2022-11-29T22:38:32.235Z","closed_at":"2022-11-29T22:44:53.528Z","type":"smart_trade"},"profit":{"volume":"380.94961957","usd":"380.94961957","percent":"3.85","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false,"trade_id":21603291,"profit_usd":"380.95","profit_percent":"3.85","to_sl":0,"volume":"6428167.63679","close_price":"1.46","closed":"2022-11-29T22:44:53.528Z","S_L":"","T_P":"4.23","in":["5m---","0.78","0.5","1","S.BUY","2.5","1","2","๐Ÿก‡","1"],"frames":"5m---","gain":"0.78","gain_st":"0.5","freq":"1","analysis":"S.BUY","tp_a":"2.5","tp_trl_a":"1","sl_a":"2","sl_trl_a":"๐Ÿก‡","tm_a":"1"},{"id":21603262,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"TRU","instant":false,"status":"SL","leverage":{"enabled":false},"position":"9914","take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"0.0416","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_0.97_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","note_raw":"5m---_0.97_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"5.526","bid":"0.04","ask":"0.0402","last":"0.0401","quote_volume":"1952957.7675"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"0.0412","average_close_price":"0.0404","average_enter_price_without_commission":"0.0412","average_close_price_without_commission":"0.0405","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"240384.0","entered_total":"9913.7246208","closed_amount":"240384.0","closed_total":"9725.816448","created_at":"2022-11-29T22:33:10.786Z","updated_at":"2022-11-29T22:33:10.786Z","closed_at":"2022-11-29T22:40:34.828Z","type":"smart_trade"},"profit":{"volume":"-187.9081728","usd":"-187.9081728","percent":"-1.9","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false,"trade_id":21603262,"profit_usd":"-187.91","profit_percent":"-1.9","to_sl":"-0.97","volume":"1952957.7675","close_price":"0.0404","closed":"2022-11-29T22:40:34.828Z","S_L":"-0.97","T_P":"","in":["5m---","0.97","0.5","1","S.BUY","2.5","1","2","๐Ÿก‡","1"],"frames":"5m---","gain":"0.97","gain_st":"0.5","freq":"1","analysis":"S.BUY","tp_a":"2.5","tp_trl_a":"1","sl_a":"2","sl_trl_a":"๐Ÿก‡","tm_a":"1"},{"id":21603266,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"TRU","instant":false,"status":"SL","leverage":{"enabled":false},"position":"9867","take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"0.0419","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_0.96_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","note_raw":"5m---_0.96_0.5_1_S.BUY_2.5_1_2_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"5.526","bid":"0.04","ask":"0.0402","last":"0.0401","quote_volume":"1952957.7675"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"0.0414","average_close_price":"0.0415","average_enter_price_without_commission":"0.0414","average_close_price_without_commission":"0.0416","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"238095.0","entered_total":"9866.990133","closed_amount":"238095.0","closed_total":"9894.847248","created_at":"2022-11-29T22:34:16.901Z","updated_at":"2022-11-29T22:34:16.901Z","closed_at":"2022-11-29T22:37:23.134Z","type":"smart_trade"},"profit":{"volume":"27.857115","usd":"27.857115","percent":"0.28","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false,"trade_id":21603266,"profit_usd":"27.86","profit_percent":"0.28","to_sl":"-1.21","volume":"1952957.7675","close_price":"0.0415","closed":"2022-11-29T22:37:23.134Z","S_L":"-1.21","T_P":"","in":["5m---","0.96","0.5","1","S.BUY","2.5","1","2","๐Ÿก‡","1"],"frames":"5m---","gain":"0.96","gain_st":"0.5","freq":"1","analysis":"S.BUY","tp_a":"2.5","tp_trl_a":"1","sl_a":"2","sl_trl_a":"๐Ÿก‡","tm_a":"1"}]

Are there any considerations to the fact this will be be large (and constantly growing) data set? Is basic file storage (vs. db) sufficient? The one thing that definitely should be done is elimination of all unnecessary fields; I am only interested in a handful of variables (you can see them in the table setup).

<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/luxon@3.0.4/build/global/luxon.min.js"></script>
<div id="analytics"></div>
<script>

    var analyticsTable = new Tabulator("#analytics", {
     	height:900, 
     	columnCalcs:"table",
   // 	textSize : 10,
   //  	groupBy:"pair",
    
        headerSortElement:"",
     	layout:"fitColumns",
     	columns:[ //Define Table Columns
     	    {title:"Status", field:"status", hozAlign:"center", width:60, headerFilter:"list", headerFilterParams:{valuesLookup:true, clearable:true}},
    	 	{title:"Coin", field:"pair", hozAlign:"center", width:65},
//            {title:"%", field:"profit.percent", hozAlign:"center", width:65, bottomCalc:"avg", bottomCalcParams:{precision:2}, formatter: function(cell, formatterParams){var value = cell.getValue(); if(value == null) {return ""}; if (value !== null) {if(value >= 0){cell.getElement().style.color ='#609f70'} else {cell.getElement().style.color ='#ff4a68'} return value +'%';}}},
            {title:"%", field:"profit.percent", hozAlign:"center", width:65, bottomCalc:"avg", bottomCalcParams:{precision:2}, bottomCalcFormatter: function(cell, bottomCalcFormatterParams){var value = cell.getValue(); if(value == null) {return ""}; if (value !== null) {if(value >= 0){cell.getElement().style.color ='#609f70'} else {cell.getElement().style.color ='#ff4a68'} return value +'%';}},formatter: function(cell, formatterParams){var value = cell.getValue(); if(value == null) {return ""}; if (value !== null) {if(value >= 0){cell.getElement().style.color ='#609f70'} else {cell.getElement().style.color ='#ff4a68'} return value +'%';}}},            {title:"$", field:"position", hozAlign:"right", width:55, formatter:"money", formatterParams:{thousand:",", precision:0}}, 
//            {title:"ฮ”", field:"profit_usd", hozAlign:"right", width:60, bottomCalc:"sum", bottomCalcParams:{precision:2}, formatter:"money", formatterParams:{thousand:",", precision:2}},
            {title:"ฮ”", field:"profit_usd", hozAlign:"center", width:70, bottomCalc:"sum", bottomCalcParams:{precision:2}, bottomCalcFormatter: function(cell, bottomCalcFormatterParams){var value = cell.getValue(); if(value == null) {return ""}; if (value !== null) {if(value >= 0){cell.getElement().style.color ='#609f70'} else {cell.getElement().style.color ='#ff4a68'} return value;}},formatter: function(cell, formatterParams){var value = cell.getValue(); if(value == null) {return ""}; if (value !== null) {if(value >= 0){cell.getElement().style.color ='#609f70'} else {cell.getElement().style.color ='#ff4a68'} return value;}}},
     	    {title:"SL", field:"sl_a", hozAlign:"center", width:40, headerFilter:"list", formatter:"money", formatterParams:{thousand:",", precision:1}, headerFilterParams:{valuesLookup:true, clearable:true}},
            {title:"sl", field:"S_L", hozAlign:"center", width:45},
     	    {title:"l๐Ÿก‡", field:"sl_trl_a", hozAlign:"center", width:20, headerFilter:"list", headerFilterParams:{valuesLookup:true, clearable:true}},
     	    {title:"tm", field:"tm_a", hozAlign:"center", width:20, headerFilter:"list", headerFilterParams:{valuesLookup:true, clearable:true}},
     	    {title:"TP", field:"tp_a", hozAlign:"center", width:25, headerFilter:"list", headerFilterParams:{valuesLookup:true, clearable:true}},
    	    {title:"tp", field:"T_P", hozAlign:"center", width:45},
     	    {title:"p๐Ÿก…", field:"tp_trl_a", hozAlign:"center", width:45, headerFilter:"list", formatter:"money", formatterParams:{thousand:",", precision:1}, headerFilterParams:{valuesLookup:true, clearable:true}},
     	    {title:"Freq", field:"freq", hozAlign:"center", width:20, headerFilter:"list", headerFilterParams:{valuesLookup:true, clearable:true}},
     	    {title:"Gain", field:"gain_st", hozAlign:"center", width:20, headerFilter:"list", formatter:"money", formatterParams:{thousand:",", precision:1}, headerFilterParams:{valuesLookup:true, clearable:true}},
     	    {title:"RealG", field:"gain", hozAlign:"center", width:20, headerFilter:"list", formatter:"money", formatterParams:{thousand:",", precision:1}, headerFilterParams:{valuesLookup:true, clearable:true}},
     	    {title:"frames", field:"frames", hozAlign:"center", width:100, headerFilter:"list", headerFilterParams:{valuesLookup:true, clearable:true}},
     	    {title:"TDV", field:"analysis", hozAlign:"center", width:70, headerFilter:"list", headerFilterParams:{valuesLookup:true, clearable:true}},
            {title:"Price", field:"close_price", hozAlign:"right", width:70, formatter:"money", formatterParams:{thousand:",", precision:2}},
     	    {title:"Volume", field:"volume", hozAlign:"right", width:90, formatter:"money", formatterParams:{thousand:",", precision:0}},
     	    {title:"Trade", field:"trade_id", hozAlign:"center", width:70},     	    
    	 	{title:"Created", field:"closed", width:90, formatter:function(cell, formatterParams, onRendered){var value = cell.getValue(); value = moment(value).format("D: h:mm a"); return value; }}
     	],  
    });
    

//  var results = table.getCalcResults();
//  var headerFilters = table.getHeaderFilters();
 
 // table.clearHeaderFilter();
  
  (function(scope){
        scope.$watch('msg', function(msg){
              analyticsTable.setData(msg.payload, true);
        });
    })(scope);
</script>
131

The only considerations I can think of right now involve how permanent you want these in storage (i.e. long term memory retention while the flow is running vs. actual disk storage), how many records you intend to keep (anything over 100k would make me start thinking disk storage), how fast you want access to a large record (large records take time to transmit over a link) and whether or not there will be any manipulation of the record order (i.e. cutting out or rearranging parts). The table can handle whatever you throw at it, so the table itself isn't a problem. If anything at all hints at using disk storage, just go that way as it's easy to implement and takes all the other worries away.

I find file storage easier, but databases are built for this kind of thing. It's all up to your comfort level in creating either as well as what resources you have available.

Yes. Everything you can do to trim the dataset will only reduce the overhead required. And from your previous code sent, it looks like you already have a good handle on breaking out what you need. Just keep that up and make sure to have a solid organization plan so you know how everything is structured.

Like I say, once you get a solid dataset you can rely on being correct and have the ability to access whatever you need, we can work on passing it to the table if you run into any snags with that. But once you get it structured and stored, I think you'll have almost everything you need to make it work. Just let me know if/when you get stuck.

132

I've been trying to figure out how to eliminate unnecessary variables (or rather just keep the ones I need ) from the payload for a while - haven't succeeded.

133

I assumed this code is going into a function that's between the function outputting the data (or can it be merged in) and the table? We had some On Start code as well last time (I kept it and adjusted the naming(. Not seeing any data in the saved array.

134 
var toKeep = {};
toKeep.trade_id = msg.payload.trade_id;
toKeep.payload.status = msg.payload.status;
toKeep.payload.pair = msg.payload.pair;
...  //Keep assigning items you want to keep.
msg = {};
msg = toKeep;
return msg;

This is the easiest way to do it looking at your dataset. I may not have addressed what you want to keep the correct way, but just adjust the locations to get at what you want. I simply emptied msg and assigned toKeep to it, but the same principle applies to anything that is getting the data you want to hang on to.

All scripting between your inputs and your table can be handled in one function. In fact, that tends to make it easier since you can manipulate the data in one place rather than trying to split it up and send it on. No reason for multiple function nodes unless you're trying to check blocks of code for functionality before merging them.

The Start code was simply to establish a storage location for the context variable. Putting that in the start block ensures that the variable is created and usable from the beginning so there are no issues afterwards.

The easiest thing to do will be to start with a list of everything you want to keep and the mapped location to the items in the dataset (i.e. msg.payload.trade_id). Once you have that, we can build a block of code at the beginning of your function to get rid of all the unnecessary data before processing the rest. Start with that and we'll go from there.

135

Unexpectedly simple approach (I kept looking at various jsonata expressions). Here a slightly updated version of the function with a filter added that creates an "input" msg. - couldn't that be the toKeep msg?


const input = msg.payload.filter(s => s.status.type == "stop_loss_finished" || s.status.type == "finished");

input.forEach((obj) => {


        obj.pair = (String(obj.pair)).replace("USDT_", "") 
        obj.trade_id = obj.id
        obj.profit_usd = Number(obj.profit.usd).toFixed(2)
        obj.profit_percent = obj.profit.percent
        obj.position = Number(obj.position.total.value).toFixed(0)
        obj.volume = obj.data.current_price.quote_volume
        obj.close_price = obj.data.average_close_price
        obj.closed = obj.data.closed_at

        if (obj.status.type === "stop_loss_finished")  {obj.status = "SL"} else if (obj.status.type === "finished")  {obj.status = "TP"} else if (obj.status.type === "panic_sold")  {obj.status = "โžฆ"} else  {
        obj.status= obj.status.type
        }


        obj.to_sl = 0

        if (obj.stop_loss.enabled) {
            obj.to_sl = ((1 - obj.stop_loss.conditional.price.value / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "stop_loss_finished") {
            obj.S_L = (((obj.data.average_enter_price - obj.stop_loss.conditional.price.value) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "finished") {
            obj.T_P = (((obj.take_profit.steps[0].price.value - obj.data.average_enter_price) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        obj.in = obj.note.split("_")

        obj.frames = obj.in[0]
        obj.gain = obj.in[1]
        obj.gain_st = obj.in[2]
        obj.freq = obj.in[3]
        obj.analysis = obj.in[4]
        obj.tp_a = obj.in[5]
        obj.tp_trl_a = obj.in[6]
        obj.sl_a = obj.in[7]
        obj.sl_trl_a = obj.in[8]
        obj.tm_a = obj.in[9]
    

})


msg.payload = input
return msg;
136

Unfortunately I am confused. The toKeep.trade_id = msg.payload.trade_id; is meant to be inside the function? If so, shouldn't it be obj.payload.trade_id ? And shouldn't it array through the array? Right now I am just getting one record.

137

Yes, it can be whatever it needs to be. The code blocks I showed you were so you could see how the interaction took place. If you're already assigning msg.payload items to another variable, use that to eliminate the excess you don't want to store. The important part of that was just how to eliminate excess data.

The trade_id can be whatever it needs to in order to fit your needs. Right now, you're getting an array in and weeding out certain objects. Then you're going through each object and processing them. This works to get data in a position to use later on.

There are some things you have in your current function that might be causing what you want to happen to not happen. It's not that it's done wrong. It's just not set to do what you want.

First, there's nothing in the function to get rid of excess stuff. Right now, you're working with the obj object, which is the whole object itself. If you wanted to get rid of the excess stuff, the first thing you would want to do is make a temporary variable and start assigning everything you want to keep from obj to the temporary variable. Copy values from obj to temp (or whatever you want to call it) and any processing done on an obj property, assign the final result to temp. Then in the end, empty the obj and copy temp to it.

input.forEach((obj) => {

        var temp = {};
        temp.pair = (String(obj.pair)).replace("USDT_", "") 
        temp.trade_id = obj.id
        temp.profit_usd = Number(obj.profit.usd).toFixed(2)
        ...  //Do the rest of the assignment or processing here
        obj = {};
        obj = temp;
});

msg.payload = {};
msg.payload = input;
return msg

That will ensure you only keep what you want as everything else gets dumped.

But this doesn't do well for making sure you only keep what you don't already have, which brings me to my second point.

If instead of doing a forEach function on an incoming array, use a for loop. This gives you the opportunity to check it against what you already have. I'm going to use recordKeeper again like I previously set it up, with recordKeeper.trade_id being our unique key.

var check = context.get('recordKeeper');
const input = msg.payload.filter(s => s.status.type == "stop_loss_finished" || s.status.type == "finished");

for(let obj of input){
    if(!(check.hasOwnProperty(obj.id))){
        //This becomes the internals of what your forEach function has now
        check[obj.id] = obj;  //Assigns the processed object to a key equal to the obj.id for tracking.
    }
}
context.set('recordKeeper', check);
msg = {};
msg.payload = check;
return msg;

The let obj of input statement copies out the individual object from input to obj so you can work with the individual object itself rather than accessing it through its array position like you would do using let obj in input. That way if you want to copy it to something else, it becomes a lot easier to do so.

In the end, you have two things you want to end up with. The first is that you need an array to pass into your table since that's what you're trying to display. The second is you want some kind of storage for that array so you have something to reference and check against so you only keep unique records. Making these changes should help you get both of those that you can work with.

Let me know if that doesn't make sense and I'll see if I can explain it another way.

138

Unfortunately neither version is outputting data. This is what I ended up with - did I get it wrong?

var check = context.get('recordKeeper');

const input = msg.payload.filter(s => s.status.type == "stop_loss_finished" || s.status.type == "finished");

for(let obj of input){
   
   if(!(check.hasOwnProperty(obj.id))){
       
        check[obj.id] = obj;
        

        obj.pair = (String(obj.pair)).replace("USDT_", "") 
        obj.trade_id = obj.id
        obj.profit_usd = Number(obj.profit.usd).toFixed(2)
        obj.profit_percent = obj.profit.percent
        obj.position = Number(obj.position.total.value).toFixed(0)
        obj.volume = obj.data.current_price.quote_volume
        obj.close_price = obj.data.average_close_price
        obj.closed = obj.data.closed_at

        if (obj.status.type === "stop_loss_finished")  {obj.status = "SL"} else if (obj.status.type === "finished")  {obj.status = "TP"} else if (obj.status.type === "panic_sold")  {obj.status = "โžฆ"} else  {
        obj.status= obj.status.type
        }


        if (obj.stop_loss.enabled) {
            obj.to_sl = ((1 - obj.stop_loss.conditional.price.value / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "stop_loss_finished") {
            obj.S_L = (((obj.data.average_enter_price - obj.stop_loss.conditional.price.value) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "finished") {
            obj.T_P = (((obj.take_profit.steps[0].price.value - obj.data.average_enter_price) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        obj.in = obj.note.split("_")

        obj.frames = obj.in[0]
        obj.gain = obj.in[1]
        obj.gain_st = obj.in[2]
        obj.freq = obj.in[3]
        obj.analysis = obj.in[4]
        obj.tp_a = obj.in[5]
        obj.tp_trl_a = obj.in[6]
        obj.sl_a = obj.in[7]
        obj.sl_trl_a = obj.in[8]
        obj.tm_a = obj.in[9]
    
  }
}

context.set('recordKeeper', check);
msg = {};
msg.payload = check;

return msg;

This is what the input looks (a piece of it, the whole thing exceeded allowable length)

{"_msgid":"804be7c4c990acfc","settings":{"input":"2022-11-29T23:48:08.245Z","input_format":"","input_tz":"Europe/London","output_format":"","output_locale":"C","output_tz":"America/New_York"},"method":"GET","url":"https://api.3commas.io/public/api/v2/smart_trades?status=successfully_finished&order_by=closed_at&per_page=100&from=2022-11-29T18:48:08.245Z","headers":{"date":"Wed, 30 Nov 2022 23:48:09 GMT","content-type":"application/json","transfer-encoding":"chunked","connection":"close","etag":"W/\"752f6a5dd55038266b5b0ff395db6004\"","cache-control":"max-age=0, private, must-revalidate","x-request-id":"0303bd52a701d93a09ddb1d3f7581e83","x-runtime":"1.199921","access-control-allow-methods":"GET, POST, OPTIONS","access-control-allow-credentials":"true","access-control-allow-headers":"DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,apikey,signature","access-control-expose-headers":"Content-Length,Content-Range","strict-transport-security":"max-age=31536000; includeSubDomains; preload","cf-cache-status":"DYNAMIC","set-cookie":["__cf_bm=KAwj2qdahQJ1TeFurGn8NzQvIFbSaR7YszSjTDsInsQ-1669852089-0-AUsA5xatvDcx5f26v/UIqYoSXtpSBFOJNELEGPcJtzEadofX4uTZcx32BuD4y01VrMyh/v/ek//tV1G2h4HNFSA=; path=/; expires=Thu, 01-Dec-22 00:18:09 GMT; domain=.api.3commas.io; HttpOnly; Secure; SameSite=None"],"server":"cloudflare","cf-ray":"772778e03f0fd676-CDG","x-node-red-request-node":"1fa3d55c"},"payload":[{"id":21614892,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_PERP","instant":false,"status":{"type":"stop_loss_finished","basic_type":"stop_loss_finished","title":"Stop Loss finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"18682.5","editable":false},"price":{"value":"0.53571","value_without_commission":"0.53518","editable":false},"total":{"value":"10008.49885035"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"0.53749","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m--_1.37_3_3_NEUTRAL_2.5_1_2_๐Ÿก‡_1","note_raw":"5m--_1.37_3_3_NEUTRAL_2.5_1_2_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"14.848","bid":"0.52913","ask":"0.52991","last":"0.52945","quote_volume":"21672677.2004991"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"0.53571","average_close_price":"0.52848","average_enter_price_without_commission":"0.53518","average_close_price_without_commission":"0.52901","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"18682.5","entered_total":"10008.49885035","closed_amount":"18682.5","closed_total":"9873.346095675","created_at":"2022-11-30T23:19:39.929Z","updated_at":"2022-11-30T23:19:39.929Z","closed_at":"2022-11-30T23:45:19.825Z","type":"smart_trade"},"profit":{"volume":"-135.152754675","usd":"-135.152754675","percent":"-1.35","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21614628,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_ETH","instant":false,"status":{"type":"panic_sold","basic_type":"panic_sold","title":"Closed at Market Price"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"0.7732","editable":false},"price":{"value":"1296.45","value_without_commission":"1295.16","editable":false},"total":{"value":"1002.41912971"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":false},"reduce_funds":{"steps":[]},"market_close":{"id":90590103,"type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"},"units":{"value":"0.7732"},"price":{"value":"1292.94","value_without_commission":"1294.24"},"total":{"value":"1002.419129712"}},"note":"undefined","note_raw":"undefined","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"6.596","bid":"1299.43","ask":"1299.44","last":"1299.46","quote_volume":"1026947241.270657"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"1296.45","average_close_price":"1292.94","average_enter_price_without_commission":"1295.16","average_close_price_without_commission":"1294.24","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":false,"base_position_step_finished":true,"entered_amount":"0.7732","entered_total":"1002.419129712","closed_amount":"0.7732","closed_total":"999.705661632","created_at":"2022-11-30T22:44:40.964Z","updated_at":"2022-11-30T22:44:40.964Z","closed_at":"2022-11-30T22:53:20.666Z","type":"smart_trade"},"profit":{"volume":"-2.71346808","usd":"-2.71346808","percent":"-0.27","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21614623,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_ETH","instant":false,"status":{"type":"panic_sold","basic_type":"panic_sold","title":"Closed at Market Price"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"0.7725","editable":false},"price":{"value":"1296.21","value_without_commission":"1294.92","editable":false},"total":{"value":"1001.3260257"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":false},"reduce_funds":{"steps":[]},"market_close":{"id":90590102,"type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"},"units":{"value":"0.7725"},"price":{"value":"1292.94","value_without_commission":"1294.24"},"total":{"value":"1001.3260257"}},"note":"undefined","note_raw":"undefined","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"6.596","bid":"1299.43","ask":"1299.44","last":"1299.46","quote_volume":"1026947241.270657"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"1296.21","average_close_price":"1292.94","average_enter_price_without_commission":"1294.92","average_close_price_without_commission":"1294.24","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":false,"base_position_step_finished":true,"entered_amount":"0.7725","entered_total":"1001.3260257","closed_amount":"0.7725","closed_total":"998.8005996","created_at":"2022-11-30T22:43:59.626Z","updated_at":"2022-11-30T22:43:59.626Z","closed_at":"2022-11-30T22:53:17.963Z","type":"smart_trade"},"profit":{"volume":"-2.5254261","usd":"-2.5254261","percent":"-0.25","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21614064,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_PERL","instant":false,"status":{"type":"stop_loss_finished","basic_type":"stop_loss_finished","title":"Stop Loss finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"531632.1","editable":false},"price":{"value":"0.01881","value_without_commission":"0.0188","editable":false},"total":{"value":"10004.67816348"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"0.01851","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_1.05_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"5m---_1.05_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"3.518","bid":"0.01855","ask":"0.01856","last":"0.01854","quote_volume":"974695.579876"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"0.01881","average_close_price":"0.01845","average_enter_price_without_commission":"0.0188","average_close_price_without_commission":"0.01847","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"531632.1","entered_total":"10004.67816348","closed_amount":"531632.1","closed_total":"9809.425642113","created_at":"2022-11-30T21:34:46.049Z","updated_at":"2022-11-30T21:34:46.049Z","closed_at":"2022-11-30T22:25:47.738Z","type":"smart_trade"},"profit":{"volume":"-195.252521367","usd":"-195.252521367","percent":"-1.95","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21614084,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_FRONT","instant":false,"status":{"type":"finished","basic_type":"finished","title":"Finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"48631.0","editable":false},"price":{"value":"0.2042","value_without_commission":"0.204","editable":false},"total":{"value":"9930.644724"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[{"id":90587570,"order_type":"market","editable":false,"units":{"value":"48631.0"},"price":{"value":"0.206713","type":"last","percent":null},"volume":"100.0","total":"10052.659903","trailing":{"enabled":true,"percent":"-1.0"},"status":{"type":"finished","basic_type":"finished","title":"Finished"},"data":{"cancelable":false,"panic_sell_available":false},"position":1}]},"stop_loss":{"enabled":false},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_1.65_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"5m---_1.65_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"5.336","bid":"0.2048","ask":"0.2053","last":"0.2053","quote_volume":"692069.2501"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"0.2042","average_close_price":"0.2049","average_enter_price_without_commission":"0.204","average_close_price_without_commission":"0.2052","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"48631.0","entered_total":"9930.644724","closed_amount":"48631.0","closed_total":"9969.1021188","created_at":"2022-11-30T21:37:00.913Z","updated_at":"2022-11-30T21:37:00.913Z","closed_at":"2022-11-30T21:51:17.310Z","type":"smart_trade"},"profit":{"volume":"38.4573948","usd":"38.4573948","percent":"0.39","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21614068,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_DREP","instant":false,"status":{"type":"finished","basic_type":"finished","title":"Finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"29895.0","editable":false},"price":{"value":"0.3389","value_without_commission":"0.3386","editable":false},"total":{"value":"10132.569447"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[{"id":90587484,"order_type":"market","editable":false,"units":{"value":"29895.0"},"price":{"value":"0.336305","type":"last","percent":null},"volume":"100.0","total":"10053.837975","trailing":{"enabled":true,"percent":"-1.0"},"status":{"type":"finished","basic_type":"finished","title":"Finished"},"data":{"cancelable":false,"panic_sell_available":false},"position":1}]},"stop_loss":{"enabled":false},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_1.18_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"5m---_1.18_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"7.059","bid":"0.3367","ask":"0.337","last":"0.3367","quote_volume":"554817.2402"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"0.3389","average_close_price":"0.3342","average_enter_price_without_commission":"0.3386","average_close_price_without_commission":"0.3346","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"29895.0","entered_total":"10132.569447","closed_amount":"29895.0","closed_total":"9992.864133","created_at":"2022-11-30T21:35:02.961Z","updated_at":"2022-11-30T21:35:02.961Z","closed_at":"2022-11-30T21:41:05.739Z","type":"smart_trade"},"profit":{"volume":"-139.705314","usd":"-139.705314","percent":"-1.38","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21614070,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_FORTH","instant":false,"status":{"type":"finished","basic_type":"finished","title":"Finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"2922.6","editable":false},"price":{"value":"3.5","value_without_commission":"3.5","editable":false},"total":{"value":"10239.3291"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[{"id":90587498,"order_type":"market","editable":false,"units":{"value":"2922.6"},"price":{"value":"3.5353","type":"last","percent":null},"volume":"100.0","total":"10332.26778","trailing":{"enabled":true,"percent":"-1.0"},"status":{"type":"finished","basic_type":"finished","title":"Finished"},"data":{"cancelable":false,"panic_sell_available":false},"position":1}]},"stop_loss":{"enabled":false},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_2.09_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"5m---_2.09_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"4.276","bid":"3.17","ask":"3.18","last":"3.17","quote_volume":"2321988.6728"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"3.5","average_close_price":"3.48","average_enter_price_without_commission":"3.5","average_close_price_without_commission":"3.49","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"2922.6","entered_total":"10239.3291","closed_amount":"2922.6","closed_total":"10189.674126","created_at":"2022-11-30T21:35:16.147Z","updated_at":"2022-11-30T21:35:16.147Z","closed_at":"2022-11-30T21:38:28.008Z","type":"smart_trade"},"profit":{"volume":"-49.654974","usd":"-49.654974","percent":"-0.48","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21614045,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_FORTH","instant":false,"status":{"type":"finished","basic_type":"finished","title":"Finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"2994.01","editable":false},"price":{"value":"3.42","value_without_commission":"3.42","editable":false},"total":{"value":"10249.7537142"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[{"id":90587384,"order_type":"market","editable":false,"units":{"value":"2994.01"},"price":{"value":"3.5361","type":"last","percent":null},"volume":"100.0","total":"10587.118761","trailing":{"enabled":true,"percent":"-1.0"},"status":{"type":"finished","basic_type":"finished","title":"Finished"},"data":{"cancelable":false,"panic_sell_available":false},"position":1}]},"stop_loss":{"enabled":false},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_1.58_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"5m---_1.58_1_3_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"4.276","bid":"3.17","ask":"3.18","last":"3.17","quote_volume":"2321988.6728"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"3.42","average_close_price":"3.48","average_enter_price_without_commission":"3.42","average_close_price_without_commission":"3.49","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"2994.01","entered_total":"10249.7537142","closed_amount":"2994.01","closed_total":"10438.6458051","created_at":"2022-11-30T21:32:46.013Z","updated_at":"2022-11-30T21:32:46.013Z","closed_at":"2022-11-30T21:38:27.609Z","type":"smart_trade"},"profit":{"volume":"188.8920909","usd":"188.8920909","percent":"1.84","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21613878,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_ERN","instant":false,"status":{"type":"stop_loss_finished","basic_type":"stop_loss_finished","title":"Stop Loss finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"4544.1","editable":false},"price":{"value":"2.18","value_without_commission":"2.178","editable":false},"total":{"value":"9906.9468498"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"2.1986","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"---_1.44_2_2_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"---_1.44_2_2_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"17.238","bid":"2.188","ask":"2.19","last":"2.19","quote_volume":"20921460.4329"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"2.18","average_close_price":"2.191","average_enter_price_without_commission":"2.178","average_close_price_without_commission":"2.194","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"4544.1","entered_total":"9906.9468498","closed_amount":"4544.1","closed_total":"9959.7856446","created_at":"2022-11-30T21:10:30.745Z","updated_at":"2022-11-30T21:10:30.745Z","closed_at":"2022-11-30T21:20:43.604Z","type":"smart_trade"},"profit":{"volume":"52.8387948","usd":"52.8387948","percent":"0.53","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21611740,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_MLN","instant":false,"status":{"type":"finished","basic_type":"finished","title":"Finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"454.834","editable":false},"price":{"value":"22.34","value_without_commission":"22.32","editable":false},"total":{"value":"10162.04677488"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[{"id":90577322,"order_type":"market","editable":false,"units":{"value":"454.834"},"price":{"value":"22.2069","type":"last","percent":null},"volume":"100.0","total":"10100.4531546","trailing":{"enabled":true,"percent":"-1.0"},"status":{"type":"finished","basic_type":"finished","title":"Finished"},"data":{"cancelable":false,"panic_sell_available":false},"position":1}]},"stop_loss":{"enabled":false},"reduce_funds":{"steps":[]},"market_close":{},"note":"-15m--_3.04_3_1_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"-15m--_3.04_3_1_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"3.55","bid":"20.97","ask":"21.02","last":"21.0","quote_volume":"1099984.39531"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"22.34","average_close_price":"22.06","average_enter_price_without_commission":"22.32","average_close_price_without_commission":"22.09","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"454.834","entered_total":"10162.04677488","closed_amount":"454.834","closed_total":"10037.23577694","created_at":"2022-11-30T18:02:23.278Z","updated_at":"2022-11-30T18:02:23.278Z","closed_at":"2022-11-30T18:04:31.769Z","type":"smart_trade"},"profit":{"volume":"-124.81099794","usd":"-124.81099794","percent":"-1.23","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21610742,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_FIS","instant":false,"status":{"type":"stop_loss_finished","basic_type":"stop_loss_finished","title":"Stop Loss finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"32450.0","editable":false},"price":{"value":"0.3076","value_without_commission":"0.3073","editable":false},"total":{"value":"9981.856885"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"0.3035","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_1.02_0.5_2_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"5m---_1.02_0.5_2_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"2.881","bid":"0.3097","ask":"0.3108","last":"0.3107","quote_volume":"486589.9993"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"0.3076","average_close_price":"0.3006","average_enter_price_without_commission":"0.3073","average_close_price_without_commission":"0.301","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"32450.0","entered_total":"9981.856885","closed_amount":"32450.0","closed_total":"9757.68255","created_at":"2022-11-30T16:37:08.711Z","updated_at":"2022-11-30T16:37:08.711Z","closed_at":"2022-11-30T17:34:47.300Z","type":"smart_trade"},"profit":{"volume":"-224.174335","usd":"-224.174335","percent":"-2.25","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false},{"id":21610660,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_STEEM","instant":false,"status":{"type":"stop_loss_finished","basic_type":"stop_loss_finished","title":"Stop Loss finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"56179.7","editable":false},"price":{"value":"0.1771","value_without_commission":"0.177","editable":false},"total":{"value":"9953.7507069"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"0.176","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":1},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"5m---_0.62_0.5_2_S.BUY_1.5_1_1.5_๐Ÿก‡_1","note_raw":"5m---_0.62_0.5_2_S.BUY_1.5_1_1.5_๐Ÿก‡_1","skip_enter_step":false,"data":{"editable":false,"current_price":{"editable":false,"current_price":{"day_change_percent":"0.985","bid":"0.041","ask":"0.0411","last":"0.041","quote_volume":"3497944.3767"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"0.0414","average_close_price":"0.0415","average_enter_price_without_commission":"0.0414","average_close_price_without_commission":"0.0416","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"238095.0","entered_total":"9866.990133","closed_amount":"238095.0","closed_total":"9894.847248","created_at":"2022-11-29T22:34:16.901Z","updated_at":"2022-11-29T22:34:16.901Z","closed_at":"2022-11-29T22:37:23.134Z","type":"smart_trade"},"profit":{"volume":"27.857115","usd":"27.857115","percent":"0.28","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false}],"statusCode":200,"responseUrl":"https://api.3commas.io/public/api/v2/smart_trades?status=successfully_finished&order_by=closed_at&per_page=100&from=2022-11-29T18:48:08.245Z","redirectList":[],"retry":0,"responseCookies":{"__cf_bm":{"path":"/","expires":"Thu, 01-Dec-22 00:18:09 GMT","domain":".api.3commas.io","SameSite":"None","value":"KAwj2qdahQJ1TeFurGn8NzQvIFbSaR7YszSjTDsInsQ-1669852089-0-AUsA5xatvDcx5f26v/UIqYoSXtpSBFOJNELEGPcJtzEadofX4uTZcx32BuD4y01VrMyh/v/ek//tV1G2h4HNFSA="}}}

Thanks for your patience ...

139

Not sure if "input" is capturing the data. I tested msg.payload = input ... no data

140

I can't identify what is going wrong yet. But no fear. This is an easy problem to solve. Let's take this one piece at a time. Start off commenting out your program like this:

//var check = context.get('recordKeeper');

const input = msg.payload.filter(s => s.status.type == "stop_loss_finished" || s.status.type == "finished");

return({payload:input});

/*for(let obj of input){
   
   if(!(check.hasOwnProperty(obj.id))){
       
        check[obj.id] = obj;
        

        obj.pair = (String(obj.pair)).replace("USDT_", "") 
        obj.trade_id = obj.id
        obj.profit_usd = Number(obj.profit.usd).toFixed(2)
        obj.profit_percent = obj.profit.percent
        obj.position = Number(obj.position.total.value).toFixed(0)
        obj.volume = obj.data.current_price.quote_volume
        obj.close_price = obj.data.average_close_price
        obj.closed = obj.data.closed_at

        if (obj.status.type === "stop_loss_finished")  {obj.status = "SL"} else if (obj.status.type === "finished")  {obj.status = "TP"} else if (obj.status.type === "panic_sold")  {obj.status = "โžฆ"} else  {
        obj.status= obj.status.type
        }


        if (obj.stop_loss.enabled) {
            obj.to_sl = ((1 - obj.stop_loss.conditional.price.value / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "stop_loss_finished") {
            obj.S_L = (((obj.data.average_enter_price - obj.stop_loss.conditional.price.value) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "finished") {
            obj.T_P = (((obj.take_profit.steps[0].price.value - obj.data.average_enter_price) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        obj.in = obj.note.split("_")

        obj.frames = obj.in[0]
        obj.gain = obj.in[1]
        obj.gain_st = obj.in[2]
        obj.freq = obj.in[3]
        obj.analysis = obj.in[4]
        obj.tp_a = obj.in[5]
        obj.tp_trl_a = obj.in[6]
        obj.sl_a = obj.in[7]
        obj.sl_trl_a = obj.in[8]
        obj.tm_a = obj.in[9]
    
  }
}

context.set('recordKeeper', check);
msg = {};
msg.payload = check;

return msg;*/

We'll start off by testing one block at a time. This will see if your input assignment is working as it should be. If you get something out that matches what you're trying to filter, then you know that you're getting input working. Once you get that working, move the comment block (everything between /* and */) from where it's at now to just inside the for loop. Comment everything out of your for loop and just have the for loop spit out obj so you can see it's separating everything as it should. It will look like this:

//var check = context.get('recordKeeper');

const input = msg.payload.filter(s => s.status.type == "stop_loss_finished" || s.status.type == "finished");

for(let obj of input){
   node.send(obj);

   /*if(!(check.hasOwnProperty(obj.id))){
       
        check[obj.id] = obj;
        

        obj.pair = (String(obj.pair)).replace("USDT_", "") 
        obj.trade_id = obj.id
        obj.profit_usd = Number(obj.profit.usd).toFixed(2)
        obj.profit_percent = obj.profit.percent
        obj.position = Number(obj.position.total.value).toFixed(0)
        obj.volume = obj.data.current_price.quote_volume
        obj.close_price = obj.data.average_close_price
        obj.closed = obj.data.closed_at

        if (obj.status.type === "stop_loss_finished")  {obj.status = "SL"} else if (obj.status.type === "finished")  {obj.status = "TP"} else if (obj.status.type === "panic_sold")  {obj.status = "โžฆ"} else  {
        obj.status= obj.status.type
        }


        if (obj.stop_loss.enabled) {
            obj.to_sl = ((1 - obj.stop_loss.conditional.price.value / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "stop_loss_finished") {
            obj.S_L = (((obj.data.average_enter_price - obj.stop_loss.conditional.price.value) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "finished") {
            obj.T_P = (((obj.take_profit.steps[0].price.value - obj.data.average_enter_price) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        obj.in = obj.note.split("_")

        obj.frames = obj.in[0]
        obj.gain = obj.in[1]
        obj.gain_st = obj.in[2]
        obj.freq = obj.in[3]
        obj.analysis = obj.in[4]
        obj.tp_a = obj.in[5]
        obj.tp_trl_a = obj.in[6]
        obj.sl_a = obj.in[7]
        obj.sl_trl_a = obj.in[8]
        obj.tm_a = obj.in[9]
    
  }*/
}

//context.set('recordKeeper', check);
//msg = {};
//msg.payload = check;

//return msg;

You're going to want to make sure your debug node on the output is set to display the whole message object and not just msg.payload. That should spit out several objects from the input. Once you know that's working, uncomment the variable assignment for check and anything associated with it like saving check to recordKeeper and uncomment just inside the if statement. See if you can assign obj objects to check. Then see if you can save it to recordKeeper. Just keep uncommenting blocks a piece an operation at a time until you find something that isn't working right. Keep moving your node.send statement as well to keep up with where you're trying to debug. There's going to be a syntax error in there somewhere and I just can't see what it is right now.

141

I narrowed it down to this; just commenting out /if(!(check.hasOwnProperty(obj.id))){/` outputs one record.

var check = context.get('recordKeeper');

const input = msg.payload.filter(s => s.status.type == "stop_loss_finished" || s.status.type == "finished");

for(let obj of input){
   node.send(obj);

   /*if(!(check.hasOwnProperty(obj.id))){
       
 */       check[obj.id] = obj;
        

        obj.pair = (String(obj.pair)).replace("USDT_", "") 
        obj.trade_id = obj.id
        obj.profit_usd = Number(obj.profit.usd).toFixed(2)
        obj.profit_percent = obj.profit.percent
        obj.position = Number(obj.position.total.value).toFixed(0)
        obj.volume = obj.data.current_price.quote_volume
        obj.close_price = obj.data.average_close_price
        obj.closed = obj.data.closed_at

        if (obj.status.type === "stop_loss_finished")  {obj.status = "SL"} else if (obj.status.type === "finished")  {obj.status = "TP"} else if (obj.status.type === "panic_sold")  {obj.status = "โžฆ"} else  {
        obj.status= obj.status.type
        }


        if (obj.stop_loss.enabled) {
            obj.to_sl = ((1 - obj.stop_loss.conditional.price.value / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "stop_loss_finished") {
            obj.S_L = (((obj.data.average_enter_price - obj.stop_loss.conditional.price.value) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "finished") {
            obj.T_P = (((obj.take_profit.steps[0].price.value - obj.data.average_enter_price) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        obj.in = obj.note.split("_")

        obj.frames = obj.in[0]
        obj.gain = obj.in[1]
        obj.gain_st = obj.in[2]
        obj.freq = obj.in[3]
        obj.analysis = obj.in[4]
        obj.tp_a = obj.in[5]
        obj.tp_trl_a = obj.in[6]
        obj.sl_a = obj.in[7]
        obj.sl_trl_a = obj.in[8]
        obj.tm_a = obj.in[9]
    
  }
//}

context.set('recordKeeper', check);
msg = {};
msg.payload = check;

return msg;

Note that while all of the "non-object" message parts have been eliminated, the object still has all of the fields. Also not seeing anything saved in context.

{"id":21620117,"version":2,"account":{"id":32144558,"type":"paper_trading","name":"Paper Account 1006740","market":"Paper trading account","link":"/accounts/32144558"},"pair":"USDT_LTC","instant":false,"status":{"type":"stop_loss_finished","basic_type":"stop_loss_finished","title":"Stop Loss finished"},"leverage":{"enabled":false},"position":{"type":"buy","editable":false,"units":{"value":"126.678","editable":false},"price":{"value":"79.11","value_without_commission":"79.04","editable":false},"total":{"value":"10022.64174912"},"order_type":"market","status":{"type":"finished","basic_type":"finished","title":"Finished"}},"take_profit":{"enabled":true,"steps":[]},"stop_loss":{"enabled":true,"breakeven":false,"order_type":"market","editable":false,"price":{"value":null},"conditional":{"price":{"value":"77.1465","type":"last","percent":null},"trailing":{"enabled":true,"percent":null}},"timeout":{"enabled":true,"value":3},"status":{"type":"finished","basic_type":"finished","title":"Finished"}},"reduce_funds":{"steps":[]},"market_close":{},"note":"--30m_2.08_0.5_1_S.BUY_2_1_2.5_๐Ÿก‡_3","note_raw":"--30m_2.08_0.5_1_S.BUY_2_1_2.5_๐Ÿก‡_3","skip_enter_step":false,"data":{"editable":false,"current_price":{"day_change_percent":"-0.645","bid":"76.95","ask":"76.96","last":"76.97","quote_volume":"89408754.47852"},"target_price_type":"price","base_order_finished":true,"missing_funds_to_close":0,"liquidation_price":null,"average_enter_price":"79.11","average_close_price":"76.58","average_enter_price_without_commission":"79.04","average_close_price_without_commission":"76.66","panic_sell_available":false,"add_funds_available":false,"reduce_funds_available":false,"force_start_available":false,"force_process_available":false,"cancel_available":false,"finished":true,"base_position_step_finished":true,"entered_amount":"126.678","entered_total":"10022.64174912","closed_amount":"126.678","closed_total":"9701.42434452","created_at":"2022-12-01T14:23:58.707Z","updated_at":"2022-12-01T14:23:58.707Z","closed_at":"2022-12-01T15:30:49.222Z","type":"smart_trade"},"profit":{"volume":"-321.2174046","usd":"-321.2174046","percent":"-3.2","roe":null},"margin":{"amount":null,"total":null},"is_position_not_filled":false,"_msgid":"8fc1ef6a2af03e45"}
142 
var check = context.get('recordKeeper');
node.send({bundle:"Original Message", msg});
const input = msg.payload.filter(s => s.status.type == "stop_loss_finished" || s.status.type == "finished");
node.send({bundle:"Filtered Message", input});
for(let obj of input){
   node.send({bundle:"Object", obj});

   /*if(!(check.hasOwnProperty(obj.id))){
       
 */       check[obj.id] = obj;
        

        obj.pair = (String(obj.pair)).replace("USDT_", "") 
        obj.trade_id = obj.id
        obj.profit_usd = Number(obj.profit.usd).toFixed(2)
        obj.profit_percent = obj.profit.percent
        obj.position = Number(obj.position.total.value).toFixed(0)
        obj.volume = obj.data.current_price.quote_volume
        obj.close_price = obj.data.average_close_price
        obj.closed = obj.data.closed_at

        if (obj.status.type === "stop_loss_finished")  {obj.status = "SL"} else if (obj.status.type === "finished")  {obj.status = "TP"} else if (obj.status.type === "panic_sold")  {obj.status = "โžฆ"} else  {
        obj.status= obj.status.type
        }


        if (obj.stop_loss.enabled) {
            obj.to_sl = ((1 - obj.stop_loss.conditional.price.value / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "stop_loss_finished") {
            obj.S_L = (((obj.data.average_enter_price - obj.stop_loss.conditional.price.value) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        if (obj.status.type == "finished") {
            obj.T_P = (((obj.take_profit.steps[0].price.value - obj.data.average_enter_price) / obj.data.average_enter_price) * 100).toFixed(2)
        }

        obj.in = obj.note.split("_")

        obj.frames = obj.in[0]
        obj.gain = obj.in[1]
        obj.gain_st = obj.in[2]
        obj.freq = obj.in[3]
        obj.analysis = obj.in[4]
        obj.tp_a = obj.in[5]
        obj.tp_trl_a = obj.in[6]
        obj.sl_a = obj.in[7]
        obj.sl_trl_a = obj.in[8]
        obj.tm_a = obj.in[9]
    
  }
//}

context.set('recordKeeper', check);
msg = {};
msg.payload = check;

return({bundle:"Final Message", msg});

Try this. I slipped in outputs for each time you expect a value to be used. I also added the "bundle" property so you know where in the code that message came from. This way we can figure out what is sending the message and what isn't. We'll also have a paper trail on how things have changed during the program. When you get the debug output, be sure to expand all the outputs enough to at least see what the bundle property is.

143

Here it is

image
image567ร—981 27.2 KB

144

Perfect. So we're getting through the filtered array. What about a lot of bundles labeled "Object"? Are there a lot of those?

145

55 ... that's how many transaction have taken place in the last 24 hours or so.

146

Ok. If that's the case, we're looping through the "input" variable like we should. Do we get the "Final Message" bundle in the end? If so, what does it contain?