Message queue node that can be indexed

Trying_to_learn · 19 January 2020 01:23

Problem:

small screen. Can only display 1 message.
Big size of messages to queue

I want the node to store/queue messages.
If it gets a next command, it shows the next message.
If it gets a previous command, it shows the previous message.
Wipe wipes the current message.
Wipe ALL wipes all messages.

Messages are output as requested.

This node nearly does it.
Nearly!

TotallyInformation · 19 January 2020 10:57

Not difficult to do in a function node.

Trying_to_learn · 19 January 2020 11:00

(Blush) Yet beyond my skills at this time.

I get the idea of what to do, but just not the details like how to keep the queue and step through it forward and backward.

Then there is the trick of deleting an entry.

dceejay · 19 January 2020 11:51

I guess the best place to start is to learn about arrays - https://www.w3schools.com/js/js_arrays.asp
and then the capabilities you have with them, index, push, pop, delete etc

Trying_to_learn · 19 January 2020 12:02

Its 23:00 local here. Maybe tomorrow.

But thanks.

I may also look at the simple-queue node which I have and see if I can learn from it too.

dceejay · 19 January 2020 12:28

Well that is a queue (which probably uses an array ) - so will be discarding things once they are sent so you can't go back - but yes I guess it may be possible to modify. As the readme says it is based off a flow I wrote that is in the flows library so that may also be useful (as it just uses normal nodes and a function block to do the smarts)

Trying_to_learn · 19 January 2020 12:29

Well, I may look at it and get a feel for how the array is made.

The stepping through may not be too good as it - as you say - wipes once read.
I'm wanting to keep it.

Just saw an edit happen I think.

dceejay · 19 January 2020 12:30

Yes learn about arrays first.

cymplecy · 19 January 2020 15:50

Nice little programming exercise

One thing I'd suggest is to use the concept of a control topic that @drmibell queue nodes use

e.g use a topic called control (it can be anything you like)

So any messages that don't have this as a topic get added to an array

Any messages with topic of control are the ones used to select which message in the array is output or wiped

TotallyInformation · 19 January 2020 17:47

Oops, should have posted this earlier, sorry.

In JavaScript, a queue can be as simple as a list. Better known as an array.

There are JS functions to add/remove to/from the beginning/end of an array, you can look them up easily on the Mozilla Developer Network (MDN) which is the best reference for JS statements and functions.

So the data from a new msg would be added as a new entry to the end of an array (myarray.push(....)). You may wish to trim the array so that it doesn't grow infinitely and eventually run out of memory. You can do that by dropping the first entry of the array, there is a standard function for that (myarray.shift()).

You know the length of the array at any point because there is the myarray.length property. The first element is myarray[0].

To get the last but one element of an array, you can do myarray[myarray.length - 2] or myarray.slice(-2,-1) - you should be able to work out how to access other parts of the array from that.

machadotiago · 19 January 2020 18:48

Well, maybe you could use a FIFO for that, but since you want to access the previous values, the correct way is to play with arrays (as @TotallyInformation brilliantly described above)

Trying_to_learn · 20 January 2020 04:19

I'm about to look at the other nodes I/we mentioned.

But this is what I have so far.

Strangely it doesn't work.
When I try to push data onto/into the array the index is never incremented.

var output = "";                        //  This is what returns the indexed value.
var i = 0;
let queue_array = [];
let index = context.get('SIZE') || 0;
if (msg.topic == "CONTROL")
{
    //  Stuff to be done with  array.
    if (msg.payload == "NEXT")
    {
        //  Display next item
    } else
    if (msg.payload == "PREVIOUS")
    {
        //  Display previous item
        i = index - 1;
        output = queue_array[i];
        msg.payload = output
        return msg;
    } else
    if (msg.payload == "WIPE")
    {
        //  Delete this item
    } else
    if (msg.payload == "WIPE ALL")
    {
        //  Wire the whole array
    }
}

node.warn("saving data to array");

//  This is where I hope the data is added to the array.
queue_array.push(msg.payload);
index = queue_array.length;

//  This never gets above 1
node.warn("Index " + index);

context.set('pointer',index);
context.set('SIZE',index);
return;
//return msg;

On the length part:
From site:
Java script stuff - as earlier mentioned
var x = cars.length; // The length property returns the number of elements

index = queue_array.length;

Looks good enough to me.

(Yes, I have a bad habit of using reserved words, but in the code none of them show up as being commands so I hope that isn't a problem)

Trying_to_learn · 20 January 2020 04:33

I just tried the flow.

Ok.

Observation:

Output 1 The first item on the top of the stack. It is sent only when msg.topic == get .

~~I am sending payloads with `topic == put` and I'm getting output 1 active.~~

Ok, that is behaving itself now.

Output 2
The complete queue in Array format. It is sent every time a new item is added, removed or when msg.topic == list

Likewise. I am getting output form it when I sending msg.topic == put.

Not quite what I wrote, but here's the screen shot.

Trying_to_learn · 20 January 2020 04:53

Please don't take it as I'm upset.

I'm curious.

Looking inside your subflow:
The put node:

var queue = flow.get("queue")


if (!Array.isArray(queue)){
    queue = [];
}else if ( queue.length >= env.get("Buffer_Size") ){
    queue.pop(1);
}

queue.unshift(msg.payload);

flow.set("queue", queue);
msg.payload = queue;

return msg;

Ok,
(physical line numbers)
line 1: Beyond me. But I'm guessing it is getting the queue.
line 4: Ok, if it isn't an array, make it an array.
line 6: Check if the length has changed. I'm not up to speed on env.get but kind of get it.
line 7: POP! That is usually for taking things OFF a stack. As opposed to PUSH.
But that's going back to my machine programming on the Z80. Anyway.
line 10: queue.unshift? Yeah, ok.
line 12: flow.set("queue",queue) Yeah. queue defined in line 10 - kind of.

Nah. Way beyond my skill set.

I'll play with it but it is all Greek to me.
I looked at the other node and it is nearly the same code and at first looks it is just as confusing.

I may try to work out where it is different and see if I can associate that as to being the opposite of what this is doing.

Colin · 20 January 2020 08:00

What do you see if you add a warn to show the length before the push?

Trying_to_learn · 20 January 2020 08:07

It is 0.

Me suspects I am constantly re-making the array, rather than checking if it exists - as per the code in the FIFO subflow.

Thanks.
Stupid me didn't think of that.

Ok, not that easy.

If I replace the single push with this:

queue_array.push("Message 1");
queue_array.push("Message 2");
queue_array.push("Message 3");

I see a starting index of 0 and an ending index of 3. Which seems good.
But if I subsequently inject another message to invoke another 3 writes to the array.....
0 - 3.

So, I am either still wiping the array or something else.

This is how I modified to code:

var output = "";                        //  This is what returns the indexed value.
var i = 0;                              //  This is for steping through the lists.
//let queue_array = [];

var queue_array = flow.get("queue")     //  I'm guessing this checks if the array exists.
if (!Array.isArray(queue_array))
{
    queue_array = [];
}
//      Yeah, I am not using the `else` option here.  Becuase I don't understand it.


let index = context.get('SIZE') || 0;   //  I hope this part works.  (See below)

index = queue_array.length;             //  This should get the *length* of the array.

node.warn("Index " + index);

if (msg.topic == "CONTROL")
{
    //  Stuff to be done with  array.
    if (msg.payload == "NEXT")
    {
        //  Display next item
    } else
    if (msg.payload == "PREVIOUS")
    {
        //  Display previous item
        i = index - 1;
        output = queue_array[i];
        msg.payload = output
        return msg;
    } else
    if (msg.payload == "WIPE")
    {
        //  Delete this item
    } else
    if (msg.payload == "WIPE ALL")
    {
        //  Wire the whole array
    }
}

node.warn("saving data to array");

//queue_array.push(msg.payload);
queue_array.push("Message 1");
queue_array.push("Message 2");
queue_array.push("Message 3");

index = queue_array.length;

node.warn("Index " + index);            //  This shows me 3

context.set('pointer',index);           //  Save the value - for what ever reason.
context.set('SIZE',index);              //  This gets loaded next time (see above)
return;
//return msg;

I'll do a bit more digging/scratching.

Colin · 20 January 2020 08:25

Methinks thee is right.

Trying_to_learn · 20 January 2020 08:32

So when I add the code:

var queue_array = flow.get("queue")     //  I'm guessing this checks if the array exists.
if (!Array.isArray(queue_array))
{
    queue_array = [];
}

That would only happen if it didn't exist.
(As per the FIFO flow)

That is all I can resolve as the test to see if it is an array.

The queue.pop( ) isn't covered in the link supplied about JS arrays. The only pop is for JS popup which is a popup message.

So that has thrown me a curve ball.

And as I said: I take pop as taking something off a stack/queue/what ever. and push is to put things on.
Yeah, ok. I need to learn the difference. But pop can't be a both way command - can it?

So unshift( ) is like a push but rather than putting at the end of the array, it puts it at the start. Hmmmmm......

dceejay · 20 January 2020 08:42

googling " javascript array pop " should be within you by now...
but yes I see you have found unshift... push and pop work on the end of the array and unshift and shift work on the front. (and delete works anywhere)

Trying_to_learn · 20 January 2020 08:46

@dceejay, yeah, I've found the commands.

The order (queueing) I put them in the array is a bit academic to me at this stage.

The problem is the detecting the existence of the array.
if (!Array.isArray(queue))
Seems to be a sticking point.

I added code so if it sets up the array, it node.warn( ) me.
I inject and it sets up the array.
Pressing it a second time, it does the same.

So to me the problem is that - but am I right?

(Remember it is all Greek to me. I simply try to follow what I see.)

Topic		Replies	Views
Queueing input messages Developing Nodes	4	333	2 September 2020
Message debug function node Share Your Projects	7	517	5 January 2022
Help with message queuing (maybe a repeat) General	7	77	27 August 2024
[ANNOUNCE] node-red-contrib-multiple-queue Share Your Nodes	8	792	16 January 2021
HTTP post queue General	7	577	2 May 2021

Message queue node that can be indexed

Related topics