Dynamic schedule management

#1

Hi everyone,

So I've been working on a small application with Node-Red and UIbuilder and hit a little wall...

I'm trying to create a list of timers / schedules dynamicly ( it kinda looks like Android alarm clock list ) .

I set up a time and choose which days it should run and enable / disable switch.

But how do I manage the timers?

I've looked at the Later node and some schedulers, but I probably need to have an instance of the node fore each timer I create?

Or do I add nodes dynamically? I saw the Add / Remove nodes documentation, but I'm not sure that's the way to go.

Please point me in the right direction.

Perhaps it would be easier w/o Node-Red...

Thanks

Is there a scheduler/timer that will tell "now" what event will be "next"?
#2

This is something I've been thinking about for a while. All of the timer/scheduler features we have right now are actually fairly simplistic. That is good for most uses of course but it doesn't work for some things including the use case you've mentioned.

I have some design ideas captured but they are still developing and I don't currently have time to cut any code.

So as things stand right now, it is almost certainly easier to do outside of Node-RED with a stand-alone node.js API microservice.

As long as your JavaScript programming capabilities are up to it, using Later.js should make it - certainly not simple - but it would do some of the heavy lifting for you.

If you've the time, I could share the design outline with you - it is just some bullet points - to see if you wanted to create something for Node-RED.

#3

That's what I was afraid of...

Let say I'll go the Node.JS microservice way and use eather Later.js, node-cron or node-schedule to manage the timers.
How would I then interact between NR and this microservice?

What about the idea of adding nodes dynamicly for each timer?

Regardung your design, I'm afraid I don't have enought expirience for that king of thing yet, especially being a C/C++ programmer :slight_smile:
But perhaps you can post it here so people can talk about additional use cases, and maybe some one will want to create that.

Thanks!

#4

You would create an http API. To do that, you simply need to implement ExpressJS into your microservice. At least, that's the easiest approach. Then you implement a "route" which is exposed as a URL. In your front-end code, you can easily interact with that API from a browser on the same network. If you need interactions over the Internet, that's a little more complex as you will need some additional security.

I wouldn't do that. A front-end, browser-based client - which is easily served up by Node-RED - would interact with your API directly and you would manage all of the timers there. You would implement a schedule API which would return the list of active timers so that it could be displayed to the user.

Well, no C would be harmed in the making! It would be all JavaScript. But that's OK, I'll keep thinking about the best approach. When I have time, I'll try to start putting it together, it is a very challenging task to get right and I'm far from sure that I have all the skills I need myself.

I will indeed post the design once I've put some more thought into it, there are lots of people in this forum with far more skills and brain power that me. I want to be sure that I can explain my thinking clearly first.

#5

So once this node.js microservice is up and running I will interact with it trough simple REST commands to manage the timers, and once the timer has fired I would need to listen on NR HTTP in node to receive the payload, correct?

And it's not really important where I locate it in terms of folders on the machine (Raspberry PI ) because it talks through HTTP api?

#6

Yep.

That is certainly one way to do it. Or you could get your microservice to push to MQTT or indeed create a websocket link between your service and Node-RED.

I'd probably use MQTT by preference assuming you are using MQTT. That provides some reliability and resilience - does some of the heavy lifting for you. Of course, Node-RED works very well with MQTT. You can then connect multiple end points to that MQTT topic to pick up the event from multiple services if you ever wanted to. You can even use MQTT in the front-end UI as well if you wanted to.

Absolutely, you could put it on a different device too. As long as it is on the same network, it doesn't really matter. The joys of working with standards-based solutions!

#7

This sounds like a great project, and I will certainly cheer it on -- from the sidelines, of course. With all due respect, however, you may be making it more difficult than necessary, at least in the near term. If all that @scotie needs right now is an "alarm clock" that has multiple alarms that can be added, deleted, and enabled/disabled for each day of the week, I can see ways of doing that inside a function node. If that's not enough, and you want the full capabilities of a bigtimer or schedex node, then by all means go for it.

#8

Of course! Where would be the fun in doing something simple :smile:

I've been thinking about a comprehensive scheduler for Node-RED for quite some time. There are plenty of simple timing and switching nodes but sometimes you need something rather more powerful. That's what I'm thinking about.

Of course, no idea whether I'm capable of implementing it so we will see.

#9

Wouldn't this approach be similar to creating a microservice in terms of managing the timers elsewhere? I mean I would still need to use some module such as node-cron or node-sheduler inside the function node, wouldn't I? Or am I missing something?

#10

I've started making a microservice with express and created a simple API. I tried scheduling a timer with node-cron and it worked.

But to edit this timer later or delete it I need to either return a reference to that timer or interface it in some other way. Perhaps a database or something like that

#11

Nope, you've nailed it, you need an embedded microservice - thankfully that is exactly what Node.js is good at and Node-RED has everything at hand.

While you don't have to use a module, it most probably makes sense.

You want to avoid a database if you possibly can because it adds overheads that you probably don't need. You shouldn't need one here, you can keep the references in memory using JSON structures. You will have to rebuild the schedule on a restart anyway. Using JSON has added benefits since, if you do it right, you can allow overrides and/or schedule replacements to be sent via an inbound msg.

I've started discussing how to build a full-featured scheduler with @Andrei in a private chat. I'll see if I can add you to that chat if you are interested. I've laid out some design ideas and suggested the same approach as you've done - start with a stand-alone microservice to test out the ideas before we start on the actual node.

I wanted to do this outside the general forum simply so that we could agree on a basic approach without too much chatter but if anyone is interested in collaborating, I'm sure Andrei wouldn't mind, I certainly don't.

So I'd suggest using this thread to continue the general sharing of ideas and thoughts and the private chat to begin exploring possible solutions until we are ready to share more widely. Of course, I'm not suggesting that other people might not want to persue their own ideas - I will say this, the basic design idea that I've set out so far is fairly complex in order to achieve a powerful, flexible, general-purpose scheduler for Node-RED. Undoubtedly there is still room for other approaches as well.

@scotie, let me know if you would like adding to the private thread, I think I can do that.

#12

Scotie, indeed you need to store that reference. It is much like using setTimeOut in JavaScript. It will return an ID value of the timer that you may use later on to cancel the timer with clearTimeout() . I can provide you later one (leaving for lunch now) a testing flow that shows how to do that with node-cron inside node-RED function node. It is pretty easy.

#13

This was my original approach. I keep all the relevant parameters in a json file. Every time I make a change - I send the updated version to NR, but I don't yet have a reference to the created timer in the microservice.

That's what I thought, but when i do:

var task = cron.schedule.scheduleJob(schedule, () => {
        console.log("Timer fired");
      }, {
        scheduled: true
});
console.log(task);

I get this:

ScheduledTask {
  start: [Function],
  stop: [Function],
  getStatus: [Function],
  destroy: [Function],
  task: [Function],
  tick:
   Timeout {
     _idleTimeout: 107,
     _idlePrev: [TimersList],
     _idleNext: [TimersList],
     _idleStart: 3246,
     _onTimeout: [Function],
     _timerArgs: undefined,
     _repeat: null,
     _destroyed: false,
     [Symbol(refed)]: true,
     [Symbol(asyncId)]: 15,
     [Symbol(triggerId)]: 14 },
  status: 'scheduled' }

That'll be great, thanks!

If I can be of any help at all, sure thing

#14

Hi @scotie, having a look in your code I realized we are using different libraries. Unfortunately, they are known by the same name cron-node but the one I use is cron.js . My flow example would be useless for you.

In the example you provided you console log the object and you get it done. I can't see what is wrong. Did you call the method start, something like task.start(), to initialize the schedule ?

EDIT: please disregard the question above. You stated the timer is working for you, so I can assume that you used the .start() method. All you want is to know how to reference the timer later to delete it. If this is the case I change my question to: Did you try to use the method .stop() or .destroy() to stop the timer? Perhaps what you want is to use the flow context to store the reference. In such case you would store it in a flow variable, like flow.set("task", task). Later on you can retrieve by doing a flow.get("task") and handle the delete with task.destroy().

#15

I think we are on the same page here. NR, either in a function node or a custom node, can do the job without having to use mqtt/http/websockets/whatever to communicate with another process.

Something that works well in this sort of situation is the Map data structure (a not-as-well-known-as-it-should-be feature of ES6), which makes it easy to add, update, and delete objects. The map can saved in a context variable and made persistent over restarts. There is an example in the function node I posted in another thread.

#16

I think that using a stand-alone microservice is more about playing with the possibilities without having to worry about all of the boilerplate you have to take care of with a custom node. Once you have the basics sorted, then you can port the code back into your node and integrate properly.

Unfortunately, that doesn't work with all browsers nor with all versions of Node.js. So you have to make sure you can restrict yourself and your users to appropriate versions.

In truth though, some careful thought as to your object layout and you won't need Map at all.

#17

I can't argue with any of that.

Always a good thing, and in this case I think it will force you to specify carefully the object representing a schedule and the operations performed on it. Custom nodes always tempt me to shove one more entry into the edit dialog and add a bit of logic...

As you imply, Map really helps keep things organized. It was introduced for good reason, so we can hope the world catches up with it.

#18

Hi @Andrei, that makes sense now. I see that cron.js is a lot more popular and seems to have a lot more options. I will definitely try it out.

Meanwhile I used uuid to generate timer id and stored it timers[timerId] = task
That's the only part I was missing. Once I have a task saved somewhere I can do task.destroy()

#19

@TotallyInformation unless I missed some context in this thread, using map in a function node should be fine as it would run server side & map has support in nodejs is since at least v4.

1 Like
#20

It would be fine if you are using a new enough version of Node.js. Not sure when Map came into node.js I'm afraid.