Dumb question about editing a node

location.js currently has:

RED.httpAdmin.get('/location/:cmd/:id', RED.auth.needsPermission('location.read'), function(req, res){
    console.log("req.params.id = " + req.params.id);
    var node = RED.nodes.getNode(req.params.id);
    let circles = node.server.getCircles();
    //let circles = node.server.getCirclesBack(function (circles) {
    //    node.sendCircles(circles);
    //    });
    console.log("circles = " + JSON.stringify(circles));
...

That's what prints the promise json (fulfilled:false,rejected:false) to the log.

I could call res.json(circles) in that code in location.js, but I actually want to process the contents of circles first (to extract the names and IDs to create a select list when I pass it back to the client, because all I want to do is store that selection for the location node to use in filtering the events that the node spits out downstream). The problem is that I can't seem to access the circles content. I currently do:

    let circles_select = {};
    circles_select['lentest'] = 'length = ' + circles.length;
    res.json(circles_select);

and I have a loop to extract the names and IDs from circles, but it never executes because circles.length is null.

It's funny. That commented code was an experiment where I mimicked other code in the repo and was able to get the node to spit out the circles data, which shows up in the debug panel. I just can't seem to pass it to the edit panel...

I just tried your suggestion and moved my loop (to process the circle data) into the oneditprepare code that calls that method on location.js. It ends up just adding that promise json to the select list:

<select id="node-input-circle">
  <option value="test">test</option>
  <option value="sanity"> check </option>
  <option value="isFulfilled"> false</option>
  <option value="isRejected"> false</option>
</select>

To be honest I had a quick look at your node, but it is not clear to me how it works. Have been investigating a lot of nodes in the last years, and they all have a rather similar setup. But not the one you have forked. A bit too confusing for me after a heavy day at work ...

Yes that is executed when you open the edit panel of your node, so in the flow editor (client).

It has an endpoint, so it is server side...

If you want to start debugging your client side nodes, you can easily get started via the chrome debugger (see my tutorial here). For your client side code, you can start the developer tools when the flow editor is open (after you have added a debugger statement e.g. to your oneditprepare function...).

From your code from yesterday:

life360.circles(session)
                .then(...

circles is a promise and not an array, so it has no length.

If I were you I would read a book about Javascript, and start developing/forking a less complex node. And then continue with this node. Otherwise this will become a very long discussion ...

Sigh, you're probably right. I'll probably keep banging my head against this though until I get it, TBH. But point taken.

1 Like

It was not my intention to demotivate you in some way! You have already achieved quite a lot. But you have started with some heavy stuff to be honest...
Just start writing a simple node, to get the feeling with both the server and client part. As soon as you know the basics, start drawing all the parts of that node you have forked. The circles and arrows on your drawing will make things much clearer, of how it works, which functionality/data is located where (client/server). And then determine what you want to change...
And meanwhile there is a bunch of nice guys here to answer your questions, during your journey :wink:

1 Like

I always name my nodes files the same as the name of the node to avoid confusing myself. So uibuilder.js and uibuilder.html for example. The .js file is the runtime and the .html file is the admin/config Editor component. Both run from the "server" which is Node-RED, but the .html file is loaded to your browser as part of the Node-RED Editor, the runtime .js file is back-end only. Server, of course, being a rather overloaded term since it may also apply to the device as well as the service.

I find it better to talk about front-end and back-end. The Node-RED Editor runs in the front-end (your browser) and the Node-RED runtime runs in the back-end (your server device).

It is doubly confusing when you write a node that calls an external service - which, of course, also runs on a (different) server.

Since a Node-RED module (the collection of files with a package.json that defines your nodes and can be installed with npm) may contain multiple nodes, I find it best to avoid using generic filenames.

1 Like

I'm learning more and more as I go here. You guys probably already know this, but node.server.getCircles() (called from location.js via the $.getJSON in the oneditprepare section of location.html) is dealing with promises and as I understand it, the return I get back from circles = node.server.getCircles() will never be the circle data because it executes asynchronously (like a system() call in perl (my native tongue)). So it's like forking a child process. And if I understand correctly, the only way to "get" the data from that call is to supply a callback method and that method is what needs to contain all the code of what I want to do with the retrieved data (e.g. make the select list items).

So my problem seems to be that I don't know how to supply a function to node.server.getCircles() to create the select list so that it updates the "gui" (location.html). I have learned how to edit that function to accept a callback function and actually call it. I've successfully tested it to have it print the circles to the log.

I was looking at the method in oneditprepare in location.html:

        oneditprepare: function() {
            var node = this;
            // Load the available categories from the server
            $.getJSON('location/circles/' + node.id, function(data) {
                // Show all available circles in the dropdown
                let selectedCircle = "";
                for (const [circleId, circleName] of Object.entries(data)) {
                    selectedCircle = circleId;
                    $("<option value='" + circleId + "'> " + circleName + "</option>").appendTo("#node-input-circle");
                }
            });
        }

And I looked at what I understand is the method in location.js that receives that call. I tried to make it retrieve and process the data to set the data parameter of the function passed in(?: function(data) from the first snippet). It seems to set that parameter via res.json(...).

    RED.httpAdmin.get('/location/:cmd/:id', RED.auth.needsPermission('location.read'), function(req, res){
        var node = RED.nodes.getNode(req.params.id);
        let circles = node.server.getCircles();
        let circles_select = {};
        for (var c = 0; c < circles.length; c++) {
            let circle = circles[c];
            let circleId = circle['id'];
            let circleName = circle['name'];
            circles_select[circleId] = circleName;
        }
        if (req.params.cmd === "circles") {
            // Return a hash of all available circle IDs/names
            res.json(circles_select);
        }
    }

The above doesn't work because circles is a promise, not the hoped-for circles object. However, I can enter static values and it does update the select list, so it works in that sense. I just can't populate it from the circle data this way.

Given my current understanding, I need to supply a callback function to the call to getCircles that will do the setting of the select list options, i.e. it will make those .append() calls. I have a copy of the function that does accept a callback function. However, I don't know how to supply that function(data) function from the $.getJSON call in the oneditprepare section of location.html as a callback function. I don't even fully understand the relationship between function(req,res) and function(data). My naive interpretation is that res.json(whatever) is a call to the function(data) method. Is that close to correct? And if so, how do I pass that function "alias" (res.json()) as a callback function to node.server.getCircles(callback)?

I experimented quite a bit to see how things behave, including trying out async-await patterns to see if I could wrangle the code I wrote to work, however with my various attempts at using await, I only ever got back either undefined or {} and I couldn't figure out why. E.g., I tried this in location.js:

    let circles = node.server.getCirclesWait();

with this in server.js:

    async getCirclesWait() {
        var node = this;
        return await node.updateSession(async function (session) {
            return await life360.circles(session)
                .then(circles => {
                    console.log("this gets called after the end of the main stack. the value received and returned is: " + JSON.stringify(circles));
                    return circles;
                })
        });
    }

Result: {}

The closest I got was putting this in location.js:

        var circles;
    // We start an 'async' function to use the 'await' keyword
    let circles = (async function(){
      var result = await node.server.getCirclesWait()
      circles = result;
      console.log('Woo done!', result)
      return result;
    })()
    console.log("circles = " + JSON.stringify(circles));

That is the only code written in location.js that actually had access to the circles data (After "Woo done!"), but the console message right after that showed circles to be undefined.

Oh My God! I got it to work! (Well, occasionally anyway, after a delay.) Plus, it's a hacky way to do it and I'd like it to work better, but as a proof of concept, this is very satisfying. Check it out:

I'd added a data member to the class/instance called circles_for_select. The code to set it is:

        (async function(node){
          var result = await node.server.getCirclesWait()
          node.circles_for_select = result;
          console.log('Woo done!', result)
        })(node)
        sleep(2000);
        let circles = node.circles_for_select;

I had tried multiple versions of setting this node.circles_for_select value, even with sleeps, but none of them worked, so I was surprised when I suddenly saw the select list populate. I think the key was passing in the node.

It's weird though. Sometimes I open up the edit and it never populates. Other times, I open it and it's initially empty, but populates after a few seconds.

I still have all my other various attempts commented in the code. I wonder if some tweak to those strategies based on this (sending the node) would work... But for now, I'm going to stop on a high, save these files, and come back to it later.

The POC solution has inspired me to think of a better solution. I could just save the circles and anything else I need for the dialog (e.g. places in each circle) in the server node. Then I can just access it directly from the location node without any of this promise stuff.

You might want to read something like https://www.loginradius.com/engineering/blog/callback-vs-promises-vs-async-await/ to better understand what you're doing.

With your "pedal to the metal" approach of hacking you might get enough by just skimming through and checking the sample codes for each way of handling async code in JavaScript.

2 Likes

Thanks so much for the tip @ristomatti. I like that characterization too. "Pedal to the metal" approach. It embodies a mix of both pride and shame. :wink: I skimmed a couple similar blog posts. I'll take a look. Although I imagine that I can avoid the promise stuff entirely if I just use the promise callback approach in the server to set a server data member. Then I can just access it directly from the location node and check if its set or not. The location node would then save the selection the user makes (though I haven't gotten to that point yet). I'll have to figure out a way to occassionally refresh that data in the server, but I'll cross that bridge when I come to it.

2 Likes

You seem to be constantly talking about callbacks, promises and async/await like your doing all at once so hence the link. Async/await greatly simplifies handling promises but it's worth noting that a function (or an arrow function) marked as async implicitly returns a Promise. So in essence both are working with promises.

Callbacks in the other hand are what Promises abstract but Promises don't have such a tight relation to callback based syntax. But as in this case it is known getCircles() returns a Promise, you can pretty much forget about callbacks.

Right. getCircles() returns a promise. I did comprehend that. But if I understand correctly (which I may well not), the problem was that in order for me to turn that promise into the circles data (in the context where I was calling it), I needed an await. But, I am inside that RED.httpAdmin.get() (or perhaps that anonymous function(data) that's passed(?) to it(?)) where the select list building code was so I could pass it back to the code in the oneditprepare, so that I could use that data to construct the select list. Since I can't(?) use the await there (because it's not an async function), I needed the callback to be able to set a value I could access.

Alternatively, I could pass the select list construction code as a callback to the server method that retrieves that data, but I don't know how to pass that RED.httpAdmin.get function (or it's anonymous function parameter(?)). (I'm probably using the wrong names of these things.) So since I couldn't use await in a non-async function and I didn't know how to pass the function that constructs the select list, I tried that strategy of setting an instance data member and slept until it was set.

It worked, but yeah, I'm sure I'm not doing it "correctly". Would setting a data member in the server instance be more "correct"?

And don't get me wrong. I do appreciate the link. I will read it. I just spent 6 hours this morning to get to my first hacky success and need to get some real work done. I'll come back to this though.

What if you just modify this first attempt to get the data from the Promise, does it help at all? Something like:

RED.httpAdmin.get('/location/:cmd/:id', RED.auth.needsPermission('location.read'), function(req, res){
    const node = RED.nodes.getNode(req.params.id);
    node.server.getCircles().then((circles) => {
        let circles_select = {};
        for (let circle of circles) {
            circles_select[circle.id] = circle.name;
        }
        if (req.params.cmd === "circles") {
            // Return a hash of all available circle IDs/names
            res.json(circles_select);
        }
      }).catch((err) => console.log(err));
    }

That's nice and clean and explains a lot that seemed like magic to me. I will definitely try that next time I touch the code. I wasn't inclined to think that using things like res and req like that inside the asynchronous .then would work because I honestly have no clue how that's connected to the code in the html file that has that anonymous function(data) method. I had tried using other variables inside a .then, but I was still in a synchronous mindset. Either way, it was clear to me that the code in that anonymous function(data) function (defined in the html file) runs on the value you supply to res.json(). So is res.json() a call to function(data) (the anonymous function that's supplied to $.getJSON)? Because the html that that constructs clearly operates on the contents of the circles_select hash(/associative array(?)). If res.json is a call to that function, then this makes sense. It's just baffling to me why those are connected, because none of these functions involved have the same number of parameters. They're visually different, which makes it hard to follow what's going on. It's like the arguments to function calls and the functions' parameters are different. It's rather disorienting.

I in turn have slight issues following your message as there's no line breaks :wink:

But anyway. it should help you understand what's going on if you read Express documentation which Node-RED uses internally for HTTP API's.

The parameters request and response come with an incoming request passed by the Express framework. Response parameter acts as a callback so the client sending the request will only get a response from Express until it's called. At least this is how I would simplify it.

That's really helpful. And yeah, I'm not the best communicator, but thanks for your patience.

Your code works! At first, I thought it wasn't working, but I learned something that seems to explain why it didn't initially. My hacky code does the same thing.

Neither attempt works until there's been a deploy of the location node. I figure that's because there's no real node returned from RED.nodes.getNode(req.params.id); until after a deploy. Or, perhaps it's because the node's constructor hasn't run? - because the error I see in the log is TypeError: Cannot read property 'server' of null.

So I tried calling node = new LocationNode(req.config); in a conditional, but I get the error:

TypeError: Cannot read property 'id' of undefined

referring to this line (line 19, column 23) in the constructor:

            RED.nodes.createNode(this, config);

So I take it that 'id' is referring to the this. Am I calling the constructor incorrectly - or am I calling it inappropriately (like in the wrong context)?

Can't help with this as I haven't developed any nodes myself, I just know JavaScript. But I'm pretty confident you'll figure it out sooner or later yourself. :slightly_smiling_face:

1 Like

Thanks for the vote oC.

So I found this post which is pretty much about the same thing and I tried to see how it's done. Basically, they're looking up ports using the "server". Following the code, it looks like the ports come from here:

...and ports are returned by serialp. I'm guessing that serialp is the analog of my server. However, that serialp "server" via a require. There's no constructor in the file. I read that require is a node red function to basically import modules.

What I don't understand is what is list() here returning?:

My take-away is that I need to provide access to the server in the same way serialp is providing access in the example and that I need to have a function(?) akin to list that returns circles? Is that on the right track? Because trying to instantiate the location class isn't setting the server data member inside the location object.