Subflow: errors not being caught by catch node


This is a subflow I put together to make it easier for me to log data to google sheets. I can share more if anyone is interested in what's going on here.
The problem I'm trying to debug is why "append catch" node is missing errors being thrown by the GSheet - Append node.
HEre is how the catch node is configured to show that I have it set up just to catch errors from the GSheet - Append node.

Here is an example of my subflow usage:

And here is the output in the sidebar from when the error slipped through.
error

I did click through the error to get to the GSheet - Append node.

The only thing I can think of is that it might have to do with using the same subflow twice in the same flow. But when I try searching for more info about subflows and catch blocks, I can't find anything that matches my use case.

What I'm expecting is for each subflow to instantiate a set of nodes for each instance of the subflow and that each instance of the catch node is tied to the corresponding instance of the node it is catching errors from.
Do things work differently from what I expect?

Can you catch that error if you use the gsheet node outside of a subflow?

It may simply be the way that node is logging the error isn't catchable.

Yes, I can catch errors from that type of node. I prototyped in a separate flow, then tested small scale with the subflow. This is my first test involving using the subflow twice in the same sheet. That's why I think it might be an issue related to multiple instances of the subflow within a flow. But I'm not able to easily find answers about the behavior of catch nodes within subflows.

I have no experience with the gsheet nodes. But I have had a lot of experience with other nodes that interface with something outside the flow.

First impression, wow. Try not to take anything that follows as offensive. I really am trying to help. I'm looking at the flows and I'm seeing function after function after function. There's a lot of stuff to get lost on working with that. So my first question, is there a reason you can't combine some of the serial functions into one function? Also, is there a reason you have functions before debugs? Are you trying to purge insignificant information? There may be some stuff you're losing by doing that if so.

If one function does something and then another function manipulates that data from the first, you can put that data manipulation in the first and have one function. Depending on what you're joining, that can be a simple .concat() line in your function. Reacting to multiple inputs can be done the same way. There are several ways around having serial functions and nodes that are very simple and easy to implement. So that's my first curiosity.

Second, yes. I never have a lot of success with multiple outputs to the same source. I've made it a goal to always use a single output. If I have multiple inputs going to that output and I need to condition things (i.e. transmission timing and stuff), I'll institute a "gatekeeper" to make sure traffic is handled correctly. That way I avoid duplicate calls and traffic collisions.

Third, I see nodes in there to limit transmission rates. Also something I've never had good luck with. Essentially you're counting on everything working at normal speed without delays. Delays always happen and they tend to go outside your delay window. So I try to work on transaction feedback whenever possible. When one transaction finishes, I start the next rather than start one transaction, wait an expected period then start the next. I don't know how many transaction returns (also called callbacks) there are in these nodes, but that may help things.

Fourth, your error is an authorization error. I don't know if it's an error thrown in your gsheet nodes or somewhere else Node-Red related. Unfortunately, there's no node information in the error. When you mouse-over the error, what node is highlighted? That should be the one that is throwing the error. Is it your append node or a different one?

Is there a reason it has to work this way? I'm just trying to figure out the why of what you're doing. Having a subflow of nodes doesn't make a lot of sense to me when there's something that can simply be done in the main flow. It seems like you're doing a simple read/respond/process/append setup based on what you've shared. First glance makes it seem really overcomplicated.

Again, this is all first impression stuff. I really am trying to be helpful and not sound like I'm tearing code apart. My understanding is limited in this situation and what I do see doesn't match what I would expect to see in a similar circumstance. If I'm wrong about what I'm assuming, I'll gladly take correction. Just let me know.

Your unsolicited criticism, acknowledgment of not understanding the design goals, and not having an answer to my question are noted.

I don't see the Catch node in the subflow image you posted

My first gut feeling would be, you are waiting for a catch to be shown in the debug. But too my knowledge debug node is not working in subflow.

I've been investigating more to see if I can understand what's going on. The issue seems to be intermittent. One of the jobs of the function nodes after the catch nodes is to use node.warn() to emit a marker string I can look for.

I added a path to force the google services to report an error and I can catch that every time. So it appears on the surface that everything is working. I added a catch all node as a sort of backstop and I've been testing to see what comes through there. The catch all node is limited to the subflow and it does indeed appear that the catch nodes are limited to the particular instance they are part of (they have subflow context).

What I think I'm facing now is a logging issue. I need to learn more about ways to send my node.warn() and error messages elsewhere. I'm running Node-Red as a service on a headless Raspberry Pi so I don't think sending messages to the console is a viable solution. I'm also not seeing anything when I run the command "node-red-log".

The reason it's important to catch the errors is because they trigger a retry mechanism. The google sheets API will eventually have errors, it's just the nature of 'cloud' based services.

edit: I seem to have stumbled onto an unofficial log file names openaq.log. This will make those marker messages really useful :slight_smile:

The subflow is the first image. There is append catch and clear catch.
The gsheet node has 4 methods of operation, clear cells, get cells, append row, and update cells. Unfortunately, I can't tell the node which method I want to use in the message so I need to steer the message to the correct node for what I want to do. That also means that unless I want to write code to figure out which of the nodes actually threw an error, it's easiest to have a dedicated node for each of the gsheet nodes.

Some more investigation into the issues I've been having leads me to believe I can can actually remove a lot of the set lock, clear lock, and append data gate nodes which also means I can remove the clear catch node.
It turns out that the google sheets API append method is kind of janky and not great at figuring out what the 'next blank row' is but I thought I was seeing a race condition. It turns out all I have to do is to give the method a 'hint' as to where to look and things are much more consistent.