My google skills are failing me. I upgraded to MacOS Sonoma and I screwed up the system perl modules when trying to re-install dependencies of scripts I wrote that are called from osascripts that are called from a couple NR exec nodes.
I since installed perlbrew (which I should have done in the first place, long ago - I also shouldn't have installed modules in the system perl, but it is what it is...), and perlbrew's modules are installed and working correctly. I can call the same osascript from the command line and it works without issue, but that same command in an exec node still apparrently is using /usr/bin/perl.
I tried grepping through the various config files and settings.js, but I cannot figure out how to modify the PATH that node-red uses to find system executables.
I could modify the osascript to add the absolute path to the perl I need, but I don't like hard-coding absolute paths like that.
How are you running node-red? In Ubuntu or Pi OS, if running as a service, then the path is setup in the systemd script, /lib/systemd/system/nodered.service.
I'm running in macos sonoma. My node red version is pretty old. I'm running it with pm2. I plan to upgrade in the near future.
But anyway, I have worked around this issue by soft-linking perlbrew's perl executable in /usr/local/bin, however, I'm now presented with a new problem...
For whatever reason, I now get an error about not being able to find an input file (via a glob) from the perl script, which I cannot reproduce on the command line, and I'm scratching my head on this one...
The globbed path is $ENV{HOME}/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-*.sqlite.
The only hunch I have is that maybe it has to do with an old perl module I wrote called CommandLineInterface. If I remember correctly, it used to have a bug WRT spaces in globbed file paths, but I fixed that a pretty long time ago, and that doesn't explain why it works on the command line.
The only way I can reproduce the error is by running it via the flow. The glob is a hard-coded default in the script. It would be useful if I could just add the --debug flag to find out what's happening, but the output is too copious to view in node-red...
I guess I could output temporarily to a file. What's the easiest way to view STDERR from an exec node?
Install node-red-contrib-flogger, then you can easily send it to a file. Or just configure a debug node to send to the console then it will appear in the node red log.
1 Like
OK. I have identified the root of the problem that determines the difference in behavior of the command line versus the node-RED exec node on Sonoma (i.e. the behavior was the same in my previous MacOS - and it could just have to do with config?).
The difference is boiled down to the use File::Glob ':glob' behavior. My CommandLineInterface.pm module uses a call like this to glob strings into file paths:
perl -e 'use File::Glob ":glob";print(bsd_glob("/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-*-*.sqlite",GLOB_CSH),"\n")'
On my command line, it outputs this:
[golrath:~] robleach% perl -e 'use File::Glob ":glob";print(bsd_glob("/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-*-*.sqlite",GLOB_CSH),"\n")'
/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-A0D24954-1339-494B-9E7D-9D0911C096F8.sqlite/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-B826F8F1-0153-4318-8666-F1C8EBBFCE96.sqlite/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-D8328338-F382-4E34-8C39-E1DC35D60A32.sqlite
But when the same command is run via an exec node (and I confirmed it uses the same perl executable):
The result is:
Now I just want to confirm whether the modules being used are the same or not...
Yep. It's the same version:
[golrath:~] robleach% perl -e 'use File::Glob ":glob";print("$File::Glob::VERSION\n")'
1.37
It must have something to do with the csh/tcsh version that node-RED has access to???
No. Even the csh versions are the same!
[golrath:~] robleach% csh --version
tcsh 6.21.00 (Astron) 2019-05-08 (x86_64-apple-darwin) options wide,nls,dl,bye,al,kan,sm,rh,color,filec
[golrath:~] robleach% tcsh --version
tcsh 6.21.00 (Astron) 2019-05-08 (x86_64-apple-darwin) options wide,nls,dl,bye,al,kan,sm,rh,color,filec
This is so weird.
I posted a stack question about this.
Someone on stack suspects its a permissions issue (the interplay between Sonoma and node-RED). My boss suspects it's an environment issue. Any insights, given those thoughts?
Looks like they were right. Tried just doing an ls in the exec node at every point along the path, and I get Operation not permitted for everything under Group Containers. So it looks like (according to this article I found), I need to grant full disk access to the node-red process/executable. Anything I need to know about that node-red process to do this? Is it only /usr/local/bin/node-red that I need to give access to?
Well, it's not as simple as granting the node-red executable full disk access (i.e. the red.js file that /usr/local/bin/node-red links to). I tried that, restarted node-red, and I still cannot ls the target directories from the exec node.
Any insights on how to grant node-red disk access would be appreciated... Has anyone had to do this before?
What do these show, from cli and node red?
ls -l /Users/robleach/Library
and
ls -l "/Users/robleach/Library/Group Containers"
They show the list of files in both cases with robleach as the owner and group as staff (apple default group for users). They also show owner permissions of rwx. The group and world permissions are either r-xr-x or ------, depending on how deep you go. When I try /Users/robleach/Library/Group Containers/group.com.apple.reminders (or anything deeper), I only get the list of files with the same permissions on the command line. In Node-RED, I get Operation not permitted.
Apple has apparently applied some sort of restrictions on top of what the owner/group permissions you see in the inodes. Even as root, you cannot mess with system files. I had run into a similar issue on a different Sonoma mac last week, but those were low level files. This is the first issue I ran into under a user home directory.
Oh, I forgot I'd left my VNC session going. Here's a snippet of exactly what it shows on the command line:
[golrath:~] robleach% ls -l "/Users/robleach/Library/Group Containers/"
total 0
...
drwx------@ 5 robleach staff 160 Jul 21 11:27 group.com.apple.reminders
...
And here's the reminders one:
[golrath:~] robleach% ls -l "/Users/robleach/Library/Group Containers/group.com.apple.reminders"
total 0
drwxr-xr-x 6 robleach staff 192 Jul 21 17:45 Container_v1
drwx------ 6 robleach staff 192 Feb 18 2022 Library
Oh yeah, and Library is:
drwx------ 70 robleach staff 2240 Jul 22 11:28 Group Containers
The exact output from node-red is:
ls: /Users/robleach/Library/Group Containers/group.com.apple.reminders: Operation not permitted
The other 2 directories' output via node-red appears exactly the same. I didn't look in the log for the precise directories in question.
What does the command whoami show when run in an exec node?
I'll have to check when I get home, but pm2 gave me the process ID 501 and I did an id on my account and it was also 501, so my bet is that whoami will return robleach, but that's a good test. I will report back.
Well, I solved it temporarily in a heavy handed way by granting the node executable full disk access. I posted an answer on stack. It's worth noting that I'm not sure how to grant access to Node-RED specifically, but the MacOS permissions structure is rather complex and has avenues for developers to trigger the system to prompt the user to allow access, instead of it just either silently or which a Operation not permitted error. It involves creating some sort of bookmark file. I didn't dig into it, but a response to my Apple Developer forum cross-post on this issue elicited a helpful response with a link to a permissions explainer post.
Did whoami give the right user?
