I'm using the exec node to call a perl script that uses the DateTime module. I installed it and all its dependencies using cpan and the script works on the command line, but when I run it from the exec node in node-red, it complains that it can't find the module. I suppose I could set -I in the command line call, but there must be a way to get node-red to see or inherit the PERL5LIB environment variable?
Out of curiosity, what is this perl script doing?
It is parsing a tvpi file from titantv.com, building a json object that the cronplus node takes as input, and makes an adjustment from GMT to EST time zones. I'm going to use it to schedule HDHomeRun recordings by clicking the record buttons on titantv listings. It works if I call the script with perl -I/home/pi/perl5/lib/perl5 ....
doesnt sound impossible to achieve in native node-red without having to spawn perl scripts? Is it a very complex process?
Perl is my native tongue. It would have taken me much longer to do it natively in node red. I had posted earlier to ask if anyone knew of a way to do datetime math / time zone conversion in node red, but I was too impatient to wait for a response, so I just did it in perl. I'll post about it when I'm done.
I also wrote an applescript folder action to make a webhook call to send the tvpi file contents to node red whenever I click a record button in a listing on titantv.com. This was after spending awhile figuring out how to use VLC to record off my new HDHomeRun.
It's all working now, aside from needing to deal with URL-encoded strings. And I want to write a Siri Shortcut so I can schedule recordings from my phone too.
My original efforts were planned out here: Scheduling VLC Recordings from an HDHomeRun Connect Duo - #5 by hepcat72
Am I right in thinking the tvpi file is just XML?
I found this on a forum...
<tv-program-info version="1.0">
<program>
<station>KCRA-DT</station>
<tv-mode>digital_cable</tv-mode>
<program-title>KCRA 3 News at 6:30pm</program-title>
<start-date>20200513</start-date>
<start-time>01:30</start-time>
<end-date>20200513</end-date>
<end-time>02:00</end-time>
<duration>00:30</duration>
<rf-channel>703</rf-channel>
</program>
</tv-program-info>
so I pushed it through an XML node...
Took less than 1m to convert to JS object.
Still not sure why you would want to spawn out to perl script?
A few things...
- The times in all tvpi files are GMT, so in order to make it work in my timezone, the times must be converted. (This might be possible in cronplus, but I'm having a hard time following the documentation.)
- The structure of that JSON is not compatible with where I need it to go. Here's an example of the JSON that needs to go into the cronplus node:
{ "command": "add", "name": "Law_and_Order", "expression": "0 0 5 8 3 * 2021", "expressionType": "cron", "payloadType": "json", "payload": {"filename": "Law_and_Order_WPPX-TV_61.1_2021.3.8.5.0", "channel": "61.1", "duration": "3590"}, "limit": "null" }
Given this example tvpi input:
<tv-program-info version="1.0">
<program>
<station>WPPX-TV</station>
<tv-mode>digital</tv-mode>
<program-title>Law & Order</program-title>
<episode-title>Evil Breeds</episode-title>
<program-description>Investigating New York City Police Detectives Lennie Briscoe and Ed Green will discover a sinister Nazi agenda could be involved in the gruesome death of an innocent elderly woman, who was violently beaten within her own home.</program-description>
<start-date>20210308</start-date>
<start-time>10:00</start-time>
<end-date>20210308</end-date>
<end-time>11:00</end-time>
<duration>01:00</duration>
<rf-channel>34</rf-channel>
<psip-major>61</psip-major>
<psip-minor>1</psip-minor>
<stream-number>1</stream-number>
</program>
</tv-program-info>
Note that duration is in seconds and the content has been un-url-encoded (e.g. the & was changed to and). The date and time need to be converted to a cron expression and the duration needs to be calculated in seconds. The internal payload is what cronplus outputs at the scheduled time to record with VLC.
While it's a simple matter the straight-convert XML to JSON, and I could have easily done that with node red, as you demonstrate, probably with not much more time investment than you, the time input for me would balloon to figure out all the details of everything else.
3. "Spawn[ing] out to perl" doesn't seem like a huge burden either. It runs in under a second and would be infrequently used. I don't see why someone wouldn't use it if they can do everything they need to do in orders of magnitude less effort and time. While it may be trivial for you to figure out given your knowledge of node red, it is a big ask of me, and I would not have gotten it working yesterday, or at least, wouldn't have gotten the numerous other things I did yesterday done as well.
If you have the interest, and manage to convert all of my efforts into node, and it worked as well or better, that would be really cool, though I wouldn't expect you to do that. But at the same time, I'd also still like to know how to make exec nodes aware of environment variables with PERL5LIB or anything else I care to have it know.
But you would have learned so much (based on your earlier statement "Perl is my native tongue")
none of what you have said (the format conversions etc) are difficult at all in any language tbh.
I would just push that --> XML --> Function node --> done.
At the end of the day, it was just a suggestion - I often see "how do i run this python" "how do I get a value from my C++ program" "how can i run this curl" etc etc etc - they jump through all sorts of hoops & all their script/app really does is to grab a value or format a string!
If you are happy not to learn node-red but jump out to perl scripts when things get hairy, that is your prerogative, but I suspect you are missing out. Each to their own.
If I knew as much about me and what my time constraints and knowledge and past efforts in node red were as you do, I might be tempted to think the same thing you do about me. But I try not to make assumptions like that about people. You're assuming I'm not willing to dive in and learn about node red and javascript, which comes with the implicit assumption that I have ample time in the moment to indulge that longer effort. Do you think that's true? Have I shared information with you that suggests that's wrong? I could make plenty of assumptions about your assumptions, but instead, I try to give people the benefit of the doubt and try to answer the question they asked instead of call their character as a coder into question.
Hi, not sure how I've offended you, it wasn't intended, i only tried to encourage a native solution. I thought that was obvious (but I guess not). I don't think I called your character into question? However, I cannot ignore the fact you are clearly are put out, so i apologise. As you said, I don't know you & you don't know me, but rest assured, I wouldn't do that internationally. I meant no offence.
I missed this question. ↑
How do you launch node-red?
From command line / terminal or does it run as a service?
Also, what account runs node-red?
Try adding this flow - what do you see in debug window?
printenv...
[{"id":"67000189.da769","type":"inject","z":"5e6c8b.7f38b374","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":1420,"y":880,"wires":[["7e0062b2.c73adc"]]},{"id":"7e0062b2.c73adc","type":"exec","z":"5e6c8b.7f38b374","command":"printenv","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"","x":1578,"y":880,"wires":[["bd65cfa7.0626a"],[],[]]},{"id":"bd65cfa7.0626a","type":"function","z":"5e6c8b.7f38b374","name":"","func":"var lines = msg.payload.split(\"\\n\")\nvar envvars = {};\nfor(let i = 0; i < lines.length; i++) {\n let line = lines[i];\n let parts = line.split(\"=\");\n envvars[parts[0]]=parts[1];\n}\nmsg.payload = envvars;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":1754,"y":880,"wires":[["36b59d59.44c5a2"]]},{"id":"36b59d59.44c5a2","type":"debug","z":"5e6c8b.7f38b374","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1924,"y":880,"wires":[]}]
then from a terminal where your perl script works, enter printenv - how do the env vars differ?
lastly,
stop node-red service (assumption) and from the terminal where your perl script works, enter node-red to start up node-red. run the above flow once more - do the correct env vars appear this time?
If so, then you can modify the service to include extra env vars required.
While not a full solution you might gleen some ideas from it.
Edit...
you could pass the env var into the exec to get you moving for now (not a solution, more of a work around)
e.g. in the command, put export PERL5LIB=/home/my/lib && perl /home/app.pl
Just to follow up. Your response inspired the following tests (after saving my sorted user environment in a file on the command line):
[{"id":"81ea3e84.bb722","type":"exec","z":"8b5ff0a3.decbe8","command":"env | sort | diff /home/pi/tmp.env - && exit 0","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"diff env file","x":250,"y":80,"wires":[["95bc6e3b.ff0d48","369ad5ad.dc0352"],["95bc6e3b.ff0d48"],["95bc6e3b.ff0d48"]]},{"id":"95bc6e3b.ff0d48","type":"debug","z":"8b5ff0a3.decbe8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":450,"y":60,"wires":[]},{"id":"71c6a30f.42003c","type":"inject","z":"8b5ff0a3.decbe8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":100,"y":80,"wires":[["81ea3e84.bb722"]]},{"id":"369ad5ad.dc0352","type":"file","z":"8b5ff0a3.decbe8","name":"","filename":"/home/pi/tmp.nr.env.update.diff","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":510,"y":100,"wires":[[]]},{"id":"f542aff2.b9cc18","type":"inject","z":"8b5ff0a3.decbe8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":100,"y":180,"wires":[["3c26a4e4.3455d4"]]},{"id":"3c26a4e4.3455d4","type":"exec","z":"8b5ff0a3.decbe8","command":"perl -e 'use DateTime'","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"perl module env test","x":280,"y":180,"wires":[["33ed4241.31df7e"],["33ed4241.31df7e"],["33ed4241.31df7e"]]},{"id":"33ed4241.31df7e","type":"debug","z":"8b5ff0a3.decbe8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":470,"y":180,"wires":[]}]
Looks like to get the environment, I need to add the --update-env option to pm2.
Thanks for following up.
1 Like
Ah ha, that makes sense now.
Glad its sorted.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
