Running Python in node-red

sd_dracula · 1 November 2021 16:12

I have a script as follows:

#!/usr/bin/python3
# Script gathering product data from Sofar Solar Inverter (K-TLX) via WiFi logger module LSW-3
# by Michalux

import sys
import socket
import binascii
import libscrc
import json
import os

def padhex(s):
    return '0x' + s[2:].zfill(4)

def hex_zfill(intval):
    hexvalue=hex(intval)
    return '0x' + str(hexvalue)[2:].zfill(4)

os.chdir(os.path.dirname(sys.argv[0]))

inverter_ip=str('192.168.0.90')
inverter_port=int(8899)
inverter_sn=int(1234569)
lang=str('EN')
verbose=str('1')
# END CONFIG

# PREPARE & SEND DATA TO INVERTER via LOGGER MODULE
output="{" # initialise json output

pini=33029 # START REGISTER
pfin=33084 # END REGISTER

SN="\""
SV="\""
HV="\""
DSPV="\""

# Data logger frame begin
start = binascii.unhexlify('A5') # Logger Start code
length=binascii.unhexlify('1700') # Logger frame DataLength
controlcode= binascii.unhexlify('1045') # Logger ControlCode
serial=binascii.unhexlify('0000') # Serial
datafield = binascii.unhexlify('020000000000000000000000000000') # com.igen.localmode.dy.instruction.send.SendDataField
# Modbus request begin
pos_ini=str(hex_zfill(pini)[2:])
pos_fin=str(hex_zfill(pfin-pini+1)[2:])
businessfield= binascii.unhexlify('0104' + pos_ini + pos_fin) # Modbus data to count crc
if verbose=="1": print('Modbus request: 0104 ' + pos_ini + " " + pos_fin +" "+str(padhex(hex(libscrc.modbus(businessfield)))[4:6])+str(padhex(hex(libscrc.modbus(businessfield)))[2:4]))
crc=binascii.unhexlify(str(padhex(hex(libscrc.modbus(businessfield)))[4:6])+str(padhex(hex(libscrc.modbus(businessfield)))[2:4])) # CRC16modbus
# Modbus request end
checksum=binascii.unhexlify('00') #checksum F2
endCode = binascii.unhexlify('15')# Logger End code
inverter_sn2 = bytearray.fromhex(hex(inverter_sn)[8:10] + hex(inverter_sn)[6:8] + hex(inverter_sn)[4:6] + hex(inverter_sn)[2:4])
frame = bytearray(start + length + controlcode + serial + inverter_sn2 + datafield + businessfield + crc + checksum + endCode)
if verbose=="1":
    print("Hex string to send: A5 1700 1045 0000 " + hex(inverter_sn)[8:10] + hex(inverter_sn)[6:8] + hex(inverter_sn)[4:6] + hex(inverter_sn)[2:4] + " 020000000000000000000000000000 " + "0104" + pos_ini + pos_fin + str(hex(libscrc.modbus(businessfield))[3:5]) + str(hex(libscrc.modbus(businessfield))[2:3].zfill(2)) + " 00 15")
if verbose=="1": print("Data sent: ", frame);
# Data logger frame end

checksum = 0
frame_bytes = bytearray(frame)
for i in range(1, len(frame_bytes) - 2, 1):
    checksum += frame_bytes[i] & 255
frame_bytes[len(frame_bytes) - 2] = int((checksum & 255))

# OPEN SOCKET
for res in socket.getaddrinfo(inverter_ip, inverter_port, socket.AF_INET, socket.SOCK_STREAM):
                 family, socktype, proto, canonname, sockadress = res
                 try:
                  clientSocket= socket.socket(family,socktype,proto);
                  clientSocket.settimeout(15);
                  clientSocket.connect(sockadress);
                 except socket.error as msg:
                  print("Could not open socket - inverter/logger turned off");
                  if prometheus=="1": prometheus_file.close();
                  sys.exit(1)

# SEND DATA
clientSocket.sendall(frame_bytes);

ok=False;
while (not ok):
 try:
  data = clientSocket.recv(1024);
  ok=True
  try:
   data
  except:
   print("No data - Exit")
   sys.exit(1) #Exit, no data
 except socket.timeout as msg:
  print("Connection timeout - inverter and/or gateway is off");
  sys.exit(1) #Exit

# PARSE RESPONSE (start position 56, end position 60)
if verbose=="1": print("Data received: ", data);
i=pfin-pini # Number of registers
a=0 # Loop counter
response=str(''.join(hex(ord(chr(x)))[2:].zfill(2) for x in bytearray(data))) #+'  '+re.sub('[^\x20-\x7f]', '', '')));
if verbose=="1":
    hexstr=str(' '.join(hex(ord(chr(x)))[2:].zfill(2) for x in bytearray(data)))
    print("Hex string received:",hexstr.upper())

with open("./SOFARHWMap.xml") as txtfile:
   parameters=json.loads(txtfile.read())
while a<=i:
 p1=56+(a*4)
 p2=60+(a*4)
 responsereg=response[p1:p2]
 val1=chr(int(str(responsereg[0:2]),16))
 val2=chr(int(str(responsereg[2:4]),16))
 hexpos=str("0x") + str(hex(a+pini)[2:].zfill(4)).upper()
 if verbose=="1": print("Register:",hexpos+" ("+str(int(hexpos,16))+"), Value: "+str("0x"+responsereg)+" ("+str(int(str(responsereg),16))+")");
 for parameter in parameters:
  for item in parameter["items"]:
    if lang=="PL":
       title=item["titlePL"]
    else:
       title=item["titleEN"]
    value_type=item["value_type"]
    for register in item["registers"]:
     if register==hexpos:
      for option in item["optionRanges"]:
       if option["key"] == int(str(responsereg),16):
           if lang=="PL":
               output=output+"\""+ title + "\":" + str('"'+option["valuePL"]+'"')+","
           else:
               output=output+"\""+ title + "\":" + str('"'+option["valueEN"]+'"')+","
      if value_type=="SN": SN+=val1+val2;
      if value_type=="SV": SV+=val1+val2;
      if value_type=="HV": HV+=val1+val2;
      if value_type=="DSPV": DSPV+=val1+val2;
 a+=1
if lang=="PL":
    output=output+"\"Numer seryjny" + "\":" + SN +"\","
    output=output+"\"Wersja oprogramowania" + "\":" + SV +"\","
    output=output+"\"Wersja sprzętowa" + "\":" + HV +"\","
    output=output+"\"Wersja DSP" + "\":" + DSPV +"\","
else:
    output=output+"\"Serial Number" + "\":" + SN +"\","
    output=output+"\"Software Version" + "\":" + SV +"\","
    output=output+"\"Hardware Version" + "\":" + HV +"\","
    output=output+"\"DSP Version" + "\":" + DSPV +"\","
output=output[:-1]+"}"
jsonoutput=json.loads(output)
print(json.dumps(jsonoutput, indent=4, sort_keys=False, ensure_ascii=False))

Is something like that possible or not really because of all the imports?

jbudd · 1 November 2021 16:18

You can call your Python script from a Node-Red exec node. It's output will be copied to msg.payload.

Steve-Mcl · 1 November 2021 16:21

By the looks of that code it's simply doing modbus reads & data conversion.

You can achieve this with the modbus nodes & buffer-perser in node red without shelling out to python at all.

sd_dracula · 1 November 2021 16:25

That would be great, I just need the values back but it is a little over my skillset, the python was not written by me but I have managed to get it to work and tweak it enough to send the data I need.

Is there a how to for running python that I can reference, thanks?

TotallyInformation · 2 November 2021 11:01

Category changed to general as this isn't about creating custom nodes.

krambriw · 2 November 2021 13:18

Not really but is has been discussed many, many times before. So a simple search from your side would have given you many hits

Anyway, here are two examples, assuming you would like to start a run your script "as is", externally from NR

The first example will run your script hidden and hopefully provide it's output to the debug node for inspection. The second example, will run the script visible on your desktop so this could be better to start off until you have it working as you want

To make it more a real NR flow, I would agree with @Steve-Mcl, there are nodes that most likely will help you to solve the task in a more elegant NR way, besides other features NR will provide (like reconnection handling, status monitoring etc)

sd_dracula · 2 November 2021 14:48

Doesn't like the libscrc module it seems

"Traceback (most recent call last):↵  File "/data/example.py", line 8, in <module>↵    import libscrc↵ModuleNotFoundError: No module named 'libscrc'↵"

I am running NR inside docker and tried a pip3 install libscrc but I get this:

Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement libscrc (from versions: none)
ERROR: No matching distribution found for libscrc

I assume I am doing something wrong

Steve-Mcl · 2 November 2021 16:19

Honestly, ditch the python, read the posts mentioning modbus - its not difficult.

davidz · 2 November 2021 16:49

Yes I agree with Steve.

Since Node-RED has the interface already, it the the most efficient way to use the node directly instead of relying on the EXEC node. The EXEC node adds another layer of uncertainty.

You may use Python to do some analytic work later on (Python has lots of such libraries). Or just write your own analytic functions with JS if you want some adventures .

sd_dracula · 2 November 2021 17:23

I would like to but it looks like some non standard modbus communication and nothing worked so far except this script.
See the code above, it uses socket instead?

dceejay · 2 November 2021 19:01

Life is short. If it works go with it. Optimise it later.

wb666greene · 3 November 2021 01:03

In the immortal words of Donald Knuth:
"Premature optimization is the root of all programming evil"

krambriw · 3 November 2021 06:03

I would assume this too, that is something happening from time to time

You say you made the script working, now you say it does not!? How & where did you run it when it worked? Outside of Docker??
If you run it in Docker you need to have all the dependencies installed there. The script must work also stand-alone inside Docker before you can expect it to work when started by NR. You have to google for advices how to install all of them
What platform are you running on?

sd_dracula · 3 November 2021 09:55

So I can run the code on windows via VS code just fine and it outputs as expected, in node-red I am not able to, that runs in docker (which is on windows) but when I try to install the modules from CLI I get the error above.
I suppose the first thing to resolve is this:

How do I install these guys in docker NR?

import sys
import socket
import binascii
import libscrc
import json

TotallyInformation · 3 November 2021 10:38

:sigh: Docker problems. Do you actually NEED Docker?

If not, get rid of it since it is just getting in the way. Not as though it is hard to install Node.js and then node-red directly in Windows and it really has no impact on other parts of Windows.

sd_dracula · 3 November 2021 11:17

I am only using it until I can get the code to run on NR and then I plan to move the code to my Home Assistant NR instance.
Reason I don't test there is because I broke NR there a few times and I need it always running for now.

TotallyInformation · 3 November 2021 11:30

I would recommend running 2 instances of Node-RED if you only have 1 device available. One live and one for dev. Then you can mess with your dev version as much as you like without impacting live and move stuff over to live when you know it is safe.

NR is really stable generally so a live instance should be usable over long periods with no issues. Docker doesn't really help there.

krambriw · 3 November 2021 12:21

Of course, this is mandatory if you still want it to work in Docker. And how to is really not a NR topic

A bit of advise, verify first what you need to install. In Docker, open a CLI, type python3
Then type import sys and check. Then the others and you will find those that needs to be installed. But how to this is something you really should ask elsewhere, in a python/docker forum or so

sd_dracula · 3 November 2021 13:56

Looks like I only need to install libscrc as the others are already available. This is the part I'm stuck at but even when running the python code from within NR it was complaining about all those imports even though they are there.
I'm not married to docker by any means, just had it installed already.
I've spun up a windows instance via njs also

system · 2 January 2022 13:56

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Run python script in Node red and export with MQTT General	16	8423	7 November 2020
Python Script into .JSON file for node red? Developing Nodes	92	3149	24 September 2021
Help me run this python script General	7	184	2 March 2023
Python in Node Red General	11	4011	1 September 2020
Running python in node-red, passing back a msg General	61	4772	12 November 2020

Running Python in node-red

Related topics