Join outputs from two python scripts into one message

ED209 · 28 August 2022 22:16

Hi,
First time poster here so be gentle. I'm not a programmer and just dabbling in python and JS to try and learn a bit more.
I have the Docker Pi Sensor Hub board for my Pi and the first script is the sensorhub.py script (which I found from DockerPi SensorHub and Dashboard (flow) - Node-RED) and outputs the sensor information into a json dump.
The second script just reads a file to capture another value and also outputs to a json dump. I want to join the two outputs so that I can process further in a function.
However I'm having trouble joining the outputs of these two separate python scrips into one message. I've tried the join node but not having much luck. I'm changing settings with out really knowing what I'm doing and need so guidance.

I would appreciate any help .

Node-RED version: v3.0.2
Node.js version: v18.8.0

craigcurtin · 28 August 2022 22:57

OK looks like you are on the right track using a join node - can you post up a screenshot of the debug output of the join node and make sure the debug node is set to complete message object (and give it a name so we can identify it)

Craig

ED209 · 29 August 2022 13:53

Hi Craig,
Cheers for the reply. I get two outputs from the debug which I've pasted below.

{"topic":"","_msgid":"55a9749d50875f88","payload":{"topic":"","payload":"{"extTemp": 27, "Brightness": 17, "intTemp": 33, "intHumid": 33, "intSensor": "true", "baroSensor": "true", "baroTemp": 36, "baroPress": 101569, "humanDet": "false"}\n","rc":{"code":0}},"rc":{"code":0}}

{"topic":"","_msgid":"c7b4fb5bacee2a38","payload":{"topic":"","payload":"{"CURRENT_TEMP_STATUS": 25}\n","rc":{"code":0}},"rc":{"code":0}}

zenofmud · 29 August 2022 14:58

see the \n in there? That is going to cause the string not to be able to be transformed into an object.

It might be a result ot the python program. Put a debug and (set to display the complete msg object) on the output of the exec node and see if the output payload shows the \n. If so you could try adding a change node on the output of the exec node and search the payload for \n and replace it with nothing. You can stick another debug node on the change nodes output to see if that works.

ED209 · 30 August 2022 14:56

Think I have the \n removed now but still don't think the messages are being joined. The function isn't working (code pasted below) but not sure if that's because of the join or my shoddy coding.
I also have the json property set to "Always convert to JavaScript object".

{"topic":"","_msgid":"d84dc6e099a5145d","payload":{"topic":"","payload":"{"extTemp": 27, "Brightness": 9, "intTemp": 35, "intHumid": 25, "intSensor": "true", "baroSensor": "true", "baroTemp": 40, "baroPress": 101851, "humanDet": "true"}","rc":{"code":0}},"rc":{"code":0}}

{"topic":"","_msgid":"ec83091c5b25344a","payload":{"topic":"","payload":"{"CURRENT_TEMP_STATUS": 25}","rc":{"code":0}},"rc":{"code":0}}

var x = msg.payload.extTemp;
var d = new Date();     // create current date object for the time string
var baseline = 25;
var last_temp_recorded = msg.payload.CURRENT_TEMP_STATUS;
var tstring = d.toString().substring(0,4) + d.toString().substring(15,21);
 
if (x > baseline){       
    if (x > last_temp_recorded) {
        msg.payload = {
        "chatId": xxxxxxxxxxxxx,
        "sentMessageId": 46,
        "type": "message",
        "content": "Too HOT!! --- The temperature is currently: " + msg.payload.extTemp + " °"
        
        }
    return msg;
    }
return msg;    
}

zenofmud · 30 August 2022 15:22

this is still not a valid object. The second payload is a string with an object imbedded and that is causing an issue.

How are you building it?
Why do you have a payload inside a payload?
where did you find the python code?

ED209 · 31 August 2022 15:22

I'm just using the script that was provided at

Sensors onboard - DockerPi Sensor Hub - NotEnoughTech

Don't know why the output is a payload within a payload

zenofmud · 31 August 2022 18:09

Looking at that site I don’t see anything pointing to a python script. I did not follow every link on the page so if you could indicate the link to the code it would be helpful

ED209 · 31 August 2022 21:02

Sorry, there was a project link to a GitHub post. Here's the direct link to the script

github.com

notenoughtech/NodeRED-Projects/blob/master/DockerPi Sensor Board/SensorBoard_script.py

import smbus
import json

DEVICE_BUS = 1
DEVICE_ADDR = 0x17

TEMP_REG = 0x01
LIGHT_REG_L = 0x02
LIGHT_REG_H = 0x03
STATUS_REG = 0x04
ON_BOARD_TEMP_REG = 0x05
ON_BOARD_HUMIDITY_REG = 0x06
ON_BOARD_SENSOR_ERROR = 0x07
BMP280_TEMP_REG = 0x08
BMP280_PRESSURE_REG_L = 0x09
BMP280_PRESSURE_REG_M = 0x0A
BMP280_PRESSURE_REG_H = 0x0B
BMP280_STATUS = 0x0C
HUMAN_DETECT = 0x0D

This file has been truncated. show original

zenofmud · 31 August 2022 22:07

And (I should have asked this before) can you copy/paste in your version?

Colin · 1 September 2022 06:16

Also add a debug node showing what the output of the script looks like at the point where it gets into node red.

ED209 · 1 September 2022 09:45

I'm actually using the exact same code and haven't modified anything in there..

import smbus
import json

DEVICE_BUS = 1
DEVICE_ADDR = 0x17

TEMP_REG = 0x01
LIGHT_REG_L = 0x02
LIGHT_REG_H = 0x03
STATUS_REG = 0x04
ON_BOARD_TEMP_REG = 0x05
ON_BOARD_HUMIDITY_REG = 0x06
ON_BOARD_SENSOR_ERROR = 0x07
BMP280_TEMP_REG = 0x08
BMP280_PRESSURE_REG_L = 0x09
BMP280_PRESSURE_REG_M = 0x0A
BMP280_PRESSURE_REG_H = 0x0B
BMP280_STATUS = 0x0C
HUMAN_DETECT = 0x0D

bus = smbus.SMBus(DEVICE_BUS)

aReceiveBuf = []

aReceiveBuf.append(0x00)

for i in range(TEMP_REG,HUMAN_DETECT + 1):
    aReceiveBuf.append(bus.read_byte_data(DEVICE_ADDR, i))

if aReceiveBuf[STATUS_REG] & 0x01 :
    extTemp = "overrange"
    #print("Off-chip temperature sensor overrange!")
elif aReceiveBuf[STATUS_REG] & 0x02 :
    extTemp = "false"
    #print("No external temperature sensor!")
else :
    extTemp = aReceiveBuf[TEMP_REG]
    #print("Current off-chip sensor temperature = %d Celsius" % aReceiveBuf[TEMP_REG])


if aReceiveBuf[STATUS_REG] & 0x04 :
    brightness = "overrange"
    #print("Onboard brightness sensor overrange!")
elif aReceiveBuf[STATUS_REG] & 0x08 :
    brightness = "false"
    #print("Onboard brightness sensor failure!")
else :
    brightness = (aReceiveBuf[LIGHT_REG_H] << 8 | aReceiveBuf[LIGHT_REG_L])
    # print("Current onboard sensor brightness = %d Lux" % (aReceiveBuf[LIGHT_REG_H] << 8 | aReceiveBuf[LIGHT_REG_L]))
intSensor = "true"
#print("Current onboard sensor temperature = %d Celsius" % aReceiveBuf[ON_BOARD_TEMP_REG])
intTemp = aReceiveBuf[ON_BOARD_TEMP_REG]
#print("Current onboard sensor humidity = %d %%" % aReceiveBuf[ON_BOARD_HUMIDITY_REG])
intHumid = aReceiveBuf[ON_BOARD_HUMIDITY_REG]

if aReceiveBuf[ON_BOARD_SENSOR_ERROR] != 0 :
    intSensor = "false"
    #print("Onboard temperature and humidity sensor data may not be up to date!")

if aReceiveBuf[BMP280_STATUS] == 0 :
    baroSensor = "true"
    #print("Current barometer temperature = %d Celsius" % aReceiveBuf[BMP280_TEMP_REG])
    baroTemp = aReceiveBuf[BMP280_TEMP_REG]
    #print("Current barometer pressure = %d pascal" % (aReceiveBuf[BMP280_PRESSURE_REG_L] | aReceiveBuf[BMP280_PRESSURE_REG_M] << 8 | aReceiveBuf[BMP280_PRESSURE_REG_H] << 16))
    baroPress = (aReceiveBuf[BMP280_PRESSURE_REG_L] | aReceiveBuf[BMP280_PRESSURE_REG_M] << 8 | aReceiveBuf[BMP280_PRESSURE_REG_H] << 16)
else :
    baroPress = "false"
    #print("Onboard barometer works abnormally!")

if aReceiveBuf[HUMAN_DETECT] == 1 :
    humanDet = "true"
    #print("Live body detected within 5 seconds!")
else:
    humanDet = "false"
    #print("No humans detected!")

result = {
    "extTemp" : extTemp,
    "Brightness" : brightness,
    "intTemp" : intTemp,
    "intHumid" : intHumid,
    "intSensor" : intSensor,
    "baroSensor" : baroSensor,
    "baroTemp" : baroTemp,
    "baroPress" : baroPress,
    "humanDet" : humanDet
}
y = json.dumps(result)
print(y)

And these are the standard out and rc debug messages from the script (exec node). Nothing from standard error.

Standard out

{"topic":"","_msgid":"a92ce3cc42379bd7","payload":"{"extTemp": 25, "Brightness": 84, "intTemp": 30, "intHumid": 38, "intSensor": "true", "baroSensor": "true", "baroTemp": 33, "baroPress": 101501, "humanDet": "false"}\n","rc":{"code":0}}

RC out

{"topic":"","_msgid":"9306b59f21d05a80","payload":{"code":0}}

zenofmud · 1 September 2022 11:09

Why did you originally show code from two python programs?
I’ll take a look at this a little later, I have a date with a two year old I have to take for her second Covid shot

UPDATE:
I just did a file compare and found 4 differenced. Three are meaningless and involve comments but line 67 of the original code is
baroSensor = "false"
while in your code it is
baroPress = "false"

ED209 · 1 September 2022 11:58

The second script reads a file for another value and I want to join this output with the output from the sensorhub output and pass them to a function node.

This is the second script.

### Temperature checker
### v.1.0
### 27-Aug-2022

import os, io, sys, zipfile, shutil, csv, datetime, random, hashlib, ctypes, logging, subprocess, fnmatch, json, pwd

from os import path
from zipfile import ZipFile
from shutil import make_archive
from os import listdir
from os.path import isfile, join
from sys import platform
from pathlib import Path

#global variables
LOG_FILE = ''
LOGFILE_DIRECTORY = ''
TEMP_CHECK_DIR = '/home/pi/RPi_Cam_Web_Interface/Motion Workfiles'
TEMP_CHECK_FILENAME = TEMP_CHECK_DIR + '/Temp_count.txt'

NOW = datetime.datetime.now()


#check what the count number is in the file and add to variable as an integer
with open(TEMP_CHECK_FILENAME, 'r') as file:
    current_temp = file.read().rstrip()
    current_temp_int = int(current_temp)
    #print('File contains: ' + str(current_temp_int))

result = {
    "CURRENT_TEMP_STATUS" : current_temp_int}

z = json.dumps(result)
print(z)

zenofmud · 1 September 2022 13:27

Ok, the \nis just because the exec returns a string. What settings do you have in the JSON node? Is your set like this:
Screen Shot 2022-09-01 at 9.25.51 AM

and what are you doing in the change nodes?

Please export your flow and attach it to a reply.

zenofmud · 2 September 2022 10:07

One other test. Can you install this test python program and call it instead of your current progran. Just change the exec node to point to the location where you install the program

import json

extTemp = "false"
brightness = 15
intSensor = "true"
intTemp = 55
intHumid = 67
baroSensor = "true"
baroTemp = 33
baroPress = 99
humanDet = "true"

result = {
    "extTemp" : extTemp,
    "Brightness" : brightness,
    "intTemp" : intTemp,
    "intHumid" : intHumid,
    "intSensor" : intSensor,
    "baroSensor" : baroSensor,
    "baroTemp" : baroTemp,
    "baroPress" : baroPress,
    "humanDet" : humanDet
}
y = json.dumps(result)
print(y)

as you can see it just simulates the data being passed into NR.

What happens when you run it?
why did you change the line of code I noted in a prior response?

ED209 · 2 September 2022 19:38

Hi,
Sorry I have a newborn here who is taking up most my spare time so only having a quick look at this again. Actually the whole point of this little project was to try and make my own baby monitor.

Honestly I can't remember why I changed the code, it was probably just to know that that particular piece of code is for pressure measurement.

For the change node, I'm removing the \n by using the reg expression option as you had said the join would not work with this there. And for JSON I have the "Format JSON string" box ticked. I have the below screenshot for these settings here: (I'm limited to the amount of images I can upload so merging some of them into one screenshot)

If I run the script you provided with the hard coded values, this is the output I get from :

Change Debug

{"topic":"","_msgid":"9781807e87f5fa51","payload":"{"extTemp": "false", "Brightness": 15, "intTemp": 55, "intHumid": 67, "intSensor": "true", "baroSensor": "true", "baroTemp": 33, "baroPress": 99, "humanDet": "true"}","rc":{"code":0}}

Join Debug
{"topic":"","_msgid":"c263f98d86c34252","payload":{"topic":"","payload":"{"extTemp": "false", "Brightness": 15, "intTemp": 55, "intHumid": 67, "intSensor": "true", "baroSensor": "true", "baroTemp": 33, "baroPress": 99, "humanDet": "true"}","rc":{"code":0}},"rc":{"code":0}}

JSON Debug

{
"topic": "",
"payload": "{"extTemp": "false", "Brightness": 15, "intTemp": 55, "intHumid": 67, "intSensor": "true", "baroSensor": "true", "baroTemp": 33, "baroPress": 99, "humanDet": "true"}",
"rc": {
"code": 0
}
}

And here's the exported json file (I removed the Telegram group ID's). You can ignore the Camera and Configuration flows as they're just something else I'm looking into.

Red-Node-BabyCam-Bkup-02Sep2022.json (30.2 KB)

zenofmud · 3 September 2022 12:06

the exec nodes are returning a valid JSON string that needs to be changed into an object. So after the 'stdout' on the exec nodes - Sensorhub and Read current temp - you need to have a json node connected. You have it for the Sensorhub so exchange the change node on the output of the Read current temp node with a json node.
In the Sensorhub flow - baroSensor path, you have a function node

var x = msg.payload.baroPress / 100;
msg.payload = x;
return msg;

The value shown in the debug for msg.payload.baroPress is 99 so the result of this function will cause msg.payload (note not msg.payload.baroPress) to be .99 In the gauge you need to change the Value format from msg.payload.baroPressure to just msg.payload and you eithor need to change the Range or change the computation in the function node so it will fall within the range you specify in the function node.

The extTemp guage node is trying to display msg.payload.extTemp but that is a true/false according to your debug so it will not display in the gauge.
consider use the link out - /link in nodes to clean up the wires.

Let me kow how it goes and I'll be happy to help you out some more.

system · 2 November 2022 12:06

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Problem with join Node and AHOY DTU PV data General	3	316	31 December 2022
Join messages from individual ''sensors'' into one json object Developing Nodes	25	3635	31 October 2020
Join output from two function nodes General	16	3252	10 April 2020
Combine node outputs General	3	222	31 July 2021
Function node with 2 different outputs General	8	574	6 March 2019

Join outputs from two python scripts into one message

Related topics