Stream analytics

[Alex95]

Hello everyone. I have the following problem: i want to send data (temperature,humidity and water level) in Stream Analytics of Azure IoT Hub and after Stream analytics, in power bi. Everytime i send it, the stream analytics recognize only one value (for example, the humidity). So, in "Table" of Stream Analytics i have only one column instead of three columns. What can i do for it?

By the way in node red my outputs are like these:

And my nodes, like these:

Fianally, the function has the following code:

msg.payload = Number(msg.payload)
if (msg.payload>20 && msg.payload<45)
msg.payload = {"temp":msg.payload};
if (msg.payload>=45)
msg.payload = {"humidity":msg.payload};
if (msg.payload>=0 && msg.payload<20)
msg.payload = {"mmwaterValue":msg.payload};
return msg;

[Steve-Mcl]

You only get one value because of the way you have written your function. I can help but first....

What sends the serial values?

What format is the serial data sent? (Show debug output of serial node)

Are you in control of the format of the serial data? If you are, you possibly need to change it.

[Alex95]

hmm ok i understand. Actually, COM5 has the following form:

So, my input deliver ascii strings.
If i answer wrong, i apologize you.

[Steve-Mcl]

You really didn't answer any of my questions...

@Steve-Mcl

What sends the serial values?

What format is the serial data sent? (Show debug output of serial node)

Are you in control of the format of the serial data? If you are, you possibly need to change it.

1. What device sends the serial values?

2. What format is the serial data sent?

   • Show debug output of serial node)

3. Are you in control of the format of the serial data?

   • If you are, you possibly need to change it.

[Alex95]

Actually i am new, so i will think first your questions and then i'll revert you later.

[Colin]

By Debug output is meant feed the serial node into a Debug node and show us what the debug node displays.

As you are a beginner I recommend watching this playlist: Node-RED Essentials. The videos are done by the developers of node-red. They're nice & short and to the point. You will understand a whole lot more in about 1 hour. A small investment for a lot of gain.

[Alex95]

In 1 question i can answer you. The other 2 i will think more. The serial values sended from 2 Arduino. Also, i have added 2 Dragino in every Arduino and i have LoRa communication. So, i send data (temperature,humidity, water leve) from Arduino to node red via LoRa.

[Alex95]

I will see the videos, i believe that it will help me. Thank you very much!!

[Alex95]

As i said yesterday, the serial values sended from 2 Arduino. Also, i have added 2 Dragino in every Arduino and i have LoRa communication. So, i send data (temperature,humidity, water leve) from Arduino to node red via LoRa. So, the first debug message without function is like this:

In this form:

[Steve-Mcl]

The problem with that is any one of those values could be temperature or humidity. There is nothing to indicate which value is which. If the serial read is out of step with your flow, the values can (and will) get assigned to the wrong variable.

You say "from Arduino to node red via LoRa" but I am fairly certain that is not how LoRa sends data (it is normally a fixed length buffer of bytes - where they are always in the same order)

Can you show me the code in your Arduino where you write the values to serial port?

[Alex95]

Hmm ok i undestand this, but why this way is not suitable? Yes i send you the 2 codes Arduino (client and server of LoRa) to see it.

CLIENT:

#include "DHT.h"
#define DHTPIN 4     // what digital pin we're connected to
#define DHTTYPE DHT22   // DHT 11
#include <SPI.h>
#include <RH_RF95.h>
RH_RF95 rf95;
DHT dht(DHTPIN, DHTTYPE);
const int waterSensor = A0; //set water sensor variable
int waterValue = 0;
int mmwaterValue = 0;

void setup() {
  Serial.begin(115200);
  dht.begin();
  if (!rf95.init())
    Serial.println("init failed");
  rf95.setFrequency(868.0);
}
void loop() {
  float h = dht.readHumidity();
  float t = dht.readTemperature();
  float f = dht.readTemperature(true);
  //float a = dht.readHumidity(true);
  int waterValue = analogRead(waterSensor); // get water sensor value
  mmwaterValue = map(waterValue, 0, 1023, 0, 40);
  //Serial.println(mmwaterValue);
  if (isnan(h) || isnan(t)) {
    Serial.println("Failed to read from DHT sensor!");
    return;
  }
  Serial.print("Humidity: ");
  Serial.print(h);
  Serial.print(" %\t");
  Serial.print("Temperature: ");
  Serial.print(t);
  Serial.println(" *C ");
  Serial.print("this is water depth: ");
  Serial.print(mmwaterValue);
  Serial.println("mm");
  String data = String(h) + "-" + String(t)+ "-" +String(mmwaterValue);
  
  int dataLength = data.length(); dataLength++;
  uint8_t total[dataLength]; //variable for data to send
  data.toCharArray(total, dataLength); //change type data from string ke uint8_t
  Serial.println(data);
  rf95.send(total, dataLength); //send data
  rf95.waitPacketSent();
  delay(10000);
}

SERVER:

#include <Adafruit_BusIO_Register.h>
#include <Adafruit_I2CDevice.h>
#include <Adafruit_I2CRegister.h>
#include <Adafruit_SPIDevice.h>

#include <SPI.h>
#include <RH_RF95.h>
#include <Wire.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>
#include <Fonts/FreeMonoBold12pt7b.h>

RH_RF95 rf95;
int led = 13;
unsigned long int millisBefore;
int turn = 0;
int h, t, mmwaterValue;

#define SCREEN_WIDTH 128 // OLED display width, in pixels
#define SCREEN_HEIGHT 32 // OLED display height, in pixels
// Declaration for an SSD1306 display connected to I2C (SDA, SCL pins)
#define OLED_RESET     -1 // Reset pin # (or -1 if sharing Arduino reset pin)
Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, OLED_RESET);
void setup()
{
  pinMode(led, OUTPUT);
  Serial.begin(115200);
  while (!Serial) ; // Wait for serial port to be available
  if (!rf95.init())
    Serial.println("init failed");
  rf95.setFrequency(868.0);
  // SSD1306_SWITCHCAPVCC = generate display voltage from 3.3V internally
  if (!display.begin(SSD1306_SWITCHCAPVCC, 0x3C)) { // Address 0x3D for 128x64
    Serial.println(F("SSD1306 allocation failed"));
    for (;;); // Don't proceed, loop forever
  }
  // Clear the buffer
  display.clearDisplay();
  printText();
  delay(1500);
}
void loop()
{
  // Now wait for a reply
  uint8_t buf[RH_RF95_MAX_MESSAGE_LEN];
  uint8_t len = sizeof(buf);
  if (rf95.waitAvailableTimeout(500))
  {
    if (rf95.recv(buf, &len))
    {
      
      String dataTotal = (char*)buf;
  
      explode(dataTotal);
    }
    else
    {
      Serial.println("recv failed");
    }
  }
  printText();
  display.display();
  delay(300);
  display.clearDisplay();
  
}
void explode(String req) {
  char str[20];
  req.toCharArray(str, 20);
  char * pch;
  pch = strtok (str, "-");
  
  while (pch != NULL)
  {
    String sementara = pch;
    turn++;
    if (turn == 1) {
      
      //Serial.println(h);
      h = sementara.toFloat();
    }
    
    if (turn == 2) {
      
      //Serial.println(t);
      t = sementara.toFloat();
    }
    if (turn ==3) {
      
      //Serial.println(mmwaterValue);
      mmwaterValue = sementara.toFloat();
    }
    pch = strtok (NULL, "-");
    delay(100);
  }
   Serial.println(h);
   delay(5000); // vazoyme to delay etsi wste i node red na ksexwrisei ta dedomena apo to Arduino 
   Serial.println(t);
   delay(5000);  //vazoyme to delay etsi wste i node red na ksexwrisei ta dedomena apo to Arduino
   Serial.println(mmwaterValue);
   delay(5000); //vazoyme to delay etsi wste i node red na ksexwrisei ta dedomena apo to Arduino
  turn = 0;
  
}

void printText() {
  display.setFont(&FreeMonoBold12pt7b);
  display.setTextColor(WHITE);        // Draw white text
  display.setCursor(0, 28);            // Start at top-left corner
  display.print(t);
  display.drawCircle(34, 8, 3, WHITE);
  display.setCursor(40, 27);
  display.print("C");
  display.setCursor(80, 28);
  display.print(h);
  display.print("%");
}

[Steve-Mcl]

What part of this code (in either SERVEr or CLIENT) writes to the serial port where node-red reads it?

[Alex95]

Τhe part of code (of server) is the following:

Serial.println(h);
delay(5000); // vazoyme to delay etsi wste i node red na ksexwrisei ta dedomena apo to Arduino
Serial.println(t);
delay(5000); //vazoyme to delay etsi wste i node red na ksexwrisei ta dedomena apo to Arduino
Serial.println(mmwaterValue);
delay(5000); //vazoyme to delay etsi wste i node red na ksexwrisei ta dedomena apo to Arduino

[Steve-Mcl]

ok, so what can happen is...

When node-red gets a value from the serial port, you can get out of step (due to buffering or intermittent comms)

As all you send is value\nvalue\nvalue\nvalue\nvalue\nvalue\nvalue\nvalue\nvalue\nvalue\n there is no way to understand which value is which value.

So, if you change the code to send valid JSON then all you need to do in node-red is pass the data through a JSON node.

↑ that sounds more complicated than it is...

If you do something like this in the SERVER code...

String json = "{\"humidity\": " + String(h) + ", \"temperature\": " + String(t)+ ", \"water\": " +String(mmwaterValue) + "}";  //make VALID JSON string
Serial.println(json); //Send JSON to node-red

then in node-red, you do this...

SERIAL NODE --> JSON NODE

then you can access the values like msg.payload.humidity and msg.payload.temperature without having to worry about which value is which.

---

ALTERNATIVELY (but not recommended)

you could write the values follewed by a marker (like newline)

Serial.write(h);
Serial.write(t);
Serial.write(mmwaterValue);
Serial.write(13); // << newline

however, the problem with this is if the temperature is 13 or the humidity is 13, you have a different problem!

[Alex95]

ok perfect i will try this and thank you very much!!! So, if this happens, do i have to change the code of function? or to do something else?

[Alex95]

By the way i did this change in the server code and the debug message of node-red is the following:

in this form:

Maybe, is there any problem in this part of code?? i was waiting numbers instead of "}".

[Steve-Mcl]

@Alex95

By the way i did this change in the server code and the debug message of node-red is the following:

As we do a Serial.println(json) it will automatically add a carriage return \r and a newline \n

Serial.println()

Description

Prints data to the serial port as human-readable ASCII text followed by a carriage return character (ASCII 13, or '\r') and a newline character (ASCII 10, or '\n')

So you need to set the split character on the serial port.

[Alex95]

Unfortunately I don't know why it is wrong. I did this change to input:

and i added this part of code in SERVER:

as you said me and the debug is "}" again.

[Steve-Mcl]

That is very odd. Can you change it to split on } as a test. Show me what you get.

[Alex95]

i get exactly this:

And i have made all the changes you told me.