Extract specific text and associated value from a string?

General
1

Hi,

I’m struggling with this one;

I able to receive a very long string on the status of my HDMI matrix, and what I want to do is extract sections of it to create new/separate payloads to then work with .

Here’s an example of the string produced,

========================================================================================================= =Systems STATUS*= =---------------------------------------- ---------------------------------------------------------------= = Systems information F/W Version : 1.10 = =------------------------------------------------------------------------------------ -------------------= = Power :ON = = Cascading Mode :Disable = =------------------ -------------------------------------------------------------------------------------= = Network Setting Status : = = MAC Address = 7Q:CB:E2:90:05:CA = = Host IP Address = 192.168.111.172 = = Net Mask = 255.255.255.000 = = Router IP Address = 192.168.111.001 = = TCP Port = 0023 = = DHCP : ON = =-------------------------------------------------------------------------------------------------------= = Video Input Setup Status : = = Video Input 01 : EDID = DEFAULT 01, LINK = ON = = Video Input 02 : EDID : User01 EDID, LINK = ON = = Video Input 03 : EDID : User01 EDID, LINK = ON = = Video Input 04 : EDID = DEFAULT 01, LINK = OFF = =-------------------------------------------------- -----------------------------------------------------= = Video Output Setup Status : = = Video Output 01 : Input = 01, Output = ON , LINK = OFF = = Video Output 02 : Input = 01, Output = ON , LINK = ON = = Video Output 03 : Input = 02, Output = ON , LINK = OFF = = Video Output 04 : Inp ut = 01, Output = ON , LINK = OFF = =-------------------------------------------------------------------------------------------------------= = Audio Output Setup Status : = = Audio Output 01 : Analog = Enabled, Digital = Enabled = = Audio Output 02 : Analog = Enabled, Digital = Enabled = = Audio Output 03 : Analog = Enabled, Digital = Enabled = = Audio Output 04 : Analog = Enabled, Digitaled ------------------------------ =*******========================================

I’d like to have created an individual payload for each of the video outputs, one for the text and one focused on the specific output value,

So, for example, 8 individual payloads would be created out of the long string above,

Payload 1 = Video Output 01 : Input = 03
Payload 2 = 3

Payload 3 = Video Output 02 : Input = 02
Payload 4 = 2

Payload 5 = Video Output 03 : Input = 02
Payload 6 = 2

Payload 7 = Video Output 04 : Input = 01
Payload 8 = 1

2

To be certain what we are dealing with can you feed that into a debug node and show us what it looks like?

3

I'd start with regex.

Go Here and develop a capture regex then copy regex from that into a function that outputs your payloads.

Google is your friend. Loads of how tos for js + regex.

If you get stuck, post your progress and others will assist.

4

Hi @Colin

it’s a bit of a mess in the debug window, as the matrix sends the information in a batch, which i then merge/join into one long string to work with ;

Here’s how it ‘a first received (after it’s converted to a string)

image
0C3E9717-F007-43A3-889F-A43270A9966B.jpeg1189×1662 512 KB

5

You could use a change node to
change msg.payload 'replace' all = with nothing and
change msg.payload replace all * with nothing and
change msg.payload replace all * with nothing and
change msg.payload replace all Video Output Setup Status with 'VO Setup Statuschange msg.payload replace allVideo Outputwith '*Video Output

feed that into a split node and split on *

Toss out the first message - that will remove a bunch of the clutter. The next four msg's will have your info and you can interrogate it and so what you want with it.

6

Thanks, i’ll look into that, I also will look again at regex now that I have that link, that looks pretty cool,

Using that tool, I could quickly see that the following regex could match to the parts within the string I’m looking for. But what node do I use to extract those parts so they become their own individual payloads ?

Video Output 0\d : Input = 0\d”

Also - looking at the debug, i’ve just noticed that each payload sent by my Matrix is 255 characters ? Is that something I can work with;, can I set some sort of buffer to capture them all, in order, rather than process them one by one ?

7

There is a section in settings.js that will allow you to show longer data fields in the debug sidebar:

// The maximum length, in characters, of any message sent to the debug sidebar tab
debugMaxLength: 1000,

you can increase it but you will need to stop/start NR.

8

Thanks, that’s good to know, but by the looks of things it’s the HDMI Matrix that’s choosing to send its Status report in chunks ? Are you aware how I can seem less combine them all into one automatically ?

9

I thought you were already joining the data together??

10

Yep, I’m using a join node - but I’m noticing each join introduces a space to the eventual combined string; so if any of the 255 character payloads from the Matrix are broken up and sent mid-word - I will get part of it in one payload and the rest in the other - with that space in between.

You can see examples in the completed joined up string posted earlier - the word “input” became “inp’ and “ut”

Just curious if there was another way to do it.

11

you can set any character you like to join strings with

12

You could pass the input msgs thru node-red-contrib-string (using the method trimLeft()) before joinging the msgs together. That would trim out all leading spaces.

13

I have had a play but before the grand reveal, some questions...

  1. What is the make and model of this device - perhaps they have an alterative interface or output format?
  2. Is this data streaming continuously? Serial? TCP???. If so, then perhaps you could find a suitable character (or characters) to split on so that data is broken up more meaningfully.

So, what i did...
To get you started, I have written some base code you can use to build usable objects from that string e.g...

The results will be JS objects that look like this...

//the video output object

{ output1: { input: 1, output: 'ON', link: 'OFF' },
  output2: { input: 1, output: 'ON', link: 'ON' },
  output3: { input: 2, output: 'ON', link: 'OFF' } 
}

//the audio output object
{
  output1: { analog: 'Enabled', digital: 'Enabled' },
  output2: { analog: 'Enabled', digital: 'Enabled' },
  output3: { analog: 'Enabled', digital: 'Enabled' }  
}

Meaning you can access them like this...
msg.payload.output1.input //1
msg.payload.output2.output) // ON
msg.payload.output3.link // OFF


NOTE...
/Your data appears to have corruption hence not finding output4 on the Video and the Audio string.


The nitty gritty...
If you add the below code to a function block with 2 outputs, you should get Video outputs on output pin 1 & Audio outputs on output pin 2 (I'll let you extend it)

The code for the function block...

var s = msg.payload; //the string to parse

//the regex we use to find video output parts
var videoOutputRegex = /Video Output\D*(\d*).*?Input\D*(\d*).*?Output\W*=\W*(\w*).*?LINK\W*=\W*(\w*)/gm;
// Alt format, less strict
// var videoOutputRegex = /Video Output (\d*) :\W*\D*(\d*)\W*.*?= (\w*)\W*.*?= (\w*)|/gm;

//the regex we use to find audio output parts
var audioOutputsRegex = /Audio Output\D*(\d*).*?Analog\W*=\W*(\w*).*?Digital\W*=\W*(\w*)/gm;


//function to get all matches and thier group items
function getMatches(string, regex) {
  let _matches = [];
  let _match;
  while (_match = regex.exec(string)) {
    _matches.push([..._match]);
  }
  return _matches;
}

//node.warn(matches)


//build an object named video and add properties .output1, .output2 etc.
var video = {};
var vomatches = getMatches(s,videoOutputRegex);
vomatches.forEach(function(m){
  let no = parseInt(m[1]);
  let name = "output" + no;
  if(m.length >= 4){
    video[name] = {
      input: parseInt(m[2]),
      output: m[3],
      link: m[4]
    }
  }
})


//build an object named audio and add properties .output1, .output2 etc.
var audio = {};
var aomatches = getMatches(s,audioOutputsRegex);
aomatches.forEach(function(m){
  let no = parseInt(m[1]);
  let name = "output" + no;
  if(m.length >= 4){
    audio[name] = {
      analog: m[2],
      digital: m[3]
    }
  }
})

//now you can access elements like...
// video.output1.input  // 1
// video.output2.output // ON
// video.output3.link   // OFF

//view the whole object
node.warn(video)
/*
{ output1: { input: 1, output: 'ON', link: 'OFF' },
  output2: { input: 1, output: 'ON', link: 'ON' },
  output3: { input: 2, output: 'ON', link: 'OFF' } 
}

*/
//view the whole object
node.warn(audio)
/*
{
  output1: { analog: 'Enabled', digital: 'Enabled' },
  output2: { analog: 'Enabled', digital: 'Enabled' },
  output3: { analog: 'Enabled', digital: 'Enabled' }  
}
*/

//build the 2 output messages for pin1 and pin2
let msg1 = {
  topic: "video/outputs",
  payload: video
}
let msg2 = {
  topic: "audio/outputs",
  payload: audio
}

return [msg1,msg2]; //remember to set 2 outputs for this function node

Want to see it in action & have a play before incorporating it into node-red?...
see node js playground here.

14

Thanks @Steve-Mcl

I’ll take a look at your suggestion

Here’s the answers to your questions.

15

Ok, this is looking very interesting now ....

I’ve added your code to a function, but with only one debug node, as I’m not sure what you mean about adding two; when there’s only one outlet coming from the function mode ?

image
8FC45B4C-DE8D-4543-B33E-864F22A3C095.jpeg721×2040 373 KB

16

Where you enter the code in the function node, below the code window there is a number spinner. Set it to two.

Alternatively, you can output the two objects on 1 msg...

msg.videoOutputs = video;
msg.audioOutputs = audio;
return msg;