# Does anyone do a lot of string word parsing?

**URL:** <https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881>\
**Category:** General\
**Created:** [18 May 2020 21:12 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881 "2020-05-18T21:12:44Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![cymplecy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/cymplecy/32/2773_2.png) [@cymplecy](https://discourse.nodered.org/u/cymplecy)\
**Post date:** [18 May 2020 21:12 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/1 "2020-05-18T21:12:44Z")

</div>

If I switch over to using "Alexa Simon Says" for my house #IoT stuff then I'm going to have to handle all such Alexa requests myself and cope with things such as saying:

"alexa simon says skybox pause"

and getting  
"alexa simon says sky bot paws"

So I'd be wanting to do stuff like

contains "sky b" or "skyb" followed by "pause" or "paws"

or even cope with

"alexa simon says pause skybox"

So - has anyone done any lexilogical parsing type stuff using NodeRED?

Or if not, got a good direction I should look to go in?

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [18 May 2020 23:15 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/2 "2020-05-18T23:15:55Z")

</div>

one way, no need for parsing

create a command object, that is stored in the context data.

```auto
//function to create context data need to edit settings json to make it survive reloads
var utterances = {
"skybox_pause": "command a",
"sky_bot_paws" :"command a",
"pause_skybox": "command a",
"another_1": "command b",
"another_2": "command b",
"more_1": "command c",
"more_2": "command c",
"more_3": "command c"
};
global.set("utterances", utterances);
return;

```

You can then add multiple phrases to one command. By removing the simon says from the incoming alexa phrase, you can retrieve the command for that phrase from the context data.

```auto
//function retrieve command
var incoming_phrase = "simon says skybox pause"
msg.payload = global.get("utterances." + incoming_phrase.substr(11).replace(/ /g, "_"));

```

output msg.payload returns "command a"

Or do similar with a database

Edit/ to fix white space and syntax error

---

<div class="post-metadata">

**Author:** ![JGKK](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/jgkk/32/18515_2.png) [@JGKK](https://discourse.nodered.org/u/JGKK)\
**Post date:** [19 May 2020 05:26 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/3 "2020-05-19T05:26:26Z")

</div>

I have 🙈 You can use fuzzy string matching to cope with some variations and get the most likely/close result from a list of possible sentences. When I did it i included [fuzzball](https://www.npmjs.com/package/fuzzball) in my settings js file and used it in a function node. Its pretty simple to use and really fast even if you compare to a list of thousands of possible strings.  
The other way you could go would be a lot more sophisticated. You could use an nlu engine of some kind. There you could either use something external to nodered like [rasa-nlu](https://rasa.com/docs/rasa/nlu/using-nlu-only/) which is python based or there is [node-red-contrib-ecolect](https://flows.nodered.org/node/node-red-contrib-ecolect) which only works for english which is why i have no experience with that one as I needed it in german for my use case.  
Another way i did which is probably best for very few commands is to use regexes with test() statements in a switch statement. The regex approach works best in a cascading style where you first do a rough filter/sort in a first function and than do the actual intent parsing in a second function for each intent.  
I hope this gives you some ideas 💡  
Johannes

---

<div class="post-metadata">

**Author:** ![cymplecy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/cymplecy/32/2773_2.png) [@cymplecy](https://discourse.nodered.org/u/cymplecy)\
**Post date:** [19 May 2020 22:03 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/4 "2020-05-19T22:03:28Z")

</div>

Like the idea here but its not working for me  
PS \_ I'm stripping off the simon says before it gets to your function node

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/0/1/01200deec98c54f89056715cadce24883106939a.png)

I think we can't have spaces in the global.get line

i.e global.get("utterances." + "skybox pause")

isn't valid because of the space in "skybox pause"

---

<div class="post-metadata">

**Author:** ![cymplecy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/cymplecy/32/2773_2.png) [@cymplecy](https://discourse.nodered.org/u/cymplecy)\
**Post date:** [19 May 2020 22:08 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/5 "2020-05-19T22:08:13Z")

</div>

> [@JGKK](#):
>
> You can use fuzzy string matching

That looks very promising but for having to require fuzzball in settings.js

I always try to avoid things like that as although I could do it - its not easily repeatable for others and I like to share and enjoy anything I do as much as possible

Maybe it'll prompt me into writing my first contrib node - although after several years - I've still not done that 🙂

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [19 May 2020 22:18 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/6 "2020-05-19T22:18:06Z")

</div>

Spaces are fine there.  
Have you stripped the simon says correctly , i.e. have you left white space at beginning

edit/ strick that, it may not like space as i never tested with your phrases.

try Using \_ in comand object keys, then replace all white space with \_ , in the simon says phrase.

---

<div class="post-metadata">

**Author:** ![cymplecy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/cymplecy/32/2773_2.png) [@cymplecy](https://discourse.nodered.org/u/cymplecy)\
**Post date:** [19 May 2020 22:19 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/7 "2020-05-19T22:19:59Z")

</div>

Well -I'm manually setting the string

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/7/2/72329cf9942f09f1df0001dbacbd47de0595e4f9.jpeg)

and without a space it all works

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/7/a/7a91ba4b93bb3dfec0650cc041d57c61f503fffa.jpeg)

So I'm thinking that spaces aren't allowed in object properties (but I'm no JS expert BTW)

My utterances def BTW

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/1/c/1c9a8dfa8ca53decca927b243ac90f8c37d08b5c.jpeg)

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [19 May 2020 22:23 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/8 "2020-05-19T22:23:35Z")

</div>

edited previous post

---

<div class="post-metadata">

**Author:** ![cymplecy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/cymplecy/32/2773_2.png) [@cymplecy](https://discourse.nodered.org/u/cymplecy)\
**Post date:** [19 May 2020 22:27 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/9 "2020-05-19T22:27:35Z")

</div>

> [@E1cid](#):
>
> try Using \_ in comand object keys, then replace all white space with \_ , in the simon says phrase.

Obvious solution - ta 🙂

---

<div class="post-metadata">

**Author:** ![cymplecy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/cymplecy/32/2773_2.png) [@cymplecy](https://discourse.nodered.org/u/cymplecy)\
**Post date:** [21 May 2020 10:40 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/10 "2020-05-21T10:40:13Z")

</div>

One issue I'm having is the when I say "alexa simon says turn tv sound off" - I sometimes get it returning "simon says turn t. v. sound up"

And when I say "alexa simon says turn tv sound on" - I sometimes get it returning "simon says turn t. v. sound up" as well ☹

So I'm going to have to track whether the sound is on or off at the time and act accordingly! 🙂

---

<div class="post-metadata">

**Author:** ![ristomatti](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ristomatti/32/5857_2.png) [@ristomatti](https://discourse.nodered.org/u/ristomatti)\
**Post date:** [21 May 2020 15:35 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/11 "2020-05-21T15:35:18Z")

</div>

> [@cymplecy](#):
>
> So I'm thinking that spaces aren't allowed in object properties

Spaces are OK when accessing the properties using a string key in brackets, e.g. `obj['property key']`. So in this case instead of:

```auto
global.get("utterances." + "skybox pause");

```

you could write:

```auto
global.get("utterances")["skybox pause"];

```

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [22 May 2020 18:25 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/12 "2020-05-22T18:25:32Z")

</div>

Clarification

Does this one just fetch the value of `utterances.skybox_pause` from context?

```auto
global.get("utterances." + "skybox_pause");

```

Does this one fetch the whole object from context and then select the value of "`skybox pause`" from the object.

```auto
global.get("utterances")["skybox pause"];

```

What i am asking is does one have more overhead?

---

<div class="post-metadata">

**Author:** ![ristomatti](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ristomatti/32/5857_2.png) [@ristomatti](https://discourse.nodered.org/u/ristomatti)\
**Post date:** [22 May 2020 18:30 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/13 "2020-05-22T18:30:27Z")

</div>

> [@E1cid](#):
>
> What i am asking is does one have more overhead?

You are correct in that the first one fetches the value directly and the second one fetches the whole object. The performance difference should be neglible though as only the object reference is fetched. My gut feeling is that something like this could matter only If there were tens or hundreds of thousands of these operations happening in a seconds time.

I wouldn't even think about compromising the code readability for something like this.

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [22 May 2020 18:32 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/14 "2020-05-22T18:32:17Z")

</div>

No matter the number, i would say it is best to use as little overhead as possible.  
It does not affect the code just the way you store the keys.

---

<div class="post-metadata">

**Author:** ![ristomatti](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ristomatti/32/5857_2.png) [@ristomatti](https://discourse.nodered.org/u/ristomatti)\
**Post date:** [22 May 2020 18:43 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/15 "2020-05-22T18:43:06Z")

</div>

There's a famous quote ["premature optimization is the root of all evil"](https://stackify.com/premature-optimization-evil/) related to this. If I would be worried about performance on a microsecond level, I would not be using Node-RED. 🙂

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [22 May 2020 18:47 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/16 "2020-05-22T18:47:07Z")

</div>

“Every minute you spend in planning saves 10 minutes in execution; this gives you a 1,000 percent return on energy!”

Quote wars!!!

---

<div class="post-metadata">

**Author:** ![ristomatti](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ristomatti/32/5857_2.png) [@ristomatti](https://discourse.nodered.org/u/ristomatti)\
**Post date:** [22 May 2020 19:23 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/17 "2020-05-22T19:23:07Z")

</div>

> [@E1cid](#):
>
> “Every minute you spend in planning saves 10 minutes in execution; this gives you a 1,000 percent return on energy!”

Exactly. Out of curiosity I wrote a simple function to test this:

```auto
const utterances = {
  "skybox_pause": "command a",
  "sky_bot_paws" :"command a",
  "pause skybox": "command a",
  "another_1": "command b",
  "another_2": "command b",
  "more_1": "command c",
  "more_2": "command c",
  "more_3": "command c"
};
global.set("utterances", utterances);

// Test 1 - fetch nested context value directly
console.log('Fetch nested value directly');
let key1 = 'skybox_pause';

console.time('test 1' );
for (let i = 0; i < 10000; i++) {
  global.get('utterances.' + key1);
}
console.timeEnd('test 1');

console.log();

// Test 2
console.log('Fetch full object, access property');
let key2 = 'pause skybox';

console.time('test 2' );
for (let i = 0; i < 10000; i++) {
  global.get('utterances')[key2];
}
console.timeEnd('test 2');

console.log();

return msg;

```

If you test this with the Node-RED console open you might find the results interesting 🕶.

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [22 May 2020 19:38 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/18 "2020-05-22T19:38:31Z")

</div>

Or you could post your results to save everyone else who might be interested the time.

---

<div class="post-metadata">

**Author:** ![ristomatti](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ristomatti/32/5857_2.png) [@ristomatti](https://discourse.nodered.org/u/ristomatti)\
**Post date:** [22 May 2020 20:01 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/19 "2020-05-22T20:01:54Z")

</div>

I modified the test to allow the JS engine to optimization to settle a bit by running both tests 5 times:

```auto
const utterances = {
  "skybox_pause": "command a",
  "sky_bot_paws" :"command a",
  "pause skybox": "command a",
  "another_1": "command b",
  "another_2": "command b",
  "more_1": "command c",
  "more_2": "command c",
  "more_3": "command c"
};
global.set("utterances", utterances);

for (let i = 1; i <= 5; i++) {
  console.log('--- Round ' + i)
  test1();
  test2();
} 

return msg;

function test1() {
  console.log('Fetch nested value directly');
  let key1 = 'skybox_pause';
  
  console.time('test 1' );
  for (let i = 0; i < 10000; i++) {
    global.get('utterances.' + key1);
  }
  console.timeEnd('test 1');
  console.log();
}

function test2() {
  console.log('Fetch full object, access property');
  let key2 = 'pause skybox';
  
  console.time('test 2' );
  for (let i = 0; i < 10000; i++) {
    global.get('utterances')[key2];
  }
  console.timeEnd('test 2');
  console.log();
}

```

Result:

```auto
--- Round 1
Fetch nested value directly
test 1: 101.016ms

Fetch full object, access property
test 2: 15.530ms

--- Round 2
Fetch nested value directly
test 1: 25.770ms

Fetch full object, access property
test 2: 8.852ms

--- Round 3
Fetch nested value directly
test 1: 18.072ms

Fetch full object, access property
test 2: 9.439ms

--- Round 4
Fetch nested value directly
test 1: 16.101ms

Fetch full object, access property
test 2: 7.403ms

--- Round 5
Fetch nested value directly
test 1: 13.952ms

Fetch full object, access property
test 2: 10.313ms

```

My theory why test 2 performs better is the fact that it doesn't involve string concatenation. But the reason could be also related to how nested object fetching is implemented in `global.get`.

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [22 May 2020 20:22 UTC](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881/20 "2020-05-22T20:22:55Z")

</div>

Now try it with 1000 entries in the object  
and write the obect[property] to a var, then write the var to null. Do same with direct property to.

```auto
Fetch nested value directly
test 1: 24.651ms

Fetch full object, access property
test 2: 46.148ms

```

so for smaller object  
`global.get('utterances')['key']`

for large objects  
`global.get('utterances.' + 'key')`

I removed the concat for the small test, it did reduce, but still the full object was better for small objects.

I put it back in for the 1000 test

[Next page](https://discourse.nodered.org/t/does-anyone-do-a-lot-of-string-word-parsing/26881.md?page=2)