Node-red-contrib-voice2json

first part in the installation where i was errorless :slight_smile:
I am gonna make your HE PIPS wav-files this evening. Get two guest, so 6 times 10 wav files....
After that i need your email, i will transfer the zip to you with wetransfer

1 Like

Hello,
I just stumbled upon a solution for the precise problem in a thread on the Rhasspy forum.
If you go to the precise folder and than open the setup.py script and look for the requirements and than change the line containing h5py to h5py<3.0.0 it will work. Just change that line and than run the setup.sh script again. It will now install a previous version of h5py and you won’t get the decode error anymore.

hi Johannes, Thanks for the tip! I wil try it this weekend. Was a bit busy, new job...

1 Like

I did a precise-train-incremental hey-bram.net /home/pi/mycroft-precise/hey-bram/ -r /home/pi/mycroft-precise/ruis-chunks/ -e 50 -th 0.4 -s 0.5

I think it worked,
i got files on
/home/pi/mycroft-precise/hey-bram/test/not-wake-word/generated

and did

a. cp /home/pi/mycroft-precise/hey-bram/test/not-wake-word/generated/* /home/pi/mycroft-precise/hey-bram/not-wake-word/generated
b. precise-convert hey-bram.net

c. i had on /home/pi/mycroft-precise/hey-bram and renamed that to _org
d. i made a fresh /home/pi/mycroft-precise/hey-bram
e. cp /home/pi/mycroft-precise/hey-bram.pb
/home/pi/mycroft-precise/hey-bram/

f. So what now, @JGKK?
a. think i must make a change in the kaldi profile in node-red to use hey-bram.pb ? where can i find how to do that, i tried myself but does not work...

by adding this to my kaldi profile in node-red? does not work...?

"wake": {
  "system": "precise",
  "precise": {
    "model": "hey-bram.pb",
    "sensitivity": 0.5,
    "trigger_level": 3,
    "chunk_size": 2048
  }
}

name: "nl_kaldi-cgn"
version: "1.1"

language:
name: "dutch"
code: "nl"

text-to-speech:
espeak:
voice: "nl"

speech-to-text:
acoustic-model-type: "kaldi"
kaldi:
model-type: "nnet3"

wake: {
system: "precise",
precise: {
model: "hey-bram.pb",
sensitivity: 0.5,
trigger_level: 3,
chunk_size: 2048
}
}

training:
acoustic-model-type: "kaldi"
kaldi:
model-type: "nnet3"
large-files:
- !env "{profile_dir}/base_dictionary.txt" - !env "{profile_dir}/base_language_model.txt"
- !env "{profile_dir}/base_language_model.fst" - !env "{profile_dir}/g2p.fst"
- !env "{profile_dir}/acoustic_model/model/final.mdl" - !env "{profile_dir}/acoustic_model/model/graph/HCLG.fst"

I think, i got it working! endly !

Only thing i must improbe is that it reacts to offten on non-wakewords amd i get this log:

Logging:
wake-word detetected but ignoring as as audio is already beeing forwarded or listening paused

Question how can i make the wakeword better? so that it does not respond to other commands?

For others:
This is the part to add to the profile for kaldi

wake-word:
sensitivity: 0.5

precise:
# Path to precise-engine executable.
# Use $PATH if empty.
engine-executable: ""
model-file: !env "/home/pi/mycroft-precise/hey-bram/hey-bram.pb"

Sorry just saw it. You can drop the !env and just use the path :+1:t2:

  • add more random audio to train against
  • decrease the noise you add to the duplicates by adding a -v argument to the noise script that decreases the level of the noise in relation to the wake word:
#!/bin/bash

NOISEDIR=$1

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

for f in *.wav
do 
    NOISEFILE=$(find ${NOISEDIR} -type f | shuf -n 1)
    
    sox -m $f -v 0.5 ${NOISEFILE} noise.$f trim 0 `soxi -D $f`
done
  • decrease the -th or -s value in the training, not sure which one was responsible for overal sensitivity
  • add more wake word samples

this is because it triggers the wakeword on you speaking your commands. So record exactly this, you speaking commands and ad those recordings to the random audio for training.

When everything is working i will publish my procedure and flows and scripts.

At the end i made a folder /home/pi/mycroft-precise/hey-bram-exec
and placed the created hey-bram.pb* in it.

Can i delete below files from the precise folder
( i have a backup on home/pi for:

  • 5minfiles
  • wakewords
  • chunked files (small fragments from 5 min files)

I like to delete
Hey-bram* (log, net/epoch/pb*/trained*) on /home/pi/mycroft-precise
( i copied hey-bram.pb* in the executionfolder that is used by node-red)

The only thing you need are a folder with all files that have a pb.
Those are three:

  • *.pb
  • *.pb.params
  • *.pbtxt

You can put that folder anywhere accessible by voice2json.
You can delete anything else.

Ok,
I made a small procedure that uses
three scripts
record.sh (for recording wakewords)
chunk.sh (for making chunks from the 5minutes noisefile)
mergefiles.sh (for merging wakewords with noise)

my wake word= hey-bram

and a flow (5 minutes recordings)

After creating every again (with this procedure), with more noise files of 5minutes (and chunks)
Recognition of my-bram is none, so i deleted the wake word the part in the profile and everythings works ok with hey-mycroft.
I think i better wait for someone to make a general wakeword like hey-computer.
Pitty that the wakeword is not configurable like the others voice commands, by adding a sentence in a file. That recognition parts for my devices works great, and looks like the same functionality...

Procedure

	a. Create on home/pi/hey-bram
		cd /home/pi
		mkdir /home/pi/5min-ruisfile
		mkdir /home/pi/ruis-chunks

		mkdir /home/pi/hey-bram/wake-word
		mkdir /home/pi/hey-bram/not-wake-word
		mkdir /home/pi/hey-bram/test
		mkdir /home/pi/hey-bram/test/wake-word
		mkdir /home/pi/hey-bram/test/not-wake-word
	
	c. Copy scripts
		cp /home/pi/scripts/record.sh /home/pi/hey-bram/wake-word

	d. Record 4 * 10 wakewords  (4 differtent persons) 
		cd /home/pi/hey-bram/wake-word
		bash record.sh 1 

	e. Record 10-20 hours ruis files and make parts of it
		Activate flow  (made a flow for it)

	f.  Copy 
		cp -r /home/pi/hey-bram /home/pi/mycroft-precise
		cp /home/pi/scripts/mergefiles.sh /home/pi/mycroft-precise/hey-bram/wake-word

	g.  Mergen 
		cd /home/pi/mycroft-precise/hey-bram/wake-word
		bash  ./mergefiles.sh /home/pi/ruis-chunks

	h.  cp /home/pi/mycroft-precise/hey-bram/wake-word/* /home/pi/mycroft-precise/hey-bram/test/wake-word/
	 (and delete 80% of the files)
          rm  /home/pi/mycroft-precise/hey-bram/wake-word/mergefiles.sh

	i.  trainen
		cd /home/pi/mycroft-precise
		source .venv/bin/activate
		precise-train hey-bram.net /home/pi/mycroft-precise/hey-bram/ -e 100 -s 0.5

	j.  trainen
		cd /home/pi/mycroft-precise
		source .venv/bin/activate
		precise-train-incremental hey-bram.net /home/pi/mycroft-precise/hey-bram/ -r /home/pi/ruis-chunks/ -e 50 -th 0.4 -s 0.5

	k.  Copy optional (copy test/not-wake-words/generated/* to to the /not-wake-words/generated /* and retrain with the first command from i, then j again)
		cp /home/pi/mycroft-precise/hey-bram/test/not-wake-word/generated/* /home/pi/mycroft-precise/hey-bram/not-wake-word/generated
		restart procedure form i:)

	l. When finished convert to pb: 
		cd /home/pi/mycroft-precise
		precise-convert hey-bram.net

	m.  copy your-wake-word.pb, your-wake-word.pbparams and your-wake-word.pbtxt to the exec folder:
		mkdir /home/pi/mycroft-precise/hey-bram-exec (must be specified in kaldi profile)
		cp  /home/pi/mycroft-precise/hey-bram.pb* /home/pi/mycroft-precise/hey-bram-exec

	n. remove old files
  	    sudo rm /home/pi/mycroft-precise/hey-bram.*
	    sudo rm -r /home/pi/mycroft-precise/hey-bram.logs
	o. ready. 
            you can start the listener in Node Red

Another thing that looks weird (but no errors) is the merge script.
The size of the merged files is the same as the original wakeword. I expected a size much greater because the chunk should be added.

This is the original merge script i use with
bash mergefiles.sh /home/pi/ruis-chunks

#!/bin/bash

NOISEDIR=$1

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

for f in *.wav
do 
    NOISEFILE=$(find ${NOISEDIR} -type f | shuf -n 1)
    
    sox -m $f ${NOISEFILE} noise.$f trim 0 `soxi -D $f`
done

You might be interested in something I just released:

This is an awesome new project I found that is build on an article by snips about how they build their personal wake word. I contacted the node-personal-wakeword author and build a node-red node to integrate it here in cooperation with him.
It works as a drop in replacement of the voice2json wait-wake node.

2 Likes

Hi jonathan, i was of the hook for a time, new job and some health issues.
I am gonna look for it !

1 Like