Python Function Node UnicodeDecodeError

Hello everyone, I am running into a problem when I import a flow I had built on the IBM Cloud Node Red to my local Windows 10 Node Red. The flow is as follows:


When I run this on the IBM cloud, it works perfectly. When I run it locally on Windows 10, I get the following debug outputs:

image
As you can see, the Speech to Text node is working fine and the Debug Text outputs "Hello". However, the Python Text to Morse node, which is a node-red-contrib-python-function node, seems to be where the errors happen. This is the code inside:

#Every letter must be followed by a duration of 3 dots, and every word is followed by a duration of 7 dots 
#Thus, to represent correct morse, we first separate letters and words by this morse spacing
transcriptWithSpacedWords = []
for letter in msg['payload']:
    if letter != ' ': #if we are inside a word
        transcriptWithSpacedWords.append(letter) #add the letter
        transcriptWithSpacedWords.append(' ') #3 dots
    if letter == ' ': #if the word is over
            transcriptWithSpacedWords.append('_') #7 dots


morse = ''

for letter in transcriptWithSpacedWords:
    if str(letter) == ' ':  #if letter is over then 3 dots
        morse = morse + '>'
        
    if str(letter) == '_': #if word is over then 4 dots (because there was already a space (3) before)
        morse = morse + '<'
        
    if str(letter) == 'a' or str(letter) == 'A':
        morse = morse + '.-'
        
    if str(letter) == 'b' or str(letter) == 'B':
        morse = morse + '-...'
        
    if str(letter) == 'c' or str(letter) == 'C':
        morse = morse + '-.-.'
        
    if str(letter) == 'd' or str(letter) == 'D':
        morse = morse + '-..'
        
    if str(letter) == 'e' or str(letter) == 'E':
        morse = morse + '.'
    
    if str(letter) == 'f' or str(letter) == 'F':
        morse = morse + '..-.'
        
    if str(letter) == 'g' or str(letter) == 'G':
        morse = morse + '--.'
        
    if str(letter) == 'h' or str(letter) == 'H':
        morse = morse + '....'
        
    if str(letter) == 'i' or str(letter) == 'I':
        morse = morse + '..'
    
    if str(letter) == 'j' or str(letter) == 'J':
        morse = morse + '.---'
        
    if str(letter) == 'k' or str(letter) == 'K':
        morse = morse + '-.-'
        
    if str(letter) == 'l' or str(letter) == 'L':
        morse = morse + '.-..'
        
    if str(letter) == 'm' or str(letter) == 'M':
        morse = morse + '--'
        
    if str(letter) == 'n' or str(letter) == 'N':
        morse = morse + '-.'
    
    if str(letter) == 'o' or str(letter) == 'O':
        morse = morse + '---'
        
    if str(letter) == 'p' or str(letter) == 'P':
        morse = morse + '.--.'
        
    if str(letter) == 'q' or str(letter) == 'Q':
        morse = morse + '--.-'
        
    if str(letter) == 'r' or str(letter) == 'R':
        morse = morse + '.-.'
        
    if str(letter) == 's' or str(letter) == 'S':
        morse = morse + '...'
        
    if str(letter) == 't' or str(letter) == 'T':
        morse = morse + '-'
        
    if str(letter) == 'u' or str(letter) == 'U':
        morse = morse + '..-'
        
    if str(letter) == 'v' or str(letter) == 'V':
        morse = morse + '...-'
        
    if str(letter) == 'w' or str(letter) == 'W':
        morse = morse + '.--'
        
    if str(letter) == 'x' or str(letter) == 'X':
        morse = morse + '-..-'
        
    if str(letter) == 'y' or str(letter) == 'Y':
        morse = morse + '-.--'
        
    if str(letter) == 'z' or str(letter) == 'Z':
        morse = morse + '--..'


msg['payload'] = morse
return msg

Any help would be greatly appreciated, thank you in advance!
The output I am expecting is this (this image is from the cloud output):
image
and I receive the same output if I run the same code on JupyterNotebook (but instead of using msg['payload'] I just use a string like Transcript = "Hello ")

I seem to remember others have had problems with this node, my suggestion would be to replace it with a normal Function node using javascript.

... or an exec node calling an external python script.

Thank you for the suggestion! The problem is that I am not familiar with the JS syntax so when I tried to replace Python with it, I couldn't get all the functions in my flow to do what I want :broken_heart:
I was able to convert the function in the question to the following:

//variables to be used to seperate morse words.
var word = '';
var morseText = [];

//Loop through every letter in the received message
for (var letter in msg.payload)
{
    //if it is part of a word
    if (msg.payload[letter] === '.' || msg.payload[letter] === '-' || msg.payload[letter] == '>')
        {
            word = word + msg.payload[letter]; //Then add it to the variable word
        }
    //if it is the signal that a word is over
    if (msg.payload[letter] === '<') 
        {
            morseText.push(word); //First, add the word to the morseText array.
            morseText.push('<'); //Then, add the signal.
            word = ""; //Finally, clear the word variable to start holding a new word.
        }
}

//variables to be used to transform morse words to english text.
var englishText = '';
var morseLetter = '';

//For every morse words saved in morseText array
for (var morseWord in morseText)
{
    //Loop through the morse signs in the word
    for (var sign in morseText[morseWord])
    {
        //if the sign is a dit or dah
        if (morseText[morseWord[sign]] === '.' || morseText[morseWord[sign]] == '-')
            {
                morseLetter = morseLetter + morseText[morseWord[sign]];  //then add it as part of a letter.
            }
        //if the sign is signalling the end of a word
        else if (morseText[morseWord[sign]] === '<') 
        {
            englishText = englishText + ' '; //then add a space.
            morseLetter = ''; //and clear the the letter variable to accept letters from the next word.
        }
        //if the sign is signalling the end of the letter  
        else if (morseText[morseWord[sign]] === '>')
        {
            //then figure out what that letter is in english
            if (morseLetter === '.-')
                {englishText = englishText + 'a';}

            else if (morseLetter === '-...')
                {englishText = englishText + 'b';}

            else if (morseLetter === '-.-.')
                {englishText = englishText + 'c';}

            else if (morseLetter === '-..')
                {englishText = englishText + 'd';}

            else if (morseLetter === '.')
                {englishText = englishText + 'e';}

            else if (morseLetter === '..-.')
                {englishText = englishText + 'f';}

            else if (morseLetter === "--.")
                {englishText = englishText + "g";}

            else if (morseLetter === '....')
                {englishText = englishText + 'h';}

            else if (morseLetter === '..')
                {englishText = englishText + 'i';}

            else if (morseLetter === '.---')
                {englishText = englishText + 'j';}

            else if (morseLetter === '-.-')
                {englishText = englishText + 'k';}

            else if (morseLetter === '.-..')
                {englishText = englishText + 'l';}

            else if (morseLetter === '--')
                {englishText = englishText + 'm';}

            else if (morseLetter === '-.')
                {englishText = englishText + 'n';}

            else if (morseLetter == '---')
                {englishText = englishText + 'o';}

            else if (morseLetter == '.--.')
                {englishText = englishText + 'p';}

            else if (morseLetter == '--.-')
                {englishText = englishText + 'q';}

            else if (morseLetter == '.-.')
                {englishText = englishText + 'r';}

            else if (morseLetter == '...')
                {englishText = englishText + 's';}

            else if (morseLetter == '-')
                {englishText = englishText + 't';}

            else if (morseLetter == '..-')
                {englishText = englishText + 'u';}

            else if (morseLetter == '...-')
                {englishText = englishText + 'v';}

            else if (morseLetter == '.--')
                {englishText = englishText + 'w';}

            else if (morseLetter == '-..-')
                {englishText = englishText + 'x';}

            else if (morseLetter == '-.--')
                {englishText = englishText + 'y';}

            else if (morseLetter == '--..')
                {englishText = englishText + 'z';}
            
        morseLetter = ''; //then clear the the letter variable to accept the next letter.
        }
    }
}
msg.payload = englishText;
return msg;

It is giving the correct result, but it shows a yellow error sign at the } right before msg.payload = englishText; that says "too many errors. 97% scanned"

Similarly, I have this python function to go from Morse to Text:

word = ''
morseText = []

for letter in msg['payload']:
    if str(letter) == '.' or str(letter) == '-' or str(letter) == '>':
        word = word + letter
    if str(letter) == '<': #if it's the signal that word is over then add a space
        morseText.append(word) 
        morseText.append(' ')
        word = ""


englishText = ''
morseLetter = ''

for word in morseText:
    for sign in word:
        if str(sign) == '.' or str(sign) == '-':
            morseLetter = morseLetter + sign
            
        if str(sign) == ' ': 
            englishText = englishText + ' '
            morseLetter = ''
                
        if str(sign) == '>': 
            #then the letter is over and we need to figure out what the it is in english now
            if str(morseLetter) == '.-':
                englishText = englishText + 'a'

            if str(morseLetter) == '-...':
                 englishText = englishText + 'b'

            if str(morseLetter) == '-.-.':
                 englishText = englishText + 'c'

            if str(morseLetter) == '-..':
                 englishText = englishText + 'd'

            if str(morseLetter) == '.':
                 englishText = englishText + 'e'

            if str(morseLetter) == '..-.':
                 englishText = englishText + 'f'

            if str(morseLetter) == '--.':
                 englishText = englishText + 'g'

            if str(morseLetter) == '....':
                 englishText = englishText + 'h'

            if str(morseLetter) == '..':
                 englishText = englishText + 'i'

            if str(morseLetter) == '.---':
                 englishText = englishText + 'j'

            if str(morseLetter) == '-.-':
                 englishText = englishText + 'k'

            if str(morseLetter) == '.-..':
                 englishText = englishText + 'l'

            if str(morseLetter) == '--':
                 englishText = englishText + 'm'

            if str(morseLetter) == '-.':
                 englishText = englishText + 'n'

            if str(morseLetter) == '---':
                 englishText = englishText + 'o'

            if str(morseLetter) == '.--.':
                 englishText = englishText + 'p'

            if str(morseLetter) == '--.-':
                 englishText = englishText + 'q'

            if str(morseLetter) == '.-.':
                 englishText = englishText + 'r'

            if str(morseLetter) == '...':
                 englishText = englishText + 's'

            if str(morseLetter) == '-':
                 englishText = englishText + 't'

            if str(morseLetter) == '..-':
                 englishText = englishText + 'u'

            if str(morseLetter) == '...-':
                 englishText = englishText + 'v'

            if str(morseLetter) == '.--':
                 englishText = englishText + 'w'

            if str(morseLetter) == '-..-':
                 englishText = englishText + 'x'

            if str(morseLetter) == '-.--':
                 englishText = englishText + 'y'

            if str(morseLetter) == '--..':
                 englishText = englishText + 'z'
            #then make the letter var empty again to accept a new letter
            morseLetter = ''


msg['payload'] = englishText
return msg

And I am trying to replace it with this:

//variables to be used to seperate morse words.
var word = '';
var morseText = [];

//Loop through every letter in the received message
for (var letter in msg.payload)
{
    //if it is part of a word
    if (msg.payload[letter] === '.' || msg.payload[letter] === '-' || msg.payload[letter] == '>')
        {
            word = word + msg.payload[letter]; //Then add it to the variable word
        }
    //if it is the signal that a word is over
    if (msg.payload[letter] === '<') 
        {
            morseText.push(word); //First, add the word to the morseText array.
            morseText.push('<'); //Then, add the signal.
            word = ""; //Finally, clear the word variable to start holding a new word.
        }
}

//variables to be used to transform morse words to english text.
var englishText = '';
var morseLetter = '';

//For every morse words saved in morseText array
for (var morseWord in morseText)
{
    //Loop through the morse signs in the word
    for (var sign in morseText[morseWord])
    {
        //if the sign is a dit or dah
        if (morseText[morseWord[sign]] === '.' || morseText[morseWord[sign]] == '-')
            {
                morseLetter = morseLetter + morseText[morseWord[sign]];  //then add it as part of a letter.
            }
        //if the sign is signalling the end of a word
        else if (morseText[morseWord[sign]] === '<') 
        {
            englishText = englishText + ' '; //then add a space.
            morseLetter = ''; //and clear the the letter variable to accept letters from the next word.
        }
        //if the sign is signalling the end of the letter  
        else if (morseText[morseWord[sign]] === '>')
        {
            //then figure out what that letter is in english
            if (morseLetter === '.-')
                {englishText = englishText + 'a';}

            else if (morseLetter === '-...')
                {englishText = englishText + 'b';}

            else if (morseLetter === '-.-.')
                {englishText = englishText + 'c';}

            else if (morseLetter === '-..')
                {englishText = englishText + 'd';}

            else if (morseLetter === '.')
                {englishText = englishText + 'e';}

            else if (morseLetter === '..-.')
                {englishText = englishText + 'f';}

            else if (morseLetter === "--.")
                {englishText = englishText + "g";}

            else if (morseLetter === '....')
                {englishText = englishText + 'h';}

            else if (morseLetter === '..')
                {englishText = englishText + 'i';}

            else if (morseLetter === '.---')
                {englishText = englishText + 'j';}

            else if (morseLetter === '-.-')
                {englishText = englishText + 'k';}

            else if (morseLetter === '.-..')
                {englishText = englishText + 'l';}

            else if (morseLetter === '--')
                {englishText = englishText + 'm';}

            else if (morseLetter === '-.')
                {englishText = englishText + 'n';}

            else if (morseLetter == '---')
                {englishText = englishText + 'o';}

            else if (morseLetter == '.--.')
                {englishText = englishText + 'p';}

            else if (morseLetter == '--.-')
                {englishText = englishText + 'q';}

            else if (morseLetter == '.-.')
                {englishText = englishText + 'r';}

            else if (morseLetter == '...')
                {englishText = englishText + 's';}

            else if (morseLetter == '-')
                {englishText = englishText + 't';}

            else if (morseLetter == '..-')
                {englishText = englishText + 'u';}

            else if (morseLetter == '...-')
                {englishText = englishText + 'v';}

            else if (morseLetter == '.--')
                {englishText = englishText + 'w';}

            else if (morseLetter == '-..-')
                {englishText = englishText + 'x';}

            else if (morseLetter == '-.--')
                {englishText = englishText + 'y';}

            else if (morseLetter == '--..')
                {englishText = englishText + 'z';}
            
        morseLetter = ''; //then clear the the letter variable to accept the next letter.
        }
    }
}
msg.payload = englishText;
return msg;

But when I inject "--.>--->--->-..><-->--->.-.>-.>..>-.>--.><" it returns " " instead of the expected "good morning " and I cannot seem to figure out where the error is.

Oh, I haven't tried these nodes before, thank you for the suggestion! I'll read more about them and see if I can figure them out

I don't know what the problem is with the 'too many errors' message, it seems to be a bug in the syntax checker used in the function node, but not certain. @dceejay will have to comment on that I think.

For the second issue with the function just not doing what it should then an easy way to do simple debugging is to use node.warn() to show the progress through. So for instance you could add at appropriate points lines like
node.warn("morseLetter is " + morseLetter)

Thank you for this! It seems the problem was trying to say morseText[morseWord[sign]] and now that I have replaced it, everything is working again.

Oh, I see. Unfortunately, I am now getting this same message in my second function node too, so I would really appreciate your take on it if it's not a bug @dceejay , thank you!

It seems to be triggered by all the else if lines. That code would be better written using the javascript switch statement so it might be worth changing over to that anyway. https://www.w3schools.com/js/js_switch.asp

Ok, so I tried switch now and the yellow error moved to an earlier place in the message (at letter q) and now says 70% scanned instead of 97% but it is still showing.

However, now that I have a version with if and one with switch and need to decide which one to keep; why do you think Switch is better? the one with if statement seems to be shorter (131 lines instead of the 142 i have with switch) and is serving the same purpose, right? :thinking:

Isn't the switch version easier to follow? I don't know whether the switch generates more efficient code, it may well do. It is more about using the appropriate construct for the requirement. Where you want to test a variable for equality to a number of different possibilities then that is what switch is designed for. You can make the code a bit more concise by using, for example
englishText += 'h'
also you generally don't need the semicolons any more.
Could you post the switch version please, I am interested to see where the problem now shows itself.

Oh, I see. That's true, you're right!

Absolutely, here you go:

//variables to be used to seperate morse words.
var word = '';
var morseText = [];

//For every letter in the received message
for (var letter in msg.payload)
{
    switch (msg.payload[letter])
    {//if it is part of a word
        case '.':
        case '-':
        case '>':
            word = word + msg.payload[letter]; //Then add it to the variable word
             break;
        case '<': //if it is the end of word signal
            morseText.push(word); //First, add the word to the morseText array.
            morseText.push('<'); //Then, add the signal.
            word = ""; //Finally, clear the word variable to start holding a new word.
            break;
        default:
            break;
    }
}

//variables to be used to transform morse words to english text.
var englishText = '';
var morseLetter = '';

//For every morse word saved in morseText array
for (var i in morseText)
{
    var morseWord = morseText[i];
    //Loop through the morse signs in the word
    for (var sign in morseWord)
    { 
        switch (morseWord[sign])
        {
            //if the sign is a dit or dah
            case '.':
            case '-':
                morseLetter = morseLetter + morseWord[sign];  //then add it as part of a letter.
                break;
            //if the sign is signaling the end of a word
            case '<':
                englishText = englishText + ' '; //then add a space.
                morseLetter = ''; //and clear the the letter variable to accept letters from the next word.
                break;
            //if the sign is signaling the end of the letter  
            case '>':
                //then figure out what that letter is in English
                switch (morseLetter)
                {
                    case '.-':
                        englishText = englishText + 'a';
                        break;
                    case '-...':
                        englishText = englishText + 'b';
                         break;
                    case '-.-.':
                        englishText = englishText + 'c';
                        break;
                    case '-..':
                        englishText = englishText + 'd';
                        break;
                     case '.':
                        englishText = englishText + 'e';
                        break;
                    case '..-.':
                        englishText = englishText + 'f';
                        break;
                    case "--.":
                        englishText = englishText + "g";
                        break;
                    case '....':
                        englishText = englishText + 'h';
                        break;
                    case '..':
                        englishText = englishText + 'i';
                        break;
                    case '.---':
                        englishText = englishText + 'j';
                        break;
                    case '-.-':
                        englishText = englishText + 'k';
                        break;
                    case '.-..':
                        englishText = englishText + 'l';
                        break;
                    case '--':
                        englishText = englishText + 'm';
                        break;
                    case '-.':
                        englishText = englishText + 'n';
                        break;
                    case '---':
                        englishText = englishText + 'o';
                        break;
                    case '.--.':
                        englishText = englishText + 'p';
                        break;
                    case '--.-':
                        englishText = englishText + 'q';
                        break;
                    case  '.-.':
                        englishText = englishText + 'r';
                        break;
                    case '...':
                        englishText = englishText + 's';
                        break;
                    case '-':
                        englishText = englishText + 't';
                        break;
                    case '..-':
                        englishText = englishText + 'u';
                        break;
                     case '...-':
                        englishText = englishText + 'v';
                        break;
                    case '.--':
                        englishText = englishText + 'w';
                        break;
                    case '-..-':
                        englishText = englishText + 'x';
                        break;
                    case '-.--':
                        englishText = englishText + 'y';
                        break
                    case '--..':
                        englishText = englishText + 'z';
                        break;
                    default: 
                        englishText = englishText;
                        break;
                }
                morseLetter = ''; //and clear the the letter variable to accept the next letter.
            default: 
        }
    }
}
msg.payload = englishText;
return msg;

I thought I might have found the cause of the warning triangle. In javascript it is not recommended to use for .. in .. for iterating an array, though in practice it may work. The recommended way is to replace, for example
for (var letter in msg.payload)
with
for (var letter=0; letter<msg.payload.length; letter++)
But unfortunately it appears that it is not that which is confusing the syntax checker. I have started a new thread for that issue to see if we can get to the bottom of it: "Too many errors" in function node, but works ok

1 Like

I will check this out, thank you so much!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.