Substituting a variable in the JSON object used to for an http call

General
1

I am setting up the parameters for a http request node. For example,

msg.payload={ 
    'client_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    'grant_type': 'refresh_token',
    'refresh_token': currentRefreshToken 
};

For my flow the variable currentRefreshToken needs to be set to the contents of a global variable. For example,

var currentRefreshToken = global.get("StoredValue");

StoredValue contains "vvvvvvvvvvvvvvvvvv" but the msg.payload passed to the http node does not contain the correct JSON object and the http call returns an error.

If I add an extra line below the global.get

currentRefreshToken = "vvvvvvvvvvvvvvvvv"

the msg.payload passed to the http node is correctly formed and works perfectly.

I am sure the answer is simple but unfortunately I do not have a good understanding of the difference between JSON, string variables etc etc.

Any help would be much appreciated.

2

Are you returning msg at the end of the function?

return msg

Is there any async/callback code in the function node?

Would be better if you can show us the full function code (redacted where necessary)

Also, use the likes of node.warn({currentRefreshToken, otherVarToDebug}) in your code at various places to get a greater insight.

3

Yes there is a return msg at the end and a debug node to check the output.

correctly formed it is
21/01/2025, 09:02:54 msg.payload : Object

object

client_id: "xxxxxxxxxxxxxxxxxxxxxxxxxx"

grant_type: "refresh_token"

refresh_token: "vvvvvvvvvvvvvvvvvvvvvvvv"

When substituting the variable the output is

21/01/2025, 10:35:22 msg.payload : Object

object

client_id: "xxxxxxxxxxxxxxxxxxxxxxx"

grant_type: "refresh_token"

refresh_token: string

vvvvvvvvvvvvvvvvvvvvvvvvvvv

which is rejected by the server and gives an invalid message.

4

Can you verify. Does the debug show the expected values and are they what you pass to the HTTP Req node? its just that before, you said...

Also, you have not provided detail of the API endpoint. For example, many I have used expect the token to be in a header (not the payload). Do you have documentation?

Additionally, share a demo flow (redacted) if you can (se we can check things like HTTP method is GET/POST etc)

What is the message? What is the status code?

5

The problem is only that the refresh token is passed as a string when the variable is substituted. I am sorry if I have made the post too complicated.

Everything works absolutely fine when the currentRefreshToken hard coded but not when it is derived from a variable via the global.get. There must be a way to set currentRefreshToken from the stored value and avoid it being read as a string.

6

Yes, of course, however we dont have enough clear information to help you further.

for example, I have no idea what value you have stored (if any) in context. whether it is a string/object/array? Or what it is even supposed to be? Or whether it is actually the right value?

have you checked the value in the context viewer is as you expect it to be?

Of course there is, but you havent shown us what a "good message" looks like vs your "not working message". Perhaps annotated screenshots would help us to help you?

Lastly, showing use the actual error code, error messages & providing API documentation would help.

Tips

  1. Switch your debug nodes to output the complete message (so you can see errors and status codes etc)

  2. When copying the values, dont drag your mouse over the text, use the provided copy Value button

There’s a great page in the docs (Working with messages : Node-RED) that will explain how to use the debug panel to find the right path/value for any data item.

Pay particular attention to the part about the buttons that appear under your mouse pointer when you over hover a debug message property in the sidebar.

BX00Cy7yHi

7

The correctly formed message is at the top of the post and the badly formed one just below it. There is only a subtle difference but one works fine but is hard coded and the second which uses a variable does not work correctly. Thanks very much for your help. I am sure it is a very simple fix if you are familiar with JSON and js.

8

When I say top of the post it is my first reply to you. These have been taken from the side bar and show the difference in the two debug outputs. The API call works fine when fed with the correct information. It can only be a simple problem but I am not very familiar with JSON / js.

9

It should indeed be a very simple thing:

const currentRefreshToken = global.get("StoredValue"); 

msg.payload={ 
    'client_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    'grant_type': 'refresh_token',
    'refresh_token': currentRefreshToken 
}

return msg;

Since you say it's not working for you, you must be doing something wrong.

Do you see this? Check for spelling mistakes and incorrect case in the variable name and your function.

image
image449Ă—468 10.1 KB

10

Unfortunately, as you have not followed my requests for debug information in the format I requested (i.e. not using the correct "copy value" button as I requested), you have not confirmed if the value in context is correct (I asked you to use node.warn and look in the context viewer), you have not shown any annotated screen shots (of debug output or flows), you have not shared the exact details of the error (by setting the debug to show full message) etc etc - so I am really struggling to explicitly understand the issue and provide you with a simple answer.

I can certainly help but unfortunately without either full access to your flows OR you responding to my requests for information etc, I can only make assumptions.

11

Thank you very much for your help and time spent, it is much appreciated. I think I have made a really bad job at explaining the problem I have. My flows have been running my home automation system for around 1 year so I do not think it is worthwhile expanding beyond the very localised problem I have. Eventually, I will find why hard coding a value works, but taking the same value from a global variable does not. Debug nodes of the variable visibly show the same information but the API call can detect the difference. The API error message is very clear, it does not like the value arising from the global variable. The debug windows previously posted also clearly show the differences in the full messages being passed to the API and provide a clear clue as to what is wrong - but I do not know the subtleties to correct it.
Thanks again, but I am happy to leave the problem for now. When I find the solution I will post the answer.

12

image
image609Ă—264 19.2 KB

The essential fact seems to be that when you use a context variable the debug shows [implies?] datatype payload.refresh_token as string, and it's value, without quotes.
This is in contrast to payload.client_id.

It would be interesting to know what causes debug to display like this.
Maybe someone with a knowledge of the editor source code can answer.

We have your assertion StoredValue contains "vvvvvvvvvvvvvvvvvv" but despite being asked you have not shown us the value of the variable in the context data sidebar, the method you use to set it or the precise code of the function where you use it.

13

This is the exact (redacted) code that works. The side bar results are as earlier in the thread.

var currentRefreshToken = global.get("xxxxxRefreshToken");

currentRefreshToken = '5500cdc17174fea39e2f209754814182';

msg.headers={ 
    'Content-Type': 'application/x-www-form-urlencoded'
};
msg.payload = {};

msg.payload={ 
    'client_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    'grant_type': 'refresh_token',
    'refresh_token': currentRefreshToken 
};

return msg;

This is the code that gives rise to the 'string' problem. The sidebar results are as earlier in the thread as you have highlighted. This the problem and there will be a simple answer.

var currentRefreshToken = global.get("xxxxxRefreshToken");
//remove hard coding fix
//currentRefreshToken = '5500cdc17174fea39e2f209754814182';

msg.headers={ 
    'Content-Type': 'application/x-www-form-urlencoded'
};
msg.payload = {};

msg.payload={ 
    'client_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    'grant_type': 'refresh_token',
    'refresh_token': currentRefreshToken 
};


return msg;

This code retrieves the global variable and outputs msg.payload

msg.payload = global.get("xxxxxRefreshToken")
return msg;

This the side panel to show the information is the 'same?' as that hardcoded

21/01/2025, 16:56:32node: debug 170
msg.payload : string[33]
"5500cdc17174fea39e2f209754814182
"

The problem is my misuse of variables, strings, JSON, js. There will be a simple answer but I have not found it yet. Thanks

14

Your last quoted debug output appears to show that there is a line break at the end of your string.
It's not \n since that would be visible as the carriage return symbol in the debug output
image

It may be \r, leading to

image
image377Ă—118 2.82 KB

Try this to get rid of it (or similar before you save it to context)

var currentRefreshToken = global.get("xxxxxRefreshToken")
currentRefreshToken = currentRefreshToken.replace(/[\n\r\t]/gm, "")
msg.payload={ 
    'client_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    'grant_type': 'refresh_token',
    'refresh_token': currentRefreshToken 
}
15

Thanks very much for the suggestion. It gives exactly the same result and using copy from the debug window shows both are the same with no new line or other character.
Another suggestion is that there is a difference between the string being in single or double quotes. Hard coding with both options shows no difference but both show in the debug window as double dits. Therefore the code editor may be automatically sorting this. I have now found other threads referencing a similar problem to mine but as yet no answer.

16

Can you show how you store this xxxxxRefreshToken to the global scope in the first place.

17

The refresh token is stored in a .csv file which is read by a read file node with properties:-
Filename msg. filename
Output a msg per line
include all existing properties in each msg
Encoding default

The csv file has two columns
Col 1 = Token
Col 2 = Value of refresh Token
Refresh Token 5500cdc17174fea39e2f209754814182

The msg.payload is Refresh Token, 5500cdc17174fea39e2f209754814182

This processed via a switch node (contains Token)
It is then processed and stored by the following code:-

var str = msg.payload
var array = str.split(',');
var token = (array[1])
global.set("xxxxxRefreshToken", token )
msg.payload = token
 
return msg;

visually at least it appears to have the correct value

Debuging it

msg.payload = global.get("ViessRefreshToken")

return msg;

Gives:-

21/01/2025, 23:12:57node: debug 172
msg.payload : string[33]
string[33]
5500cdc17174fea39e2f209754814182

Hope this helps - the problem is starting to amaze me.

18

It only appears to have the correct value if you wilfully ignore the repeated "string[33]" in the output.
This is not the way a simple string payload appears in the debug.

How come you are only now revealing where the token comes from and the code you use to extract it?

19

Debug and print out the str (msg.payload) you're getting in before splitting it.
Also trim the token value before setting it.
Also perform some check after splitting the string as accessing index of an array that might not exist will throw errors.

20

When you hard code the string without using the stored value the debug window looks very similar, it's a string.

22/01/2025, 00:11:33node: debug 172
msg.payload : string[32]
"5500cdc17174fea39e2f209754814182"

Except that the length is now 32 not 33 which is the reported length when derived from the globally stored value. This will probably be the key to the problem I am having. I need to find and remove the non visible character that is adding 1 to the length.

Thank you very much for your help. I can assure you that I was not trying to mislead anyone.

Hopefully when the problem is finally solved I will report back here. As far as I can see the overall design using a file to store data required between starts or after a system crash is perfectly valid. The system has been running for a year and the odd crash does happen especially on Windows. I am only revisiting to try and get rid of the hard code solution which has a limited life.