HTTP-Request cookies

Need a little help with a challenge. I have posted similar issues but did not go about describing it the best way.

I am developing a temporary middleware application in between two applications. I'm basically taking one SOAP request and converting it to a JSON lookup/validation request. Both sides of the process have 3 steps. Login, lookup/validate, logoff. My problem exists with the second part because of the way the API is setup.

When I successfully use the HTTP request to login, I get a response with a sessionID in the body that I can use. The problem is with the login process cookies. In POSTMan or CURL there are multiple cookies that are passed and used for the session, the sessionID isn't actually used in the body of other calls. I have been unable to get those cookies to pass across to the subsequent HTTP requests. Is there a way to get the verbose output of the HTTP request to get the cookies and create them or pass them along?

Have you tried adding a debug node (set to show complete message) to the output of the HTTP Request? There are a lot of properties NOT in msg.payload including cookies. You could capture them and add them into subsequent calls by placing them in msg.cookies before the HTTP Request.

Maybe this cookbook example will help: Work with cookies : Node-RED

Indeed I have, neither the debug window nor enabling the console option shows the full details. Here's a sample. Note the session is dynamic per request process.

23 Feb 14:06:53 - [info] Started flows
23 Feb 14:06:55 - [info] [debug:656505c4ebf42be3]
{ version: '7.44.0', session: '03D40B9E-C7FD-4BAB-905A-85B404D8581D' }

Here is a sample of the cookies created in the same POSTMan request.

AV-Desktop
!H8+YAucXyotcIMPlLa8hgtWxoYxNybH0D4lc3cMHc+f56RWnyiErl08zaodFmZfqn8KxIq8yfR5METg=

AV-Cookie
!gC0tfdYGPYq26oLlLa8hgtWxoYxNycO0yRii6Qomm20r/8hFEaGwYnsbuQhsYkwV02QCVYxqGignBQ==

TS01a6bd37
01cf42f8a6e7682b434f8c3322c012c9e4daf64911ef58f0fc55ccab406f30e5f6c57287d0337da518de804026aa14bce3ebc111a548c2a85752bf3bf9fda4d28479046061c561b723ca602c20aba6e5ff20eeb19c

session
8C5B8D81-8DB6-4E5E-B9D5-4DAC2DC53952

I can use the execute node to run a CURL command and get the full content but parsing it has been a significant challenge because of carriage returns and some hidden characters (can't convert it to an object).

Can you copy the full message (using the "Copy Value" button on the debug msg) returned from a HTTP Request node and past it into a reply.


NOTE:

In order to make code readable and usable it is necessary to surround your code with three backticks (also known as a left quote or backquote ```)

``` 
   code goes here 
```

You can edit and correct your post by clicking the pencil :pencil2: icon.

See this post for more details - How to share code or flow json

1 Like

I changed the debug node to view more details. Is it possible I need to use the set node with each one of these items in order to carry them along?

{"_msgid":"876d9747f9a37124","payload":{"version":"7.44.0","session":"CAAFFA2B-7218-46B1-973D-80490AF97500"},"topic":"","headers":{"cache-control":"no-cache","keep-alive":"timeout=5","content-type":"application/json; charset=utf-8","set-cookie":["session=CAAFFA2B-7218-46B1-973D-80490AF97500; Secure; HttpOnly; Path=/;samesite=none","AV-Cookie=!hJRfQ5msWTjMbJBEpB17PCYniyldec5SpX1tVfMgQcVZb7K5fz4Ns+sbvDf0Iw9f7z4kh2IV4q9g0w==; path=/; Httponly; Secure;samesite=none","TS01a6bd37=01cf42f8a6d425db3c58d3c5db549b69ce575e01073077f8312806e7047253d9d7198f9058366e8ec80e1c13cde58b493bf60576cdbe1294ae168432701cff11eeac32060e33b34c36010663329536bdfecb2459e6; Path=/; Secure; HTTPOnly;samesite=none"],"date":"Wed, 23 Feb 2022 19:20:27 GMT","content-length":"69","connection":"close","x-frame-options":"SAMEORIGIN","x-xss-protection":"1; mode=block","x-content-type-options":"nosniff","content-security-policy":"script-src 'unsafe-inline' 'unsafe-eval' 'self' https://*; object-src 'self'","strict-transport-security":"max-age=16070400; includeSubDomains","vary":"Accept-Encoding"},"statusCode":200}

So when you said ...

You had not set the debug to show complete message as i asked

Because i can see AV-Cookie in the headers under "msg.headers["set-cookie"]


Have you tried setting this back into msg.headers in your subsequent calls?


If you can show me what the 2nd HTTP Req looks link in postman, i can tell you where they are supposed to go.

2 Likes

That's correct, I hadn't enabled the full message. That was my misunderstanding.

I believe when I am adding the accept json header as required that it is replacing all headers. Now I need to figure out how to add the header. I suspect I need to pull them all out and re-add them.

In a function you can update headers like this...

msg.headers = msg.headers || {}; //make headers an object if empty
msg.headers["my-new-header"] = 123;
msg.headers["another-new-header"] = msg.whatever;
msg.headers["and-another-new-header"] = global.get("a-header-i-stored-in-global-context")
return msg;
1 Like

In the Help tab of the http-request is says that if you pass cookies in msg.cookies it must be an object

Cookie handling

The cookies property passed to the node must be an object of name/value pairs. The value can be either a string to set the value of the cookie or it can be an object with a single value property.

but the cookies in @zuasive's request reply is an array
If you tested passing them as an array and fails
.. you can try to join the cookies in the header with

...
msg.headers = msg.headers || {}; //make headers an object if empty
msg.headers["Cookie"] = msg.headers["set-cookie"].join(";")
...
2 Likes

That is exactly why I asked the OP to show us the Postman request details

Otherwise, we are guessing :slight_smile:

true .. i didnt have a chance to test this before as most the questions on the Forum involved a single cookie, as far as i remember

Thanks guys, clearly I wasn't digging hard enough into the logging (credit to Steve-Mcl). I was under the impression it wasn't utilizing everything. Now I just need to do some modifying and likely combination. I'll test everything here and give credit to the solutions while showing the combined work to assist with help searches in the future.

2 Likes

Okay, this is awesome. Thank you both for helping me out, I really appreciate it. I'm still relatively new at all of this. Running into some weird issues but sorting through them. Not ironically enough, the service I'm connecting to that I don't control is running Node. I know this because I got a Node error when testing in POSTman. Guess it shouldn't be surpising, Node in general seems to be the best for dynamic APIs.

Anyway, here's the new issue I'm working on. I have been changing things around and modifying based on the suggestions to get a new error. Now it is telling me a header is missing but it's not. Check this out.

Logon request response:

{"_msgid":"a462e36c74f078ac","payload":{"version":"7.44.0","session":"82CEE509-23BF-4936-BD96-B8E409A77D1C"},"topic":"","headers":{"cache-control":"no-cache","keep-alive":"timeout=5","content-type":"application/json; charset=utf-8","set-cookie":["session=82CEE509-23BF-4936-BD96-B8E409A77D1C; Secure; HttpOnly; Path=/;samesite=none","AV-Cookie=!h0K7l+7wJaOuvbBEpB17PCYniylded024VTS7y4R/Yx3wwyAj2nkGVKIQIafGa3zdlCFPHcuoB/pOQ==; path=/; Httponly; Secure;samesite=none","TS01a6bd37=01cf42f8a6fe90faa7ac977ef0a4aadce1d9f8019506b98aa562ef5924c51499b77b689b1a53b69dffdb418af987099fcf03b200fa53b9e423b7c7f1216a77e6d2eb35840bd06c73ee7a6445b38e27a683091f6580; Path=/; Secure; HTTPOnly;samesite=none"],"date":"Thu, 24 Feb 2022 05:11:40 GMT","content-length":"69","connection":"close","x-frame-options":"SAMEORIGIN","x-xss-protection":"1; mode=block","x-content-type-options":"nosniff","content-security-policy":"script-src 'unsafe-inline' 'unsafe-eval' 'self' https://*; object-src 'self'","strict-transport-security":"max-age=16070400; includeSubDomains","vary":"Accept-Encoding"},"statusCode":200}

Lookup content:

{"_msgid":"a462e36c74f078ac","payload":{"actions":[{"method":"mark","params":{"ticket_number":"13780448510"}}],"set":{"Options::performance_ids":"AE104AF2-EE91-4BD4-8919-62262CAF0778","Options::marking_options":"status"},"get":["Ticket","Seat","Order","Customer","Options","Items","ItemDetails","Result"],"objectname":"myticket"},"headers":{"cache-control":"no-cache","keep-alive":"timeout=5","content-type":"application/json; charset=utf-8","set-cookie":["session=82CEE509-23BF-4936-BD96-B8E409A77D1C; Secure; HttpOnly; Path=/;samesite=none","AV-Cookie=!h0K7l+7wJaOuvbBEpB17PCYniylded024VTS7y4R/Yx3wwyAj2nkGVKIQIafGa3zdlCFPHcuoB/pOQ==; path=/; Httponly; Secure;samesite=none","TS01a6bd37=01cf42f8a6fe90faa7ac977ef0a4aadce1d9f8019506b98aa562ef5924c51499b77b689b1a53b69dffdb418af987099fcf03b200fa53b9e423b7c7f1216a77e6d2eb35840bd06c73ee7a6445b38e27a683091f6580; Path=/; Secure; HTTPOnly;samesite=none"],"date":"Thu, 24 Feb 2022 05:11:40 GMT","connection":"close","x-frame-options":"SAMEORIGIN","x-xss-protection":"1; mode=block","x-content-type-options":"nosniff","content-security-policy":"script-src 'unsafe-inline' 'unsafe-eval' 'self' https://*; object-src 'self'","strict-transport-security":"max-age=16070400; includeSubDomains","vary":"Accept-Encoding","accept":"application/json"}}

The funniest part is the response from the lookup.

{"_msgid":"a462e36c74f078ac","payload":{"version":"7.44.0","session":"E1B60202-FF64-473B-B5ED-21A68BA93840","errorCode":"2","message":"The 'Accept' header must be 'application/json'"},"headers":{"cache-control":"no-cache","keep-alive":"timeout=5","content-type":"application/json; charset=utf-8","set-cookie":["session=E1B60202-FF64-473B-B5ED-21A68BA93840; Secure; HttpOnly; Path=/;samesite=none","AV-Cookie=!ka+Js+whGN8z6vlEpB17PCYniyldef/8RpFCvRBd/zrkSY8a7oxLhHsGQ3wSG9G5v4iXqTHjNOQ8imc=; path=/; Httponly; Secure;samesite=none","TS01a6bd37=01cf42f8a6a9c6f93adae53747f5c917d71ffb1d099d9fdec469e5a69ea1f885deb654d92d2625cb1e72e9feba5e0b6f6d0ef9335ee0023051c55f36b12b2460642e4f51e074f27d7c719413d5b6d07b72fd8be506; Path=/; Secure; HTTPOnly;samesite=none"],"date":"Thu, 24 Feb 2022 05:11:40 GMT","content-length":"144","connection":"close","x-frame-options":"SAMEORIGIN","x-xss-protection":"1; mode=block","x-content-type-options":"nosniff","content-security-policy":"script-src 'unsafe-inline' 'unsafe-eval' 'self' https://*; object-src 'self'","strict-transport-security":"max-age=16070400; includeSubDomains","vary":"Accept-Encoding"},"statusCode":400}

Also yes, I started using the function node to delete content to get to this point. Specifically that content length was throwing a JSON error on their side. Also, it doesn't seem to matter if I capitalize Accept.

msg.headers = msg.headers || {}; //make headers an object if empty
msg.headers["accept"] = 'application/json';
delete msg.topic;
delete msg.statusCode;
delete msg.headers["content-length"];
return msg;

I've tested it in Postman and verified the "Accept" isn't really required at least not in there.

I found, it. There is an additional cookie that is not carried along. The fourth cookie is clearly security token. In Postman it appears as
TS01a6bd37
01cf42f8a6e7682b434f8c3322c012c9e4daf64911ef58f0fc55ccab406f30e5f6c57287d0337da518de804026aa14bce3ebc111a548c2a85752bf3bf9fda4d28479046061c561b723ca602c20aba6e5ff20eeb19c

Digging in to see if I can get it.

I've stepped this back to just doing a simple logon/logoff trying both the http request node and the www-request node. Neither one of them actually work in passing the session along like it does in Postman. That session cookie and their process is giving me the rage.

Logon:

{"_msgid":"be53dbf48c4f43cd","payload":{"version":"7.44.0","session":"BCA37503-98D5-4F60-9707-ED4E806E4569"},"topic":"","headers":{"cache-control":"no-cache","keep-alive":"timeout=5","content-type":"application/json; charset=utf-8","set-cookie":["session=BCA37503-98D5-4F60-9707-ED4E806E4569; Secure; HttpOnly; Path=/;samesite=none","AV-Cookie=!EZKCo6hbbsP3Gh4ueq1NBsctUyzZDnStWnnF14kSnhdWPVeCPF8yGfWqDL+x1lEhRQPML0/0TpX4irY=; path=/; Httponly; Secure;samesite=none","TS01a6bd37=01cf42f8a6db393158b9a026eaef9d2e8ded77074a6080bc70f43c0aa2be2e36648f1ca4267d153e2e35dd27182d76c0fbabb2b2f819c77b04ab26a10d1db1490b95b9dd18079ea2ce6fda0825a76d38b976ae64c7; Path=/; Secure; HTTPOnly;samesite=none"],"date":"Fri, 25 Feb 2022 04:04:47 GMT","content-length":"69","x-frame-options":"SAMEORIGIN","x-xss-protection":"1; mode=block","x-content-type-options":"nosniff","content-security-policy":"script-src 'unsafe-inline' 'unsafe-eval' 'self' https://*; object-src 'self'","strict-transport-security":"max-age=16070400; includeSubDomains","vary":"Accept-Encoding"},"statusCode":200}

Content sent to logoff after I have filtered out the body:

{"_msgid":"be53dbf48c4f43cd","headers":{"cache-control":"no-cache","keep-alive":"timeout=5","content-type":"application/json; charset=utf-8","set-cookie":["session=BCA37503-98D5-4F60-9707-ED4E806E4569; Secure; HttpOnly; Path=/;samesite=none","AV-Cookie=!EZKCo6hbbsP3Gh4ueq1NBsctUyzZDnStWnnF14kSnhdWPVeCPF8yGfWqDL+x1lEhRQPML0/0TpX4irY=; path=/; Httponly; Secure;samesite=none","TS01a6bd37=01cf42f8a6db393158b9a026eaef9d2e8ded77074a6080bc70f43c0aa2be2e36648f1ca4267d153e2e35dd27182d76c0fbabb2b2f819c77b04ab26a10d1db1490b95b9dd18079ea2ce6fda0825a76d38b976ae64c7; Path=/; Secure; HTTPOnly;samesite=none"],"date":"Fri, 25 Feb 2022 04:04:47 GMT","x-frame-options":"SAMEORIGIN","x-xss-protection":"1; mode=block","x-content-type-options":"nosniff","content-security-policy":"script-src 'unsafe-inline' 'unsafe-eval' 'self' https://*; object-src 'self'","strict-transport-security":"max-age=16070400; includeSubDomains","vary":"Accept-Encoding","accept":"application/json"}}

Response from the logoff: (Notice the session ID is completely different like it ignored the session cookie)

{"_msgid":"be53dbf48c4f43cd","headers":{"cache-control":"no-cache","keep-alive":"timeout=5","content-type":"application/json; charset=utf-8","set-cookie":["session=; Max-Age=0; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Secure; HttpOnly; Path=/;samesite=none","AV-Cookie=!RHBtVxZF/z0QZUEueq1NBsctUyzZDlCzxfW/jyg/PwRxR7wveZVJp+wLueX5mtCxGx0MBBhVgKzvPZQ=; path=/; Httponly; Secure;samesite=none","TS01a6bd37=01cf42f8a68387746f4ab4a80e8c8c1ab00503543805db8dbe3f93ecf7355441ae70097058395951d53c6d621f841f93f82e1049de5dbafe1f6c4a82235043737c30d5de6e9544afc0a86d22d6f923f8ab6ccbce7b; Path=/; Secure; HTTPOnly;samesite=none"],"date":"Fri, 25 Feb 2022 04:04:47 GMT","content-length":"69","x-frame-options":"SAMEORIGIN","x-xss-protection":"1; mode=block","x-content-type-options":"nosniff","content-security-policy":"script-src 'unsafe-inline' 'unsafe-eval' 'self' https://*; object-src 'self'","strict-transport-security":"max-age=16070400; includeSubDomains","vary":"Accept-Encoding"},"payload":{"version":"7.44.0","session":"CD59909A-2A3A-4589-9F1D-9D5298F58C30"},"statusCode":200}

This is the only thing I did in the function node:

msg.headers = msg.headers || {}; //make headers an object if empty
msg.headers["accept"] = 'application/json';
delete msg.topic;
delete msg.statusCode;
delete msg.headers["content-length"];
delete msg.payload
return msg;

What are your thoughts? It seems clear to me that the session cookie is carrying along just fine but for some reason their API treats it like it doesn't exist.

Thanks

yes .. it looks like the session is ignored most probably because you dont have anything in your Function to send it. You are just sending back with the http-request node whatever headers you received and also adding msg.headers["accept"] = 'application/json'; .. but is this enough for the server to consider it as the same session ?

Correct if im wrong but with the server reply you receive some session data which is received in header set-cookie and then you send it back with msg.headers = msg.headers || {}; again in the header set-cookie .. shouldnt that be in header Cookie instead ? set-cookie is a header for the receiving client side to set the cookies in the browser - if there was a browser .. in this case there isnt so we need to manually send the session data on each request (something that would happen automatically in case of a browser)

As Steve said show us step by step what is being sent and works in Postman (including headers) so we can try to replicate it for Node-red.
In postman you have a convenient Code icon image
that shows how a request should be constructed for different languages .. NodeJs - Axios is a close enough library to how request are prepared for NR.

Alright, I broke down and started forcing everything through a monitoring proxy (fiddler). What I have found is that the cookies are being dropped in miscellaneous instead of in Cookies. I'm including screenshots.

Here is the Postman logoff

Here is the Node-Red logoff

I'd like nothing more than to share completely but I'm limited with what I can share due to proprietary information.

If I am understanding what you are saying and what has been said before, I first need to convert the cookies to objects and then put them in the subsequent requests instead of just forwarding them along which makes sense.