# Http request with http aouth01 to get a binary buffer

**URL:** <https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081>\
**Category:** General\
**Created:** [17 November 2020 17:33 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081 "2020-11-17T17:33:24Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 17:33 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/1 "2020-11-17T17:33:24Z")

</div>

I want to create an endpoint API to display the image in Twitter DM  
like in this [URL](https://developer.twitter.com/en/docs/twitter-api/v1/direct-messages/message-attachments/guides/retrieving-media)  
I tried to make a request using HTTP request but it's not working because it needs oauth01  
I also tried oauth 01 node it's retrieving data as JSON object and I want to retrieve data as a binary buffer.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 18:00 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/3 "2020-11-17T18:00:15Z")

</div>

Hi @mohamedEmadEldien, welcome to the forum.

Please note, tagging a person after 3m of posting your first ever post is not really cricket. We are all unpaid helpers and have lives outside of node-red.

So if I read this right, you managed to get a response (in JSON) ?

Why is this no good? What did the JSON include? If it was a valid response, the data you want might be in there.

Can you post a screenshot of the output (debug node)

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 19:27 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/4 "2020-11-17T19:27:11Z")

</div>

Thanks @Steve-Mcl for your prompt response

the below image contains my flow but I receive the response as a JSON object instead of binary buffer so I got the below error when I use HTTP out

 ![flow](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/5/5/55a9eb72010161f981c9d12f7632bcaa943dcf39.png)

TypeError: Converting circular structure to JSON --\> starting at object with constructor 'ClientRequest' | property 'socket' -\> object with constructor 'TLSSocket' --- property '\_httpMessage' closes the circle

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 19:30 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/5 "2020-11-17T19:30:55Z")

</div>

Forget the endpoint for now (use just a simple inject and debug output)

e.g..

inject --\> function --\> OAuth --\> Debug (set to show complete message)

then press the inject & expand the DEBUG message to show me what is inside.

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 19:38 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/6 "2020-11-17T19:38:09Z")

</div>

ok, @Steve-Mcl

I added debug node, the below image contains the message

 ![Data](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/0/3/03b6b0b6bcf8592e6a3f72e87b724b5cf0b4c7d0.png)

the below is the json object  
the object contains an image in the data field

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 19:40 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/7 "2020-11-17T19:40:24Z")

</div>

so msg.data contains the image you want right? (I cant quite see)

Could you collapse the cookies and header objects so i can see the top msg part.

Next you want to automate this so that it is returned to an endpoint - correct?

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 19:47 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/8 "2020-11-17T19:47:17Z")

</div>

@Steve-Mcl, Yes msg.data contains the image  
the below image contains the header object  
Yes, I want to automate this to return to the end point

 ![header object](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/c/5/c5fb7ea199042e33ee7daab5b2ed148f1e997b86.png)

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 19:53 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/9 "2020-11-17T19:53:34Z")

</div>

> [@Steve-Mcl](#):
>
> Could you collapse the cookies and header objects so i can see the top msg part.

^^^^^^^^^^^^^^ ??? ^^^^^^^^^^^^^^^

I need to see the msg & its 1st level properties of the debug msg

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 20:00 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/10 "2020-11-17T20:00:36Z")

</div>

Here is the full object and the request part

 ![object](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/9/8/986166cab68bfb022561d7caab30db9a2ddb9802.png)

 ![request](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/0/5/05bb897803725521e5952719aba588cc663f547c.png)

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 20:03 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/11 "2020-11-17T20:03:36Z")

</div>

ok, I see the issue. the payload that comes out of the OAuth node has all that content in - you only need the data part...

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/5/d/5d359ccf11905aac7777834d66b384736948b506.png)

function content...

```auto
msg.payload = msg.payload.data;
return msg;

```

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 20:12 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/12 "2020-11-17T20:12:48Z")

</div>

I really appreciate your support it's worked to return the response  
but I still have an issue I want to add an image tag in my HTML page and tag image call this endpoint to retrieve the image  
like the last example in this [url](https://developer.twitter.com/en/docs/twitter-api/v1/direct-messages/message-attachments/guides/retrieving-media)  
but the content is like the below image of the response

 ![conetnt](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/5/f/5f33ccdf41783df641edcdaac16768929c082333.png)

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 20:22 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/13 "2020-11-17T20:22:04Z")

</div>

so now you need to convert the string image to either a binary buffer or base64 data url.

without having the data I cant try things out but you will likely need to use the Buffer.from() function.  
you will likely need to set headers in the response to match the data type you use.

as a test, try changing the function to

```auto
msg.payload = Buffer.from(msg.payload.data);
return msg;

```

and add a header to match

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/1/d110e70d9d9fd24d7fbffc85fb270c4d05ace603.png)

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 20:23 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/14 "2020-11-17T20:23:58Z")

</div>

Actually, there is something odd

according to your earlier screen shot, msg.payload.data is an object NOT a string (so the above wont work)

could you expand `msg.payload.data` and screenshot once more?

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 20:40 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/15 "2020-11-17T20:40:15Z")

</div>

Thanks @Steve-Mcl  
Yes, It's not worked the below image contains msg.payload.data

 ![msg_data](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/8/0/80bfcd24a132185d01a1997cc25b72a2dad2590d.png)

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 20:41 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/16 "2020-11-17T20:41:23Z")

</div>

Also I got this error when I added Buffer.from(msg.payload.data);

"TypeError [ERR\_INVALID\_ARG\_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined"

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 20:45 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/17 "2020-11-17T20:45:32Z")

</div>

oh, that is a big mess.

> [@mohamedEmadEldien](#):
>
> Also I got this error when I added Buffer.from(msg.payload.data);

Yes, I knew it would as `msg.payload.data` is an object (I said so above)

> [@Steve-Mcl](#):
>
> according to your earlier screen shot, msg.payload.data is an object NOT a string (so the above wont work)

Need more info...

- could you expand msg.payload.data.PNG & screenshot once more
- could you expand the next object down (below payload.data.PNG) & screenshot that too

NOTE...  
I am not certain we will be able to re-assemble that data into a usable image - we might have to [post an issue](https://github.com/aldian/node-red-contrib-oauth1a/issues) on the github repo.

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 20:50 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/18 "2020-11-17T20:50:55Z")

</div>

Here is the screen shots

but data is not json object it's a binary file may be some chartchters made like curly braces "{"  
so it might appear as an object

 ![data2](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/1/3/1319be29b1ff9c5a86c6b5d89f9d760038429ce7.png)

 ![Data](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/0/3/03b6b0b6bcf8592e6a3f72e87b724b5cf0b4c7d0.png)

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 21:08 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/19 "2020-11-17T21:08:33Z")

</div>

> [@mohamedEmadEldien](#):
>
> but data is not json object it's a binary file may be some chartchters made like curly braces "{"  
> so it might appear as an object

It is definitely an object. node-red would have shown it as a buffer or string if were a buffer or string.

note - I do understand what you are saying but the fact is, whatever the data **WAS** , it has been returned by that node AS AN OBJECT. See [here](https://github.com/aldian/node-red-contrib-oauth1a/blob/9f5404fe0bfc90f3b8a35d821673b7702b0a9b06/oauth1a.js#L56) what the author does to the data.

you can see what a binary PNG should start with [here](http://www.libpng.org/pub/png/spec/1.2/PNG-Rationale.html#R.PNG-file-signature) - see anything familiar?

We can try to solve it or post an issue.

~~Try setting the function code to this...~~

~~`~~ ~~var data = JSON.stringify(msg.payload.data).trim(); //undo the parse-ing that the node does~~ ~~data = data.substring(1, data.length-1); //remove 1st and last chars (assuming they will be {" and~~}) ~~~~ msg.payload = Buffer.from(msg.payload.data); ~~~~ return msg; ~~~~ `~~

I have been looking through the nodes code - the data _MIGHT_ be present in another place - un touched.

could you expand msg.payload and look for another 'data' object/string that contains something like "PNG" or a buffer or an array?

---

<div class="post-metadata">

**Author:** ![mohamedEmadEldien](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/mohamedemadeldien/32/32137_2.png) [@mohamedEmadEldien](https://discourse.nodered.org/u/mohamedEmadEldien)\
**Post date:** [17 November 2020 21:20 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/20 "2020-11-17T21:20:49Z")

</div>

I tried this code  
var data = JSON.stringify(msg.payload.data).trim();  
//undo the parse-ing that the node does ~~~~  
data = data.substring(1, data.length-1);  
//remove 1st and last chars (assuming they will be {" and ~~})~~ ~~  
msg.payload = Buffer.from(msg.payload.data.toString());  
return msg;  
but still not working  
for untouched I searched for this data in the whole object but unfortunately I didn't find it  
-I think in another idea but i'm not sure if it will work or not  
use the below flow  
http in -\> function -\> http request-\> function -\> http out  
in http request use return as a binary buffer

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 November 2020 21:26 UTC](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081/21 "2020-11-17T21:26:29Z")

</div>

> [@mohamedEmadEldien](#):
>
> I think in another idea but i'm not sure if it will work or not  
> use the below flow  
> http in -\> function -\> http request-\> function -\> http out  
> in http request use return as a binary buffer

Yes, if you have the token from the OAuth you can probably use the http request node. but thats a whole new story.

for now, I would say - raise an issue on the repo & link to this thread - the author might point out something we have missed.

Alternatively, someone else with a better understanding might pop in and help you progress this.

For now, I dont think I can take you any further.

[Next page](https://discourse.nodered.org/t/http-request-with-http-aouth01-to-get-a-binary-buffer/36081.md?page=2)