Http request with http aouth01 to get a binary buffer

I want to create an endpoint API to display the image in Twitter DM
like in this URL
I tried to make a request using HTTP request but it's not working because it needs oauth01
I also tried oauth 01 node it's retrieving data as JSON object and I want to retrieve data as a binary buffer.

Hi @mohamedEmadEldien, welcome to the forum.

Please note, tagging a person after 3m of posting your first ever post is not really cricket. We are all unpaid helpers and have lives outside of node-red.

So if I read this right, you managed to get a response (in JSON) ?

Why is this no good? What did the JSON include? If it was a valid response, the data you want might be in there.

Can you post a screenshot of the output (debug node)

1 Like

Thanks @Steve-Mcl for your prompt response

the below image contains my flow but I receive the response as a JSON object instead of binary buffer so I got the below error when I use HTTP out

TypeError: Converting circular structure to JSON --> starting at object with constructor 'ClientRequest' | property 'socket' -> object with constructor 'TLSSocket' --- property '_httpMessage' closes the circle

Forget the endpoint for now (use just a simple inject and debug output)

e.g..

inject --> function --> OAuth --> Debug (set to show complete message)

then press the inject & expand the DEBUG message to show me what is inside.

ok, @Steve-Mcl

I added debug node, the below image contains the message

the below is the json object
the object contains an image in the data field

so msg.data contains the image you want right? (I cant quite see)

Could you collapse the cookies and header objects so i can see the top msg part.

Next you want to automate this so that it is returned to an endpoint - correct?

@Steve-Mcl, Yes msg.data contains the image
the below image contains the header object
Yes, I want to automate this to return to the end point

^^^^^^^^^^^^^^ ??? ^^^^^^^^^^^^^^^

I need to see the msg & its 1st level properties of the debug msg

Here is the full object and the request part

ok, I see the issue. the payload that comes out of the OAuth node has all that content in - you only need the data part...

function content...

msg.payload = msg.payload.data;
return msg;
1 Like

I really appreciate your support it's worked to return the response
but I still have an issue I want to add an image tag in my HTML page and tag image call this endpoint to retrieve the image
like the last example in this url
but the content is like the below image of the response

so now you need to convert the string image to either a binary buffer or base64 data url.

without having the data I cant try things out but you will likely need to use the Buffer.from() function.
you will likely need to set headers in the response to match the data type you use.

as a test, try changing the function to

msg.payload = Buffer.from(msg.payload.data);
return msg;

and add a header to match

image

1 Like

Actually, there is something odd

according to your earlier screen shot, msg.payload.data is an object NOT a string (so the above wont work)

could you expand msg.payload.data and screenshot once more?

Thanks @Steve-Mcl
Yes, It's not worked the below image contains msg.payload.data

Also I got this error when I added Buffer.from(msg.payload.data);

"TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined"

oh, that is a big mess.

Yes, I knew it would as msg.payload.data is an object (I said so above)

Need more info...

  • could you expand msg.payload.data.PNG & screenshot once more
  • could you expand the next object down (below payload.data.PNG) & screenshot that too

NOTE...
I am not certain we will be able to re-assemble that data into a usable image - we might have to post an issue on the github repo.

Here is the screen shots

but data is not json object it's a binary file may be some chartchters made like curly braces "{"
so it might appear as an object

It is definitely an object. node-red would have shown it as a buffer or string if were a buffer or string.

note - I do understand what you are saying but the fact is, whatever the data WAS, it has been returned by that node AS AN OBJECT. See here what the author does to the data.

you can see what a binary PNG should start with here - see anything familiar?

We can try to solve it or post an issue.

Try setting the function code to this...

~~ ~~var data = JSON.stringify(msg.payload.data).trim(); //undo the parse-ing that the node does~~ ~~data = data.substring(1, data.length-1); //remove 1st and last chars (assuming they will be {" and ~~})~~ ~~msg.payload = Buffer.from(msg.payload.data);~~ ~~return msg;~~ ~~

I have been looking through the nodes code - the data MIGHT be present in another place - un touched.

could you expand msg.payload and look for another 'data' object/string that contains something like "PNG" or a buffer or an array?

I tried this code
var data = JSON.stringify(msg.payload.data).trim();
//undo the parse-ing that the node does~~ ~~
data = data.substring(1, data.length-1);
//remove 1st and last chars (assuming they will be {" and }) ~~
msg.payload = Buffer.from(msg.payload.data.toString());
return msg;
but still not working
for untouched I searched for this data in the whole object but unfortunately I didn't find it
-I think in another idea but i'm not sure if it will work or not
use the below flow
http in -> function -> http request-> function -> http out
in http request use return as a binary buffer

Yes, if you have the token from the OAuth you can probably use the http request node. but thats a whole new story.

for now, I would say - raise an issue on the repo & link to this thread - the author might point out something we have missed.

Alternatively, someone else with a better understanding might pop in and help you progress this.

For now, I dont think I can take you any further.

1 Like