Broken http request flow

Hello Node-Red community,
I have basic flow that triggers 5hours from a website everyday at 00:00 and write them in a times.txt file.
It was working fine in last few months but I think something changed on the target website which broke the flow as you can see on the screenshot I am getting an error message.
The target website where it takes hours is: Presidency of Religious Affairs | Prayer Time for Montreal

I don't have a lot of programming skills, just trying to understand where to start and get this fixed... Thanks!

Did you set an user-agent in the http request node ? Without a user-agent it assumes there is no javascript available and blocks it.

This example flow produces this:

[{"id":"c905ee12da44453a","type":"html","z":"b8a01411d25c8f49","name":"","property":"payload","outproperty":"payload","tag":"#today-pray-times-row div[class=tpt-cell]","ret":"html","as":"multi","x":415,"y":100,"wires":[["63ed769d1f785e1c","6ac3f5aacd7f1664"]],"l":false},{"id":"2b2a9bbc796fbadd","type":"inject","z":"b8a01411d25c8f49","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":100,"wires":[["1316bdb5c0a8c752"]]},{"id":"1316bdb5c0a8c752","type":"http request","z":"b8a01411d25c8f49","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"https://namazvakitleri.diyanet.gov.tr/en-US/9132/prayer-time-for-montreal","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"User-Agent","keyValue":"","valueType":"other","valueValue":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:89.0) Gecko/20100101 Firefox/89.0"}],"x":290,"y":100,"wires":[["c905ee12da44453a"]]},{"id":"daf1ac4bf1cd7606","type":"debug","z":"b8a01411d25c8f49","name":"debug 122","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":790,"y":100,"wires":[]},{"id":"63ed769d1f785e1c","type":"html","z":"b8a01411d25c8f49","name":"","property":"payload","outproperty":"payload","tag":"div[class=tpt-title]","ret":"html","as":"multi","x":485,"y":80,"wires":[["5f4eca7604092473"]],"l":false},{"id":"6ac3f5aacd7f1664","type":"html","z":"b8a01411d25c8f49","name":"","property":"payload","outproperty":"payload","tag":"div[class=tpt-time]","ret":"html","as":"multi","x":485,"y":120,"wires":[["5f4eca7604092473"]],"l":false},{"id":"5f4eca7604092473","type":"join","z":"b8a01411d25c8f49","name":"","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":555,"y":100,"wires":[["4198e8a4659a8f31"]],"l":false},{"id":"4198e8a4659a8f31","type":"change","z":"b8a01411d25c8f49","name":"","rules":[{"t":"move","p":"payload","pt":"msg","to":"temp","tot":"msg"},{"t":"delete","p":"payload","pt":"msg"},{"t":"set","p":"payload","pt":"msg","to":"{}","tot":"json"},{"t":"set","p":"payload.prayer","pt":"msg","to":"temp[0]","tot":"msg"},{"t":"set","p":"payload.time","pt":"msg","to":"temp[1]","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":615,"y":100,"wires":[["19d626209f440d07"]],"l":false},{"id":"19d626209f440d07","type":"join","z":"b8a01411d25c8f49","name":"","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"1","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":675,"y":100,"wires":[["daf1ac4bf1cd7606"]],"l":false}]
1 Like

Thanks @bakman2 just added the user-agent in http request node and it worked..

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.