URLDownloadToFile and Cloudflare

[ozzii]

Hi,

I try to do an URLDownloadToFile for an RSS file.
But the site is protected with cloudflare. In the downladed file I have the message 'wait for 5 secunds...'

So at the end I don't have the wanted file

Is there any way to have the file after the cloudflare check or in some way to bypass it?

[TheDewd]

Would you be willing to share the link to the RSS file?

If not, do you know any other similar RSS feeds that use Cloudflare?

It may be possible to find a workaround for your issue, but without something to test it would be difficult.

I do have an idea, but would like to test my theory first.

[malcev]

With URLDownloadToFile - no chance.
You have to solve their js puzzle by Yourself or use cookies from IE with winhttprequest or msxml2.xmlhttp, that used ie cookies by default.

[ozzii]

Thank for the answer.

It's not a puzzle it's juste the javascript test.....
But no chances for me. Too complicate for me (winhttprequest or msxml2.xmlhttp)
I will try to findan existing script with these 2 for example.

Thank you again for the answer

EDIT:
Tried this:

Code: Select all

_whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
_whr.Open("GET", URL, true)
_whr.Send()
_whr.WaitForResponse()
msgbox % _whr.ResponseText

No luck. the javascript is not executed. Same result like the URLDownloadToFile

[TheDewd]

I was going to suggest loading the link in an activex control, emulating the edge browser. I've had success with this method for other projects. Then handle the download from there.

[teadrinker]

emulating the edge browser

How did you achieve this?

[TheDewd]

How did you achieve this?

I may have not used the correct phrasing, but something like this is what I'm referring to: https://www.autohotkey.com/boards/viewtopic.php?p=365169#p365169

It's worth a try.

[teadrinker]

That is an implementation of Internet Explorer, not the Edge browser.

[malcev]

Too complicate for me (winhttprequest or msxml2.xmlhttp)

Then use IE or Chrome automation.

[ozzii]

I think that the automation is the only answer for me
I just need to see if it's possible to do it with a hidden window.
I need the script to loop every 10 minutes. So having the chrome window showing every 10 minutes while I'm on the computer it's not so "cool".

[TheDewd]

I think that the automation is the only answer for me
I just need to see if it's possible to do it with a hidden window.
I need the script to loop every 10 minutes. So having the chrome window showing every 10 minutes while I'm on the computer it's not so "cool".

That's why you try using ActiveX. Look at the link I provided. You can have it hidden. Make sure to enable FBE to ensure the browser emulation mode is using the latest IE mode.

[malcev]

I just need to see if it's possible to do it with a hidden window.

You can use IE or Chrome in hidden mode.
If You dont like this variants then You need reverse-engineering this js puzzle by Yourself or hire freelancer.

[ozzii]

Thank you all for your answers
My last thing to do it's to try to understand and use script for the hidden mode.