URLDownloadToFile and Cloudflare

Get help with using AutoHotkey and its commands and hotkeys
ozzii
Posts: 380
Joined: 30 Oct 2013, 06:04

URLDownloadToFile and Cloudflare

20 Nov 2020, 12:17

Hi,

I try to do an URLDownloadToFile for an RSS file.
But the site is protected with cloudflare. In the downladed file I have the message 'wait for 5 secunds...'

So at the end I don't have the wanted file :headwall:

Is there any way to have the file after the cloudflare check or in some way to bypass it?
User avatar
TheDewd
Posts: 1449
Joined: 19 Dec 2013, 11:16
Location: USA

Re: URLDownloadToFile and Cloudflare

20 Nov 2020, 12:30

Would you be willing to share the link to the RSS file?

If not, do you know any other similar RSS feeds that use Cloudflare?

It may be possible to find a workaround for your issue, but without something to test it would be difficult.

I do have an idea, but would like to test my theory first.
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
malcev
Posts: 829
Joined: 12 Aug 2014, 12:37

Re: URLDownloadToFile and Cloudflare

20 Nov 2020, 14:14

With URLDownloadToFile - no chance.
You have to solve their js puzzle by Yourself or use cookies from IE with winhttprequest or msxml2.xmlhttp, that used ie cookies by default.
ozzii
Posts: 380
Joined: 30 Oct 2013, 06:04

Re: URLDownloadToFile and Cloudflare

21 Nov 2020, 04:51

Thank for the answer.

It's not a puzzle it's juste the javascript test.....
But no chances for me. Too complicate for me (winhttprequest or msxml2.xmlhttp) :headwall:
I will try to findan existing script with these 2 for example.

Thank you again for the answer

EDIT:
Tried this:

Code: Select all

_whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
_whr.Open("GET", URL, true)
_whr.Send()
_whr.WaitForResponse()
msgbox % _whr.ResponseText
No luck. the javascript is not executed. Same result like the URLDownloadToFile
User avatar
TheDewd
Posts: 1449
Joined: 19 Dec 2013, 11:16
Location: USA

Re: URLDownloadToFile and Cloudflare

21 Nov 2020, 11:24

I was going to suggest loading the link in an activex control, emulating the edge browser. I've had success with this method for other projects. Then handle the download from there.
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
teadrinker
Posts: 2057
Joined: 29 Mar 2015, 09:41
Contact:

Re: URLDownloadToFile and Cloudflare

21 Nov 2020, 12:27

TheDewd wrote: emulating the edge browser
How did you achieve this?
User avatar
TheDewd
Posts: 1449
Joined: 19 Dec 2013, 11:16
Location: USA

Re: URLDownloadToFile and Cloudflare

21 Nov 2020, 12:31

teadrinker wrote:
21 Nov 2020, 12:27
How did you achieve this?
I may have not used the correct phrasing, but something like this is what I'm referring to: https://www.autohotkey.com/boards/viewtopic.php?p=365169#p365169

It's worth a try.
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
teadrinker
Posts: 2057
Joined: 29 Mar 2015, 09:41
Contact:

Re: URLDownloadToFile and Cloudflare

21 Nov 2020, 12:43

That is an implementation of Internet Explorer, not the Edge browser. :)
malcev
Posts: 829
Joined: 12 Aug 2014, 12:37

Re: URLDownloadToFile and Cloudflare

21 Nov 2020, 15:56

Too complicate for me (winhttprequest or msxml2.xmlhttp)
Then use IE or Chrome automation.
ozzii
Posts: 380
Joined: 30 Oct 2013, 06:04

Re: URLDownloadToFile and Cloudflare

22 Nov 2020, 03:30

I think that the automation is the only answer for me :headwall:
I just need to see if it's possible to do it with a hidden window.
I need the script to loop every 10 minutes. So having the chrome window showing every 10 minutes while I'm on the computer it's not so "cool".
User avatar
TheDewd
Posts: 1449
Joined: 19 Dec 2013, 11:16
Location: USA

Re: URLDownloadToFile and Cloudflare

22 Nov 2020, 10:49

ozzii wrote:
22 Nov 2020, 03:30
I think that the automation is the only answer for me :headwall:
I just need to see if it's possible to do it with a hidden window.
I need the script to loop every 10 minutes. So having the chrome window showing every 10 minutes while I'm on the computer it's not so "cool".
That's why you try using ActiveX. Look at the link I provided. You can have it hidden. Make sure to enable FBE to ensure the browser emulation mode is using the latest IE mode.
Image Bulldozer - Sokoban inspired game from 1994 recreated in AutoHotkey.
malcev
Posts: 829
Joined: 12 Aug 2014, 12:37

Re: URLDownloadToFile and Cloudflare

22 Nov 2020, 14:03

I just need to see if it's possible to do it with a hidden window.
You can use IE or Chrome in hidden mode.
If You dont like this variants then You need reverse-engineering this js puzzle by Yourself or hire freelancer.
ozzii
Posts: 380
Joined: 30 Oct 2013, 06:04

Re: URLDownloadToFile and Cloudflare

23 Nov 2020, 04:34

Thank you all for your answers :salute:
My last thing to do it's to try to understand and use script for the hidden mode.

Return to “Ask For Help”

Who is online

Users browsing this forum: Google [Bot], HELLiPOD, HiSoKa, IceBubble, Lord-9621, TAC109 and 42 guests