Before i start, i am an advanced AHK user, but absolutely horrendous at any form of web systems
the lack of any standardization, and the requirement to know HTML, CSS, JS, PHP, SQL, and f knows what else just to code a website has resulted in me having ZERO desire to be any part of that absolute mess!
as such, my knowledge on how websites and even moreso the security surrounding websites is completely lost on me, i know as much as your grandmother does
Anywho, with that out of the way;
i'm trying to pull information from a site that is using Cloudflart, ofcourse it's checking to see if it's a browser or not, and with COM controlling an IE8 instance, cloudflare should pass
i have scoured the forums and found ONE guy who appears to have got it working but only posted vague details on why and how (also was horrendously salty that nobody was helping him!)
here's the code he posted (renamed WHO to ie because it was bugging me)
Code: Select all
hdr := "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0)"
url1 := "http://google.com/" ; Referrer URL, in case it's required
ie := ComObjCreate("WinHttp.WinHttpRequest.5.1")
ie.Option(0) := hdr ; UserAgentString
ie.Option(4) := 0x3300 ; SslErrorIgnoreFlags (ignore all certificate errors)
ie.Option(6) := 1 ; EnableRedirects
ie.Option(9) := 0xA8 ; SecureProtocols: TLS 1.0=0x80, SSL 3.0=0x20, SSL 2.0=0x8
ie.Option(11) := 0 ; RevertImpersonationOverSsl
ie.Option(12) := 1 ; EnableHttpsToHttpRedirects
ie.Option(17) := 1 ; EnableHttp1_1
ie.Option(18) := 0 ; EnableCertificateRevocationCheck
ie.Open("GET", url, true) ; async mode=true
ie.SetRequestHeader("Referer", url1)
ie.SetRequestHeader("User-Agent", hdr) ; (this appears to be separate from the WinHTTP option 0)
ie.SetTimeouts(0, 60000, 30000, 120000) ; Resolve, Connect, Send, Receive.Timeout
ie.Send()
ie.WaitForResponse(120) ; only in async mode
response := ie.ResponseText
; url2 is the prepared reply URL containing the elements of the solved challenge
ie.Open("GET", url2, true) ; async mode=true
ie.SetRequestHeader("Referer", url) ; set original URL as referrer
ie.SetRequestHeader("User-Agent", hdr)
ie.SetTimeouts(0, 60000, 30000, 120000) ; Resolve, Connect, Send, Receive.Timeout
ie.Send()
ie.WaitForResponse(120) ; only in async mode
finalResponse := ie.ResponseText
ie.quit()
specifically:
Code: Select all
Error: 0x80070057 - The parameter is incorrect.
Specifically: Open
025: ie.Open("GET", url, true)
026: ie.SetRequestHeader("Referer", url1)
027: ie.SetRequestHeader("User-Agent", hdr)
028: ie.SetTimeouts(0, 60000, 30000, 120000)
029: ie.Send()
030: ie.WaitForResponse(120)
031: response := ie.ResponseText
---> 033: ie.Open("GET", url2, true)
034: ie.SetRequestHeader("Referer", url)
035: ie.SetRequestHeader("User-Agent", hdr)
036: ie.SetTimeouts(0, 60000, 30000, 120000)
037: ie.Send()
038: ie.WaitForResponse(120)
039: finalResponse := ie.ResponseText
040: ie.quit()
Code: Select all
response := ie.ResponseText
Code: Select all
<h2 data-translate="why_captcha_headline">Why do I have to complete a CAPTCHA?</h2>
<p data-translate="why_captcha_detail">Completing the CAPTCHA proves you are a human and gives you temporary access to the web property.</p>
I run Brave as a main browser, and it disables all JS iirc, this leads to me being captcha'd at every check, which is fine for me, ofcourse not fine for automation
now ofcourse COM uses IE8, so is it possible there's JS code not runnning or something?
https://www.autohotkey.com/boards/viewtopic.php?t=14218
certainly seems to be a JS issue!
https://autohotkey.com/board/topic/111944-winhttprequest-and-ddos-protection/
Progress has been made in the form of this;
https://github.com/Stickymaddness/Procurement/wiki/SessionID#chrome
seems procurement got by the issue by informing people to plug in their session ID, which isn't too bad!!
i'll look to getting this working, ofcourse any more info would be fantastic!