Download file from internet Topic is solved

[euras]

hi, my company uses firewalls, which does not allow to download a file from internet using

Code: Select all

UrlDownloadToFile, https://www.autohotkey.com/download/1.1/version.txt, C:\AutoHotkey Latest Version.txt

or

Code: Select all

whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", "https://www.autohotkey.com/download/1.1/version.txt", true)
whr.Send()
; Using 'true' above and the call below allows the script to remain responsive.
whr.WaitForResponse()
version := whr.ResponseText
MsgBox % version

but this one passes.

Code: Select all

req := ComObjCreate("Msxml2.XMLHTTP")
; Open a request with async enabled.
req.open("GET", "https://www.autohotkey.com/download/1.1/version.txt", true)
; Set our callback function [requires v1.1.17+].
req.onreadystatechange := Func("Ready")
; Send the request.  Ready() will be called when it's complete.
req.send()
/*
; If you're going to wait, there's no need for onreadystatechange.
; Setting async=true and waiting like this allows the script to remain
; responsive while the download is taking place, whereas async=false
; will make the script unresponsive.
while req.readyState != 4
    sleep 100
*/
#Persistent

Ready() {
    global req
    if (req.readyState != 4)  ; Not done yet.
        return
    if (req.status == 200) ; OK.
        MsgBox % "Latest AutoHotkey version: " req.responseText
    else
        MsgBox 16,, % "Status " req.status
    ExitApp
}

I'm curious is it possible to download a file, using that function? or it's build only for retrieving a plain text? Maybe someone can put an example how to use this function to download a file?

[teadrinker]

Code: Select all

#Persistent
url := "https://www.autohotkey.com/download/ahk-install.exe"
localPath := A_Desktop . "\ahk-install.exe"

Req := ComObjCreate("Msxml2.XMLHTTP.6.0")
Req.Open("GET", url, true)
Req.onreadystatechange := Func("Ready").Bind(Req, localPath)
Req.Send()
Return

Ready(Req, filePath) {
   if !(Req.readyState = 4 && Req.status = 200)
      Return
   
   Arr := Req.responseBody
   pData := NumGet(ComObjValue(Arr) + 8 + A_PtrSize)
   len := Arr.MaxIndex() + 1
   FileOpen(filePath, "w").RawWrite(pData + 0, len)
   MsgBox, File downloaded
}

[garry]

@teadrinker this is great , you have always a solution . I thought it only works for text. Is this faster/better than urldownloadtofile ?

[teadrinker]

Is this faster/better than urldownloadtofile ?

Its advantage is asynchrony. While the file is downloading, the script may do something else:

Code: Select all

#Persistent
url := "https://www.autohotkey.com/download/ahk-install.exe"
localPath := A_Desktop . "\ahk-install.exe"

Req := ComObjCreate("Msxml2.XMLHTTP.6.0")
Req.Open("GET", url, true)
Req.onreadystatechange := Func("Ready").Bind(Req, localPath)
Req.Send()

Loop {
   ToolTip % A_Index
   Sleep, 100
}
Return

Ready(Req, filePath) {
   if !(Req.readyState = 4 && Req.status = 200)
      Return
   
   Arr := Req.responseBody
   pData := NumGet(ComObjValue(Arr) + 8 + A_PtrSize)
   len := Arr.MaxIndex() + 1
   FileOpen(filePath, "w").RawWrite(pData + 0, len)
   MsgBox, File downloaded
}

[garry]

Interesting , thank you very much .

[euras]

Its advantage is asynchrony. While the file is downloading, the script may do something else:

thank you teadrinker! That works fine! Except trying to download geckodriver.zip. I think it's something with a sign "-" that blocks Req.Send() line.
any ideas how to pass this?

Code: Select all

FF_Driver_Version := "v0.29.0"
t_path := "C:\Users\Documents\"
url := "https://github.com/mozilla/geckodriver/releases/download/" FF_Driver_Version "/geckodriver-" FF_Driver_Version "-win64.zip"
localPath := "C:\Users\Documents\TempDrivers.zip"
    Req := ComObjCreate("Msxml2.XMLHTTP.6.0")
    Req.Open("GET", url, false)
    Req.Send()
    while Req.readyState != 4
        sleep 100
    Arr := Req.responseBody
    pData := NumGet(ComObjValue(Arr) + 8 + A_PtrSize)
    len := Arr.MaxIndex() + 1
    FileOpen(localPath, "w").RawWrite(pData + 0, len)

[teadrinker]

This URL https://github.com/mozilla/geckodriver/releases/download/v0.29.0/geckodriver-v0.29.0-win64.zip is not the direct address of the file, the file is downloaded by redirect. The code I provided does not support redirects. You can get the direct URL like this:

Code: Select all

localPath := "C:\Users\Documents\TempDrivers.zip"

FF_Driver_Version := "v0.29.0"
url := "https://github.com/mozilla/geckodriver/releases/download/" FF_Driver_Version "/geckodriver-" FF_Driver_Version "-win64.zip"

Whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
Whr.Option(WinHttpRequestOption_EnableRedirects := 6) := false
Whr.Open("GET", url, false)
Whr.Send()
MsgBox, % directUrl := Whr.GetResponseHeader("Location")

Req := ComObjCreate("Msxml2.XMLHTTP.6.0")
Req.Open("GET", directUrl, false)
Req.Send()
Arr := Req.responseBody
pData := NumGet(ComObjValue(Arr) + 8 + A_PtrSize)
len := Arr.MaxIndex() + 1
FileOpen(localPath, "w").RawWrite(pData + 0, len)

[euras]

This URL https://github.com/mozilla/geckodriver/releases/download/v0.29.0/geckodriver-v0.29.0-win64.zip is not the direct address of the file, the file is downloaded by redirect. The code I provided does not support redirects. You can get the direct URL like this:

thank you teadrinker! It works on my home PC, but on company PC I cannot use "WinHttp.WinHttpRequest.5.1".
I got an link from the home PC, using your function, and tried to feed it directly into function, but it only give me an empty zip file. Maybe the link needs to be modified?

Code: Select all

url := "https://github-releases.githubusercontent.com/25354393/3d4a5f00-5662-11eb-9fdb-9cd542e4f60a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210219%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210219T123512Z&X-Amz-Expires=300&X-Amz-Signature=e6c0cc4875b904b38dac8dfa4ca27ef87e98b1bbb1975dcdb939c37bdd5fb5a6&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=25354393&response-content-disposition=attachment%3B%20filename%3Dgeckodriver-v0.29.0-win64.zip&response-content-type=application%2Foctet-stream"

localPath := "C:\Users\Documents\TempDrivers.zip"
Req := ComObjCreate("Msxml2.XMLHTTP.6.0")
Req.Open("GET", url, false)
Req.Send()
while Req.readyState != 4
  sleep 100
Arr := Req.responseBody
pData := NumGet(ComObjValue(Arr) + 8 + A_PtrSize)
len := Arr.MaxIndex() + 1
FileOpen(localPath, "w").RawWrite(pData + 0, len)

[teadrinker]

Maybe the link needs to be modified?

I'm not sure, perhaps this link may be changed from time to time. I've got now another one:

Code: Select all

https://github-releases.githubusercontent.com/25354393/3d4a5f00-5662-11eb-9fdb-9cd542e4f60a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210219%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210219T125904Z&X-Amz-Expires=300&X-Amz-Signature=83f3d07e3f0e6a341c1ff4297162dc195b2f6e17b3fbb3c7585c2e1e920fc673&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=25354393&response-content-disposition=attachment%3B%20filename%3Dgeckodriver-v0.29.0-win64.zip&response-content-type=application%2Foctet-stream

[euras]

I'm not sure, perhaps this link may be changed from time to time. I've got now another one:

when I try to run this link on the browser, I get that HTML error. It looks like the link contains some temporary keys, which does not match second time, when you try to use it. Is there any other way to get that full link without Whr := ComObjCreate("WinHttp.WinHttpRequest.5.1") ?

Code: Select all

<?xml version="1.0" encoding="ISO-8859-1"?>
<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<X-Amz-Expires>300</X-Amz-Expires>
<Expires>2021-02-19T12:40:12Z</Expires>
<ServerTime>2021-02-19T12:59:09Z</ServerTime>
<RequestId>D8F49AE921B2C09A</RequestId>
<HostId>cj/7nXZgV4JINcioCGy3QGdQGQ5vSA5sRLzcLKkAte5+r4lAp2BdHrV70JQqiQX57IV8UUQ1a4s=</HostId>
</Error>

[teadrinker]

Code: Select all

localPath := "C:\Users\Documents\TempDrivers.zip"

FF_Driver_Version := "v0.29.0"
url := "https://github.com/mozilla/geckodriver/releases/download/" FF_Driver_Version "/geckodriver-" FF_Driver_Version "-win64.zip"

XmlHttp := ComObjCreate("MSXML2.ServerXMLHTTP.6.0")
XmlHttp.Open("GET", url, false)
XmlHttp.Send()

Arr := XmlHttp.responseBody
pData := NumGet(ComObjValue(Arr) + 8 + A_PtrSize)
len := Arr.MaxIndex() + 1

FileOpen(localPath, "w").RawWrite(pData + 0, len)

[euras]

It works on my home PC, but on company PC I still get an error on XmlHttp.Send() part and the script fails to download the file :/

[malcev]

Code: Select all

url := "https://github.com/mozilla/geckodriver/releases/download/v0.29.0/geckodriver-v0.29.0-win64.zip"
req := ComObjCreate("Msxml2.XMLHTTP")
req.open("GET", "http://expandurl.com/api/v1/?url=" url)
req.send()
while req.readyState != 4
   sleep 100
msgbox % req.ResponseText

[garry]

@malcev I tried this ( all scripts worked for me )

Code: Select all

;  from malcev expandurlxxx
localPath := a_desktop . "\TempDrivers.zip"
url1      := "https://github.com/mozilla/geckodriver/releases/download/v0.29.0/geckodriver-v0.29.0-win64.zip"
;----------------
;req := ComObjCreate("Msxml2.XMLHTTP")
req := ComObjCreate("MSXML2.ServerXMLHTTP.6.0")                
req.open("GET", "http://expandurl.com/api/v1/?url=" url1)
req.send()
while req.readyState != 4
   sleep 100
url2:=req.ResponseText
msgbox, url2=%url2%`n-------------------`n
;req.Open("GET", url1, false)   ;- works also        with  MSXML2.ServerXMLHTTP.6.0
 req.Open("GET", url2, false)   ;- expandurl also OK with  Msxml2.XMLHTTP
req.Send()
Arr := req.responseBody
pData := NumGet(ComObjValue(Arr) + 8 + A_PtrSize)
len := Arr.MaxIndex() + 1
FileOpen(localPath, "w").RawWrite(pData + 0, len)
msgbox,DONE
run,%localpath%
exitapp
;====================================================

[euras]

@malcev I tried this ( all scripts worked for me )

The script works now guys, but I have another problem. Trying to download Chrome or Edge webdrivers, it takes 90-100 seconds to pass the line Req.Send(). Is it a way to make it faster? It happens if you connect the first time. If you delete the file after download and try to run it second time, it takes 1-2 seconds to pass the line Req.Send(). But first time it's really slow.

Code: Select all

Driver_Version := "88.0.705.68"
url := "https://msedgedriver.azureedge.net/" Driver_Version "/edgedriver_win64.zip"
localPath := "C:\Users\Documents\TempDrivers.zip"
StartTime := A_TickCount
    Req := ComObjCreate("Msxml2.XMLHTTP.6.0")
    Req.Open("GET", url, false)
    Req.Send()
    while Req.readyState != 4
        sleep 100
    ElapsedTime := A_TickCount - StartTime
    MsgBox,  % "Total waiting time: " ElapsedTime / 1000
    Arr := Req.responseBody
    pData := NumGet(ComObjValue(Arr) + 8 + A_PtrSize)
    len := Arr.MaxIndex() + 1
    FileOpen(localPath, "w").RawWrite(pData + 0, len)
    Msgbox done

[teadrinker]

Can't reproduce the issue. For me it takes about 6 seconds for the first time. Remove

Code: Select all

    while Req.readyState != 4
        sleep 100

It's unnecessary in this case (Req.Open("GET", url, false)).

[euras]

Can't reproduce the issue. For me it takes about 6 seconds for the first time. Remove

Code: Select all

    while Req.readyState != 4
        sleep 100

It's unnecessary in this case (Req.Open("GET", url, false)).

This line does not affect the speed. I went to the microsoft webdrivers storage and tried to download the new webdriver manually from there. And it takes the same 90-100 seconds to complete the download. It stops at 99% after a couple of seconds and then you wait the rest of time until you get 100%. Interesting part is that if you try to download the same webdriver seconds time, it takes just a couple of seconds to complete the downloading part...

https://msedgewebdriverstorage.z22.web.core.windows.net/?prefix=77.0.188.0/