[solved] UrlDownloadToFile slows down GUI response

Get help with using AutoHotkey and its commands and hotkeys
Azevedo
Posts: 81
Joined: 07 Feb 2014, 11:35

[solved] UrlDownloadToFile slows down GUI response

29 Aug 2015, 14:47

Hi,

I wrote this script to download files from a website.
However, using UrlDownloadToFile will slow down the GUI, which may stop responding.
For example, if UrlDownloadToFile is in progress, dragging the window by the title bar pauses :wtf: the download while the window is being dragged.

Code: Select all

 ; GUI becomes irresponsive 
 ; UrlDownloadToFile, % url, % filepath
 
 ; GUI smooth as butter
RunWait, %A_ScriptDir%\download.vbs "%url%" "%filepath%"
To fix that I wrote a standalone VBScript that solves the problem:

download.vbs:

Code: Select all

	Dim xHttp		: Set xHttp = CreateObject("WinHttp.WinHttpRequest.5.1")
	Dim bitStream	: Set bitStream = CreateObject("Adodb.Stream")
	
	xHttp.Open "GET", WScript.Arguments(0), False
	xHttp.Send
	If xHttp.status = 200 Then
		With bitStream
			.type = 1 'binary
			.Open
			.Write xHttp.responseBody
			.SaveToFile WScript.Arguments(1) , 2
			.Close
		End With
	End If
It works, the GUI doesn't hang anymore as the download is being made in a different proccess (wscript.exe).

Is there a way to download *binary* (not text) files in AHK without hanging the GUI?
Last edited by Azevedo on 30 Aug 2015, 01:39, edited 1 time in total.
gilliduck
Posts: 265
Joined: 06 Oct 2014, 15:01

Re: UrlDownloadToFile slows down GUI response

29 Aug 2015, 14:57

Maybe Run a second script that does the actual UrlDownloadToFile instead of it being in the main script? Depending on the amount of data being downloaded it can seriously impede the progress of the script. If it's running in an unrelated script, then it won't bother your main GUI. Don't #Include it, otherwise it's the same as putting it in the main script, just Run it.
Azevedo
Posts: 81
Joined: 07 Feb 2014, 11:35

Re: UrlDownloadToFile slows down GUI response

29 Aug 2015, 15:10

I thought of that but downloading with vbscript (WSH) is way lighter than calling AutoHotkeyU64.exe (900Kb) again which will load a thousand things into memory before actually downloading the file.

Imagine a cycle of 100 files. poor cpu..
gilliduck
Posts: 265
Joined: 06 Oct 2014, 15:01

Re: UrlDownloadToFile slows down GUI response

29 Aug 2015, 19:27

How many UrlDownloadToFile are you doing? Specifically how many unique downloads are you doing (are any being repeated and get you the same data as the first time it was run)?
Azevedo
Posts: 81
Joined: 07 Feb 2014, 11:35

Re: UrlDownloadToFile slows down GUI response

29 Aug 2015, 19:34

It is a queue loop. may be 10 or 200.

Code: Select all

For index, image in this.images {

			RegExMatch(image, "imgur\.com\/(.*)", current) ; current file name
			current := current1
			filepath := this.destination . "\" . current
			
			if ( inStr(this.skip,  current) == 0 ) { ; file was not downloaded yet
				
				try {
					;UrlDownloadToFile, % image, % filepath ; GUI becomes irresponsive 
					RunWait, %A_ScriptDir%\download.vbs "%image%" "%filepath%"
				} catch e {
					MsgBox, 16, , % "Error downloading file `n" . image
				}
				
				if ( FileExist(filepath) != "A") {
					this.iFail++
					MsgBox, 16, , % "Error downloading file `n" . image
				} else {
					this.iDone++
				}
				
			} else {  ; file already exists
				this.iDone++
			}
			
			if this.paused 
				return
		}
lexikos
Posts: 6207
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: UrlDownloadToFile slows down GUI response

29 Aug 2015, 20:58

There are two examples in the help file for HTTP requests which do not make the script unresponsive. To get binary data, you can use ResponseBody instead of ResponseText.
Azevedo wrote:I thought of that but downloading with vbscript (WSH) is way lighter than calling AutoHotkeyU64.exe (900Kb) again
How do you figure? If file size is relevant (doubtful), consider that wscript.exe+vbscript.dll = 735KB, and who knows what else vbscript.dll loads.

Either way it's irrelevant if a single child process does all the downloading - you only need to initialize once. For instance, you can write the list of files to a script and have it download everything, or you can have the script read filenames from stdin one at a time.

Also, I think your assumption that VBScript will load faster is false. My benchmarks with a single-line script showed that RunWait test.vbs and Run test.ahk performed exactly the same (for 100 iterations). If you were to rewrite your vbs script in ahk, it would probably perform much the same. VBScript may or may not be more optimized, but it's likely that any difference in language performance will be greatly outweighed by network latency/download time and the overhead of running a new process.
Azevedo
Posts: 81
Joined: 07 Feb 2014, 11:35

Re: UrlDownloadToFile slows down GUI response

29 Aug 2015, 22:03

lexikos wrote:There are two examples in the help file for HTTP requests which do not make the script unresponsive. To get binary data, you can use ResponseBody instead of ResponseText.
I've seen all these examples. None of them show how to save a binary stream to a file (like I did in vbscript in the 1st post). Do you know how to do that in AHK?
So I am a dumb to come here and ask a question that is in the help file? Where is in the help file "save a binary stream to a file using HTTPRequest"? Please don't imply that I'm lazy and I didn't looked in the help file.

PS: If AHK performs better than VBS/WSH in your computer then good for you! I'm not the one to prove contrary.
tmplinshi
Posts: 1304
Joined: 01 Oct 2013, 14:57

Re: UrlDownloadToFile slows down GUI response

29 Aug 2015, 23:09

Code: Select all

WinHttp_DownloadToFile("http://ahkscript.org/static/ahk_logo.png", "logo.png")

WinHttp_DownloadToFile(Url, FileName) {
	whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
	whr.Open("GET", Url, True)
	whr.Send()
	If whr.WaitForResponse()
	{
		ado := ComObjCreate("ADODB.Stream")
		ado.Type := 1 ; adTypeBinary
		ado.Open
		ado.Write(whr.ResponseBody)
		ado.SaveToFile(FileName, 2)
		ado.Close
	}
}
Azevedo
Posts: 81
Joined: 07 Feb 2014, 11:35

Re: UrlDownloadToFile slows down GUI response

30 Aug 2015, 01:25

Thanks tmplinshi!
Worked perfectly.
And the GUI didn't froze.
just me
Posts: 5824
Joined: 02 Oct 2013, 08:51
Location: Germany

Re: [solved] UrlDownloadToFile slows down GUI response

30 Aug 2015, 08:30

Seemingly there's no need to use ADO in this case:

Code: Select all

WinHttp_DownloadToFileBin(Url, FileName) {
   WHR := ComObjCreate("WinHttp.WinHttpRequest.5.1")
   WHR.Open("GET", Url, True)
   WHR.Send()
   WHR.WaitForResponse()
   ResponseLength := WHR.ResponseBody.MaxIndex()
   FileLength := 0
   VarSetCapacity(Index, 4, 0)
   If !DllCall("OleAut32.dll\SafeArrayPtrOfIndex", "Ptr", ComObjValue(WHR.ResponseBody), "Ptr", &Index, "PtrP", PtrOfIndex) {
      hFile := FileOpen(FileName, "w")
      hFile.RawWrite(PtrOfIndex + 0, ResponseLength)
      FileLength := hFile.Pos
      hFile.Close()
   }
   Return FileLength
}
Azevedo
Posts: 81
Joined: 07 Feb 2014, 11:35

Re: [solved] UrlDownloadToFile slows down GUI response

30 Aug 2015, 10:43

hmm interesting. I'll try that too!
Danke
tmplinshi
Posts: 1304
Joined: 01 Oct 2013, 14:57

Re: [solved] UrlDownloadToFile slows down GUI response

30 Aug 2015, 10:56

just me wrote:Seemingly there's no need to use ADO in this case:

Code: Select all

WinHttp_DownloadToFileBin(Url, FileName) {
   WHR := ComObjCreate("WinHttp.WinHttpRequest.5.1")
   WHR.Open("GET", Url, True)
   WHR.Send()
   WHR.WaitForResponse()
   ResponseLength := WHR.ResponseBody.MaxIndex()
   FileLength := 0
   VarSetCapacity(Index, 4, 0)
   If !DllCall("OleAut32.dll\SafeArrayPtrOfIndex", "Ptr", ComObjValue(WHR.ResponseBody), "Ptr", &Index, "PtrP", PtrOfIndex) {
      hFile := FileOpen(FileName, "w")
      hFile.RawWrite(PtrOfIndex + 0, ResponseLength)
      FileLength := hFile.Pos
      hFile.Close()
   }
   Return FileLength
}
nice :)
lexikos
Posts: 6207
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: UrlDownloadToFile slows down GUI response

30 Aug 2015, 23:06

Azevedo wrote:I've seen all these examples.
You didn't give any indication that you had, so I assumed you had not. Why would I tell you something I think you already know?
None of them show how to save a binary stream to a file (like I did in vbscript in the 1st post). Do you know how to do that in AHK?
Yes, using ResponseBody as I indicated. ResponseBody is an array of bytes, so you can do whatever you want with the data, including writing to a file. In fact, you already had code to deal with ResponseBody in your first post - it would just need minimal translation for use in AutoHotkey.
PS: If AHK performs better than VBS/WSH in your computer then good for you!
My implication was the opposite, but that it was irrelevant. It was based on assumptions and not real benchmarks.
Azevedo
Posts: 81
Joined: 07 Feb 2014, 11:35

Re: [solved] UrlDownloadToFile slows down GUI response

30 Aug 2015, 23:49

Ok. It was a misunderstanding. My bad.
Thanks buddy!
User avatar
Taurus
Posts: 51
Joined: 20 Jan 2015, 10:31

Re: [solved] UrlDownloadToFile slows down GUI response

31 May 2016, 10:46

Hi, thats a great script. I am interested, why UrlDownloadToFile is making the script unresponseable. Where is the difference to WinHttpRequest? (heavier load?)

At the moment i am developing an app, which downloads a 50 MB file. So i am interested, which has the best performance?

UrlDownloadToFile vs WinHttpRequest?

Does WinHttpRequest: ResponseBody + ADODB.Stream vs ResponseStream + omObjQuery(STREAM, "{0000000c-0000-0000-C000-000000000046}") make a difference?

Thanks for your answers/ideas!
Dev for a better world :) > PHP for Web > AHK H for Local > with KISS (Keep it Short and Simple) on Win 10 Pro (Version 1803) x64
User avatar
jeeswg
Posts: 5738
Joined: 19 Dec 2016, 01:58
Location: UK

Re: [solved] UrlDownloadToFile slows down GUI response

15 Apr 2018, 00:50

I was doing some testing recently and I believe that:
ResponseLength := WHR.ResponseBody.MaxIndex()
should be:
ResponseLength := 1+WHR.ResponseBody.MaxIndex()
because the array is 0-based.

For a 0-byte file, oHTTP.ResponseBody.MaxIndex() returns blank.
For a 1-byte file, oHTTP.ResponseBody.MaxIndex() returns 0.

So, some code might be something like this:

Code: Select all

;vUrl := "https://raw.githubusercontent.com/f5devcentral/f5-irule-editor/master/iRuler/Templates/Blank.txt" ;1 byte
;vUrl := "https://raw.githubusercontent.com/SparkPost/java-sparkpost/master/src/main/resources/empty.txt" ;0 bytes

if IsObject(oHTTP.ResponseBody)
	vSize := 1+oHTTP.ResponseBody.MaxIndex()
else
	vSize := 0
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA

Return to “Ask For Help”

Who is online

Users browsing this forum: DRocks, Google [Bot], swagfag, TAC109, therealbuzzkill and 54 guests