100 replies to this topic

[Chris]

This should have been done earlier, but I've finally linked to this topic from the UrlDownloadToFile page.

By the way, at the top of this topic, it says "Not yet fully functional!". It might be good to revise that because it gives the impression that even basic operation is lacking.

Thanks for sharing your work.

[majkinetor]

It can screw up caller script as it will not return AutoTrim correctly

There should be use of A_AutoTrim in the func.

[corrupt]

This should have been done earlier, but I've finally linked to this topic from the UrlDownloadToFile page.

I had hoped that you would build in the functionality instead of linking to it.

[Chris]

Unfortunately, I haven't had the time to develop this or much of anything else lately. However, it's still planned to have it built-in or at least part of the standard library.

[corrupt]

Unfortunately, I haven't had the time to develop this or much of anything else lately.

I didn't mean today . Sorry if my response sounded like that. No rush . The main reason I was thinking that this functionality might get built-in instead of being added to a Standard Library is that it seems like an option could be used to either output the data to a file or to a var since the programming for retrieving the data already exists. In general, output to a var seems more useful in most scripts and avoids the use of temporary files.

[HuBa]

I have a serious problem with this script!

I tried to download a file that contained only 2 bytes.

And this function returned with empty string.

Why is this line so necessary?

StringTrimRight, res, res, 2

It deletes 2 bytes from the end of the file!!!

Another thing:
I think it would be useful to allow the lpszAgent to be supplied with an optional parameter.

[olfen]

I'm too lazy, to delve into this again.

You may want to try the COM alternative instead.

; Requires Windows Vista, Windows XP SP1, or Windows 2000 Professional SP3 and later.
; Requires WinHTTP 5.0 and Internet Explorer 5.01 or later on Windows XP, Windows 2000, and Windows NT 4.0.
; http://msdn2.microsoft.com/en-us/library/aa384106.aspx

#NoEnv
#include CoHelper.ahk ; http://www.autohotkey.com/forum/topic16631.html
CoInitialize()
pWHR := ActiveXObject("WinHttp.WinHttpRequest.5.1") ; CreateObject

SetTimeouts()
SetTimeouts(Resolve=0, Connect=60000, Send=30000, Receive=30000) {
  ; http://msdn2.microsoft.com/en-us/library/aa384061.aspx
  global pWHR
  DllCall(VTable(pWHR, 23), "Uint", pWHR, "Int", Resolve, "Int", Connect, "Int", Send, "Int", Receive)
}

; Use "HEAD" method, if it is not neccesary to retrieve ResponseText
sMethod := "GET", Ansi2Unicode(sMethod, wMethod) 
sUrl := "http://microsoft.com", Ansi2Unicode(sUrl, wUrl)
DllCall(VTable(pWHR, 9), "Uint", pWHR, "Str", wMethod, "Str", wUrl) ; Open

; Set UserAgent
sUserAgent := "A WinHttpRequest Example Program"
;sUserAgent := "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
pUserAgent := SysAllocString(sUserAgent)
DllCall(VTable(pWHR, 20), "Uint", pWHR, "UInt", WinHttpRequestOption_UserAgentString := 0 , "int64", 8, "int64", pUserAgent) ; put_Option

DllCall(VTable(pWHR, 13), "Uint", pWHR) ; Send

DllCall(VTable(pWHR, 12), "Uint", pWHR, "UIntP", pAllResponseHeaders) ; GetAllResponseHeaders
Unicode2Ansi(pAllResponseHeaders, sAllResponseHeaders)
SysFreeString(pAllResponseHeaders)
MsgBox % sAllResponseHeaders

sHeader := "content-length", Ansi2Unicode(sHeader, wHeader)
DllCall(VTable(pWHR, 11), "Uint", pWHR, "Str", wHeader, "UIntP", pValue) ; GetResponseHeader
Unicode2Ansi(pValue, sValue)
SysFreeString(pValue)
MsgBox % sValue

VarSetCapacity(StatusCode, 4, 0)
DllCall(VTable(pWHR, 14), "Uint", pWHR, "UInt", &StatusCode) ; Status
MsgBox % NumGet(StatusCode)

DllCall(VTable(pWHR, 15), "Uint", pWHR, "UIntP", pStatusText) ; StatusText
Unicode2Ansi(pStatusText, sStatusText)
SysFreeString(pStatusText)
MsgBox % sStatusText

If (sMethod != "HEAD") {
DllCall(VTable(pWHR, 16), "Uint", pWHR, "UIntP", pResponseText) ; ResponseText
Unicode2Ansi(pResponseText, sResponseText)
SysFreeString(pResponseText)
MsgBox % sResponseText
}

Release(pWHR)
CoUninitialize()


/* Settings for put_Option method: http://msdn2.microsoft.com/EN-US/library/aa383998.aspx

WinHttpRequestOption_UserAgentString := 0
WinHttpRequestOption_URL := 1
WinHttpRequestOption_URLCodePage := 2
WinHttpRequestOption_EscapePercentInURL := 3
WinHttpRequestOption_SslErrorIgnoreFlags := 4
WinHttpRequestOption_SelectCertificate := 5
WinHttpRequestOption_EnableRedirects := 6
WinHttpRequestOption_UrlEscapeDisable := 7
WinHttpRequestOption_UrlEscapeDisableQuery := 8
WinHttpRequestOption_SecureProtocols := 9
WinHttpRequestOption_EnableTracing := 10
WinHttpRequestOption_RevertImpersonationOverSsl := 11
WinHttpRequestOption_EnableHttpsToHttpRedirects := 12
WinHttpRequestOption_EnablePassportAuthentication := 13
WinHttpRequestOption_MaxAutomaticRedirects := 14
WinHttpRequestOption_MaxResponseHeaderSize := 15
WinHttpRequestOption_MaxResponseDrainSize := 16
WinHttpRequestOption_EnableHttp1_1 := 17
WinHttpRequestOption_EnableCertificateRevocationCheck := 18

Edit: Added GetAllResponseHeaders, GetResponseHeader, Get Status, Get StatusText
Edit: Added method to set the UserAgent (and other options)
Edit: Added SetTimeouts method

[HuBa]

Wonderfull script, thanx!

But it's just a workaround. I still don't know why is it necessary to trim the result value by 2 bytes in the original function.

[olfen]

Sorry, I currently don't want to spend my time on it.

[HuBa]

Sorry, I currently don't want to spend my time on it.

Ok

[Tekl]

Does this solution also block the whole script like UrlDownloadToFile does? It's a pain when network is unreachable that UrlDownloadToFile will block everything even if it runs in a thread.

[olfen]

Does this solution also block the whole script like UrlDownloadToFile does?

Yes, same problem here.

It's a pain when network is unreachable that UrlDownloadToFile will block everything even if it runs in a thread.

I added SetTimeouts(), so you can at least define how long it blocks the script. See MSDN url for details...

[z3lig]

Greetings,

I'm new to programming as well as AHK. I'm finding myself growing more and more obsessed with it as I come up with new ideas that AHK can make real. At any rate, if its not too much trouble, would someone more seasoned please post a functional version of this script? It would be helpful for my understanding if you could make it do the following:

1. open box with edit field to input URL. button to submit and trigger download.

Ultimately, I'm looking to parse web pages for specific content (if you can provide me a quick example of how to do this say with <!-- w -->www.google.com<!-- w --> it would be appreciated, otherwise, with time I'm sure I'll figure it out)
and then write an XML file with the content for an RSS reader to interpret and display.


Thanks in advance. Z

[engunneer]

while I understand why you posted in this thread, you may want to post in the ask for help section in the future, and include a link to this topic to show you did the research.

In any case, I would suggest starting with this thread: <!-- m -->http://www.autohotke...pic.php?t=20397<!-- m -->

and adding a Gui or an InputBox or two.

[Sean]

You may want to try the COM alternative instead.

Here is a demonstration of Asynchronous, i.e., Event/Callback, alternative.

NEED the latest COM Standard Library and DebugView to display the response text.

#Persistent
OnExit, CleanUp

COM_Init()
pwhr    := COM_CreateObject("WinHttp.WinHttpRequest.5.1")
psink   := ConnectIWinHttpRequestEvents(pwhr)

COM_Invoke(pwhr, "Open", "GET", "http://www.autohotkey.com/", "True")
COM_Invoke(pwhr, "Send")
Sleep, 3000
COM_Invoke(pwhr, "Open", "GET", "http://www.autohotkey.com/forum/", "True")
COM_Invoke(pwhr, "Send")
Return

CleanUp:
COM_Unadvise(NumGet(psink+8), NumGet(psink+12))
COM_Release(NumGet(psink+8))
COM_Release(pwhr)
COM_Term()
ExitApp

IWinHttpRequestEvents(this, nStatus = "", pType = "")
{
	Critical
	If	A_EventInfo = 0
		NumPut(this,pType+0)
;	Else If	A_EventInfo = 1
;	Else If	A_EventInfo = 2
	Else If	A_EventInfo = 3
		OutputDebug, % "[START]`t" . nStatus . ": " . COM_Ansi4Unicode(pType) . "`n`n"
;	Else If	A_EventInfo = 4
	Else If	A_EventInfo = 5
		OutputDebug, % COM_Invoke(NumGet(this+4), "ResponseText") . "`n[END]`n"	
	Else If	A_EventInfo = 6
		OutputDebug, % "[ERROR]`t" . nStatus . ": " . COM_Ansi4Unicode(pType) . "`n`n"
	Return	0
}

ConnectIWinHttpRequestEvents(pwhr)
{
	Static	IWinHttpRequestEvents
	If Not	VarSetCapacity(IWinHttpRequestEvents)
	{
		VarSetCapacity(IWinHttpRequestEvents,28,0), nParams=3113213
		Loop,	Parse,	nParams
		NumPut(RegisterCallback("IWinHttpRequestEvents","",A_LoopField,A_Index-1),IWinHttpRequestEvents,4*(A_Index-1))
	}
	pconn:=COM_FindConnectionPoint(pwhr,IID_IWinHttpRequestEvents:="{F97F4E15-B787-4212-80D1-D380CBBF982E}")
	psink:=COM_CoTaskMemAlloc(16), NumPut(pwhr,NumPut(&IWinHttpRequestEvents,psink+0))
	NumPut(COM_Advise(pconn,psink),NumPut(pconn,psink+8))
	Return	psink
}