AutoHotkey Community

It is currently May 27th, 2012, 6:21 am

All times are UTC [ DST ]




Post new topic Reply to topic  [ 82 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
 Post subject: Re: UrlDownloadToVar
PostPosted: February 6th, 2010, 7:59 pm 
Offline
User avatar

Joined: December 21st, 2007, 3:14 pm
Posts: 3826
Location: Louisville KY USA
guest3456 wrote:
where are these unresolved issues documented? is there a post in this thread that i'm missing?
um seriosuly did you actually read the thread?
http://www.autohotkey.com/forum/viewtop ... 911#125911
guest3456 wrote:
where can i read about these differences?
microsoft.com these are both microsoft API's i just googled WinHTTP or WinInet and got plenty of good materials
its best not to play with fire if you dont know weather to use gas or wood to fuel it

_________________
No matter what your oppinion Please join this discussion
Formal request to Polyethene
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: February 7th, 2010, 3:50 am 
thank you sir


Report this post
Top
  
Reply with quote  
PostPosted: June 15th, 2010, 4:57 am 
Offline

Joined: March 27th, 2009, 3:32 am
Posts: 24
Since the CoHelper.ahk is retired, in fact, I did not find it in forum. Then I revised the codes below and wrap it to a function. I also remove the parts to get header and status, only keeping the part to get webpage content.

Hope useful. Thanks!

Code:
;downloaded from http://www.autohotkey.com/forum/viewtopic.php?t=10466&postdays=0&postorder=asc&start=15
;Modified by steadycookie 06/14/2010
;****usage: UrlDownloadToVar(URL, sResponseText)

; Requires Windows Vista, Windows XP SP1, or Windows 2000 Professional SP3 and later.
; Requires WinHTTP 5.0 and Internet Explorer 5.01 or later on Windows XP, Windows 2000, and Windows NT 4.0.
; http://msdn2.microsoft.com/en-us/library/aa384106.aspx

;#NoEnv
;Need COM library: http://www.autohotkey.com/forum/viewtopic.php?t=22923

UrlDownloadToVar(URL, byRef sResponseText)
{   
   COM_CoInitialize()
   pWHR := COM_ActiveXObject("WinHttp.WinHttpRequest.5.1") ; CreateObject
   
   SetTimeouts()
   
   ; Use "HEAD" method, if it is not neccesary to retrieve ResponseText
   sMethod := "GET", COM_Ansi2Unicode(sMethod, wMethod)
   sUrl := URL, COM_Ansi2Unicode(sUrl, wUrl)
      ;msgbox % sUrl
   DllCall(COM_VTable(pWHR, 9), "Uint", pWHR, "Str", wMethod, "Str", wUrl) ; Open
   
   ; Set UserAgent
   sUserAgent := "A WinHttpRequest Example Program"
   ;sUserAgent := "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
   pUserAgent := COM_SysAllocString(sUserAgent)
   DllCall(COM_VTable(pWHR, 20), "Uint", pWHR, "UInt", WinHttpRequestOption_UserAgentString := 0 , "int64", 8, "int64", pUserAgent) ; put_Option
   
   DllCall(COM_VTable(pWHR, 13), "Uint", pWHR) ; Send

   ;get URL webpage      
   If (sMethod != "HEAD") {
   DllCall(COM_VTable(pWHR, 16), "Uint", pWHR, "UIntP", pResponseText) ; ResponseText
   COM_Unicode2Ansi(pResponseText, sResponseText)
   COM_SysFreeString(pResponseText)
   ;MsgBox % sResponseText
   }
   
   COM_Release(pWHR)
   COM_CoUninitialize()
}

SetTimeouts(Resolve=0, Connect=60000, Send=30000, Receive=60000) {
  ; http://msdn2.microsoft.com/en-us/library/aa384061.aspx
  global pWHR
  DllCall(COM_VTable(pWHR, 23), "Uint", pWHR, "Int", Resolve, "Int", Connect, "Int", Send, "Int", Receive)
}


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: June 15th, 2010, 7:41 am 
Offline

Joined: January 12th, 2007, 4:30 am
Posts: 531
Location: Norway
steadycookie: Thanks for sharing. I'm going to wait and see what yoda has to say about this, but if it's better than the default URLDownloadToVar, I again recommend editing the first post in this thread to link to this script.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: June 15th, 2010, 12:46 pm 
Offline
User avatar

Joined: December 21st, 2007, 3:14 pm
Posts: 3826
Location: Louisville KY USA
I know the answer already but do you?
i guess im Yoda?
looks like an un tested function i wonder how the average user retreives sResponseText

_________________
No matter what your oppinion Please join this discussion
Formal request to Polyethene
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: June 15th, 2010, 12:58 pm 
Offline

Joined: January 12th, 2007, 4:30 am
Posts: 531
Location: Norway
tank wrote:
i wonder how the average user retreives sResponseText

sResponseText seems to be the return value of the function i.e. the source code of the webpage.

This seems to work just fine, should it supersede the original function?


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: June 15th, 2010, 1:09 pm 
Offline

Joined: February 12th, 2007, 7:54 am
Posts: 2462
The interface used in the top post was abandoned long-time ago. The equivalent to that for COM.ahk is as follows.
Code:
sURL := "http://www.autohotkey.com/"

COM_Init()
pwhr := COM_CreateObject("WinHttp.WinHttpRequest.5.1")
COM_Invoke(pwhr, "Open", "GET", sURL)
COM_Invoke(pwhr, "Send")
sResponseText := COM_Invoke(pwhr, "ResponseText")
COM_Release(pwhr)
COM_Term()

MsgBox, % sResponseText


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: June 15th, 2010, 1:10 pm 
yet there is no return statement


Report this post
Top
  
Reply with quote  
 Post subject:
PostPosted: June 15th, 2010, 1:20 pm 
Offline

Joined: January 12th, 2007, 4:30 am
Posts: 531
Location: Norway
tanknlid: Wtf? sResponseText is a byref parameter so you would use it like this:

Code:
 
#include com.ahk
UrlDownloadToVar("http://www.google.com", googleSource)
 msgbox % googleSource
Return


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: June 15th, 2010, 1:23 pm 
Offline
User avatar

Joined: December 21st, 2007, 3:14 pm
Posts: 3826
Location: Louisville KY USA
well isnt i special i guess the frack up fairy visited my house this morning i didnt even look at the parameters

_________________
No matter what your oppinion Please join this discussion
Formal request to Polyethene
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: September 20th, 2010, 8:55 pm 
Offline

Joined: October 9th, 2004, 8:55 pm
Posts: 217
Location: Budapest, Hungary
I tried the script in post

http://www.autohotkey.com/forum/viewtop ... 470#166470

and it works fine. Only one little useful thing is missing. Is there a way to a give an id to the async request, so if I start several requests at the same time then I know in the callback which request the current response belongs to?


Edit: Seems like the script cannot handle multiple simultaneous url retrievals at the same time, because only the last one is retrieved if I start more than one at the same time. Would it be hard to add the ability to handle multiple requests?


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 8th, 2010, 4:33 pm 
Offline

Joined: August 25th, 2009, 4:52 pm
Posts: 321
Sean wrote:
Some users may be perplexed how to save the binary data then. I used File.ahk Standard Library for that, however, it's a common practice to use ADODB.Stream object in scripting like:

Code:
sURL :=   "http://www.autohotkey.com/download/AutoHotkeyInstall.exe"
sFile:=   A_Temp "\AutoHotkeyInstall.exe"
bOverWrite := False

COM_Init()
pwhr :=   COM_CreateObject("WinHttp.WinHttpRequest.5.1")
COM_Invoke(pwhr, "Open", "GET", sURL)
COM_Invoke(pwhr, "Send")

If   psfa :=   COM_Invoke(pwhr, "ResponseBody")
{
;   pData:=   NumGet(psfa+12)
;   nSize:=   NumGet(psfa+16)
   pstm :=   COM_CreateObject("ADODB.Stream")
   COM_Invoke(pstm, "Type", 1)   ; adTypeBinary:=1, adTypeText:=2
   COM_Invoke(pstm, "Open")
   COM_Invoke_(pstm, "Write", 0x2011, psfa) ; VT_ARRAY | VT_UI1
   COM_Invoke(pstm, "SaveToFile", sFile, bOverWrite ? 2:1)
   COM_Invoke(pstm, "Close")
   COM_Release(pstm)
   COM_SafeArrayDestroy(psfa)
}

COM_Release(pwhr)
COM_Term()
What would be the best way to save downloaded data into a file directly on the fly? With this method, the script's memory size corresponds to increase the size of the downloading file. Therefore, if a huge file is specified to download, the script memory size becomes huge.

I've tried your UrlDownloadToFile Progress but it uses URLDownloadToFile API and it does not support asynchronous mode so it freezes the script's GUI windows while downloading.

A quick google search gave me a reference to URLOpenPullStream to use asyncronous mode. But I'm not skilled enough to implement it.

To me this COM method is the most preferable since it can also set a timeout. So I'm wondering if it is possible to download data into a file without increasing(keep minimum increase) the script memory size.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 9th, 2010, 1:17 am 
Offline

Joined: February 12th, 2007, 7:54 am
Posts: 2462
You want to download large files with these? These solutions are not suitable to download large files, it will be slow. Anyway, use the asynchronous version of WinHttpRequest posted in this thread, by implementing currently commented out OnResponseDataAvailable (A_EventInfo = 4). This is similar to, but simpler than, URLOpenStream which is less cumbersome (but less controllable) than URLOpenPullStream.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 9th, 2010, 10:37 am 
Offline

Joined: August 25th, 2009, 4:52 pm
Posts: 321
Sean wrote:
You want to download large files with these? These solutions are not suitable to download large files, it will be slow. Anyway, use the asynchronous version of WinHttpRequest posted in this thread, by implementing currently commented out OnResponseDataAvailable (A_EventInfo = 4). This is similar to, but simpler than, URLOpenStream which is less cumbersome (but less controllable) than URLOpenPullStream.
Thanks, I tried it but I don't seem to be able to modify it for the Unicode and 64bit build. :?

Edit: I found something relevant: ResponseStream


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 29th, 2011, 12:21 am 
Offline

Joined: February 19th, 2010, 8:07 pm
Posts: 614
here is the code for Unicode and Ansi builds of AHK_L

Code:
UrlDownloadToVar(URL, ByRef Result, UserAgent = "", Proxy = "", ProxyBypass = "") {
    ; Requires Windows Vista, Windows XP, Windows 2000 Professional, Windows NT Workstation 4.0,
    ; Windows Me, Windows 98, or Windows 95.
    ; Requires Internet Explorer 3.0 or later.
    pFix:=a_isunicode ? "W" : "A"
    hModule := DllCall("LoadLibrary", "Str", "wininet.dll")

    AccessType := Proxy != "" ? 3 : 1
    ;INTERNET_OPEN_TYPE_PRECONFIG                    0   // use registry configuration
    ;INTERNET_OPEN_TYPE_DIRECT                       1   // direct to net
    ;INTERNET_OPEN_TYPE_PROXY                        3   // via named proxy
    ;INTERNET_OPEN_TYPE_PRECONFIG_WITH_NO_AUTOPROXY  4   // prevent using java/script/INS

    io := DllCall("wininet\InternetOpen" . pFix
    , "Str", UserAgent ;lpszAgent
    , "UInt", AccessType
    , "Str", Proxy
    , "Str", ProxyBypass
    , "UInt", 0) ;dwFlags

    iou := DllCall("wininet\InternetOpenUrl" . pFix
    , "UInt", io
    , "Str", url
    , "Str", "" ;lpszHeaders
    , "UInt", 0 ;dwHeadersLength
    , "UInt", 0x80000000 ;dwFlags: INTERNET_FLAG_RELOAD = 0x80000000 // retrieve the original item
    , "UInt", 0) ;dwContext

    If (ErrorLevel != 0 or iou = 0) {
        DllCall("FreeLibrary", "UInt", hModule)
        return 0
    }

    VarSetCapacity(buffer, 10240, 0)
    VarSetCapacity(BytesRead, 4, 0)

    Loop
    {
        ;http://msdn.microsoft.com/library/en-us/wininet/wininet/internetreadfile.asp
        irf := DllCall("wininet\InternetReadFile", "UInt", iou, "UInt", &buffer, "UInt", 10240, "UInt", &BytesRead)
        VarSetCapacity(buffer, -1) ;to update the variable's internally-stored length

        BytesRead_ = 0 ; reset
        Loop, 4  ; Build the integer by adding up its bytes. (From ExtractInteger-function)
            BytesRead_ += *(&BytesRead + A_Index-1) << 8*(A_Index-1) ;Bytes read in this very DllCall

        ; To ensure all data is retrieved, an application must continue to call the
        ; InternetReadFile function until the function returns TRUE and the lpdwNumberOfBytesRead parameter equals zero.
        If (irf = 1 and BytesRead_ = 0)
            break
        Else ; append the buffer's contents
        {
            a_isunicode ? buffer:=StrGet(&buffer, "CP0")
            Result .= SubStr(buffer, 1, BytesRead_ * (a_isunicode ? 2 : 1))
        }

        /* optional: retrieve only a part of the file
        BytesReadTotal += BytesRead_
        If (BytesReadTotal >= 30000) ; only read the first x bytes
        break                      ; (will be a multiple of the buffer size, if the file is not smaller; trim if neccessary)
        */
    }

    DllCall("wininet\InternetCloseHandle",  "UInt", iou)
    DllCall("wininet\InternetCloseHandle",  "UInt", io)
    DllCall("FreeLibrary", "UInt", hModule)
}


Tested on unicode but not on ansi.

_________________
Main Project: AutoHotkey Toolkit
Unless specified otherwise all my code is written for the latest Autohotkey L Unicode version.


Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 82 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next

All times are UTC [ DST ]


Who is online

Users browsing this forum: Bing [Bot], JamixZol, rbrtryn, Stigg and 19 guests


You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group