43 replies to this topic

[SKAN]

Download URLGet.exe 64 KiB
See virusTotal results: <!-- m -->http://goo.gl/0CPAU<!-- m -->

Features ( 0.9.0.0 ):
[*:1zt2us2i] IE based ( urlmon/URLDownloadToCacheFileW ). Tested fine for me in: XP SP3 IE6
[*:1zt2us2i] Small footprint 64 KiB, low processor usage ( but memory around 4 MiB per instance ).
[*:1zt2us2i] No need to specify target filename. Auto-Increments target filename.
[*:1zt2us2i] Automatically decodes content encoding.
[*:1zt2us2i] Progress in TitleBar.
[*:1zt2us2i] IPC via WM.
Credit: Many thanks to Sean for URLDownloadToCacheFile

Author: SKAN ( Suresh Kumar A N ).
License: Public Domain
FileType: 32bit Console
FileVersion: 0.9.00 ( MD5: 6a529f13ca4261cab359a0333b9572dd )

Usage Example:

URLGet "http://google.co.in/"
will download into IE cache and copy the file as google.co.htm into your current folder ( A_WorkingDir ) and returns the fullpath of the copied file.

URLGet "http://google.co.in/" 1
will omit the FileCopy and instead, will return the fullpath to the cache file.

________________________________________________________________________________________

Source code

This utility was written and compiled in Gentee: Open Source Free Programming Language

Source code released: <!-- m -->http://gentee.com/ph...hp?p=9822#p9822<!-- m -->
View the code: <!-- m -->http://dl.dropbox.co... ... 0/URLGet.g<!-- m -->

[SKAN]

AutoHotkey and URLGet 0.9.0.0

URLGet can report the progress status as 'Window Messages' when called in Four Parameter Mode.

The parameters:
[*:3bjenlk3]URL : A valid url wrapped in double quotes
[*:3bjenlk3]NoFileCopy : 0 or 1. 1 will omit file copy. By default, downloaded file is copied from cache into the current folder ( A_WorkingDir )
[*:3bjenlk3]CallerWnd : A handle to Window. ( hWnd of GUI )
[*:3bjenlk3]CallerMsg : OnMessage Num, like 0x7FFF ( refer Application defined Message Identifier )



URLGet will Send a combination of UGM ( URLGet Message ) and redirects of BINDSTATUS callback.
[*:3bjenlk3]When wParam of OnMessage() is greater than 0x400 ( WM_USER ), it is UGM - otherwise, a redirect of BINDSTATUS
[*:3bjenlk3]Use lParam to recieve Numbers. For Text, you will have to retrieve it from the Console Window Titlebar.
[*:3bjenlk3]For each BINDSTATUS Message, OnMessage() should return 0 for URLGet to proceed, or may return 0x80004004 ( E_ABORT ) to cancel the download

[color=darkred]; URLGet Messages:[/color]
UGM_INIT               := 0x401 ; lParam = hwnd of Console
UGM_CACHEFILE          := 0x404 ; use WinGetTitle on Console Window
UGM_TARGETFILE         := 0x405 ; use WinGetTitle on Console Window
UGM_DEINIT             := 0x406 ; lParam = result from URLDownloadToCacheFile
UGM_CANCEL             := 0x410 ; lParam = 1 for acknowledgement

[color=darkred]; BINDSTATUS[/color]

[color=indigo]BINDSTATUS_FINDINGRESOURCE                     :=  1
BINDSTATUS_CONNECTING                          :=  2
BINDSTATUS_REDIRECTING                         :=  3
BINDSTATUS_BEGINDOWNLOADDATA                   :=  4
BINDSTATUS_DOWNLOADINGDATA                     :=  5
BINDSTATUS_ENDDOWNLOADDATA                     :=  6
[/color]BINDSTATUS_BEGINDOWNLOADCOMPONENTS             :=  7
BINDSTATUS_INSTALLINGCOMPONENTS                :=  8
BINDSTATUS_ENDDOWNLOADCOMPONENTS               :=  9
BINDSTATUS_USINGCACHEDCOPY                     := 10
[color=indigo]BINDSTATUS_SENDINGREQUEST                      := 11[/color]
BINDSTATUS_CLASSIDAVAILABLE                    := 12
[color=indigo]BINDSTATUS_MIMETYPEAVAILABLE                   := 13
BINDSTATUS_CACHEFILENAMEAVAILABLE              := 14[/color]
BINDSTATUS_BEGINSYNCOPERATION                  := 15
BINDSTATUS_ENDSYNCOPERATION                    := 16
BINDSTATUS_BEGINUPLOADDATA                     := 17
BINDSTATUS_UPLOADINGDATA                       := 18
BINDSTATUS_ENDUPLOADINGDATA                    := 19
BINDSTATUS_PROTOCOLCLASSID                     := 20
BINDSTATUS_ENCODING                            := 21
BINDSTATUS_VERFIEDMIMETYPEAVAILABLE            := 22
BINDSTATUS_CLASSINSTALLLOCATION                := 23
[color=indigo]BINDSTATUS_DECODING                            := 24[/color]
BINDSTATUS_LOADINGMIMEHANDLER                  := 25
BINDSTATUS_CONTENTDISPOSITIONATTACH            := 26
BINDSTATUS_FILTERREPORTMIMETYPE                := 27
BINDSTATUS_CLSIDCANINSTANTIATE                 := 28
BINDSTATUS_IUNKNOWNAVAILABLE                   := 29
BINDSTATUS_DIRECTBIND                          := 30
BINDSTATUS_RAWMIMETYPE                         := 31
BINDSTATUS_PROXYDETECTING                      := 32
BINDSTATUS_ACCEPTRANGES                        := 33
BINDSTATUS_COOKIE_SENT                         := 34
BINDSTATUS_COMPACT_POLICY_RECEIVED             := 35
BINDSTATUS_COOKIE_SUPPRESSED                   := 36
BINDSTATUS_COOKIE_STATE_UNKNOWN                := 37
[color=indigo]BINDSTATUS_COOKIE_STATE_ACCEPT                 := 38
BINDSTATUS_COOKIE_STATE_REJECT                 := 39
[/color]BINDSTATUS_COOKIE_STATE_PROMPT                 := 40
BINDSTATUS_COOKIE_STATE_LEASH                  := 41
BINDSTATUS_COOKIE_STATE_DOWNGRADE              := 42
BINDSTATUS_POLICY_HREF                         := 43
BINDSTATUS_P3P_HEADER                          := 44
BINDSTATUS_SESSION_COOKIE_RECEIVED             := 45
BINDSTATUS_PERSISTENT_COOKIE_RECEIVED          := 46
BINDSTATUS_SESSION_COOKIES_ALLOWED             := 47
BINDSTATUS_CACHECONTROL                        := 48
BINDSTATUS_CONTENTDISPOSITIONFILENAME          := 49
BINDSTATUS_MIMETEXTPLAINMISMATCH               := 50
BINDSTATUS_PUBLISHERAVAILABLE                  := 51
BINDSTATUS_DISPLAYNAMEAVAILABLE                := 52
BINDSTATUS_SSLUX_NAVBLOCKED                    := 53
BINDSTATUS_SERVER_MIMETYPEAVAILABLE            := 54
BINDSTATUS_SNIFFED_CLASSIDAVAILABLE            := 55
BINDSTATUS_64BIT_PROGRESS                      := 56
BINDSTATUS_LAST = BINDSTATUS_64BIT_PROGRESS    := 57
BINDSTATUS_RESERVED_0                          := 58
BINDSTATUS_RESERVED_1                          := 59
BINDSTATUS_RESERVED_2                          := 60
BINDSTATUS_RESERVED_3                          := 61
BINDSTATUS_RESERVED_4                          := 62
BINDSTATUS_RESERVED_5                          := 63
BINDSTATUS_RESERVED_6                          := 64
BINDSTATUS_RESERVED_7                          := 65
BINDSTATUS_RESERVED_8                          := 66
BINDSTATUS_RESERVED_9                          := 67

Terminating download:
URLGet process can be terminated by ControlSending ^c to the Console Window, however:
When simultaneously running multiple instances of URLGet with FileCopy in effect ( NoFileCopy=0 ), every instance will compete for filename in current folder. Providing for such cases, every copy of URLGet increments* filename to find an available one and immediately creates a zero-byte-file to reserve it for FileCopy, before the download can even commence. Cancelling a download will delete the empty file, whereas, terminating URLGet will leave orphaned ones.

*Filename will be incremented in following pattern: filename.ext, filename.1.ext, filename.2.ext ... filename.100.ext


DEMONSTRATION:

SetWorkingDir %A_ScriptDir%
URL := "http://www.autohotkey.net/~Lexikos/AutoHotkey_L/AutoHotkey_L_Install.exe"

Gui +LastFound +AlwaysOnTop
hGUI := WinExist()

Gui, Font, S10, Tahoma
Gui, Add, Text,    w270 h20 0x200, Initiating..                 ; Static1
Gui, Add, Text,    x+3 w42 hp 0x201,                            ; Static2
Gui, Add, Progress,xm y+3 w315 h10 cF73D00 h8, 1                ; msctls_progress321
Gui, Add, Text,    xm y+3 w42 h25 y+5 c808080 0x200, Bytes:     ; Static3
Gui, Add, Text,    x+2 w80 h25 Right 0x200,                     ; Static4
Gui, Add, Text,    x+2 w42 h25 Center c808080 0x200             ; Static5
Gui, Add, Text,    x+2 w80 h25 Left 0x200                       ; Static6
Gui, Add, Button,  x+5 w60 hp gCancelDownload,       &Cancel    ; Button1
Gui, Show,, URLGet Demo

OnMessage( 0x7FFF, "URLGet_Callback" )
[color=#CC0000]Run URLGet "%URL%" 0 %hGUI% 0x7FFF,, Hide[/color]
Return ;                                                 // end of auto-execute section //


[color=darkred]GuiClose:
GuiEscape:
[/color] SendMessage, 0x7FFF, 0x410, False,, ahk_id %hGUI% ; UGM_CANCEL before ExitApp
 hwndConsole := ErrorLevel
 While DllCall( "IsWindow", UInt,hwndConsole )     ; Instead of WinWaitClose to avoid
   Sleep 100                                       ; 'DetectHiddenWindows, On'
 ExitApp
Return


[color=darkred]CancelDownload[/color]:
 SendMessage, 0x7FFF, 0x410, False,, ahk_id %hGUI%
 IfEqual, A_GuiControl, &Close, ExitApp
Return


[color=darkred]WinGetTitle[/color]( hWnd ) { ; Instead of WinGetTitle command, to avoid 'DetectHiddenWindows, On'
 VarSetcapacity( Title, 4096 ), DllCall( "GetWindowText", UInt,hwnd, Str,Title, Int,4096 )
Return Title
}


[color=darkred]UrlGet_Callback[/color]( wParam, lParam, Msg, hGUI ) {
 static hwndConsole, dataSz, RetVal=0

 If ( wParam = 0x401 ) {     ; UGM_INIT
   hwndConsole := lparam
 }

 If ( wParam = 0x405 ) {     ; UGM_TARGETFILE
   TargetFileName := WinGetTitle( hwndConsole )
   SplitPath, TargetFileName, TargetFileName
   GuiControl,,Static1, %TargetFilename%
 }

 If ( wParam = 0x406 ) {     ; UGM_DEINIT
   hwndConsole := dataSz := Retval := 0
   GuiControl, ,  Button1, &Close
 }

 If ( wParam = 0x410 && DllCall( "IsWindow", UInt,hwndConsole ) ) {   ; UGM_CANCEL
   If ( lParam = False ) {
       RetVal := 0x80004004 ; E_ABORT
       GuiControl,,Static1, Cancelling download..
       GuiControl, Disable, Button1
       Return hwndConsole
   } Else {
       RetVal := 0
       GuiControl,,Static1, Download cancelled!
       GuiControl, Enable,  Button1
       GuiControl, ,  Button1, &Close
   }
 }

 If ( wParam = 5 || wParam = 6 ) ; BINDSTATUS_DOWNLOADINGDATA
   {                             ; BINDSTATUS_ENDDOWNLOADDATA
   GuiControl,, Static4, %lParam%
   If ( dataSz > 0 ) {
     prc := Round( (lParam/dataSz)*100 )
     GuiControl,,Static2,           %prc%`%
     GuiControl,,msctls_progress321, %prc%
   }
 }

 If ( wParam = 1 )  {            ; BINDSTATUS_FINDINGRESOURCE
   Domain := WinGetTitle( hwndConsole )
   GuiControl,,Static1, Finding %Domain%
 }

 If ( wParam = 4 )  {            ; BINDSTATUS_BEGINDOWNLOADDATA
   If ( dataSz := lParam ) {     ; Download data size
   GuiControl,,Static5, out of
   GuiControl,,Static6, %dataSz%
   }
}
Return RetVal
}

[TheGreatSwami Woo]

just wondering could you use this to download a video from youtube if you gave it the correct url ?

[SKAN]

URLGet can download any URL.
In case of large files ( > 10 MiB ), it is better that URLGet does not handle the FileCopy.

Calling it with 2nd parameter as 1 ( eg. URLGet "<url>" 1 ) will download the file to cache and return the fullpath which can be used with FileMove command.

[garry]

interesting
urldownloadtofile not worked in this case (can't see played songs) :

urldownloadtofile,http://109.169.26.78:9246/played.html,test.txt

script with urlget.exe :

;-------- http://www.autohotkey.com/forum/viewtopic.php?p=482932#482932 ---
SetWorkingDir %A_ScriptDir%

URL=http://109.169.26.78:9246/played.html

;-- used this because URLGet.exe makes extension with 3 characters ( html > htm )  
    SplitPath,url, name, dir, ext, name_no_ext, drive
    stringmid,ext,ext,1,3
F1=%name_no_ext%.%ext%
ifexist,%f1%
  filedelete,%f1%
;----------------------------------------------------

Runwait, URLGet.exe "%URL%",,hide

aaa=
FileRead,aaa,%f1%
xx4=Current Song
StringReplace,aaa,aaa,</tr>,$, All
Loop,parse,aaa,$,
      {
      if A_loopfield contains %xx4%
            {
            aa:=RegExReplace( A_loopfield, "<.*?>" )
            stringreplace,aa,aa,Current Song,,all
            stringmid,aa,aa,9,200
            break
            }
      }
msgbox,%aa%
exitapp

script from IsNull , AHK_L
urldownloadToVAR /RAM

;-- MODIFIED =20110909
;-- tested with AHK_L    1.1.03.00
;-- IsNull
;-- http://de.autohotkey.com/forum/post-71924.html#71924
;-- UrlDownloadToVar
;========================================================

F1=http://109.169.26.78:9246/played.html
;F1:=http://130.166.82.14:8006/played.html

httpQuery(aaa,F1)
xx4=Current Song
StringReplace,aaa,aaa,</tr>,$, All
Loop,parse,aaa,$,
      {
      if A_loopfield contains %xx4%
            {
            aa:=RegExReplace( A_loopfield, "<.*?>" )
            stringreplace,aa,aa,Current Song,,all
            stringmid,aa,aa,9,200
            break
            }
      }
msgbox,%aa%
exitapp


httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{
   WebRequest := ComObjCreate("WinHttp.WinHttpRequest.5.1")
   WebRequest.Open("GET", lpszUrl)
   WebRequest.setRequestHeader("Content-Type", "application/x-www-form-urlencoded")
   WebRequest.Send(POSTDATA)
   Result := WebRequest.ResponseText
   WebRequest := ""
}

/*
<tr><td>11:10:18</td><td>Florante Aguilar and Lori Abucayan - Lahat Ng Araw (Silayan) <td><b>Current Song</b></td></tr>
*/

[SKAN]

Dear garry, thanks for testing

script with urlget.exe :

;-------- http://www.autohotkey.com/forum/viewtopic.php?p=482932#482932 ---
SetWorkingDir %A_ScriptDir%

URL=http://109.169.26.78:9246/played.html

    SplitPath,url, name, dir, ext, name_no_ext, drive
    stringmid,ext,ext,1,3
F1=%name_no_ext%.%ext%
ifexist,%f1%
  filedelete,%f1%

Runwait,%comspec% /c URLGet.exe "%URL%",,hide

aaa=
FileRead,aaa,%f1%
xx4=Current Song
StringReplace,aaa,aaa,</tr>,$, All
Loop,parse,aaa,$,
      {
      if A_loopfield contains %xx4%
            {
            aa:=RegExReplace( A_loopfield, "<.*?>" )
            stringreplace,aa,aa,Current Song,,all
            stringmid,aa,aa,9,200
            break
            }
      }
msgbox,%aa%
exitapp

comspec will be required only if you want to redirect stdout to a file, in order to capture the downloaded filename, like in:
Runwait, %COMSPEC% /c URLGet.exe "%URL%" > filename.txt,, Hide
In your example script, we know for sure that downloaded file is played.htm
The following should suffice:

SetWorkingDir %A_ScriptDir%
FileDelete, played.htm
Runwait, URLGet.exe "http://109.169.26.78:9246/played.htm[color=darkred]l[/color]",, Hide

PS:
I am able to reboot my 'beetel 220 BXI ADSL2 Modem' with a desktop shortcut to:
URLGet.exe "http://username:password@192.168.1.1/rebootinfo.cgi"

[garry]

thank you SKAN for comments, script, examples
I removed %comspec%
your suggestion filedelete is enough for this example, but I used often URLGet so I have the correct filename with splitpath / stringmid ( variable F1 )

[SKAN]

The download completed so quickly that the progress bar never left the starting gate and no percentage appears above it.

Usually, we cannot retrieve filesize for webpages and hence progress will not be available.

point out to me where in your code I need to make changes

Replace all &Close with &Done
Add the following line

If ( wParam = 0x406 ) {     ; UGM_DEINIT
   hwndConsole := dataSz := Retval := 0
[color=red]   GuiControl, ,  Button1, &Done[/color]
 }

[polyethene]

How is this advantageous to curl or wget?

[SKAN]

How is this advantageous to curl or wget?

I do not see any advantage.
I wrote this to overcome a few limitations of URLDownloadToFile:

1) As good as a browser download. No 'User-agent' problems with certain sites that requires it.
2) Decodes 'Content Encoded' data ( like the torrent files from torcache )
3) Does not block or hang the calling script, and can be simply killed from taskbar.

As for anybody wanting to distribute it:
1) It is light weight
2) Public Domain

[ahknby]

Very nice work SKAN!

Working perfect here!

btw, win7 x64 IE9

CUmps!

[SKAN]

btw, win7 x64 IE9

Glad to know it. Thanks for the feedback