 |
AutoHotkey Community Let's help each other out
|
| View previous topic :: View next topic |
| Author |
Message |
twhyman
Joined: 07 Dec 2005 Posts: 339
|
Posted: Wed Feb 10, 2010 6:07 pm Post subject: [AHK_L] HTTPQuery not working |
|
|
[Moderator's note: Topic split from AutoHotkey_L.]
Hi,
It seems that HTTPQuery does not work with AHK_L.
HTTPQuery
HTTPQuery is one of my most used functions in AHK, Is that fixable?
Here a sample script i included the HTTPQuery function inside the script to make it easier to run.
| Code: |
; exmpl.searchAHKforum.httpQuery.ahk
; Searches the forum for a given Phrase: in this case httpQuery
#noenv
;#include %A_ScriptDir%\httpQuery.ahk
html := ""
URL := "http://google.co.il"
length := httpQuery(html,URL)
varSetCapacity(html,-1)
Gui,Add,Edit,w600 +Wrap r25, %html%
Gui,Show
Return
GuiClose:
GuiEscape:
ExitApp
;### HTTP QUERY ###
; httpQuery-0-3-5.ahk
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{ ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
; updates Aug, 28 2008
; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
; handling this might be set using the httpQueryDwFlags variable as global
global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
, httpQueryDwFlags
; Get any missing default Values
defaultOps =
(LTrim Join|
httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
)
Loop,Parse,defaultOps,|
{
RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
if StrLen(%httpOption%)=0
%httpOption% := httpDefault
}
; Load Library
hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")
; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
offset_name_length:= "4-lpszScheme-255|16-lpszHostName-1024|28-lpszUserName-1024|"
. "36-lpszPassword-1024|44-lpszUrlPath-1024|52-lpszExtrainfo-1024"
VarSetCapacity(URL_COMPONENTS,60,0)
; Struc Size ; Scheme Size ; Max Port Number
NumPut(60,URL_COMPONENTS,0), NumPut(255,URL_COMPONENTS,12), NumPut(0xffff,URL_COMPONENTS,24)
Loop,Parse,offset_name_length,|
{
RegExMatch(A_LoopField,"(?P<Offset>\d+)-(?P<Name>[a-zA-Z]+)-(?P<Size>\d+)",iCU_)
VarSetCapacity(%iCU_Name%,iCU_Size,0)
NumPut(&%iCU_Name%,URL_COMPONENTS,iCU_Offset)
NumPut(iCU_Size,URL_COMPONENTS,iCU_Offset+4)
}
; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
DllCall("WinINet\InternetCrackUrlA","Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"uInt",&URL_COMPONENTS)
; Update variables to retrieve results
Loop,Parse,offset_name_length,|
{
RegExMatch(A_LoopField,"-(?P<Name>[a-zA-Z]+)-",iCU_)
VarSetCapacity(%iCU_Name%,-1)
}
nPort:=NumGet(URL_COMPONENTS,24,"uInt")
; Import any set dwFlags
dwFlags := httpQueryDwFlags
; For some reasons using a selfsigned https certificates doesnt work
; such as an own webmin service - even though every security is turned off
; https with valid certificates works when
if (lpszScheme = "https")
dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)
; Check for Header and drop exception if unknown or invalid URL
if (lpszScheme="unknown") {
Result := "ERR: No Valid URL supplied."
Return StrLen(Result)
}
; Initialise httpQuery's use of the WinINet functions.
; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
hInternet := DllCall("WinINet\InternetOpenA"
,"Str",httpAgent,"UInt"
,(httpProxy != 0 ? INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)
; Open HTTP session for the given URL
; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
hConnect := DllCall("WinINet\InternetConnectA"
,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
,"Str",lpszUserName, "Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
,"uInt",0,"uInt*",0)
; Do we POST? If so, check for header handling and set default
if (Strlen(POSTDATA)>0) {
HTTPVerb:="POST"
if StrLen(Headers)=0
Headers:="Content-Type: application/x-www-form-urlencoded"
} else ; otherwise mode must be GET - no header defaults needed
HTTPVerb:="GET"
; Form the request with proper HTTP protocol version and create the request handle
; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
hRequest := DllCall("WinINet\HttpOpenRequestA"
,"uInt",hConnect,"Str",HTTPVerb,"Str",lpszUrlPath . lpszExtrainfo
,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
,"uInt",dwFlags,"uInt",Context:=0 )
; Send the specified request to the server
; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
sRequest := DllCall("WinINet\HttpSendRequestA"
, "uInt",hRequest,"Str",Headers, "uInt",Strlen(Headers)
, "Str",POSTData,"uInt",Strlen(POSTData))
VarSetCapacity(header, 2048, 0) ; max 2K header data for httpResponseHeader
VarSetCapacity(header_len, 4, 0)
; Check for returned server response-header (works only _after_ request been sent)
; http://msdn.microsoft.com/en-us/library/aa384238.aspx
Loop, 5
if ((headerRequest:=DllCall("WinINet\HttpQueryInfoA","uint",hRequest
,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
break
If (headerRequest=1) {
VarSetCapacity(res,headerLength:=NumGet(header_len),32)
DllCall("RtlMoveMemory","uInt",&res,"uInt",&header,"uInt",headerLength)
Loop,% headerLength
if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
NumPut(Asc("`n"),res,a_index-1,"uChar")
VarSetCapacity(res,-1)
} else
res := "timeout"
; Get 1st Line of Full Response
Loop,Parse,res,`n,`r
{
RetValue := A_LoopField
break
}
; No Connection established - drop exception
If (RetValue="timeout") {
html := "Error: timeout"
return -1
}
; Strip protocol version from return value
RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
. "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
. " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
. "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
. "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
. "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
. "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
. "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
. "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
. "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
. "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
. "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
. " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
. "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
. "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
. "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
. "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
. "t Extended (RFC 2774)"
; Gather numeric response value
RetValue := SubStr(RetValue,1,3)
; Parse through return codes and set according informations
Loop,Parse,HttpRetCodes,|
{
HttpReturnCode := SubStr(A_LoopField,1,3) ; Numeric return value see above
HttpReturnMsg := SubStr(A_LoopField,5) ; link for additional information
if (RetValue=HttpReturnCode) {
RetMsg := HttpReturnMsg
break
}
}
; Global HttpQueryOps handling
if strlen(HTTPQueryOps)>0 {
; Show full Header response (usefull for debugging)
if (instr(HTTPQueryOps,"showHeader"))
MsgBox % res
; Save the full Header response in a global Variable
if (instr(HTTPQueryOps,"storeHeader"))
global HttpQueryHeader := res
; Check for size updates to export to a global Var
if (instr(HTTPQueryOps,"updateSize")) {
Loop,Parse,res,`n
If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
global HttpQueryFullSize := fullSize
break
}
if (fullSize+0=0)
HttpQueryFullSize := "size unavailable"
}
}
; Check for valid codes and drop exception if suspicious
if !(InStr("100 200 201 202 302",RetValue)) {
Result := RetValue " " RetMsg
return StrLen(Result)
}
VarSetCapacity(BytesRead,4,0)
fsize := 0
Loop ; the receiver loop - rewritten in the need to enable
{ ; support for larger file downloads
bc := A_Index
VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
ReadFile := DllCall("wininet\InternetReadFile"
,"uInt",hRequest,"uInt",&buffer%bc%,"uInt",1024,"uInt",&BytesRead)
ReadBytes := NumGet(BytesRead) ; how many bytes were received?
If ((ReadFile!=0)&&(!ReadBytes)) ; we have had no error yet and received no more bytes
break ; we must be done! so lets break the receiver loop
Else {
fsize += ReadBytes ; sum up all chunk sizes for correct return size
sizeArray .= ReadBytes "|"
}
if (instr(HTTPQueryOps,"updateSize"))
Global HttpQueryCurrentSize := fsize
}
sizeArray := SubStr(sizeArray,1,-1) ; trim last PipeChar
VarSetCapacity(result,fSize+1,0) ; reconstruct the result from above generated chunkblocks
Dest := &result ; to a our ByRef result variable
Loop,Parse,SizeArray,|
DllCall("RtlMoveMemory","uInt",Dest,"uInt",&buffer%A_Index%,"uInt",A_LoopField)
, Dest += A_LoopField
DllCall("WinINet\InternetCloseHandle", "uInt", hRequest) ; close all opened
DllCall("WinINet\InternetCloseHandle", "uInt", hInternet)
DllCall("WinINet\InternetCloseHandle", "uInt", hConnect)
DllCall("FreeLibrary", "UInt", hModule) ; unload the library
return fSize ; return the size - strings need update via VarSetCapacity(res,-1)
}
|
Thanks,
Twhyman _________________ (\__/) This is Bunny.
(='.'=) Cut, copy, and paste bunny onto your sig.
(")_(") Help Bunny gain World Domination.
|
|
| Back to top |
|
 |
Lexikos
Joined: 17 Oct 2006 Posts: 7293 Location: Australia
|
Posted: Thu Feb 11, 2010 4:01 am Post subject: |
|
|
Since the script deals with strings at a low level (via DllCall, NumGet, etc.), it must be updated to support Unicode strings. These issues are not unique to AutoHotkey_L. Note that the AutoHotkeyU documentation also applies to AutoHotkey_L revision 42 and later, with the exceptions listed. In particular,
| Quote: | This doesn't work in the Unicode version:
| Code: | DllCall("kernel32\DeleteFileA", "UInt", &filename)
DllCall("kernel32\DeleteFileA", "str", filename) |
Source: AutoHotkeyU - AutoHotkey |
(Read to the end before making any changes.)
First, remove the "A" suffix from each Dll function, so AutoHotkey_L will append "W" or "A" as appropriate. Note however that official AutoHotkey won't append "A" to functions which aren't located in User32.dll, Kernel32.dll, ComCtl32.dll, or Gdi32.dll. An alternative would be to do something like this:
| Code: | API_Suffix := A_IsUnicode ? "W" : "A"
...
DllCall("WinINet\InternetCrackUrl" API_Suffix,"Str",lpszUrl, ... )
| Unfortunately there's more to it.
This means that when VarSetCapacity is called to size a buffer which will hold a string, the capacity must be multiplied by 2 in Unicode build: length in characters * size of character = length in bytes. One way of supporting both builds is:
| Code: | | VarSetCapacity(%iCU_Name%,iCU_Size*(A_IsUnicode ? 2:1),0) | Alternatively, the following is sufficient for both builds but "wasteful" in ANSI builds:
| Code: | VarSetCapacity(%iCU_Name%,iCU_Size*2,0)
| Additionally, whenever NumPut or NumGet is used to manipulate a string, character offsets must also be multiplied by the size of a character, and the type must be "Char" or "UShort" as appropriate. Since this is required a few times in httpQuery(), I'd recommend defining the following near the top of the function:
| Code: | tchar := A_IsUnicode ? "UShort" : "Char"
tchar_size := A_IsUnicode ? 2 : 1
| Then, instead of the two methods demonstrated above, I would recommend the following changes:
| Code: | - VarSetCapacity(%iCU_Name%,iCU_Size,0)
+ VarSetCapacity(%iCU_Name%,iCU_Size*tchar_size,0)
|
| Code: | - Loop,% headerLength
- if (*(&res-1+a_index)=0)
- NumPut(Asc("`n"),res,a_index-1,"uChar")
+ Loop,% headerLength/tchar_size
+ if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
+ NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
| Notes:
- *(address) always reads a single byte. Since each character in a Unicode build is 2 bytes, we need to use NumGet.
- headerLength is always in bytes, even though the headers appear to be returned in Unicode (presumably because "HttpQueryInfoW" was used).
- There are more efficient ways to do this than to loop through the characters, but I won't go into that here.
We're still not done. Since httpQuery() returns the file data and html files are usually in 8-bit encodings, we won't be able to understand the output. Near the end of the function (perhaps immediately before InternetCloseHandle is called), put something like this:
| Code: | ;------------------------------------------------------------------------
; AutoHotkey_L: Convert text files into our native string format.
; First, check the headers for hints about how the file is encoded:
is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
; Now check for a byte order mark - it probably won't be present,
; but if it is we don't want it to be present in the returned string:
dest := &result
if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
dest += 3, charset := "utf-8"
else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
dest += 2, charset := "utf-16"
; else: no recognized BOM, just use charset as set by RegExMatch.
if charset in utf-8,utf-16
result := StrGet(dest, charset)
; Since it doesn't seem practical to support every possible charset name,
; simply treat all unrecognized names as this system's ANSI code page.
else if is_text && A_IsUnicode
result := StrGet(dest, "cp0")
; else:
; Content-Type indicates it's not text; or
; this is an ANSI build so the above StrGet call is unnecessary.
;------------------------------------------------------------------------
| Note: Above use of StrGet requires L46, which I've just released. It would also be feasible to write a script version, for compatibility with official AutoHotkey.
Note that if you'll only ever run the script on a Unicode build, some of this can be simplified. I won't go into that.
Finally, if you have AutoHotkey_L and COM_L, you can simply use the following:
| Code: | WebRequest := COM_CreateObject("WinHttp.WinHttpRequest.5.1")
WebRequest.Open("GET", "http://google.co.il")
WebRequest.Send()
html := WebRequest.ResponseText
WebRequest := ""
Gui,Add,Edit,w600 +Wrap r25, %html%
Gui,Show
Return
GuiClose:
GuiEscape:
ExitApp
| ...or in any build of AutoHotkey with the appropriate version of COM:
| Code: | COM_Init()
WebRequest := COM_CreateObject("WinHttp.WinHttpRequest.5.1")
COM_Invoke(WebRequest, "Open", "GET", "http://google.co.il")
COM_Invoke(WebRequest, "Send")
html := COM_Invoke(WebRequest, "ResponseText")
COM_Release(WebRequest)
Gui,Add,Edit,w600 +Wrap r25, %html%
Gui,Show
Return
GuiClose:
GuiEscape:
ExitApp
|
|
|
| Back to top |
|
 |
twhyman
Joined: 07 Dec 2005 Posts: 339
|
Posted: Thu Feb 11, 2010 6:07 am Post subject: |
|
|
Hi,
Thanks for the detailed answer, I made all the changes you said and had a problem with strget, Then I noticed you moved the thread and updtaed the AHK_L to R46
So problem is solved.
Can you have a look at this script to see why I get error 500 when running with AHK_L, This script is a POC for XML webservice interaction.
| Code: |
;#Include %A_ScriptDir%\xpath.ahk
;#include %A_ScriptDir%\httpQuery_Unicode_Support.ahk
#noenv
html := ""
URL := "http://www.w3schools.com/webservices/tempconvert.asmx?op=FahrenheitToCelsius"
POSTData := "<?xml version=""1.0"" encoding=""utf-8""?><soap12:Envelope xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xmlns:xsd=""http://www.w3.org/2001/XMLSchema"" xmlns:soap12=""http://www.w3.org/2003/05/soap-envelope""><soap12:Body><FahrenheitToCelsius xmlns=""http://tempuri.org/""><Fahrenheit>90</Fahrenheit></FahrenheitToCelsius></soap12:Body></soap12:Envelope>"
length := httpQuery(html,URL,POSTdata,"Content-Type: text/xml; charset=utf-8")
varSetCapacity(html,-1)
Gui,Add,Edit,w600 +Wrap r25,% html
Gui,Show
XPath_load(X_File, html) ; load an XML document
RESULT:=xpath(X_File,"/soap:Envelope/soap:Body/FahrenheitToCelsiusResponse/FahrenheitToCelsiusResult/text()") ; Get Converion name from the xml
msgbox, %RESULT%
Return
GuiClose:
GuiEscape:
ExitApp
;### Functions ###
;### HTTP QUERY With Unicode Support ###
; httpQuery-0-3-5.ahk
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{ ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
; updates Aug, 28 2008
; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
; handling this might be set using the httpQueryDwFlags variable as global
global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
, httpQueryDwFlags
;### Added for unicode support
API_Suffix := A_IsUnicode ? "W" : "A"
tchar := A_IsUnicode ? "UShort" : "Char"
tchar_size := A_IsUnicode ? 2 : 1
; Get any missing default Values
defaultOps =
(LTrim Join|
httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
)
Loop,Parse,defaultOps,|
{
RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
if StrLen(%httpOption%)=0
%httpOption% := httpDefault
}
; Load Library
hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")
; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
offset_name_length:= "4-lpszScheme-255|16-lpszHostName-1024|28-lpszUserName-1024|"
. "36-lpszPassword-1024|44-lpszUrlPath-1024|52-lpszExtrainfo-1024"
VarSetCapacity(URL_COMPONENTS,60,0)
; Struc Size ; Scheme Size ; Max Port Number
NumPut(60,URL_COMPONENTS,0), NumPut(255,URL_COMPONENTS,12), NumPut(0xffff,URL_COMPONENTS,24)
Loop,Parse,offset_name_length,|
{
RegExMatch(A_LoopField,"(?P<Offset>\d+)-(?P<Name>[a-zA-Z]+)-(?P<Size>\d+)",iCU_)
;VarSetCapacity(%iCU_Name%,iCU_Size*(A_IsUnicode ? 2:1),0)
VarSetCapacity(%iCU_Name%,iCU_Size*tchar_size,0)
NumPut(&%iCU_Name%,URL_COMPONENTS,iCU_Offset)
NumPut(iCU_Size,URL_COMPONENTS,iCU_Offset+4)
}
; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
DllCall("WinINet\InternetCrackUrl" . API_Suffix,"Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"uInt",&URL_COMPONENTS)
; Update variables to retrieve results
Loop,Parse,offset_name_length,|
{
RegExMatch(A_LoopField,"-(?P<Name>[a-zA-Z]+)-",iCU_)
VarSetCapacity(%iCU_Name%,-1)
}
nPort:=NumGet(URL_COMPONENTS,24,"uInt")
; Import any set dwFlags
dwFlags := httpQueryDwFlags
; For some reasons using a selfsigned https certificates doesnt work
; such as an own webmin service - even though every security is turned off
; https with valid certificates works when
if (lpszScheme = "https")
dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)
; Check for Header and drop exception if unknown or invalid URL
if (lpszScheme="unknown") {
Result := "ERR: No Valid URL supplied."
Return StrLen(Result)
}
; Initialise httpQuery's use of the WinINet functions.
; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
hInternet := DllCall("WinINet\InternetOpen" . API_Suffix
,"Str",httpAgent,"UInt"
,(httpProxy != 0 ? INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)
; Open HTTP session for the given URL
; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
hConnect := DllCall("WinINet\InternetConnect" . API_Suffix
,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
,"Str",lpszUserName, "Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
,"uInt",0,"uInt*",0)
; Do we POST? If so, check for header handling and set default
if (Strlen(POSTDATA)>0) {
HTTPVerb:="POST"
if StrLen(Headers)=0
Headers:="Content-Type: application/x-www-form-urlencoded"
} else ; otherwise mode must be GET - no header defaults needed
HTTPVerb:="GET"
; Form the request with proper HTTP protocol version and create the request handle
; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
hRequest := DllCall("WinINet\HttpOpenRequest" . API_Suffix
,"uInt",hConnect,"Str",HTTPVerb,"Str",lpszUrlPath . lpszExtrainfo
,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
,"uInt",dwFlags,"uInt",Context:=0 )
; Send the specified request to the server
; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
sRequest := DllCall("WinINet\HttpSendRequest" . API_Suffix
, "uInt",hRequest,"Str",Headers, "uInt",Strlen(Headers)
, "Str",POSTData,"uInt",Strlen(POSTData))
VarSetCapacity(header, 2048, 0) ; max 2K header data for httpResponseHeader
VarSetCapacity(header_len, 4, 0)
; Check for returned server response-header (works only _after_ request been sent)
; http://msdn.microsoft.com/en-us/library/aa384238.aspx
Loop, 5
if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"uint",hRequest
,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
break
If (headerRequest=1) {
VarSetCapacity(res,headerLength:=NumGet(header_len),32)
DllCall("RtlMoveMemory","uInt",&res,"uInt",&header,"uInt",headerLength)
;Loop,% headerLength
;if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
;NumPut(Asc("`n"),res,a_index-1,"uChar")
Loop,% headerLength/tchar_size
if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
VarSetCapacity(res,-1)
} else
res := "timeout"
; Get 1st Line of Full Response
Loop,Parse,res,`n,`r
{
RetValue := A_LoopField
break
}
; No Connection established - drop exception
If (RetValue="timeout") {
html := "Error: timeout"
return -1
}
; Strip protocol version from return value
RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
. "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
. " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
. "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
. "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
. "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
. "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
. "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
. "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
. "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
. "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
. "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
. " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
. "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
. "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
. "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
. "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
. "t Extended (RFC 2774)"
; Gather numeric response value
RetValue := SubStr(RetValue,1,3)
; Parse through return codes and set according informations
Loop,Parse,HttpRetCodes,|
{
HttpReturnCode := SubStr(A_LoopField,1,3) ; Numeric return value see above
HttpReturnMsg := SubStr(A_LoopField,5) ; link for additional information
if (RetValue=HttpReturnCode) {
RetMsg := HttpReturnMsg
break
}
}
; Global HttpQueryOps handling
if strlen(HTTPQueryOps)>0 {
; Show full Header response (usefull for debugging)
if (instr(HTTPQueryOps,"showHeader"))
MsgBox % res
; Save the full Header response in a global Variable
if (instr(HTTPQueryOps,"storeHeader"))
global HttpQueryHeader := res
; Check for size updates to export to a global Var
if (instr(HTTPQueryOps,"updateSize")) {
Loop,Parse,res,`n
If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
global HttpQueryFullSize := fullSize
break
}
if (fullSize+0=0)
HttpQueryFullSize := "size unavailable"
}
}
; Check for valid codes and drop exception if suspicious
if !(InStr("100 200 201 202 302",RetValue)) {
Result := RetValue " " RetMsg
return StrLen(Result)
}
VarSetCapacity(BytesRead,4,0)
fsize := 0
Loop ; the receiver loop - rewritten in the need to enable
{ ; support for larger file downloads
bc := A_Index
VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
ReadFile := DllCall("wininet\InternetReadFile"
,"uInt",hRequest,"uInt",&buffer%bc%,"uInt",1024,"uInt",&BytesRead)
ReadBytes := NumGet(BytesRead) ; how many bytes were received?
If ((ReadFile!=0)&&(!ReadBytes)) ; we have had no error yet and received no more bytes
break ; we must be done! so lets break the receiver loop
Else {
fsize += ReadBytes ; sum up all chunk sizes for correct return size
sizeArray .= ReadBytes "|"
}
if (instr(HTTPQueryOps,"updateSize"))
Global HttpQueryCurrentSize := fsize
}
sizeArray := SubStr(sizeArray,1,-1) ; trim last PipeChar
VarSetCapacity(result,fSize+1,0) ; reconstruct the result from above generated chunkblocks
Dest := &result ; to a our ByRef result variable
Loop,Parse,SizeArray,|
DllCall("RtlMoveMemory","uInt",Dest,"uInt",&buffer%A_Index%,"uInt",A_LoopField)
, Dest += A_LoopField
;------------------------------------------------------------------------
; AutoHotkey_L: Convert text files into our native string format.
; First, check the headers for hints about how the file is encoded:
is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
; Now check for a byte order mark - it probably won't be present,
; but if it is we don't want it to be present in the returned string:
dest := &result
if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
dest += 3, charset := "utf-8"
else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
dest += 2, charset := "utf-16"
; else: no recognized BOM, just use charset as set by RegExMatch.
if charset in utf-8,utf-16
result := StrGet(dest, charset)
; Since it doesn't seem practical to support every possible charset name,
; simply treat all unrecognized names as this system's ANSI code page.
else if is_text && A_IsUnicode
result := StrGet(dest, "cp0")
; else:
; Content-Type indicates it's not text; or
; this is an ANSI build so the above StrGet call is unnecessary.
;------------------------------------------------------------------------
DllCall("WinINet\InternetCloseHandle", "uInt", hRequest) ; close all opened
DllCall("WinINet\InternetCloseHandle", "uInt", hInternet)
DllCall("WinINet\InternetCloseHandle", "uInt", hConnect)
DllCall("FreeLibrary", "UInt", hModule) ; unload the library
return fSize ; return the size - strings need update via VarSetCapacity(res,-1)
}
;### XPATH
/*
Title: XPath Quick Reference
License:
- Version 3.13c by Titan <http://www.autohotkey.net/~Titan/#xpath>
- GNU General Public License 3.0 or higher <http://www.gnu.org/licenses/gpl-3.0.txt>
*/
/*
Function: xpath
Selects nodes and attributes from a standard xpath expression.
Parameters:
doc - an xml object returned by xpath_load()
step - a valid XPath 2.0 expression
set - (optional) text content to replace the current selection
Returns:
The XML source of the selection.
If the content was set to be modified the previous value will be returned.
Remarks:
Since multiple return values are seperated with a comma,
any found within the text content will be entity escaped as ,.
To get or set the text content of a node use the text function,
e.g. /root/node/text(); this behaviour is always assumed within predicates.
For performance reasons count() and last() do not update with the creation of new nodes
so you may need to alter the results accordingly.
*/
xpath(ByRef doc, step, set = "") {
static sc, scb = "" ; use EOT (\x04) as delimiter
If step contains %scb% ; i.e. illegal character
Return
sid := scb . &doc . ":" ; should be unique identifier for current XML object
If (InStr(step, "select:") == 1) { ; for quick selection
stsl = 1
StringTrimLeft, step, step, 7
}
If (InStr(step, "/") == 1)
str = 1 ; root selected
Else {
StringGetPos, p, sc, %sid% ; retrieve previous path
If ErrorLevel = 1
t = /
Else StringMid, t, sc, p += StrLen(sid) + 1, InStr(sc, scb, "", p + 2) - p
step = %t%/%step%
}
; normalize path e.g. /root/node/../child/. becomes /root/child:
step := RegExReplace(step, "(?<=^|/)(?:[^\[/@]+/\.{2}|\.)(?:/|$)|^.+(?=//)")
If (str == 1 or stsl == 1) { ; if not relative path and no select:
; remove last node and trailing attributes and functions:
xpr := RegExReplace(step, (str == 1 and stsl != 1 ? "/(?:\w+:)?\w+(?:\[[^\]]*])?" : "")
. "(?:/@.*?|/\w+\(\))*$")
StringReplace, xpr, xpr, [-1], [1], All ; -1 become just 1
StringReplace, xpr, xpr, [+1], [last()], All ; +1 becomes last()
StringGetPos, p, sc, %sid%
If ErrorLevel = 1
sc = %sc%%sid%%xpr%%scb% ; add record or update as necessary:
Else sc := SubStr(sc, 1, p += StrLen(sid)) . xpr . SubStr(sc, InStr(sc, scb, "", p))
}
; for unions call each operand seperately and join results:
If (InStr(step, "|")) {
StringSplit, s, step, |, `n`t `r
Loop, %s0%
res .= xpath(doc, s%A_Index%, set) . ","
Return, SubStr(res, 1, -1)
}
Else If (InStr(step, "//") == 1) { ; for wildcard selectors use regex searching mode:
StringTrimLeft, step, step, 2
re = 1
rew = 1
xp = /(?:(?:\w+:)?\w+)*/
}
NumPut(160, doc, 0, "UChar") ; unmask variable
; resolve xpath components to: absolute node path, attributes and predicates (if any)
Loop, Parse, step, /
{
s = %A_LoopField%
If (InStr(s, "*") == 1) ; regex mode for wildcards
{
re = 1
s := "(?:\w+:)?\w+" . SubStr(s, 2)
}
Else If (InStr(s, "@") == 1) { ; if current step is attribute:
StringTrimLeft, atr, s, 1
Continue
}
StringGetPos, p, s, [ ; look for predicate opening [...]
If ErrorLevel = 0
{
If s contains [+1],[-1] ; for child nodes record creation instructions in a list
{
If A_Index = 2 ; root node creation
{
StringLeft, t, s, p
t = <%t%/:: ></%t%/>
If (InStr(s, "+1")) {
If doc =
doc = .
doc = %doc%%t%
}
Else doc = .%t%%doc%
StringLeft, s, s, InStr(s, "[") - 1
}
Else {
nw = %nw%%s%
Continue
}
}
Else { ; i.e. for conditional predicates
If (InStr(s, "last()")) { ; finding last node
StringLeft, t, s, p
t := "<" . SubStr(xp, 2) . t . "/:: "
If re ; with regex:
{
os = 0
Loop
If (!os := RegExMatch(doc, t, "", 1 + os)) {
t = %A_Index%
Break
}
t--
}
Else { ; otherwise using StringReplace
StringReplace, doc, doc, %t%, %t%, UseErrorLevel
t = %ErrorLevel%
}
If (RegExMatch(s, "i)last\(\)\s*\-\s*(\d+)", a)) ; i.e. [last() - 2]
StringReplace, s, s, %a%, % t - a1
Else StringReplace, s, s, last(), %t%
}
; concat. the predicate to the list against the current absolute path:
ax = %ax%%xp%%s%
StringLeft, s, s, p
}
}
Else If (InStr(s, "()")) { ; if step is function, add to list
fn = %fn%+%s% ; use + prefix to prevent overlapping naming conflicts
Continue
}
; finally, if step is not any of the above, assume it's the name of a child node
xp = %xp%%s%/ ; ... and add to list, forming the absolute path
}
If (xp == "" or xp == "/") ; i.e. error cases
Return
; remove initial root selector (/) as this is how the parser works by default:
StringTrimLeft, xp, xp, 1
StringTrimRight, ax, ax, 1
StringTrimRight, nw, nw, 1
ct = 0 ; counter
os = 0 ; offset for main loop starts at zero
Loop {
; find offset of next element, and its closing tag offset:
If re
os := RegExMatch(doc, "<" . xp . ":: ", "", 1 + os)
, osx := RegExMatch(doc, "</" . xp . ">", rem, os) + StrLen(rem)
Else {
StringGetPos, osx, doc, </%xp%>, , os := InStr(doc, "<" . xp . ":: ", true, 1 + os)
osx += 4 + StrLen(xp)
}
If os = 0 ; stop looping when no more tags are found
Break
; predicate parser:
If ax !=
{
sk = 0
Loop, Parse, ax, ] ; for each predicate
{
; split components to: (1) path, (2) selector, (3) operator, (4) quotation char, (5) operand
If (!RegExMatch(A_LoopField, "/?(.*?)\[(.+?)(?:\s*([<>!=]{1,2})\s*(['""])?(.+)(?(4)\4))?\s*$", a))
Continue
a1 = %a1%/
If re
RegExMatch(rem, "(?<=^</)" . a1, a1)
If a2 is integer ; i.e. match only certain index
{
StringGetPos, t, a1, /, R2
StringMid, t, a1, 1, t + 1
t := InStr(SubStr(doc, 1, os), "<" . t . ":: ", true, 0)
; extract parent node:
StringMid, sub, doc, t, InStr(doc, ">", "", os) - t
xpf := "<" . a1 . ":: "
; get index of current element within parent node:
StringReplace, sub, sub, %xpf%, %xpf%, UseErrorLevel
If a2 != %ErrorLevel%
sk = 1
Continue
}
StringReplace, xp, xp, /, /, UseErrorLevel
t = %ErrorLevel%
StringReplace, a1, a1, /, /, UseErrorLevel
; extract result for deep analysis
If t = %ErrorLevel% ; i.e. /root/node[child='test']
StringMid, sub, doc, os, osx - os
Else StringMid, sub, doc
, t := InStr(SubStr(doc, 1, os), "<" . a1 . ":: ", true, 0)
, InStr(doc, "</" . a1 . ">", true, t) + 1
If a2 = position()
sub = %i%
Else If (InStr(a2, "@") == 1) ; when selector is an attribute:
RegExMatch(SubStr(sub, 1, InStr(sub, ">"))
, a3 == "" ? "\b" . SubStr(a2, 2) . "=([""'])[^\1]*?\1"
: "\b(?<=" . SubStr(a2, 2) . "=([""']))[^\1]+?(?=\1)", sub)
Else ; otherwise child node:
{
If a2 = . ; if selector is current node don't append to path:
a2 = /
Else a2 = %a2%/
StringMid, sub, sub
, t := InStr(sub, ">", "", InStr(sub, "<" . a1 . a2 . ":: ", true) + 1) + 1
, InStr(sub, "</" . a1 . a2 . ">", true) - t
}
; dynamic mini expression evaluator:
sk += !(a3 == "" ? (sub != "")
: a3 == "=" ? sub == a5
: a3 == "!=" ? sub != a5
: a3 == ">" ? sub > a5
: a3 == ">=" ? sub >= a5
: a3 == "<" ? sub < a5
: a3 == "<=" ? sub <= a5)
}
If sk != 0 ; if conditions were not met for this result, skip it
Continue
}
If nw != ; for node creation
{
If re
nwp := SubStr(rem, 3, -1)
Else nwp = %xp%
Loop, Parse, nw, ]
{
StringLeft, nwn, A_LoopField, InStr(A_LoopField, "[") - 1
nwn = %nwn%/
nwt = <%nwp%%nwn%:: ></%nwp%%nwn%>
If (t := InStr(A_LoopField, "-1")
? InStr(doc, ">", "", InStr(doc, "<" . nwp . ":: ", true, os) + 1) + 1
: InStr(doc, "</" . nwp . ">", true, os))
os := t
StringLen, osx, nwt
osx += os
doc := SubStr(doc, 1, os - 1) . nwt . SubStr(doc, os)
nwp = %nwp%%nwn%
}
StringLen, t, nwp
If (InStr(fn, "+text()") and atr == "")
os += t + 5, osx -= t + 3
}
If atr !=
{
; extract attribute offsets, with surrounding declaration if text() is not used:
If (t := RegExMatch(SubStr(doc, os, InStr(doc, ">", "", os) - os), InStr(fn, "+text()")
? "(?<=\b" . atr . "=([""']))[^\1]*?(?=\1)"
: "\b" . atr . "=([""'])[^\1]*?\1", rem))
os += t - 1, osx := os + StrLen(rem)
Else { ; create attribute
os := InStr(doc, ">", "", os + 1)
, doc := SubStr(doc, 1, os - 1) . " " . atr . "=""""" . SubStr(doc, os)
, osx := ++os + StrLen(atr) + 3
If (InStr(fn, "+text()"))
osx := os += StrLen(atr) + 2
}
}
Else If (InStr(fn, "+text()") and nw == "") ; for text content:
os := InStr(doc, ">", "", os) + 1, osx := InStr(doc, re ? rem : "</" . xp . ">", true, os)
If InStr(fn, "+index-of()") ; get index rather than content
sub = %A_Index%
Else StringMid, sub, doc, os, osx - os ; extract result
If (InStr(fn, "+count()")) ; increment counter if count() function is used
ct++
Else res = %res%%sub%, ; ... and concat to list
If (set != "" or InStr(fn, "+remove()")) ; modify or remove...
setb = %setb%%os%.%osx%| ; mark for modification
}
If setb !=
{
If (InStr(set, "node:") == 1) {
set := SubStr(xpath(doc, SubStr(set, 6) . "/rawsrc()"), 2)
StringReplace, set, set, <, <%xp%, All
StringReplace, set, set, <%xp%/, </%xp%, All
NumPut(160, doc, 0, "UChar")
}
StringTrimRight, setb, setb, 1
Loop, Parse, setb, |
{
StringSplit, setp, A_LoopField, .
StringLen, t, xp
If (InStr(fn, "+append()"))
setp2 := setp1 := setp2 - t - 3
Else If (InStr(fn, "+prepend()"))
setp2 := setp1 := InStr(doc, ">", "", setp1) + 2
doc := SubStr(doc, 1, setp1 - 1) . set . SubStr(doc, setp2) ; dissect then insert new value
}
}
If (InStr(fn, "+count()"))
res = %ct%, ; trailing char since SubStr is used below
nsid := scb . &doc . ":" ; update sid as necessary
If nsid != %sid%
StringReplace, sc, sc, %sid%, %nsid%
NumPut(0, doc, 0, "UChar") ; remask variable to prevent external editing
StringTrimRight, res, res, 1
If (InStr(fn, "+rawsrc()")) {
StringTrimLeft, t, xpr, 1
StringReplace, res, res, <%t%/, <, All
StringReplace, res, res, </%t%, <, All
StringReplace, res, res, `,, `,%scb%, All
Return, scb . res
}
; remove trailing comma and absolute paths from result before returning:
Return, RegExReplace(res, "S)(?<=<)(\/)?(?:(\w+)\/)+(?(1)|:: )", "$1$2")
}
/*
Function: xpath_save
Saves an XML document to file or returns the source.
Parameters:
doc - an xml object returned by xpath_load()
src - (optional) a path to a file where the XML document should be saved;
if the file already exists it will be replaced
Returns:
False if there was an error in saving the document, true otherwise.
If the src parameter is left blank the source code of the document is returned instead.
*/
xpath_save(ByRef doc, src = "") {
xml := RegExReplace(SubStr(doc, 2), "S)(?<=<)(\/)?(?:(\w+)\/)+(?(1)|:: )", "$1$2") ; remove metadata
xml := RegExReplace(xml, "<([\w:]+)([^>]*)><\/\1>", "<$1$2 />") ; fuse empty nodes
;xml := RegExReplace(xml, " (?=(?:\w+:)?\w+=['""])") ; remove prepending whitespace on attributes
xml := RegExReplace(xml, "^\s+|\s+$") ; remove start and leading whitespace
If InStr(xml, "<?xml") != 1 ; add processor instruction if there isn't any:
xml = <?xml version="1.0" encoding="iso-8859-1"?>`n%xml%
StringReplace, xml, xml, `r, , All ; normalize linefeeds:
StringReplace, xml, xml, `n, `r`n, All
sp := " "
StringLen, sl, sp
s =
VarSetCapacity(sxml, StrLen(xml) * 1.1)
Loop, Parse, xml, <, `n`t `r
{
If A_LoopField =
Continue
If (sb := InStr(A_LoopField, "/") == 1)
StringTrimRight, s, s, sl
sxml = %sxml%`n%s%<%A_LoopField%
If sb
StringTrimRight, s, s, sl
If (InStr(A_LoopField, "?") != 1 and InStr(A_LoopField, "!") != 1
and !InStr(A_LoopField, "/>"))
s .= sp
}
StringTrimLeft, sxml, sxml, 1
sxml := RegExReplace(sxml, "(\n(?:" . sp . ")*<((?:\w+:)?\w+\b)[^<]+?)\n(?:"
. sp . ")*</\2>", "$1</$2>")
If src = ; if save path not specified return the XML document:
Return, sxml
FileDelete, %src% ; delete existing file
FileAppend, %sxml%, %src% ; create new one
Return, ErrorLevel ; return errors, if any
}
/*
Function: xpath_load
Loads an XML document.
Parameters:
doc - a reference to the loaded XML file as a variable, to be used in other functions
src - (optional) the document to load, this can be a file name or a string,
if omitted the first paramter is used as the source
Returns:
False if there was an error in loading the document, true otherwise.
*/
xpath_load(ByRef doc, src = "") {
If src = ; if source is empty assume the out variable is the one to be loaded
src = %doc%
Else If FileExist(src) ; otherwise read from file (if it exists)
FileRead, src, %src%
If src not contains <,>
Return, false
; combined expressions slightly improve performance:
src := RegExReplace(src, "<((?:\w+:)?\w+\b)([^>]*)\/\s*>", "<$1$2></$1>") ; defuse nodes
, VarSetCapacity(doc, VarSetCapacity(xml, StrLen(src) * 1.5) * 1.1) ; pre-allocate enough space
Loop, Parse, src, < ; for each opening tag:
{
If (A_Index == 2 and InStr(A_LoopField, "?xml") == 1)
Continue
Else If (InStr(A_LoopField, "?") == 1) ; ignore all other processor instructions
xml = %xml%<%A_LoopField%
Else If (InStr(A_LoopField, " |
Desi Guest
|
Posted: Mon Apr 12, 2010 5:22 pm Post subject: I second mister twhyman |
|
|
I second mister twhyman... Please, can you give us a hand?
Thanks |
|
| Back to top |
|
 |
HotKeyIt
Joined: 18 Jun 2008 Posts: 4652 Location: AHK Forum
|
Posted: Mon Apr 12, 2010 8:37 pm Post subject: |
|
|
Above httpQuery() version works for me on another server, possibly the server does not support unicode
Here is a little modified version that should work for any AutoHotkey build. | Code: | httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{ ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
; updates Aug, 28 2008
; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
; handling this might be set using the httpQueryDwFlags variable as global
global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
, httpQueryDwFlags
static StrGet:="StrGet" ;for AHK_L and AHK_H (AutoHotkey.dll)
;### Added for unicode support
API_Suffix := A_IsUnicode ? "W" : "A"
tchar := A_IsUnicode ? "UShort" : "Char"
tchar_size := A_IsUnicode ? 2 : 1
; Get any missing default Values
defaultOps =
(LTrim Join|
httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
)
Loop,Parse,defaultOps,|
{
RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
if StrLen(%httpOption%)=0
%httpOption% := httpDefault
}
; Load Library
hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")
; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
offset_name_length:= "4-lpszScheme-255|16-lpszHostName-1024|28-lpszUserName-1024|"
. "36-lpszPassword-1024|44-lpszUrlPath-1024|52-lpszExtrainfo-1024"
VarSetCapacity(URL_COMPONENTS,60,0)
; Struc Size ; Scheme Size ; Max Port Number
NumPut(60,URL_COMPONENTS,0), NumPut(255,URL_COMPONENTS,12), NumPut(0xffff,URL_COMPONENTS,24)
Loop,Parse,offset_name_length,|
{
RegExMatch(A_LoopField,"(?P<Offset>\d+)-(?P<Name>[a-zA-Z]+)-(?P<Size>\d+)",iCU_)
;VarSetCapacity(%iCU_Name%,iCU_Size*(A_IsUnicode ? 2:1),0)
VarSetCapacity(%iCU_Name%,iCU_Size*tchar_size,0)
NumPut(&%iCU_Name%,URL_COMPONENTS,iCU_Offset)
NumPut(iCU_Size,URL_COMPONENTS,iCU_Offset+4)
}
; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
DllCall("WinINet\InternetCrackUrl" . API_Suffix,"Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"uInt",&URL_COMPONENTS)
; Update variables to retrieve results
Loop,Parse,offset_name_length,|
{
RegExMatch(A_LoopField,"-(?P<Name>[a-zA-Z]+)-",iCU_)
VarSetCapacity(%iCU_Name%,-1)
}
nPort:=NumGet(URL_COMPONENTS,24,"uInt")
; Import any set dwFlags
dwFlags := httpQueryDwFlags
; For some reasons using a selfsigned https certificates doesnt work
; such as an own webmin service - even though every security is turned off
; https with valid certificates works when
if (lpszScheme = "https")
dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)
; Check for Header and drop exception if unknown or invalid URL
if (lpszScheme="unknown") {
Result := "ERR: No Valid URL supplied."
Return StrLen(Result)
}
; Initialise httpQuery's use of the WinINet functions.
; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
hInternet := DllCall("WinINet\InternetOpen" . API_Suffix
,"Str",httpAgent,"UInt"
,(httpProxy != 0 ? INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)
; Open HTTP session for the given URL
; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
hConnect := DllCall("WinINet\InternetConnect" . API_Suffix
,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
,"Str",lpszUserName, "Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
,"uInt",0,"uInt*",0)
; Do we POST? If so, check for header handling and set default
if (Strlen(POSTDATA)>0) {
HTTPVerb:="POST"
if StrLen(Headers)=0
Headers:="Content-Type: application/x-www-form-urlencoded"
} else ; otherwise mode must be GET - no header defaults needed
HTTPVerb:="GET"
; Form the request with proper HTTP protocol version and create the request handle
; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
hRequest := DllCall("WinINet\HttpOpenRequest" . API_Suffix
,"uInt",hConnect,"Str",HTTPVerb,"Str",lpszUrlPath . lpszExtrainfo
,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
,"uInt",dwFlags,"uInt",Context:=0 )
; Send the specified request to the server
; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
sRequest := DllCall("WinINet\HttpSendRequest" . API_Suffix
, "uInt",hRequest,"Str",Headers, "uInt",Strlen(Headers)
, "Str",POSTData,"uInt",Strlen(POSTData))
VarSetCapacity(header, 2048, 0) ; max 2K header data for httpResponseHeader
VarSetCapacity(header_len, 4, 0)
; Check for returned server response-header (works only _after_ request been sent)
; http://msdn.microsoft.com/en-us/library/aa384238.aspx
Loop, 5
if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"uint",hRequest
,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
break
If (headerRequest=1) {
VarSetCapacity(res,headerLength:=NumGet(header_len),32)
DllCall("RtlMoveMemory","uInt",&res,"uInt",&header,"uInt",headerLength)
;Loop,% headerLength
;if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
;NumPut(Asc("`n"),res,a_index-1,"uChar")
Loop,% headerLength/tchar_size
if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
VarSetCapacity(res,-1)
} else
res := "timeout"
; Get 1st Line of Full Response
Loop,Parse,res,`n,`r
{
RetValue := A_LoopField
break
}
; No Connection established - drop exception
If (RetValue="timeout") {
html := "Error: timeout"
return -1
}
; Strip protocol version from return value
RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
. "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
. " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
. "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
. "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
. "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
. "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
. "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
. "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
. "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
. "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
. "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
. " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
. "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
. "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
. "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
. "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
. "t Extended (RFC 2774)"
; Gather numeric response value
RetValue := SubStr(RetValue,1,3)
; Parse through return codes and set according informations
Loop,Parse,HttpRetCodes,|
{
HttpReturnCode := SubStr(A_LoopField,1,3) ; Numeric return value see above
HttpReturnMsg := SubStr(A_LoopField,5) ; link for additional information
if (RetValue=HttpReturnCode) {
RetMsg := HttpReturnMsg
break
}
}
; Global HttpQueryOps handling
if strlen(HTTPQueryOps)>0 {
; Show full Header response (usefull for debugging)
if (instr(HTTPQueryOps,"showHeader"))
MsgBox % res
; Save the full Header response in a global Variable
if (instr(HTTPQueryOps,"storeHeader"))
global HttpQueryHeader := res
; Check for size updates to export to a global Var
if (instr(HTTPQueryOps,"updateSize")) {
Loop,Parse,res,`n
If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
global HttpQueryFullSize := fullSize
break
}
if (fullSize+0=0)
HttpQueryFullSize := "size unavailable"
}
}
; Check for valid codes and drop exception if suspicious
if !(InStr("100 200 201 202 302",RetValue)) {
Result := RetValue " " RetMsg
return StrLen(Result)
}
VarSetCapacity(BytesRead,4,0)
fsize := 0
Loop ; the receiver loop - rewritten in the need to enable
{ ; support for larger file downloads
bc := A_Index
VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
ReadFile := DllCall("wininet\InternetReadFile"
,"uInt",hRequest,"uInt",&buffer%bc%,"uInt",1024,"uInt",&BytesRead)
ReadBytes := NumGet(BytesRead) ; how many bytes were received?
If ((ReadFile!=0)&&(!ReadBytes)) ; we have had no error yet and received no more bytes
break ; we must be done! so lets break the receiver loop
Else {
fsize += ReadBytes ; sum up all chunk sizes for correct return size
sizeArray .= ReadBytes "|"
}
if (instr(HTTPQueryOps,"updateSize"))
Global HttpQueryCurrentSize := fsize
}
sizeArray := SubStr(sizeArray,1,-1) ; trim last PipeChar
VarSetCapacity(result,fSize+1,0) ; reconstruct the result from above generated chunkblocks
Dest := &result ; to a our ByRef result variable
Loop,Parse,SizeArray,|
DllCall("RtlMoveMemory","uInt",Dest,"uInt",&buffer%A_Index%,"uInt",A_LoopField)
, Dest += A_LoopField
if InStr(A_AhkVersion,".L") { ;;for AHK_L and AHK_H (AutoHotkey.dll)
;------------------------------------------------------------------------
; AutoHotkey_L: Convert text files into our native string format.
; First, check the headers for hints about how the file is encoded:
is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
; Now check for a byte order mark - it probably won't be present,
; but if it is we don't want it to be present in the returned string:
dest := &result
if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
dest += 3, charset := "utf-8"
else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
dest += 2, charset := "utf-16"
; else: no recognized BOM, just use charset as set by RegExMatch.
if charset in utf-8,utf-16
result := %StrGet%(dest, charset)
; Since it doesn't seem practical to support every possible charset name,
; simply treat all unrecognized names as this system's ANSI code page.
else if is_text && A_IsUnicode
result := %StrGet%(dest, "cp0")
; else:
; Content-Type indicates it's not text; or
; this is an ANSI build so the above StrGet call is unnecessary.
;------------------------------------------------------------------------
}
DllCall("WinINet\InternetCloseHandle", "uInt", hRequest) ; close all opened
DllCall("WinINet\InternetCloseHandle", "uInt", hInternet)
DllCall("WinINet\InternetCloseHandle", "uInt", hConnect)
DllCall("FreeLibrary", "UInt", hModule) ; unload the library
return fSize ; return the size - strings need update via VarSetCapacity(res,-1)
} |
_________________ AHK_H (2alpha) AHF TT _Struct WatchDir Yaml _Input ObjTree RapidHotkey DynaRun  |
|
| Back to top |
|
 |
sbc
Joined: 25 Aug 2009 Posts: 321
|
Posted: Fri Dec 03, 2010 10:29 am Post subject: |
|
|
| HotKeyIt wrote: | | Here is a little modified version that should work for any AutoHotkey build. | Does it work for the 64bit build? I just tested it but doesn't seem to work. I'm using the AHK_L R61 Unicode 64bit build.  |
|
| Back to top |
|
 |
sinkfaze
Joined: 18 Mar 2008 Posts: 5043 Location: the tunnel(?=light)
|
Posted: Tue Jan 11, 2011 6:26 am Post subject: |
|
|
I similarly decided to test HotkeyIt's mod of httpQuery on my Windows 7 machine without any success, it looks like the common point of failure is here:
| Code: | Loop, 5
if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"uint",hRequest
,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
break |
EDIT: The more appropriate point of failure may be here:
| Code: | hConnect := DllCall("WinINet\InternetConnect" API_Suffix
,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
,"Str",lpszUserName,"Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
,"uInt",0,"uInt*",0) |
_________________ Try Quick Search for Autohotkey or see the tutorial for newbies. |
|
| Back to top |
|
 |
HotKeyIt
Joined: 18 Jun 2008 Posts: 4652 Location: AHK Forum
|
Posted: Tue Jan 11, 2011 8:50 pm Post subject: |
|
|
I've hopefully fixed it now
requires AHK_L + AHK_H and Struct()
| Code: | httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{ ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
; updates Aug, 28 2008
; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
; handling this might be set using the httpQueryDwFlags variable as global
global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
, httpQueryDwFlags
;### Added for unicode support
static API_Suffix := A_IsUnicode ? "W" : "A"
,tchar := A_IsUnicode ? "UShort" : "UChar"
,tchar_size := A_IsUnicode ? 2 : 1
,URL_COMPONENTS:="DWORD dwStructSize,LPWSTR lpszScheme,DWORD dwSchemeLength,int nScheme,LPWSTR lpszHostName,DWORD dwHostNameLength,LONG nPort,LPWSTR lpszUserName,DWORD dwUserNameLength,LPWSTR lpszPassword,DWORD dwPasswordLength,LPWSTR lpszUrlPath,DWORD dwUrlPathLength,LPWSTR lpszExtraInfo,DWORD dwExtraInfoLength"
,STRBUFFERS:= "Scheme|SchemeLength|HostName|HostNameLength|UserName|UserNameLength|Password|PasswordLength|UrlPath|UrlPathLength|Extrainfo|ExtraInfoLength"
; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
UC:=Struct(URL_COMPONENTS)
UC.dwStructSize:=Struct(UC)
UC.dwSchemeLength:=255
UC.nPort:=0xffff
Loop,Parse,STRBUFFERS,|
If Mod(A_Index,2)
VarSetCapacity(_%A_Index%,tchar_size*1024),UC["lpsz" A_LoopField]:=&_%A_Index%
else
UC["dw" A_LoopField]:=tchar_size*1024
; Get any missing default Values
defaultOps =
(LTrim Join|
httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
)
Loop,Parse,defaultOps,|
{
RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
if StrLen(%httpOption%)=0
%httpOption% := httpDefault
}
; Load Library
hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")
; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
DllCall("WinINet\InternetCrackUrl" . API_Suffix,"Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"UPTR",UC[""])
; Import any set dwFlags
dwFlags := httpQueryDwFlags
; For some reasons using a selfsigned https certificates doesnt work
; such as an own webmin service - even though every security is turned off
; https with valid certificates works when
if (StrGet(UC.lpszScheme) = "https")
dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)
; Check for Header and drop exception if unknown or invalid URL
if (StrGet(UC.lpszScheme)="unknown") {
Result := "ERR: No Valid URL supplied."
Return StrLen(Result)
}
; Initialise httpQuery's use of the WinINet functions.
; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
hInternet := DllCall("WinINet\InternetOpen" . API_Suffix
,"Str",httpAgent,"UInt"
,(httpProxy != 0 ? INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)
; Open HTTP session for the given URL
; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
hConnect := DllCall("WinINet\InternetConnect" . API_Suffix
,"UPTR",hInternet,"UPTR",UC.lpszHostname, "Int",NC.nPort
,"UPTR",StrGet(UC.lpszUserName)?UC.lpszUserName:0
, "UPTR",StrGet(UC.lpszPassword)?UC.lpszPassword:0,"uInt",INTERNET_SERVICE_HTTP
,"uInt",0,"uInt*",0)
; Do we POST? If so, check for header handling and set default
if (Strlen(POSTDATA)>0) {
HTTPVerb:="POST"
if StrLen(Headers)=0
Headers:="Content-Type: application/x-www-form-urlencoded"
} else ; otherwise mode must be GET - no header defaults needed
HTTPVerb:="GET"
; Form the request with proper HTTP protocol version and create the request handle
; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
hRequest := DllCall("WinINet\HttpOpenRequest" . API_Suffix
,"UPTR",hConnect,"Str",HTTPVerb,"Str",StrGet(UC.lpszUrlPath) . StrGet(UC.lpszExtrainfo)
,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
,"uInt",dwFlags,"uInt",Context:=0 )
; Send the specified request to the server
; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
sRequest := DllCall("WinINet\HttpSendRequest" . API_Suffix
, "UPTR",hRequest,"Str",Headers, "uInt",Strlen(Headers)
, "Str",POSTData,"uInt",Strlen(POSTData))
VarSetCapacity(header, 2048, 0) ; max 2K header data for httpResponseHeader
VarSetCapacity(header_len, 4, 0)
; Check for returned server response-header (works only _after_ request been sent)
; http://msdn.microsoft.com/en-us/library/aa384238.aspx
Loop, 5
if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"UPTR",hRequest
,"uint",21,"UPTR",&header,"UPTR",&header_len,"uint",0))=1)
break
If (headerRequest=1) {
VarSetCapacity(res,headerLength:=NumGet(header_len),32)
t:=DllCall("RtlMoveMemory","UPTR",&res,"UPTR",&header,"uInt",headerLength)
;Loop,% headerLength
;if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
;NumPut(Asc("`n"),res,a_index-1,"uChar")
Loop,% headerLength/tchar_size
if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
VarSetCapacity(res,-1)
} else
res := "timeout"
; Get 1st Line of Full Response
Loop,Parse,res,`n,`r
{
RetValue := A_LoopField
break
}
; No Connection established - drop exception
If (RetValue="timeout") {
html := "Error: timeout"
return -1
}
; Strip protocol version from return value
RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
. "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
. " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
. "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
. "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
. "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
. "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
. "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
. "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
. "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
. "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
. "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
. " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
. "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
. "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
. "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
. "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
. "t Extended (RFC 2774)"
; Gather numeric response value
RetValue := SubStr(RetValue,1,3)
; Parse through return codes and set according informations
Loop,Parse,HttpRetCodes,|
{
HttpReturnCode := SubStr(A_LoopField,1,3) ; Numeric return value see above
HttpReturnMsg := SubStr(A_LoopField,5) ; link for additional information
if (RetValue=HttpReturnCode) {
RetMsg := HttpReturnMsg
break
}
}
; Global HttpQueryOps handling
if strlen(HTTPQueryOps)>0 {
; Show full Header response (usefull for debugging)
if (instr(HTTPQueryOps,"showHeader"))
MsgBox % res
; Save the full Header response in a global Variable
if (instr(HTTPQueryOps,"storeHeader"))
global HttpQueryHeader := res
; Check for size updates to export to a global Var
if (instr(HTTPQueryOps,"updateSize")) {
Loop,Parse,res,`n
If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
global HttpQueryFullSize := fullSize
break
}
if (fullSize+0=0)
HttpQueryFullSize := "size unavailable"
}
}
; Check for valid codes and drop exception if suspicious
if !(InStr("100 200 201 202 302",RetValue)) {
Result := RetValue " " RetMsg
return StrLen(Result)
}
VarSetCapacity(BytesRead,4,0)
fsize := 0
Loop ; the receiver loop - rewritten in the need to enable
{ ; support for larger file downloads
bc := A_Index
VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
ReadFile := DllCall("wininet\InternetReadFile"
,"UPTR",hRequest,"UPTR",&buffer%bc%,"uInt",1024,"UPTR",&BytesRead)
ReadBytes := NumGet(BytesRead) ; how many bytes were received?
If ((ReadFile!=0)&&(!ReadBytes)) ; we have had no error yet and received no more bytes
break ; we must be done! so lets break the receiver loop
Else {
fsize += ReadBytes ; sum up all chunk sizes for correct return size
sizeArray .= ReadBytes "|"
}
if (instr(HTTPQueryOps,"updateSize"))
Global HttpQueryCurrentSize := fsize
}
sizeArray := SubStr(sizeArray,1,-1) ; trim last PipeChar
VarSetCapacity(result,fSize+1,0) ; reconstruct the result from above generated chunkblocks
Dest := &result ; to a our ByRef result variable
Loop,Parse,SizeArray,|
DllCall("RtlMoveMemory","UPTR",Dest,"UPTR",&buffer%A_Index%,"uInt",A_LoopField)
, Dest += A_LoopField
;------------------------------------------------------------------------
; AutoHotkey_L: Convert text files into our native string format.
; First, check the headers for hints about how the file is encoded:
is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
; Now check for a byte order mark - it probably won't be present,
; but if it is we don't want it to be present in the returned string:
dest := &result
if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
dest += 3, charset := "utf-8"
else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
dest += 2, charset := "utf-16"
; else: no recognized BOM, just use charset as set by RegExMatch.
if charset in utf-8,utf-16
result := StrGet(dest, charset)
; Since it doesn't seem practical to support every possible charset name,
; simply treat all unrecognized names as this system's ANSI code page.
else if is_text && A_IsUnicode
result := StrGet(dest, "cp0")
; else:
; Content-Type indicates it's not text; or
; this is an ANSI build so the above StrGet call is unnecessary.
;------------------------------------------------------------------------
DllCall("WinINet\InternetCloseHandle", "UPTR", hRequest) ; close all opened
DllCall("WinINet\InternetCloseHandle", "UPTR", hInternet)
DllCall("WinINet\InternetCloseHandle", "UPTR", hConnect)
DllCall("FreeLibrary", "UPTR", hModule) ; unload the library
return fSize ; return the size - strings need update via VarSetCapacity(res,-1)
} |
EDIT:
Hopefully fixed support for 64-bit version.
12. Jan 2011 - Another fix for username and password. _________________ AHK_H (2alpha) AHF TT _Struct WatchDir Yaml _Input ObjTree RapidHotkey DynaRun  |
|
| Back to top |
|
 |
Joolee
Joined: 08 Aug 2005 Posts: 18
|
Posted: Tue Feb 08, 2011 8:33 pm Post subject: |
|
|
| HotKeyIt wrote: | I've hopefully fixed it now
requires AHK_L + AHK_H and Struct()
| Code: | httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{ ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
; updates Aug, 28 2008
; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
; handling this might be set using the httpQueryDwFlags variable as global
global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
, httpQueryDwFlags
;### Added for unicode support
static API_Suffix := A_IsUnicode ? "W" : "A"
,tchar := A_IsUnicode ? "UShort" : "UChar"
,tchar_size := A_IsUnicode ? 2 : 1
,URL_COMPONENTS:="DWORD dwStructSize,LPWSTR lpszScheme,DWORD dwSchemeLength,int nScheme,LPWSTR lpszHostName,DWORD dwHostNameLength,LONG nPort,LPWSTR lpszUserName,DWORD dwUserNameLength,LPWSTR lpszPassword,DWORD dwPasswordLength,LPWSTR lpszUrlPath,DWORD dwUrlPathLength,LPWSTR lpszExtraInfo,DWORD dwExtraInfoLength"
,STRBUFFERS:= "Scheme|SchemeLength|HostName|HostNameLength|UserName|UserNameLength|Password|PasswordLength|UrlPath|UrlPathLength|Extrainfo|ExtraInfoLength"
; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
UC:=Struct(URL_COMPONENTS)
UC.dwStructSize:=Struct(UC)
UC.dwSchemeLength:=255
UC.nPort:=0xffff
Loop,Parse,STRBUFFERS,|
If Mod(A_Index,2)
VarSetCapacity(_%A_Index%,tchar_size*1024),UC["lpsz" A_LoopField]:=&_%A_Index%
else
UC["dw" A_LoopField]:=tchar_size*1024
; Get any missing default Values
defaultOps =
(LTrim Join|
httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
)
Loop,Parse,defaultOps,|
{
RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
if StrLen(%httpOption%)=0
%httpOption% := httpDefault
}
; Load Library
hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")
; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
DllCall("WinINet\InternetCrackUrl" . API_Suffix,"Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"UPTR",UC[""])
; Import any set dwFlags
dwFlags := httpQueryDwFlags
; For some reasons using a selfsigned https certificates doesnt work
; such as an own webmin service - even though every security is turned off
; https with valid certificates works when
if (StrGet(UC.lpszScheme) = "https")
dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)
; Check for Header and drop exception if unknown or invalid URL
if (StrGet(UC.lpszScheme)="unknown") {
Result := "ERR: No Valid URL supplied."
Return StrLen(Result)
}
; Initialise httpQuery's use of the WinINet functions.
; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
hInternet := DllCall("WinINet\InternetOpen" . API_Suffix
,"Str",httpAgent,"UInt"
,(httpProxy != 0 ? INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)
; Open HTTP session for the given URL
; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
hConnect := DllCall("WinINet\InternetConnect" . API_Suffix
,"UPTR",hInternet,"UPTR",UC.lpszHostname, "Int",NC.nPort
,"UPTR",StrGet(UC.lpszUserName)?UC.lpszUserName:0
, "UPTR",StrGet(UC.lpszPassword)?UC.lpszPassword:0,"uInt",INTERNET_SERVICE_HTTP
,"uInt",0,"uInt*",0)
; Do we POST? If so, check for header handling and set default
if (Strlen(POSTDATA)>0) {
HTTPVerb:="POST"
if StrLen(Headers)=0
Headers:="Content-Type: application/x-www-form-urlencoded"
} else ; otherwise mode must be GET - no header defaults needed
HTTPVerb:="GET"
; Form the request with proper HTTP protocol version and create the request handle
; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
hRequest := DllCall("WinINet\HttpOpenRequest" . API_Suffix
,"UPTR",hConnect,"Str",HTTPVerb,"Str",StrGet(UC.lpszUrlPath) . StrGet(UC.lpszExtrainfo)
,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
,"uInt",dwFlags,"uInt",Context:=0 )
; Send the specified request to the server
; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
sRequest := DllCall("WinINet\HttpSendRequest" . API_Suffix
, "UPTR",hRequest,"Str",Headers, "uInt",Strlen(Headers)
, "Str",POSTData,"uInt",Strlen(POSTData))
VarSetCapacity(header, 2048, 0) ; max 2K header data for httpResponseHeader
VarSetCapacity(header_len, 4, 0)
; Check for returned server response-header (works only _after_ request been sent)
; http://msdn.microsoft.com/en-us/library/aa384238.aspx
Loop, 5
if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"UPTR",hRequest
,"uint",21,"UPTR",&header,"UPTR",&header_len,"uint",0))=1)
break
If (headerRequest=1) {
VarSetCapacity(res,headerLength:=NumGet(header_len),32)
t:=DllCall("RtlMoveMemory","UPTR",&res,"UPTR",&header,"uInt",headerLength)
;Loop,% headerLength
;if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
;NumPut(Asc("`n"),res,a_index-1,"uChar")
Loop,% headerLength/tchar_size
if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
VarSetCapacity(res,-1)
} else
res := "timeout"
; Get 1st Line of Full Response
Loop,Parse,res,`n,`r
{
RetValue := A_LoopField
break
}
; No Connection established - drop exception
If (RetValue="timeout") {
html := "Error: timeout"
return -1
}
; Strip protocol version from return value
RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
. "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
. " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
. "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
. "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
. "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
. "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
. "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
. "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
. "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
. "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
. "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
. " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
. "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
. "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
. "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
. "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
. "t Extended (RFC 2774)"
; Gather numeric response value
RetValue := SubStr(RetValue,1,3)
; Parse through return codes and set according informations
Loop,Parse,HttpRetCodes,|
{
HttpReturnCode := SubStr(A_LoopField,1,3) ; Numeric return value see above
HttpReturnMsg := SubStr(A_LoopField,5) ; link for additional information
if (RetValue=HttpReturnCode) {
RetMsg := HttpReturnMsg
break
}
}
; Global HttpQueryOps handling
if strlen(HTTPQueryOps)>0 {
; Show full Header response (usefull for debugging)
if (instr(HTTPQueryOps,"showHeader"))
MsgBox % res
; Save the full Header response in a global Variable
if (instr(HTTPQueryOps,"storeHeader"))
global HttpQueryHeader := res
; Check for size updates to export to a global Var
if (instr(HTTPQueryOps,"updateSize")) {
Loop,Parse,res,`n
If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
global HttpQueryFullSize := fullSize
break
}
if (fullSize+0=0)
HttpQueryFullSize := "size unavailable"
}
}
; Check for valid codes and drop exception if suspicious
if !(InStr("100 200 201 202 302",RetValue)) {
Result := RetValue " " RetMsg
return StrLen(Result)
}
VarSetCapacity(BytesRead,4,0)
fsize := 0
Loop ; the receiver loop - rewritten in the need to enable
{ ; support for larger file downloads
bc := A_Index
VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
ReadFile := DllCall("wininet\InternetReadFile"
,"UPTR",hRequest,"UPTR",&buffer%bc%,"uInt",1024,"UPTR",&BytesRead)
ReadBytes := NumGet(BytesRead) ; how many bytes were received?
If ((ReadFile!=0)&&(!ReadBytes)) ; we have had no error yet and received no more bytes
break ; we must be done! so lets break the receiver loop
Else {
fsize += ReadBytes ; sum up all chunk sizes for correct return size
sizeArray .= ReadBytes "|"
}
if (instr(HTTPQueryOps,"updateSize"))
Global HttpQueryCurrentSize := fsize
}
sizeArray := SubStr(sizeArray,1,-1) ; trim last PipeChar
VarSetCapacity(result,fSize+1,0) ; reconstruct the result from above generated chunkblocks
Dest := &result ; to a our ByRef result variable
Loop,Parse,SizeArray,|
DllCall("RtlMoveMemory","UPTR",Dest,"UPTR",&buffer%A_Index%,"uInt",A_LoopField)
, Dest += A_LoopField
;------------------------------------------------------------------------
; AutoHotkey_L: Convert text files into our native string format.
; First, check the headers for hints about how the file is encoded:
is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
; Now check for a byte order mark - it probably won't be present,
; but if it is we don't want it to be present in the returned string:
dest := &result
if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
dest += 3, charset := "utf-8"
else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
dest += 2, charset := "utf-16"
; else: no recognized BOM, just use charset as set by RegExMatch.
if charset in utf-8,utf-16
result := StrGet(dest, charset)
; Since it doesn't seem practical to support every possible charset name,
; simply treat all unrecognized names as this system's ANSI code page.
else if is_text && A_IsUnicode
result := StrGet(dest, "cp0")
; else:
; Content-Type indicates it's not text; or
; this is an ANSI build so the above StrGet call is unnecessary.
;------------------------------------------------------------------------
DllCall("WinINet\InternetCloseHandle", "UPTR", hRequest) ; close all opened
DllCall("WinINet\InternetCloseHandle", "UPTR", hInternet)
DllCall("WinINet\InternetCloseHandle", "UPTR", hConnect)
DllCall("FreeLibrary", "UPTR", hModule) ; unload the library
return fSize ; return the size - strings need update via VarSetCapacity(res,-1)
} |
EDIT:
Hopefully fixed support for 64-bit version.
12. Jan 2011 - Another fix for username and password. | Using Autohotkey_L (32 bit, unicode) v. 1.0.92.02, my app chrashes when uding your function. Don't know whats the problem yet. _________________
Redhead Technology |
|
| Back to top |
|
 |
HotKeyIt
Joined: 18 Jun 2008 Posts: 4652 Location: AHK Forum
|
|
| Back to top |
|
 |
Joolee
Joined: 08 Aug 2005 Posts: 18
|
Posted: Wed Feb 09, 2011 7:33 pm Post subject: |
|
|
I was including the dll in the wrong way, I got it working when including the dll correctly.
Eventually decided to replace the httpQuery function file with
| Code: | httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{
WebRequest := ComObjCreate("WinHttp.WinHttpRequest.5.1")
WebRequest.Open("POST", lpszUrl)
WebRequest.setRequestHeader("Content-Type", "application/x-www-form-urlencoded")
WebRequest.Send(POSTDATA)
Result := WebRequest.ResponseText
WebRequest := ""
} | I'll build in headers support when I'll start using it  _________________
Redhead Technology |
|
| Back to top |
|
 |
computerspazzz
Joined: 25 May 2010 Posts: 22
|
Posted: Wed Jan 25, 2012 1:02 am Post subject: |
|
|
Could this be converted in any way to be able to download a binary file?
I've been searching for a AHK_L compatible way to download files from http servers, need to download both text and binary files, and save them. Prefered to have an ability to monitor the download progress somehow... even if the file is just being written as its downloaded, we can monitor the file size, and the size it should be, and report the progress that way. The most important thing is just downloading the file reliably.
I had written myself a nice patcher program in the past that would download files, but not the httpQuery I was using fails to function under AHK_L.
Thanks! _________________ Computerspazzz The Technowizard
The Wizard Is In! |
|
| Back to top |
|
 |
[VxE]
Joined: 07 Oct 2006 Posts: 3254 Location: Simi Valley, CA
|
|
| Back to top |
|
 |
|
|
You can post new topics in this forum You can reply to topics in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|