AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

[AHK_L] HTTPQuery not working

 
Reply to topic    AutoHotkey Community Forum Index -> Ask for Help
View previous topic :: View next topic  
Author Message
twhyman



Joined: 07 Dec 2005
Posts: 339

PostPosted: Wed Feb 10, 2010 6:07 pm    Post subject: [AHK_L] HTTPQuery not working Reply with quote

[Moderator's note: Topic split from AutoHotkey_L.]

Hi,

It seems that HTTPQuery does not work with AHK_L.
HTTPQuery

HTTPQuery is one of my most used functions in AHK, Is that fixable?


Here a sample script i included the HTTPQuery function inside the script to make it easier to run.
Code:

; exmpl.searchAHKforum.httpQuery.ahk
; Searches the forum for a given Phrase: in this case httpQuery
#noenv
;#include %A_ScriptDir%\httpQuery.ahk

html     := ""
URL      := "http://google.co.il"

length := httpQuery(html,URL)
varSetCapacity(html,-1)


Gui,Add,Edit,w600 +Wrap r25, %html%
Gui,Show
Return

GuiClose:
GuiEscape:
   ExitApp
   
;### HTTP QUERY ###
; httpQuery-0-3-5.ahk
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{   ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
   ; updates Aug, 28 2008   
   ; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
   ; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
   ; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
   ; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
   ; handling this might be set using the httpQueryDwFlags variable as global
   global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
       , httpQueryDwFlags
   ; Get any missing default Values
   defaultOps =
   (LTrim Join|
      httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
      SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
      SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
      INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
   )
   Loop,Parse,defaultOps,|
   {
      RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
      if StrLen(%httpOption%)=0
         %httpOption% := httpDefault
   }

   ; Load Library
   hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")

   ; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
   ; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
   offset_name_length:= "4-lpszScheme-255|16-lpszHostName-1024|28-lpszUserName-1024|"
                  . "36-lpszPassword-1024|44-lpszUrlPath-1024|52-lpszExtrainfo-1024"
   VarSetCapacity(URL_COMPONENTS,60,0)
   ; Struc Size               ; Scheme Size                  ; Max Port Number
   NumPut(60,URL_COMPONENTS,0), NumPut(255,URL_COMPONENTS,12), NumPut(0xffff,URL_COMPONENTS,24)
   
   Loop,Parse,offset_name_length,|
   {
      RegExMatch(A_LoopField,"(?P<Offset>\d+)-(?P<Name>[a-zA-Z]+)-(?P<Size>\d+)",iCU_)
      VarSetCapacity(%iCU_Name%,iCU_Size,0)
      NumPut(&%iCU_Name%,URL_COMPONENTS,iCU_Offset)
      NumPut(iCU_Size,URL_COMPONENTS,iCU_Offset+4)
   }

   ; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
   ; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
   DllCall("WinINet\InternetCrackUrlA","Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"uInt",&URL_COMPONENTS)

   ; Update variables to retrieve results
   Loop,Parse,offset_name_length,|
   {
      RegExMatch(A_LoopField,"-(?P<Name>[a-zA-Z]+)-",iCU_)
      VarSetCapacity(%iCU_Name%,-1)
   }
   nPort:=NumGet(URL_COMPONENTS,24,"uInt")
   
   ; Import any set dwFlags
   dwFlags := httpQueryDwFlags
   ; For some reasons using a selfsigned https certificates doesnt work
   ; such as an own webmin service - even though every security is turned off
   ; https with valid certificates works when
   if (lpszScheme = "https")
      dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
               |SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)

   ; Check for Header and drop exception if unknown or invalid URL
   if (lpszScheme="unknown") {
      Result := "ERR: No Valid URL supplied."
      Return StrLen(Result)
   }

   ; Initialise httpQuery's use of the WinINet functions.
   ; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
   hInternet := DllCall("WinINet\InternetOpenA"
                  ,"Str",httpAgent,"UInt"
                  ,(httpProxy != 0 ?  INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
                  ,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)

   ; Open HTTP session for the given URL
   ; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
   hConnect := DllCall("WinINet\InternetConnectA"
                  ,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
                  ,"Str",lpszUserName, "Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
                  ,"uInt",0,"uInt*",0)

   ; Do we POST? If so, check for header handling and set default
   if (Strlen(POSTDATA)>0) {
      HTTPVerb:="POST"
      if StrLen(Headers)=0
         Headers:="Content-Type: application/x-www-form-urlencoded"
   } else ; otherwise mode must be GET - no header defaults needed
      HTTPVerb:="GET"   

   ; Form the request with proper HTTP protocol version and create the request handle
   ; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
   hRequest := DllCall("WinINet\HttpOpenRequestA"
                  ,"uInt",hConnect,"Str",HTTPVerb,"Str",lpszUrlPath . lpszExtrainfo
                  ,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
                  ,"uInt",dwFlags,"uInt",Context:=0 )

   ; Send the specified request to the server
   ; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
   sRequest := DllCall("WinINet\HttpSendRequestA"
                  , "uInt",hRequest,"Str",Headers, "uInt",Strlen(Headers)
                  , "Str",POSTData,"uInt",Strlen(POSTData))

   VarSetCapacity(header, 2048, 0)  ; max 2K header data for httpResponseHeader
   VarSetCapacity(header_len, 4, 0)
   
   ; Check for returned server response-header (works only _after_ request been sent)
   ; http://msdn.microsoft.com/en-us/library/aa384238.aspx
   Loop, 5
     if ((headerRequest:=DllCall("WinINet\HttpQueryInfoA","uint",hRequest
      ,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
      break

   If (headerRequest=1) {
      VarSetCapacity(res,headerLength:=NumGet(header_len),32)
      DllCall("RtlMoveMemory","uInt",&res,"uInt",&header,"uInt",headerLength)
      Loop,% headerLength
         if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
            NumPut(Asc("`n"),res,a_index-1,"uChar")
      VarSetCapacity(res,-1)
   } else
      res := "timeout"

   ; Get 1st Line of Full Response
   Loop,Parse,res,`n,`r
   {
      RetValue := A_LoopField
      break
   }
   
   ; No Connection established - drop exception
   If (RetValue="timeout") {
      html := "Error: timeout"
      return -1
   }
   ; Strip protocol version from return value
   RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
   
    ; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
   HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
              . "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
              . " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
              . "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
              . "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
              . "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
              . "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
              . "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
              . "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
              . "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
              . "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
              . "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
              . " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
              . "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
              . "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
              . "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
              . "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
              . "t Extended (RFC 2774)"
   
   ; Gather numeric response value
   RetValue := SubStr(RetValue,1,3)
   
   ; Parse through return codes and set according informations
   Loop,Parse,HttpRetCodes,|
   {
      HttpReturnCode := SubStr(A_LoopField,1,3)    ; Numeric return value see above
      HttpReturnMsg  := SubStr(A_LoopField,5)      ; link for additional information
      if (RetValue=HttpReturnCode) {
         RetMsg := HttpReturnMsg
         break
      }
   }

   ; Global HttpQueryOps handling
   if strlen(HTTPQueryOps)>0 {
      ; Show full Header response (usefull for debugging)
      if (instr(HTTPQueryOps,"showHeader"))
         MsgBox % res
      ; Save the full Header response in a global Variable
      if (instr(HTTPQueryOps,"storeHeader"))
         global HttpQueryHeader := res
      ; Check for size updates to export to a global Var
      if (instr(HTTPQueryOps,"updateSize")) {
         Loop,Parse,res,`n
            If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
               global HttpQueryFullSize := fullSize
               break
            }
         if (fullSize+0=0)
            HttpQueryFullSize := "size unavailable"
      }
   }

   ; Check for valid codes and drop exception if suspicious
   if !(InStr("100 200 201 202 302",RetValue)) {
      Result := RetValue " " RetMsg
      return StrLen(Result)
   }

   VarSetCapacity(BytesRead,4,0)
   fsize := 0
   Loop            ; the receiver loop - rewritten in the need to enable
   {               ; support for larger file downloads
      bc := A_Index
      VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
      ReadFile := DllCall("wininet\InternetReadFile"
                  ,"uInt",hRequest,"uInt",&buffer%bc%,"uInt",1024,"uInt",&BytesRead)
      ReadBytes := NumGet(BytesRead)    ; how many bytes were received?
      If ((ReadFile!=0)&&(!ReadBytes))  ; we have had no error yet and received no more bytes
         break                         ; we must be done! so lets break the receiver loop
      Else {
         fsize += ReadBytes            ; sum up all chunk sizes for correct return size
         sizeArray .= ReadBytes "|"
      }
      if (instr(HTTPQueryOps,"updateSize"))
         Global HttpQueryCurrentSize := fsize
   }
   sizeArray := SubStr(sizeArray,1,-1)   ; trim last PipeChar
   
   VarSetCapacity(result,fSize+1,0)      ; reconstruct the result from above generated chunkblocks
   Dest := &result                       ; to a our ByRef result variable
   Loop,Parse,SizeArray,|
      DllCall("RtlMoveMemory","uInt",Dest,"uInt",&buffer%A_Index%,"uInt",A_LoopField)
      , Dest += A_LoopField
     
   DllCall("WinINet\InternetCloseHandle", "uInt", hRequest)   ; close all opened
   DllCall("WinINet\InternetCloseHandle", "uInt", hInternet)
   DllCall("WinINet\InternetCloseHandle", "uInt", hConnect)
   DllCall("FreeLibrary", "UInt", hModule)                    ; unload the library
   return fSize                          ; return the size - strings need update via VarSetCapacity(res,-1)
}


Thanks,
Twhyman
_________________
(\__/) This is Bunny.
(='.'=) Cut, copy, and paste bunny onto your sig.
(")_(") Help Bunny gain World Domination.
Back to top
View user's profile Send private message
Lexikos



Joined: 17 Oct 2006
Posts: 7293
Location: Australia

PostPosted: Thu Feb 11, 2010 4:01 am    Post subject: Reply with quote

Since the script deals with strings at a low level (via DllCall, NumGet, etc.), it must be updated to support Unicode strings. These issues are not unique to AutoHotkey_L. Note that the AutoHotkeyU documentation also applies to AutoHotkey_L revision 42 and later, with the exceptions listed. In particular,
Quote:
This doesn't work in the Unicode version:
Code:
DllCall("kernel32\DeleteFileA", "UInt", &filename)
DllCall("kernel32\DeleteFileA", "str", filename)

Source: AutoHotkeyU - AutoHotkey

(Read to the end before making any changes.)
First, remove the "A" suffix from each Dll function, so AutoHotkey_L will append "W" or "A" as appropriate. Note however that official AutoHotkey won't append "A" to functions which aren't located in User32.dll, Kernel32.dll, ComCtl32.dll, or Gdi32.dll. An alternative would be to do something like this:
Code:
   API_Suffix := A_IsUnicode ? "W" : "A"
   ...
   DllCall("WinINet\InternetCrackUrl" API_Suffix,"Str",lpszUrl, ... )
Unfortunately there's more to it.
Quote:
In addition, buffers to store strings should be enlarged as well.
Source: AutoHotkeyU - AutoHotkey
This means that when VarSetCapacity is called to size a buffer which will hold a string, the capacity must be multiplied by 2 in Unicode build: length in characters * size of character = length in bytes. One way of supporting both builds is:
Code:
VarSetCapacity(%iCU_Name%,iCU_Size*(A_IsUnicode ? 2:1),0)
Alternatively, the following is sufficient for both builds but "wasteful" in ANSI builds:
Code:
VarSetCapacity(%iCU_Name%,iCU_Size*2,0)
Additionally, whenever NumPut or NumGet is used to manipulate a string, character offsets must also be multiplied by the size of a character, and the type must be "Char" or "UShort" as appropriate. Since this is required a few times in httpQuery(), I'd recommend defining the following near the top of the function:
Code:
   tchar := A_IsUnicode ? "UShort" : "Char"
   tchar_size := A_IsUnicode ? 2 : 1
Then, instead of the two methods demonstrated above, I would recommend the following changes:
Code:
-      VarSetCapacity(%iCU_Name%,iCU_Size,0)
+      VarSetCapacity(%iCU_Name%,iCU_Size*tchar_size,0)
Code:
-      Loop,% headerLength
-         if (*(&res-1+a_index)=0)
-            NumPut(Asc("`n"),res,a_index-1,"uChar")
+      Loop,% headerLength/tchar_size
+         if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
+            NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
Notes:
  • *(address) always reads a single byte. Since each character in a Unicode build is 2 bytes, we need to use NumGet.
  • headerLength is always in bytes, even though the headers appear to be returned in Unicode (presumably because "HttpQueryInfoW" was used).
  • There are more efficient ways to do this than to loop through the characters, but I won't go into that here.
We're still not done. Since httpQuery() returns the file data and html files are usually in 8-bit encodings, we won't be able to understand the output. Near the end of the function (perhaps immediately before InternetCloseHandle is called), put something like this:
Code:
   ;------------------------------------------------------------------------
   ; AutoHotkey_L: Convert text files into our native string format.
   ; First, check the headers for hints about how the file is encoded:
   is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
   ; Now check for a byte order mark - it probably won't be present,
   ; but if it is we don't want it to be present in the returned string:
   dest := &result
   if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
      dest += 3, charset := "utf-8"
   else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
      dest += 2, charset := "utf-16"
   ; else: no recognized BOM, just use charset as set by RegExMatch.
   if charset in utf-8,utf-16
      result := StrGet(dest, charset)
   ; Since it doesn't seem practical to support every possible charset name,
   ; simply treat all unrecognized names as this system's ANSI code page.
   else if is_text && A_IsUnicode
      result := StrGet(dest, "cp0")
   ; else:
   ;    Content-Type indicates it's not text; or
   ;    this is an ANSI build so the above StrGet call is unnecessary.
   ;------------------------------------------------------------------------
Note: Above use of StrGet requires L46, which I've just released. It would also be feasible to write a script version, for compatibility with official AutoHotkey.

Note that if you'll only ever run the script on a Unicode build, some of this can be simplified. I won't go into that.


Finally, if you have AutoHotkey_L and COM_L, you can simply use the following:
Code:
WebRequest := COM_CreateObject("WinHttp.WinHttpRequest.5.1")
WebRequest.Open("GET", "http://google.co.il")
WebRequest.Send()
html := WebRequest.ResponseText
WebRequest := ""

Gui,Add,Edit,w600 +Wrap r25, %html%
Gui,Show
Return

GuiClose:
GuiEscape:
   ExitApp
...or in any build of AutoHotkey with the appropriate version of COM:
Code:
COM_Init()
WebRequest := COM_CreateObject("WinHttp.WinHttpRequest.5.1")
COM_Invoke(WebRequest, "Open", "GET", "http://google.co.il")
COM_Invoke(WebRequest, "Send")
html := COM_Invoke(WebRequest, "ResponseText")
COM_Release(WebRequest)

Gui,Add,Edit,w600 +Wrap r25, %html%
Gui,Show
Return

GuiClose:
GuiEscape:
   ExitApp
Back to top
View user's profile Send private message Visit poster's website
twhyman



Joined: 07 Dec 2005
Posts: 339

PostPosted: Thu Feb 11, 2010 6:07 am    Post subject: Reply with quote

Hi,

Thanks for the detailed answer, I made all the changes you said and had a problem with strget, Then I noticed you moved the thread Very Happy and updtaed the AHK_L to R46 Razz

So problem is solved.

Can you have a look at this script to see why I get error 500 when running with AHK_L, This script is a POC for XML webservice interaction.
Code:

;#Include %A_ScriptDir%\xpath.ahk
;#include %A_ScriptDir%\httpQuery_Unicode_Support.ahk


#noenv
html     := ""

URL      := "http://www.w3schools.com/webservices/tempconvert.asmx?op=FahrenheitToCelsius"

POSTData := "<?xml version=""1.0"" encoding=""utf-8""?><soap12:Envelope xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xmlns:xsd=""http://www.w3.org/2001/XMLSchema"" xmlns:soap12=""http://www.w3.org/2003/05/soap-envelope""><soap12:Body><FahrenheitToCelsius xmlns=""http://tempuri.org/""><Fahrenheit>90</Fahrenheit></FahrenheitToCelsius></soap12:Body></soap12:Envelope>"


length := httpQuery(html,URL,POSTdata,"Content-Type: text/xml; charset=utf-8")
varSetCapacity(html,-1)

Gui,Add,Edit,w600 +Wrap r25,% html
Gui,Show

XPath_load(X_File, html) ; load an XML document
 
RESULT:=xpath(X_File,"/soap:Envelope/soap:Body/FahrenheitToCelsiusResponse/FahrenheitToCelsiusResult/text()") ; Get Converion name from the xml
msgbox, %RESULT%
Return

GuiClose:
GuiEscape:
ExitApp
   

;### Functions ###
;### HTTP QUERY With Unicode Support ###
; httpQuery-0-3-5.ahk
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{   ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
   ; updates Aug, 28 2008   
   ; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
   ; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
   ; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
   ; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
   ; handling this might be set using the httpQueryDwFlags variable as global
   global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
       , httpQueryDwFlags
   
   ;### Added for unicode support
   API_Suffix := A_IsUnicode ? "W" : "A"
   tchar := A_IsUnicode ? "UShort" : "Char"
   tchar_size := A_IsUnicode ? 2 : 1
   
   ; Get any missing default Values
   defaultOps =
   (LTrim Join|
      httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
      SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
      SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
      INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
   )
   Loop,Parse,defaultOps,|
   {
      RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
      if StrLen(%httpOption%)=0
         %httpOption% := httpDefault
   }

   ; Load Library
   hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")

   ; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
   ; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
   offset_name_length:= "4-lpszScheme-255|16-lpszHostName-1024|28-lpszUserName-1024|"
                  . "36-lpszPassword-1024|44-lpszUrlPath-1024|52-lpszExtrainfo-1024"
   VarSetCapacity(URL_COMPONENTS,60,0)
   ; Struc Size               ; Scheme Size                  ; Max Port Number
   NumPut(60,URL_COMPONENTS,0), NumPut(255,URL_COMPONENTS,12), NumPut(0xffff,URL_COMPONENTS,24)
   
   Loop,Parse,offset_name_length,|
   {
      RegExMatch(A_LoopField,"(?P<Offset>\d+)-(?P<Name>[a-zA-Z]+)-(?P<Size>\d+)",iCU_)
      ;VarSetCapacity(%iCU_Name%,iCU_Size*(A_IsUnicode ? 2:1),0)
      VarSetCapacity(%iCU_Name%,iCU_Size*tchar_size,0)
      NumPut(&%iCU_Name%,URL_COMPONENTS,iCU_Offset)
      NumPut(iCU_Size,URL_COMPONENTS,iCU_Offset+4)
   }

   ; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
   ; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
   DllCall("WinINet\InternetCrackUrl" . API_Suffix,"Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"uInt",&URL_COMPONENTS)

   ; Update variables to retrieve results
   Loop,Parse,offset_name_length,|
   {
      RegExMatch(A_LoopField,"-(?P<Name>[a-zA-Z]+)-",iCU_)
      VarSetCapacity(%iCU_Name%,-1)
   }
   nPort:=NumGet(URL_COMPONENTS,24,"uInt")
   
   ; Import any set dwFlags
   dwFlags := httpQueryDwFlags
   ; For some reasons using a selfsigned https certificates doesnt work
   ; such as an own webmin service - even though every security is turned off
   ; https with valid certificates works when
   if (lpszScheme = "https")
      dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
               |SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)

   ; Check for Header and drop exception if unknown or invalid URL
   if (lpszScheme="unknown") {
      Result := "ERR: No Valid URL supplied."
      Return StrLen(Result)
   }

   ; Initialise httpQuery's use of the WinINet functions.
   ; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
   hInternet := DllCall("WinINet\InternetOpen" . API_Suffix
                  ,"Str",httpAgent,"UInt"
                  ,(httpProxy != 0 ?  INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
                  ,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)

   ; Open HTTP session for the given URL
   ; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
   hConnect := DllCall("WinINet\InternetConnect" . API_Suffix
                  ,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
                  ,"Str",lpszUserName, "Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
                  ,"uInt",0,"uInt*",0)

   ; Do we POST? If so, check for header handling and set default
   if (Strlen(POSTDATA)>0) {
      HTTPVerb:="POST"
      if StrLen(Headers)=0
         Headers:="Content-Type: application/x-www-form-urlencoded"
   } else ; otherwise mode must be GET - no header defaults needed
      HTTPVerb:="GET"   

   ; Form the request with proper HTTP protocol version and create the request handle
   ; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
   hRequest := DllCall("WinINet\HttpOpenRequest" . API_Suffix
                  ,"uInt",hConnect,"Str",HTTPVerb,"Str",lpszUrlPath . lpszExtrainfo
                  ,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
                  ,"uInt",dwFlags,"uInt",Context:=0 )

   ; Send the specified request to the server
   ; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
   sRequest := DllCall("WinINet\HttpSendRequest" . API_Suffix
                  , "uInt",hRequest,"Str",Headers, "uInt",Strlen(Headers)
                  , "Str",POSTData,"uInt",Strlen(POSTData))

   VarSetCapacity(header, 2048, 0)  ; max 2K header data for httpResponseHeader
   VarSetCapacity(header_len, 4, 0)
   
   ; Check for returned server response-header (works only _after_ request been sent)
   ; http://msdn.microsoft.com/en-us/library/aa384238.aspx
   Loop, 5
     if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"uint",hRequest
      ,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
      break

   If (headerRequest=1) {
      VarSetCapacity(res,headerLength:=NumGet(header_len),32)
      DllCall("RtlMoveMemory","uInt",&res,"uInt",&header,"uInt",headerLength)
      ;Loop,% headerLength
         ;if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
            ;NumPut(Asc("`n"),res,a_index-1,"uChar")
      Loop,% headerLength/tchar_size
      if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
       NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
      VarSetCapacity(res,-1)
   } else
      res := "timeout"

   ; Get 1st Line of Full Response
   Loop,Parse,res,`n,`r
   {
      RetValue := A_LoopField
      break
   }
   
   ; No Connection established - drop exception
   If (RetValue="timeout") {
      html := "Error: timeout"
      return -1
   }
   ; Strip protocol version from return value
   RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
   
    ; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
   HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
              . "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
              . " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
              . "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
              . "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
              . "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
              . "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
              . "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
              . "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
              . "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
              . "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
              . "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
              . " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
              . "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
              . "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
              . "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
              . "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
              . "t Extended (RFC 2774)"
   
   ; Gather numeric response value
   RetValue := SubStr(RetValue,1,3)
   
   ; Parse through return codes and set according informations
   Loop,Parse,HttpRetCodes,|
   {
      HttpReturnCode := SubStr(A_LoopField,1,3)    ; Numeric return value see above
      HttpReturnMsg  := SubStr(A_LoopField,5)      ; link for additional information
      if (RetValue=HttpReturnCode) {
         RetMsg := HttpReturnMsg
         break
      }
   }

   ; Global HttpQueryOps handling
   if strlen(HTTPQueryOps)>0 {
      ; Show full Header response (usefull for debugging)
      if (instr(HTTPQueryOps,"showHeader"))
         MsgBox % res
      ; Save the full Header response in a global Variable
      if (instr(HTTPQueryOps,"storeHeader"))
         global HttpQueryHeader := res
      ; Check for size updates to export to a global Var
      if (instr(HTTPQueryOps,"updateSize")) {
         Loop,Parse,res,`n
            If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
               global HttpQueryFullSize := fullSize
               break
            }
         if (fullSize+0=0)
            HttpQueryFullSize := "size unavailable"
      }
   }

   ; Check for valid codes and drop exception if suspicious
   if !(InStr("100 200 201 202 302",RetValue)) {
      Result := RetValue " " RetMsg
      return StrLen(Result)
   }

   VarSetCapacity(BytesRead,4,0)
   fsize := 0
   Loop            ; the receiver loop - rewritten in the need to enable
   {               ; support for larger file downloads
      bc := A_Index
      VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
      ReadFile := DllCall("wininet\InternetReadFile"
                  ,"uInt",hRequest,"uInt",&buffer%bc%,"uInt",1024,"uInt",&BytesRead)
      ReadBytes := NumGet(BytesRead)    ; how many bytes were received?
      If ((ReadFile!=0)&&(!ReadBytes))  ; we have had no error yet and received no more bytes
         break                         ; we must be done! so lets break the receiver loop
      Else {
         fsize += ReadBytes            ; sum up all chunk sizes for correct return size
         sizeArray .= ReadBytes "|"
      }
      if (instr(HTTPQueryOps,"updateSize"))
         Global HttpQueryCurrentSize := fsize
   }
   sizeArray := SubStr(sizeArray,1,-1)   ; trim last PipeChar
   
   VarSetCapacity(result,fSize+1,0)      ; reconstruct the result from above generated chunkblocks
   Dest := &result                       ; to a our ByRef result variable
   Loop,Parse,SizeArray,|
      DllCall("RtlMoveMemory","uInt",Dest,"uInt",&buffer%A_Index%,"uInt",A_LoopField)
      , Dest += A_LoopField
     
    ;------------------------------------------------------------------------
   ; AutoHotkey_L: Convert text files into our native string format.
   ; First, check the headers for hints about how the file is encoded:
   is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
   ; Now check for a byte order mark - it probably won't be present,
   ; but if it is we don't want it to be present in the returned string:
   dest := &result
   if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
      dest += 3, charset := "utf-8"
   else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
      dest += 2, charset := "utf-16"
   ; else: no recognized BOM, just use charset as set by RegExMatch.
   if charset in utf-8,utf-16
      result := StrGet(dest, charset)
   ; Since it doesn't seem practical to support every possible charset name,
   ; simply treat all unrecognized names as this system's ANSI code page.
   else if is_text && A_IsUnicode
      result := StrGet(dest, "cp0")
   ; else:
   ;    Content-Type indicates it's not text; or
   ;    this is an ANSI build so the above StrGet call is unnecessary.
   ;------------------------------------------------------------------------
   
   
   DllCall("WinINet\InternetCloseHandle", "uInt", hRequest)   ; close all opened
   DllCall("WinINet\InternetCloseHandle", "uInt", hInternet)
   DllCall("WinINet\InternetCloseHandle", "uInt", hConnect)
   DllCall("FreeLibrary", "UInt", hModule)                    ; unload the library
   return fSize                          ; return the size - strings need update via VarSetCapacity(res,-1)
}


;### XPATH
/*
      Title: XPath Quick Reference
      
      License:
         - Version 3.13c by Titan <http://www.autohotkey.net/~Titan/#xpath>
         - GNU General Public License 3.0 or higher <http://www.gnu.org/licenses/gpl-3.0.txt>
*/

/*

      Function: xpath
         Selects nodes and attributes from a standard xpath expression.
      
      Parameters:
         doc - an xml object returned by xpath_load()
         step - a valid XPath 2.0 expression
         set - (optional) text content to replace the current selection
      
      Returns:
         The XML source of the selection.
         If the content was set to be modified the previous value will be returned.
      
      Remarks:
         Since multiple return values are seperated with a comma,
         any found within the text content will be entity escaped as ,.
         To get or set the text content of a node use the text function,
         e.g. /root/node/text(); this behaviour is always assumed within predicates.
         For performance reasons count() and last() do not update with the creation of new nodes
         so you may need to alter the results accordingly.

*/
xpath(ByRef doc, step, set = "") {
   static sc, scb = "" ; use EOT (\x04) as delimiter
   
   If step contains %scb% ; i.e. illegal character
      Return
   sid := scb . &doc . ":" ; should be unique identifier for current XML object
   If (InStr(step, "select:") == 1) { ; for quick selection
      stsl = 1
      StringTrimLeft, step, step, 7
   }
   If (InStr(step, "/") == 1)
      str = 1 ; root selected
   Else {
      StringGetPos, p, sc, %sid% ; retrieve previous path
      If ErrorLevel = 1
         t = /
      Else StringMid, t, sc, p += StrLen(sid) + 1, InStr(sc, scb, "", p + 2) - p
      step = %t%/%step%
   }
   
   ; normalize path e.g. /root/node/../child/. becomes /root/child:
   step := RegExReplace(step, "(?<=^|/)(?:[^\[/@]+/\.{2}|\.)(?:/|$)|^.+(?=//)")
   
   If (str == 1 or stsl == 1) { ; if not relative path and no select:
      ; remove last node and trailing attributes and functions:
      xpr := RegExReplace(step, (str == 1 and stsl != 1 ? "/(?:\w+:)?\w+(?:\[[^\]]*])?" : "")
         . "(?:/@.*?|/\w+\(\))*$")
      StringReplace, xpr, xpr, [-1], [1], All ; -1 become just 1
      StringReplace, xpr, xpr, [+1], [last()], All ; +1 becomes last()
      StringGetPos, p, sc, %sid%
      If ErrorLevel = 1
         sc = %sc%%sid%%xpr%%scb% ; add record or update as necessary:
      Else sc := SubStr(sc, 1, p += StrLen(sid)) . xpr . SubStr(sc, InStr(sc, scb, "", p))
   }
   
   ; for unions call each operand seperately and join results:
   If (InStr(step, "|")) {
      StringSplit, s, step, |, `n`t  `r
      Loop, %s0%
         res .= xpath(doc, s%A_Index%, set) . ","
      Return, SubStr(res, 1, -1)
   }
   Else If (InStr(step, "//") == 1) { ; for wildcard selectors use regex searching mode:
      StringTrimLeft, step, step, 2
      re = 1
      rew = 1
      xp = /(?:(?:\w+:)?\w+)*/
   }
   
   NumPut(160, doc, 0, "UChar") ; unmask variable
   
   ; resolve xpath components to: absolute node path, attributes and predicates (if any)
   Loop, Parse, step, /
   {
      s = %A_LoopField%
      If (InStr(s, "*") == 1) ; regex mode for wildcards
      {
         re = 1
         s := "(?:\w+:)?\w+" . SubStr(s, 2)
      }
      Else If (InStr(s, "@") == 1) { ; if current step is attribute:
         StringTrimLeft, atr, s, 1
         Continue
      }
      StringGetPos, p, s, [ ; look for predicate opening [...]
      If ErrorLevel = 0
      {
         If s contains [+1],[-1] ; for child nodes record creation instructions in a list
         {
            If A_Index = 2 ; root node creation
            {
               StringLeft, t, s, p
               t = <%t%/:: ></%t%/>
               If (InStr(s, "+1")) {
                  If doc =
                     doc = .
                  doc = %doc%%t%
               }
               Else doc = .%t%%doc%
               StringLeft, s, s, InStr(s, "[") - 1
            }
            Else {
               nw = %nw%%s%
               Continue
            }
         }
         Else { ; i.e. for conditional predicates
            
            If (InStr(s, "last()")) { ; finding last node
               StringLeft, t, s, p
               t := "<" . SubStr(xp, 2) . t . "/:: "
               If re ; with regex:
               {
                  os = 0
                  Loop
                     If (!os := RegExMatch(doc, t, "", 1 + os)) {
                        t = %A_Index%
                        Break
                     }
                  t--
               }
               Else { ; otherwise using StringReplace
                  StringReplace, doc, doc, %t%, %t%, UseErrorLevel
                  t = %ErrorLevel%
               }
               If (RegExMatch(s, "i)last\(\)\s*\-\s*(\d+)", a)) ; i.e. [last() - 2]
                  StringReplace, s, s, %a%, % t - a1
               Else StringReplace, s, s, last(), %t%
            }
            
            ; concat. the predicate to the list against the current absolute path:
            ax = %ax%%xp%%s%
            StringLeft, s, s, p
         }
      }
      Else If (InStr(s, "()")) { ; if step is function, add to list
         fn = %fn%+%s% ; use + prefix to prevent overlapping naming conflicts
         Continue
      }
      ; finally, if step is not any of the above, assume it's the name of a child node
      xp = %xp%%s%/ ; ... and add to list, forming the absolute path
   }
   
   If (xp == "" or xp == "/") ; i.e. error cases
      Return
   
   ; remove initial root selector (/) as this is how the parser works by default:
   StringTrimLeft, xp, xp, 1
   StringTrimRight, ax, ax, 1
   
   StringTrimRight, nw, nw, 1
   
   ct = 0 ; counter
   os = 0 ; offset for main loop starts at zero
   Loop {
      ; find offset of next element, and its closing tag offset:
      If re
         os := RegExMatch(doc, "<" . xp . ":: ", "", 1 + os)
            , osx := RegExMatch(doc, "</" . xp . ">", rem, os) + StrLen(rem)
      Else {
         StringGetPos, osx, doc, </%xp%>, , os := InStr(doc, "<" . xp . ":: ", true, 1 + os)
         osx += 4 + StrLen(xp)
      }
      If os = 0 ; stop looping when no more tags are found
         Break
      
      ; predicate parser:
      If ax !=
      {
         sk = 0
         Loop, Parse, ax, ] ; for each predicate
         {
            ; split components to: (1) path, (2) selector, (3) operator, (4) quotation char, (5) operand
            If (!RegExMatch(A_LoopField, "/?(.*?)\[(.+?)(?:\s*([<>!=]{1,2})\s*(['""])?(.+)(?(4)\4))?\s*$", a))
               Continue
            a1 = %a1%/
            If re
               RegExMatch(rem, "(?<=^</)" . a1, a1)
            
            If a2 is integer ; i.e. match only certain index
            {
               StringGetPos, t, a1, /, R2
               StringMid, t, a1, 1, t + 1
               t := InStr(SubStr(doc, 1, os), "<" . t . ":: ", true, 0)
               ; extract parent node:
               StringMid, sub, doc, t, InStr(doc, ">", "", os) - t
               xpf := "<" . a1 . ":: "
               ; get index of current element within parent node:
               StringReplace, sub, sub, %xpf%, %xpf%, UseErrorLevel
               If a2 != %ErrorLevel%
                  sk = 1
               Continue
            }
            
            StringReplace, xp, xp, /, /, UseErrorLevel
            t = %ErrorLevel%
            StringReplace, a1, a1, /, /, UseErrorLevel
            
             ; extract result for deep analysis
            If t = %ErrorLevel% ; i.e. /root/node[child='test']
               StringMid, sub, doc, os, osx - os
            Else StringMid, sub, doc
                   , t := InStr(SubStr(doc, 1, os), "<" . a1 . ":: ", true, 0)
                   , InStr(doc, "</" . a1 . ">", true, t) + 1
            
            If a2 = position()
               sub = %i%
            Else If (InStr(a2, "@") == 1) ; when selector is an attribute:
             RegExMatch(SubStr(sub, 1, InStr(sub, ">"))
                  , a3 == "" ? "\b" . SubStr(a2, 2) . "=([""'])[^\1]*?\1"
                  : "\b(?<=" . SubStr(a2, 2) . "=([""']))[^\1]+?(?=\1)", sub)
            Else ; otherwise child node:
            {
               If a2 = . ; if selector is current node don't append to path:
                  a2 = /
               Else a2 = %a2%/
               StringMid, sub, sub
                  , t := InStr(sub, ">", "", InStr(sub, "<" . a1 . a2 . ":: ", true) + 1) + 1
                  , InStr(sub, "</" . a1 . a2 . ">", true) - t
            }
            
            ; dynamic mini expression evaluator:
            sk += !(a3 == "" ? (sub != "")
               : a3 == "=" ? sub == a5
               : a3 == "!=" ? sub != a5
               : a3 == ">" ? sub > a5
               : a3 == ">=" ? sub >= a5
               : a3 == "<" ? sub < a5
               : a3 == "<=" ? sub <= a5)
         }
         If sk != 0 ; if conditions were not met for this result, skip it
            Continue
      }
      
      If nw != ; for node creation
      {
         If re
            nwp := SubStr(rem, 3, -1)
         Else nwp = %xp%
         Loop, Parse, nw, ]
         {
            StringLeft, nwn, A_LoopField, InStr(A_LoopField, "[") - 1
            nwn = %nwn%/
            nwt = <%nwp%%nwn%:: ></%nwp%%nwn%>
            If (t := InStr(A_LoopField, "-1")
               ? InStr(doc, ">", "", InStr(doc, "<" . nwp . ":: ", true, os) + 1) + 1
               : InStr(doc, "</" . nwp . ">", true, os))
               os := t
            StringLen, osx, nwt
            osx += os
            doc := SubStr(doc, 1, os - 1) . nwt . SubStr(doc, os)
            nwp = %nwp%%nwn%
         }
         StringLen, t, nwp
         If (InStr(fn, "+text()") and atr == "")
            os += t + 5, osx -= t + 3
      }
      
      If atr !=
      {
         ; extract attribute offsets, with surrounding declaration if text() is not used:
         If (t := RegExMatch(SubStr(doc, os, InStr(doc, ">", "", os) - os), InStr(fn, "+text()")
            ? "(?<=\b" . atr . "=([""']))[^\1]*?(?=\1)"
            : "\b" . atr . "=([""'])[^\1]*?\1", rem))
            os += t - 1, osx := os + StrLen(rem)
         Else { ; create attribute
            os := InStr(doc, ">", "", os + 1)
               , doc := SubStr(doc, 1, os - 1) . " " . atr . "=""""" . SubStr(doc, os)
               , osx := ++os + StrLen(atr) + 3
            If (InStr(fn, "+text()"))
               osx := os += StrLen(atr) + 2
         }
      }
      Else If (InStr(fn, "+text()") and nw == "") ; for text content:
         os := InStr(doc, ">", "", os) + 1, osx := InStr(doc, re ? rem : "</" . xp . ">", true, os)
      
      If InStr(fn, "+index-of()") ; get index rather than content
         sub = %A_Index%
      Else StringMid, sub, doc, os, osx - os ; extract result
      
      If (InStr(fn, "+count()")) ; increment counter if count() function is used
         ct++
      Else res = %res%%sub%, ; ... and concat to list
      
      If (set != "" or InStr(fn, "+remove()")) ; modify or remove...
         setb = %setb%%os%.%osx%| ; mark for modification
   }
   
   If setb !=
   {
      If (InStr(set, "node:") == 1) {
         set := SubStr(xpath(doc, SubStr(set, 6) . "/rawsrc()"), 2)
         StringReplace, set, set, <, <%xp%, All
         StringReplace, set, set, <%xp%/, </%xp%, All
         NumPut(160, doc, 0, "UChar")
      }
      StringTrimRight, setb, setb, 1
      Loop, Parse, setb, |
      {
         StringSplit, setp, A_LoopField, .
         StringLen, t, xp
         If (InStr(fn, "+append()"))
            setp2 := setp1 := setp2 - t - 3
         Else If (InStr(fn, "+prepend()"))
            setp2 := setp1 := InStr(doc, ">", "", setp1) + 2
         doc := SubStr(doc, 1, setp1 - 1) . set . SubStr(doc, setp2) ; dissect then insert new value
      }
   }
   
   If (InStr(fn, "+count()"))
      res = %ct%, ; trailing char since SubStr is used below
   
   nsid := scb . &doc . ":" ; update sid as necessary
   If nsid != %sid%
      StringReplace, sc, sc, %sid%, %nsid%
   
   NumPut(0, doc, 0, "UChar") ; remask variable to prevent external editing
   StringTrimRight, res, res, 1
   If (InStr(fn, "+rawsrc()")) {
      StringTrimLeft, t, xpr, 1
      StringReplace, res, res, <%t%/, <, All
      StringReplace, res, res, </%t%, <, All
      StringReplace, res, res, `,, `,%scb%, All
      Return, scb . res
   }
   ; remove trailing comma and absolute paths from result before returning:
   Return, RegExReplace(res, "S)(?<=<)(\/)?(?:(\w+)\/)+(?(1)|:: )", "$1$2")
}

/*

      Function: xpath_save
         Saves an XML document to file or returns the source.
      
      Parameters:
         doc - an xml object returned by xpath_load()
         src - (optional) a path to a file where the XML document should be saved;
            if the file already exists it will be replaced
      
      Returns:
         False if there was an error in saving the document, true otherwise.
         If the src parameter is left blank the source code of the document is returned instead.

*/
xpath_save(ByRef doc, src = "") {
   xml := RegExReplace(SubStr(doc, 2), "S)(?<=<)(\/)?(?:(\w+)\/)+(?(1)|:: )", "$1$2") ; remove metadata
   xml := RegExReplace(xml, "<([\w:]+)([^>]*)><\/\1>", "<$1$2 />") ; fuse empty nodes
   ;xml := RegExReplace(xml, " (?=(?:\w+:)?\w+=['""])") ; remove prepending whitespace on attributes
   xml := RegExReplace(xml, "^\s+|\s+$") ; remove start and leading whitespace
   If InStr(xml, "<?xml") != 1 ; add processor instruction if there isn't any:
      xml = <?xml version="1.0" encoding="iso-8859-1"?>`n%xml%
   StringReplace, xml, xml, `r, , All ; normalize linefeeds:
   StringReplace, xml, xml, `n, `r`n, All
   sp := "  "
   StringLen, sl, sp
   s =
   VarSetCapacity(sxml, StrLen(xml) * 1.1)
   Loop, Parse, xml, <, `n`t     `r
   {
      If A_LoopField =
         Continue
      If (sb := InStr(A_LoopField, "/") == 1)
         StringTrimRight, s, s, sl
      sxml = %sxml%`n%s%<%A_LoopField%
      If sb
         StringTrimRight, s, s, sl
      If (InStr(A_LoopField, "?") != 1 and InStr(A_LoopField, "!") != 1
         and !InStr(A_LoopField, "/>"))
         s .= sp
   }
   StringTrimLeft, sxml, sxml, 1
   sxml := RegExReplace(sxml, "(\n(?:" . sp . ")*<((?:\w+:)?\w+\b)[^<]+?)\n(?:"
      . sp . ")*</\2>", "$1</$2>")
   If src = ; if save path not specified return the XML document:
      Return, sxml
   FileDelete, %src% ; delete existing file
   FileAppend, %sxml%, %src% ; create new one
   Return, ErrorLevel ; return errors, if any
}

/*

      Function: xpath_load
         Loads an XML document.
      
      Parameters:
         doc - a reference to the loaded XML file as a variable, to be used in other functions
         src - (optional) the document to load, this can be a file name or a string,
            if omitted the first paramter is used as the source
      
      Returns:
         False if there was an error in loading the document, true otherwise.

*/
xpath_load(ByRef doc, src = "") {
   If src = ; if source is empty assume the out variable is the one to be loaded
      src = %doc%
   Else If FileExist(src) ; otherwise read from file (if it exists)
      FileRead, src, %src%
   If src not contains <,>
      Return, false
   ; combined expressions slightly improve performance:
   src := RegExReplace(src, "<((?:\w+:)?\w+\b)([^>]*)\/\s*>", "<$1$2></$1>") ; defuse nodes
      , VarSetCapacity(doc, VarSetCapacity(xml, StrLen(src) * 1.5) * 1.1) ; pre-allocate enough space
   Loop, Parse, src, < ; for each opening tag:
   {
      If (A_Index == 2 and InStr(A_LoopField, "?xml") == 1)
         Continue
      Else If (InStr(A_LoopField, "?") == 1) ; ignore all other processor instructions
         xml = %xml%<%A_LoopField%
      Else If (InStr(A_LoopField, "![CDATA[") == 1) { ; escape entities in CDATA sections
         cdata := SubStr(A_LoopField, 9, -3)
         StringReplace, cdata, cdata, ", &quot;, All
         StringReplace, cdata, cdata, &, &amp;, All
         StringReplace, cdata, cdata, ', &apos;, All
         StringReplace, cdata, cdata, <, &lt;, All
         StringReplace, cdata, cdata, >, &gt;, All
         xml = %xml%%cdata%
      }
      Else If (!pos := RegExMatch(A_LoopField, "^\/?(?:\w+:)?\w+", tag)) ; if this isn't a valid tag:
      {
         If A_LoopField is not space
            xml = %xml%&lt;%A_LoopField% ; convert to escaped entity value
      }
      Else {
         StringMid, ex, A_LoopField, pos + StrLen(tag) ; get tag name
         If InStr(tag, "/") = 1 { ; if this is a closing tag:
            xml = %xml%</%pre%%ex% ; close tag
            StringGetPos, pos, pre, /, R2
            StringLeft, pre, pre, pos + 1
         }
         Else {
            pre = %pre%%tag%/
            xml = %xml%<%pre%:: %ex%
         }
      }
   }
   StringReplace, doc, xml, `,, ,, All ; entity escape commas (which are used as array delimiters)
   NumPut(0, doc := " " . doc, 0, "UChar") ; mask variable from text display with nullbyte
   Return, true ; assume sucessful load by this point
}




And one more this can the COM_L do HTTP Post and control the HTTP User Agent, flags (no caching for instance ), and download and UpdateSize like HTTPQuery?

I would really love to bring the ease of HTTPQuery to AHK_L Unicode, I will be more then happy to do what ever you say to adjust HTTPQuery and test it and of course publish it later.

I am currently using HTTPQuery in a lot of project at work, Web site testing,Synthetic XML transction. ETC. this is why its so important to me.


Thanks again for your hard work,
Twhyman
_________________
(\__/) This is Bunny.
(='.'=) Cut, copy, and paste bunny onto your sig.
(")_(") Help Bunny gain World Domination.
Back to top
View user's profile Send private message
Desi
Guest





PostPosted: Mon Apr 12, 2010 5:22 pm    Post subject: I second mister twhyman Reply with quote

I second mister twhyman... Please, can you give us a hand?

Thanks
Back to top
HotKeyIt



Joined: 18 Jun 2008
Posts: 4652
Location: AHK Forum

PostPosted: Mon Apr 12, 2010 8:37 pm    Post subject: Reply with quote

Above httpQuery() version works for me on another server, possibly the server does not support unicode Question

Here is a little modified version that should work for any AutoHotkey build.
Code:
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{   ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
   ; updates Aug, 28 2008   
   ; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
   ; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
   ; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
   ; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
   ; handling this might be set using the httpQueryDwFlags variable as global
   global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
       , httpQueryDwFlags
   static StrGet:="StrGet" ;for AHK_L and AHK_H (AutoHotkey.dll)
   ;### Added for unicode support
   API_Suffix := A_IsUnicode ? "W" : "A"
   tchar := A_IsUnicode ? "UShort" : "Char"
   tchar_size := A_IsUnicode ? 2 : 1
   
   ; Get any missing default Values
   defaultOps =
   (LTrim Join|
      httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
      SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
      SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
      INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
   )
   Loop,Parse,defaultOps,|
   {
      RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
      if StrLen(%httpOption%)=0
         %httpOption% := httpDefault
   }

   ; Load Library
   hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")

   ; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
   ; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
   offset_name_length:= "4-lpszScheme-255|16-lpszHostName-1024|28-lpszUserName-1024|"
                  . "36-lpszPassword-1024|44-lpszUrlPath-1024|52-lpszExtrainfo-1024"
   VarSetCapacity(URL_COMPONENTS,60,0)
   ; Struc Size               ; Scheme Size                  ; Max Port Number
   NumPut(60,URL_COMPONENTS,0), NumPut(255,URL_COMPONENTS,12), NumPut(0xffff,URL_COMPONENTS,24)
   
   Loop,Parse,offset_name_length,|
   {
      RegExMatch(A_LoopField,"(?P<Offset>\d+)-(?P<Name>[a-zA-Z]+)-(?P<Size>\d+)",iCU_)
      ;VarSetCapacity(%iCU_Name%,iCU_Size*(A_IsUnicode ? 2:1),0)
      VarSetCapacity(%iCU_Name%,iCU_Size*tchar_size,0)
      NumPut(&%iCU_Name%,URL_COMPONENTS,iCU_Offset)
      NumPut(iCU_Size,URL_COMPONENTS,iCU_Offset+4)
   }

   ; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
   ; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
   DllCall("WinINet\InternetCrackUrl" . API_Suffix,"Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"uInt",&URL_COMPONENTS)

   ; Update variables to retrieve results
   Loop,Parse,offset_name_length,|
   {
      RegExMatch(A_LoopField,"-(?P<Name>[a-zA-Z]+)-",iCU_)
      VarSetCapacity(%iCU_Name%,-1)
   }
   nPort:=NumGet(URL_COMPONENTS,24,"uInt")
   
   ; Import any set dwFlags
   dwFlags := httpQueryDwFlags
   ; For some reasons using a selfsigned https certificates doesnt work
   ; such as an own webmin service - even though every security is turned off
   ; https with valid certificates works when
   if (lpszScheme = "https")
      dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
               |SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)

   ; Check for Header and drop exception if unknown or invalid URL
   if (lpszScheme="unknown") {
      Result := "ERR: No Valid URL supplied."
      Return StrLen(Result)
   }

   ; Initialise httpQuery's use of the WinINet functions.
   ; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
   hInternet := DllCall("WinINet\InternetOpen" . API_Suffix
                  ,"Str",httpAgent,"UInt"
                  ,(httpProxy != 0 ?  INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
                  ,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)

   ; Open HTTP session for the given URL
   ; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
   hConnect := DllCall("WinINet\InternetConnect" . API_Suffix
                  ,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
                  ,"Str",lpszUserName, "Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
                  ,"uInt",0,"uInt*",0)

   ; Do we POST? If so, check for header handling and set default
   if (Strlen(POSTDATA)>0) {
      HTTPVerb:="POST"
      if StrLen(Headers)=0
         Headers:="Content-Type: application/x-www-form-urlencoded"
   } else ; otherwise mode must be GET - no header defaults needed
      HTTPVerb:="GET"   

   ; Form the request with proper HTTP protocol version and create the request handle
   ; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
   hRequest := DllCall("WinINet\HttpOpenRequest" . API_Suffix
                  ,"uInt",hConnect,"Str",HTTPVerb,"Str",lpszUrlPath . lpszExtrainfo
                  ,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
                  ,"uInt",dwFlags,"uInt",Context:=0 )

   ; Send the specified request to the server
   ; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
   sRequest := DllCall("WinINet\HttpSendRequest" . API_Suffix
                  , "uInt",hRequest,"Str",Headers, "uInt",Strlen(Headers)
                  , "Str",POSTData,"uInt",Strlen(POSTData))

   VarSetCapacity(header, 2048, 0)  ; max 2K header data for httpResponseHeader
   VarSetCapacity(header_len, 4, 0)
   
   ; Check for returned server response-header (works only _after_ request been sent)
   ; http://msdn.microsoft.com/en-us/library/aa384238.aspx
   Loop, 5
     if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"uint",hRequest
      ,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
      break

   If (headerRequest=1) {
      VarSetCapacity(res,headerLength:=NumGet(header_len),32)
      DllCall("RtlMoveMemory","uInt",&res,"uInt",&header,"uInt",headerLength)
      ;Loop,% headerLength
         ;if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
            ;NumPut(Asc("`n"),res,a_index-1,"uChar")
      Loop,% headerLength/tchar_size
      if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
       NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
      VarSetCapacity(res,-1)
   } else
      res := "timeout"

   ; Get 1st Line of Full Response
   Loop,Parse,res,`n,`r
   {
      RetValue := A_LoopField
      break
   }
   
   ; No Connection established - drop exception
   If (RetValue="timeout") {
      html := "Error: timeout"
      return -1
   }
   ; Strip protocol version from return value
   RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
   
    ; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
   HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
              . "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
              . " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
              . "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
              . "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
              . "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
              . "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
              . "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
              . "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
              . "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
              . "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
              . "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
              . " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
              . "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
              . "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
              . "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
              . "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
              . "t Extended (RFC 2774)"
   
   ; Gather numeric response value
   RetValue := SubStr(RetValue,1,3)
   
   ; Parse through return codes and set according informations
   Loop,Parse,HttpRetCodes,|
   {
      HttpReturnCode := SubStr(A_LoopField,1,3)    ; Numeric return value see above
      HttpReturnMsg  := SubStr(A_LoopField,5)      ; link for additional information
      if (RetValue=HttpReturnCode) {
         RetMsg := HttpReturnMsg
         break
      }
   }

   ; Global HttpQueryOps handling
   if strlen(HTTPQueryOps)>0 {
      ; Show full Header response (usefull for debugging)
      if (instr(HTTPQueryOps,"showHeader"))
         MsgBox % res
      ; Save the full Header response in a global Variable
      if (instr(HTTPQueryOps,"storeHeader"))
         global HttpQueryHeader := res
      ; Check for size updates to export to a global Var
      if (instr(HTTPQueryOps,"updateSize")) {
         Loop,Parse,res,`n
            If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
               global HttpQueryFullSize := fullSize
               break
            }
         if (fullSize+0=0)
            HttpQueryFullSize := "size unavailable"
      }
   }

   ; Check for valid codes and drop exception if suspicious
   if !(InStr("100 200 201 202 302",RetValue)) {
      Result := RetValue " " RetMsg
      return StrLen(Result)
   }

   VarSetCapacity(BytesRead,4,0)
   fsize := 0
   Loop            ; the receiver loop - rewritten in the need to enable
   {               ; support for larger file downloads
      bc := A_Index
      VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
      ReadFile := DllCall("wininet\InternetReadFile"
                  ,"uInt",hRequest,"uInt",&buffer%bc%,"uInt",1024,"uInt",&BytesRead)
      ReadBytes := NumGet(BytesRead)    ; how many bytes were received?
      If ((ReadFile!=0)&&(!ReadBytes))  ; we have had no error yet and received no more bytes
         break                         ; we must be done! so lets break the receiver loop
      Else {
         fsize += ReadBytes            ; sum up all chunk sizes for correct return size
         sizeArray .= ReadBytes "|"
      }
      if (instr(HTTPQueryOps,"updateSize"))
         Global HttpQueryCurrentSize := fsize
   }
   sizeArray := SubStr(sizeArray,1,-1)   ; trim last PipeChar
   
   VarSetCapacity(result,fSize+1,0)      ; reconstruct the result from above generated chunkblocks
   Dest := &result                       ; to a our ByRef result variable
   Loop,Parse,SizeArray,|
      DllCall("RtlMoveMemory","uInt",Dest,"uInt",&buffer%A_Index%,"uInt",A_LoopField)
      , Dest += A_LoopField
   if InStr(A_AhkVersion,".L") { ;;for AHK_L and AHK_H (AutoHotkey.dll)
      ;------------------------------------------------------------------------
      ; AutoHotkey_L: Convert text files into our native string format.
      ; First, check the headers for hints about how the file is encoded:
      is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
      ; Now check for a byte order mark - it probably won't be present,
      ; but if it is we don't want it to be present in the returned string:
      dest := &result
      if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
         dest += 3, charset := "utf-8"
      else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
         dest += 2, charset := "utf-16"
      ; else: no recognized BOM, just use charset as set by RegExMatch.
      if charset in utf-8,utf-16
         result := %StrGet%(dest, charset)
      ; Since it doesn't seem practical to support every possible charset name,
      ; simply treat all unrecognized names as this system's ANSI code page.
      else if is_text && A_IsUnicode
         result := %StrGet%(dest, "cp0")
      ; else:
      ;    Content-Type indicates it's not text; or
      ;    this is an ANSI build so the above StrGet call is unnecessary.
      ;------------------------------------------------------------------------
   }
   
   DllCall("WinINet\InternetCloseHandle", "uInt", hRequest)   ; close all opened
   DllCall("WinINet\InternetCloseHandle", "uInt", hInternet)
   DllCall("WinINet\InternetCloseHandle", "uInt", hConnect)
   DllCall("FreeLibrary", "UInt", hModule)                    ; unload the library
   return fSize                          ; return the size - strings need update via VarSetCapacity(res,-1)
}

_________________
AHK_H (2alpha) AHF TT _Struct WatchDir Yaml _Input ObjTree RapidHotkey DynaRun Wink
Back to top
View user's profile Send private message
sbc



Joined: 25 Aug 2009
Posts: 321

PostPosted: Fri Dec 03, 2010 10:29 am    Post subject: Reply with quote

HotKeyIt wrote:
Here is a little modified version that should work for any AutoHotkey build.
Does it work for the 64bit build? I just tested it but doesn't seem to work. I'm using the AHK_L R61 Unicode 64bit build. Confused
Back to top
View user's profile Send private message Visit poster's website
sinkfaze



Joined: 18 Mar 2008
Posts: 5043
Location: the tunnel(?=light)

PostPosted: Tue Jan 11, 2011 6:26 am    Post subject: Reply with quote

I similarly decided to test HotkeyIt's mod of httpQuery on my Windows 7 machine without any success, it looks like the common point of failure is here:

Code:
   Loop, 5
     if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"uint",hRequest
      ,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
      break


EDIT: The more appropriate point of failure may be here:

Code:
   hConnect :=   DllCall("WinINet\InternetConnect" API_Suffix
          ,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
          ,"Str",lpszUserName,"Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
          ,"uInt",0,"uInt*",0)

_________________
Try Quick Search for Autohotkey or see the tutorial for newbies.
Back to top
View user's profile Send private message Send e-mail
HotKeyIt



Joined: 18 Jun 2008
Posts: 4652
Location: AHK Forum

PostPosted: Tue Jan 11, 2011 8:50 pm    Post subject: Reply with quote

I've hopefully fixed it now Very Happy
Exclamation requires AHK_L + AHK_H Exclamation and Struct()

Code:
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{   ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
   ; updates Aug, 28 2008   
   ; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
   ; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
   ; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
   ; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
   ; handling this might be set using the httpQueryDwFlags variable as global
   global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
       , httpQueryDwFlags
   ;### Added for unicode support
   static API_Suffix := A_IsUnicode ? "W" : "A"
         ,tchar := A_IsUnicode ? "UShort" : "UChar"
         ,tchar_size := A_IsUnicode ? 2 : 1
   ,URL_COMPONENTS:="DWORD dwStructSize,LPWSTR lpszScheme,DWORD dwSchemeLength,int nScheme,LPWSTR lpszHostName,DWORD dwHostNameLength,LONG nPort,LPWSTR lpszUserName,DWORD dwUserNameLength,LPWSTR lpszPassword,DWORD dwPasswordLength,LPWSTR lpszUrlPath,DWORD dwUrlPathLength,LPWSTR lpszExtraInfo,DWORD dwExtraInfoLength"
   ,STRBUFFERS:= "Scheme|SchemeLength|HostName|HostNameLength|UserName|UserNameLength|Password|PasswordLength|UrlPath|UrlPathLength|Extrainfo|ExtraInfoLength"

   ; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
   ; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
   UC:=Struct(URL_COMPONENTS)
   UC.dwStructSize:=Struct(UC)
   UC.dwSchemeLength:=255
   UC.nPort:=0xffff
   Loop,Parse,STRBUFFERS,|
      If Mod(A_Index,2)
         VarSetCapacity(_%A_Index%,tchar_size*1024),UC["lpsz" A_LoopField]:=&_%A_Index%
      else
         UC["dw" A_LoopField]:=tchar_size*1024

   ; Get any missing default Values
   defaultOps =
   (LTrim Join|
      httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
      SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
      SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
      INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
   )
   Loop,Parse,defaultOps,|
   {
      RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
      if StrLen(%httpOption%)=0
         %httpOption% := httpDefault
   }
   
   ; Load Library
   hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")

   ; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
   ; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
   
   DllCall("WinINet\InternetCrackUrl" . API_Suffix,"Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"UPTR",UC[""])
   
   ; Import any set dwFlags
   dwFlags := httpQueryDwFlags
   ; For some reasons using a selfsigned https certificates doesnt work
   ; such as an own webmin service - even though every security is turned off
   ; https with valid certificates works when
   if (StrGet(UC.lpszScheme) = "https")
      dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
               |SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)

   ; Check for Header and drop exception if unknown or invalid URL
   if (StrGet(UC.lpszScheme)="unknown") {
      Result := "ERR: No Valid URL supplied."
      Return StrLen(Result)
   }
 
   ; Initialise httpQuery's use of the WinINet functions.
   ; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx

   hInternet := DllCall("WinINet\InternetOpen" . API_Suffix
                  ,"Str",httpAgent,"UInt"
                  ,(httpProxy != 0 ?  INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
                  ,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)
   ; Open HTTP session for the given URL
   ; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
   hConnect := DllCall("WinINet\InternetConnect" . API_Suffix
                  ,"UPTR",hInternet,"UPTR",UC.lpszHostname, "Int",NC.nPort
                  ,"UPTR",StrGet(UC.lpszUserName)?UC.lpszUserName:0
                  , "UPTR",StrGet(UC.lpszPassword)?UC.lpszPassword:0,"uInt",INTERNET_SERVICE_HTTP
                  ,"uInt",0,"uInt*",0)

   ; Do we POST? If so, check for header handling and set default
   if (Strlen(POSTDATA)>0) {
      HTTPVerb:="POST"
      if StrLen(Headers)=0
         Headers:="Content-Type: application/x-www-form-urlencoded"
   } else ; otherwise mode must be GET - no header defaults needed
      HTTPVerb:="GET"   

   ; Form the request with proper HTTP protocol version and create the request handle
   ; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
   hRequest := DllCall("WinINet\HttpOpenRequest" . API_Suffix
                  ,"UPTR",hConnect,"Str",HTTPVerb,"Str",StrGet(UC.lpszUrlPath) . StrGet(UC.lpszExtrainfo)
                  ,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
                  ,"uInt",dwFlags,"uInt",Context:=0 )

   ; Send the specified request to the server
   ; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
   sRequest := DllCall("WinINet\HttpSendRequest" . API_Suffix
                  , "UPTR",hRequest,"Str",Headers, "uInt",Strlen(Headers)
                  , "Str",POSTData,"uInt",Strlen(POSTData))

   VarSetCapacity(header, 2048, 0)  ; max 2K header data for httpResponseHeader
   VarSetCapacity(header_len, 4, 0)
   
   ; Check for returned server response-header (works only _after_ request been sent)
   ; http://msdn.microsoft.com/en-us/library/aa384238.aspx
   Loop, 5
     if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"UPTR",hRequest
      ,"uint",21,"UPTR",&header,"UPTR",&header_len,"uint",0))=1)
      break

   If (headerRequest=1) {
      VarSetCapacity(res,headerLength:=NumGet(header_len),32)
      t:=DllCall("RtlMoveMemory","UPTR",&res,"UPTR",&header,"uInt",headerLength)
      ;Loop,% headerLength
         ;if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
            ;NumPut(Asc("`n"),res,a_index-1,"uChar")
      Loop,% headerLength/tchar_size
      if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
       NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
      VarSetCapacity(res,-1)
   } else
      res := "timeout"
   ; Get 1st Line of Full Response
   Loop,Parse,res,`n,`r
   {
      RetValue := A_LoopField
      break
   }
   
   ; No Connection established - drop exception
   If (RetValue="timeout") {
      html := "Error: timeout"
      return -1
   }
   ; Strip protocol version from return value
   RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
   
    ; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
   HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
              . "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
              . " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
              . "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
              . "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
              . "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
              . "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
              . "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
              . "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
              . "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
              . "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
              . "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
              . " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
              . "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
              . "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
              . "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
              . "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
              . "t Extended (RFC 2774)"
   
   ; Gather numeric response value
   RetValue := SubStr(RetValue,1,3)
 
   ; Parse through return codes and set according informations
   Loop,Parse,HttpRetCodes,|
   {
      HttpReturnCode := SubStr(A_LoopField,1,3)    ; Numeric return value see above
      HttpReturnMsg  := SubStr(A_LoopField,5)      ; link for additional information
      if (RetValue=HttpReturnCode) {
         RetMsg := HttpReturnMsg
         break
      }
   }

   ; Global HttpQueryOps handling
   if strlen(HTTPQueryOps)>0 {
      ; Show full Header response (usefull for debugging)
      if (instr(HTTPQueryOps,"showHeader"))
         MsgBox % res
      ; Save the full Header response in a global Variable
      if (instr(HTTPQueryOps,"storeHeader"))
         global HttpQueryHeader := res
      ; Check for size updates to export to a global Var
      if (instr(HTTPQueryOps,"updateSize")) {
         Loop,Parse,res,`n
            If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
               global HttpQueryFullSize := fullSize
               break
            }
         if (fullSize+0=0)
            HttpQueryFullSize := "size unavailable"
      }
   }

   ; Check for valid codes and drop exception if suspicious
   if !(InStr("100 200 201 202 302",RetValue)) {
      Result := RetValue " " RetMsg
      return StrLen(Result)
   }

   VarSetCapacity(BytesRead,4,0)
   fsize := 0
   Loop            ; the receiver loop - rewritten in the need to enable
   {               ; support for larger file downloads
      bc := A_Index
      VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
      ReadFile := DllCall("wininet\InternetReadFile"
                  ,"UPTR",hRequest,"UPTR",&buffer%bc%,"uInt",1024,"UPTR",&BytesRead)
      ReadBytes := NumGet(BytesRead)    ; how many bytes were received?
      If ((ReadFile!=0)&&(!ReadBytes))  ; we have had no error yet and received no more bytes
         break                         ; we must be done! so lets break the receiver loop
      Else {
         fsize += ReadBytes            ; sum up all chunk sizes for correct return size
         sizeArray .= ReadBytes "|"
      }
      if (instr(HTTPQueryOps,"updateSize"))
         Global HttpQueryCurrentSize := fsize
   }
   sizeArray := SubStr(sizeArray,1,-1)   ; trim last PipeChar

   VarSetCapacity(result,fSize+1,0)      ; reconstruct the result from above generated chunkblocks
   Dest := &result                       ; to a our ByRef result variable
   Loop,Parse,SizeArray,|
      DllCall("RtlMoveMemory","UPTR",Dest,"UPTR",&buffer%A_Index%,"uInt",A_LoopField)
      , Dest += A_LoopField
   ;------------------------------------------------------------------------
   ; AutoHotkey_L: Convert text files into our native string format.
   ; First, check the headers for hints about how the file is encoded:
   is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
   ; Now check for a byte order mark - it probably won't be present,
   ; but if it is we don't want it to be present in the returned string:
   dest := &result
   if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
      dest += 3, charset := "utf-8"
   else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
      dest += 2, charset := "utf-16"
   ; else: no recognized BOM, just use charset as set by RegExMatch.
   if charset in utf-8,utf-16
      result := StrGet(dest, charset)
   ; Since it doesn't seem practical to support every possible charset name,
   ; simply treat all unrecognized names as this system's ANSI code page.
   else if is_text && A_IsUnicode
      result := StrGet(dest, "cp0")
   ; else:
   ;    Content-Type indicates it's not text; or
   ;    this is an ANSI build so the above StrGet call is unnecessary.
   ;------------------------------------------------------------------------
   
   DllCall("WinINet\InternetCloseHandle", "UPTR", hRequest)   ; close all opened
   DllCall("WinINet\InternetCloseHandle", "UPTR", hInternet)
   DllCall("WinINet\InternetCloseHandle", "UPTR", hConnect)
   DllCall("FreeLibrary", "UPTR", hModule)                    ; unload the library
   return fSize                          ; return the size - strings need update via VarSetCapacity(res,-1)
}

EDIT:
Hopefully fixed support for 64-bit version.
12. Jan 2011 - Another fix for username and password.
_________________
AHK_H (2alpha) AHF TT _Struct WatchDir Yaml _Input ObjTree RapidHotkey DynaRun Wink
Back to top
View user's profile Send private message
Joolee



Joined: 08 Aug 2005
Posts: 18

PostPosted: Tue Feb 08, 2011 8:33 pm    Post subject: Reply with quote

HotKeyIt wrote:
I've hopefully fixed it now Very Happy
Exclamation requires AHK_L + AHK_H Exclamation and Struct()

Code:
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{   ; v0.3.5 (w) Sep, 8 2008 by Heresy & derRaphael / zLib-Style release
   ; updates Aug, 28 2008   
   ; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
   ; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
   ; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
   ; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
   ; handling this might be set using the httpQueryDwFlags variable as global
   global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
       , httpQueryDwFlags
   ;### Added for unicode support
   static API_Suffix := A_IsUnicode ? "W" : "A"
         ,tchar := A_IsUnicode ? "UShort" : "UChar"
         ,tchar_size := A_IsUnicode ? 2 : 1
   ,URL_COMPONENTS:="DWORD dwStructSize,LPWSTR lpszScheme,DWORD dwSchemeLength,int nScheme,LPWSTR lpszHostName,DWORD dwHostNameLength,LONG nPort,LPWSTR lpszUserName,DWORD dwUserNameLength,LPWSTR lpszPassword,DWORD dwPasswordLength,LPWSTR lpszUrlPath,DWORD dwUrlPathLength,LPWSTR lpszExtraInfo,DWORD dwExtraInfoLength"
   ,STRBUFFERS:= "Scheme|SchemeLength|HostName|HostNameLength|UserName|UserNameLength|Password|PasswordLength|UrlPath|UrlPathLength|Extrainfo|ExtraInfoLength"

   ; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
   ; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
   UC:=Struct(URL_COMPONENTS)
   UC.dwStructSize:=Struct(UC)
   UC.dwSchemeLength:=255
   UC.nPort:=0xffff
   Loop,Parse,STRBUFFERS,|
      If Mod(A_Index,2)
         VarSetCapacity(_%A_Index%,tchar_size*1024),UC["lpsz" A_LoopField]:=&_%A_Index%
      else
         UC["dw" A_LoopField]:=tchar_size*1024

   ; Get any missing default Values
   defaultOps =
   (LTrim Join|
      httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
      SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
      SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
      INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
   )
   Loop,Parse,defaultOps,|
   {
      RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
      if StrLen(%httpOption%)=0
         %httpOption% := httpDefault
   }
   
   ; Load Library
   hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")

   ; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
   ; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
   
   DllCall("WinINet\InternetCrackUrl" . API_Suffix,"Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"UPTR",UC[""])
   
   ; Import any set dwFlags
   dwFlags := httpQueryDwFlags
   ; For some reasons using a selfsigned https certificates doesnt work
   ; such as an own webmin service - even though every security is turned off
   ; https with valid certificates works when
   if (StrGet(UC.lpszScheme) = "https")
      dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
               |SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)

   ; Check for Header and drop exception if unknown or invalid URL
   if (StrGet(UC.lpszScheme)="unknown") {
      Result := "ERR: No Valid URL supplied."
      Return StrLen(Result)
   }
 
   ; Initialise httpQuery's use of the WinINet functions.
   ; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx

   hInternet := DllCall("WinINet\InternetOpen" . API_Suffix
                  ,"Str",httpAgent,"UInt"
                  ,(httpProxy != 0 ?  INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
                  ,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)
   ; Open HTTP session for the given URL
   ; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
   hConnect := DllCall("WinINet\InternetConnect" . API_Suffix
                  ,"UPTR",hInternet,"UPTR",UC.lpszHostname, "Int",NC.nPort
                  ,"UPTR",StrGet(UC.lpszUserName)?UC.lpszUserName:0
                  , "UPTR",StrGet(UC.lpszPassword)?UC.lpszPassword:0,"uInt",INTERNET_SERVICE_HTTP
                  ,"uInt",0,"uInt*",0)

   ; Do we POST? If so, check for header handling and set default
   if (Strlen(POSTDATA)>0) {
      HTTPVerb:="POST"
      if StrLen(Headers)=0
         Headers:="Content-Type: application/x-www-form-urlencoded"
   } else ; otherwise mode must be GET - no header defaults needed
      HTTPVerb:="GET"   

   ; Form the request with proper HTTP protocol version and create the request handle
   ; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
   hRequest := DllCall("WinINet\HttpOpenRequest" . API_Suffix
                  ,"UPTR",hConnect,"Str",HTTPVerb,"Str",StrGet(UC.lpszUrlPath) . StrGet(UC.lpszExtrainfo)
                  ,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
                  ,"uInt",dwFlags,"uInt",Context:=0 )

   ; Send the specified request to the server
   ; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
   sRequest := DllCall("WinINet\HttpSendRequest" . API_Suffix
                  , "UPTR",hRequest,"Str",Headers, "uInt",Strlen(Headers)
                  , "Str",POSTData,"uInt",Strlen(POSTData))

   VarSetCapacity(header, 2048, 0)  ; max 2K header data for httpResponseHeader
   VarSetCapacity(header_len, 4, 0)
   
   ; Check for returned server response-header (works only _after_ request been sent)
   ; http://msdn.microsoft.com/en-us/library/aa384238.aspx
   Loop, 5
     if ((headerRequest:=DllCall("WinINet\HttpQueryInfo" . API_Suffix,"UPTR",hRequest
      ,"uint",21,"UPTR",&header,"UPTR",&header_len,"uint",0))=1)
      break

   If (headerRequest=1) {
      VarSetCapacity(res,headerLength:=NumGet(header_len),32)
      t:=DllCall("RtlMoveMemory","UPTR",&res,"UPTR",&header,"uInt",headerLength)
      ;Loop,% headerLength
         ;if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
            ;NumPut(Asc("`n"),res,a_index-1,"uChar")
      Loop,% headerLength/tchar_size
      if NumGet(res, (a_index-1)*tchar_size, tchar) = 0
       NumPut(Asc("`n"),res,(a_index-1)*tchar_size, tchar)
      VarSetCapacity(res,-1)
   } else
      res := "timeout"
   ; Get 1st Line of Full Response
   Loop,Parse,res,`n,`r
   {
      RetValue := A_LoopField
      break
   }
   
   ; No Connection established - drop exception
   If (RetValue="timeout") {
      html := "Error: timeout"
      return -1
   }
   ; Strip protocol version from return value
   RetValue := RegExReplace(RetValue,"HTTP/1\.[01]\s+")
   
    ; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
   HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
              . "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
              . " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
              . "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
              . "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
              . "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
              . "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
              . "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
              . "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
              . "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
              . "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
              . "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
              . " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
              . "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
              . "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
              . "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
              . "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
              . "t Extended (RFC 2774)"
   
   ; Gather numeric response value
   RetValue := SubStr(RetValue,1,3)
 
   ; Parse through return codes and set according informations
   Loop,Parse,HttpRetCodes,|
   {
      HttpReturnCode := SubStr(A_LoopField,1,3)    ; Numeric return value see above
      HttpReturnMsg  := SubStr(A_LoopField,5)      ; link for additional information
      if (RetValue=HttpReturnCode) {
         RetMsg := HttpReturnMsg
         break
      }
   }

   ; Global HttpQueryOps handling
   if strlen(HTTPQueryOps)>0 {
      ; Show full Header response (usefull for debugging)
      if (instr(HTTPQueryOps,"showHeader"))
         MsgBox % res
      ; Save the full Header response in a global Variable
      if (instr(HTTPQueryOps,"storeHeader"))
         global HttpQueryHeader := res
      ; Check for size updates to export to a global Var
      if (instr(HTTPQueryOps,"updateSize")) {
         Loop,Parse,res,`n
            If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
               global HttpQueryFullSize := fullSize
               break
            }
         if (fullSize+0=0)
            HttpQueryFullSize := "size unavailable"
      }
   }

   ; Check for valid codes and drop exception if suspicious
   if !(InStr("100 200 201 202 302",RetValue)) {
      Result := RetValue " " RetMsg
      return StrLen(Result)
   }

   VarSetCapacity(BytesRead,4,0)
   fsize := 0
   Loop            ; the receiver loop - rewritten in the need to enable
   {               ; support for larger file downloads
      bc := A_Index
      VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
      ReadFile := DllCall("wininet\InternetReadFile"
                  ,"UPTR",hRequest,"UPTR",&buffer%bc%,"uInt",1024,"UPTR",&BytesRead)
      ReadBytes := NumGet(BytesRead)    ; how many bytes were received?
      If ((ReadFile!=0)&&(!ReadBytes))  ; we have had no error yet and received no more bytes
         break                         ; we must be done! so lets break the receiver loop
      Else {
         fsize += ReadBytes            ; sum up all chunk sizes for correct return size
         sizeArray .= ReadBytes "|"
      }
      if (instr(HTTPQueryOps,"updateSize"))
         Global HttpQueryCurrentSize := fsize
   }
   sizeArray := SubStr(sizeArray,1,-1)   ; trim last PipeChar

   VarSetCapacity(result,fSize+1,0)      ; reconstruct the result from above generated chunkblocks
   Dest := &result                       ; to a our ByRef result variable
   Loop,Parse,SizeArray,|
      DllCall("RtlMoveMemory","UPTR",Dest,"UPTR",&buffer%A_Index%,"uInt",A_LoopField)
      , Dest += A_LoopField
   ;------------------------------------------------------------------------
   ; AutoHotkey_L: Convert text files into our native string format.
   ; First, check the headers for hints about how the file is encoded:
   is_text := RegExMatch(res, "`nmi)^Content-Type:.*?\btext\b.*?(?:\bcharset=\K[\w-]+|$\K)", charset)
   ; Now check for a byte order mark - it probably won't be present,
   ; but if it is we don't want it to be present in the returned string:
   dest := &result
   if (NumGet(result) & 0xffffff) = 0xBFBBEF ; UTF-8 BOM: EF BB BF
      dest += 3, charset := "utf-8"
   else if (NumGet(result) & 0xffff) = 0xFEFF ; UTF-16 BOM: FF FE
      dest += 2, charset := "utf-16"
   ; else: no recognized BOM, just use charset as set by RegExMatch.
   if charset in utf-8,utf-16
      result := StrGet(dest, charset)
   ; Since it doesn't seem practical to support every possible charset name,
   ; simply treat all unrecognized names as this system's ANSI code page.
   else if is_text && A_IsUnicode
      result := StrGet(dest, "cp0")
   ; else:
   ;    Content-Type indicates it's not text; or
   ;    this is an ANSI build so the above StrGet call is unnecessary.
   ;------------------------------------------------------------------------
   
   DllCall("WinINet\InternetCloseHandle", "UPTR", hRequest)   ; close all opened
   DllCall("WinINet\InternetCloseHandle", "UPTR", hInternet)
   DllCall("WinINet\InternetCloseHandle", "UPTR", hConnect)
   DllCall("FreeLibrary", "UPTR", hModule)                    ; unload the library
   return fSize                          ; return the size - strings need update via VarSetCapacity(res,-1)
}

EDIT:
Hopefully fixed support for 64-bit version.
12. Jan 2011 - Another fix for username and password.
Using Autohotkey_L (32 bit, unicode) v. 1.0.92.02, my app chrashes when uding your function. Don't know whats the problem yet.
_________________

Redhead Technology
Back to top
View user's profile Send private message
HotKeyIt



Joined: 18 Jun 2008
Posts: 4652
Location: AHK Forum

PostPosted: Tue Feb 08, 2011 9:26 pm    Post subject: Reply with quote

Have you got an example. This seems to work fine for me:
Code:
httpQuery(result,"http://www.autohotkey.com")
VarSetCapacity(result,-1)
MsgBox % result

_________________
AHK_H (2alpha) AHF TT _Struct WatchDir Yaml _Input ObjTree RapidHotkey DynaRun Wink
Back to top
View user's profile Send private message
Joolee



Joined: 08 Aug 2005
Posts: 18

PostPosted: Wed Feb 09, 2011 7:33 pm    Post subject: Reply with quote

I was including the dll in the wrong way, I got it working when including the dll correctly.

Eventually decided to replace the httpQuery function file with
Code:
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{
   WebRequest := ComObjCreate("WinHttp.WinHttpRequest.5.1")
   WebRequest.Open("POST", lpszUrl)
   WebRequest.setRequestHeader("Content-Type", "application/x-www-form-urlencoded")
   WebRequest.Send(POSTDATA)
   Result := WebRequest.ResponseText
   WebRequest := ""
}
I'll build in headers support when I'll start using it Razz
_________________

Redhead Technology
Back to top
View user's profile Send private message
computerspazzz



Joined: 25 May 2010
Posts: 22

PostPosted: Wed Jan 25, 2012 1:02 am    Post subject: Reply with quote

Could this be converted in any way to be able to download a binary file?

I've been searching for a AHK_L compatible way to download files from http servers, need to download both text and binary files, and save them. Prefered to have an ability to monitor the download progress somehow... even if the file is just being written as its downloaded, we can monitor the file size, and the size it should be, and report the progress that way. The most important thing is just downloading the file reliably.

I had written myself a nice patcher program in the past that would download files, but not the httpQuery I was using fails to function under AHK_L.

Thanks!
_________________
Computerspazzz The Technowizard
The Wizard Is In!
Back to top
View user's profile Send private message
[VxE]



Joined: 07 Oct 2006
Posts: 3254
Location: Simi Valley, CA

PostPosted: Wed Jan 25, 2012 1:44 am    Post subject: Reply with quote

Have you seen HTTPRequest?
_________________
Ternary (a ? b : c) guide     TSV Table Manipulation Library
Post code inside [code][/code] tags!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    AutoHotkey Community Forum Index -> Ask for Help All times are GMT
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group