 |
AutoHotkey Community Let's help each other out
|
| View previous topic :: View next topic |
| Should a more generic function be released which will include the already available web functions, such as header queries, uri encoding, base encoding, etc ? |
| yes, i'd like to have one function to get all neccessary http functionalities in one instead of collecting each for my own |
|
93% |
[ 14 ] |
| no, i prefer collecting the functions i need |
|
0% |
[ 0 ] |
| neither nor ... explained in post |
|
6% |
[ 1 ] |
|
| Total Votes : 15 |
|
| Author |
Message |
DerRaphael
Joined: 23 Nov 2007 Posts: 445 Location: Heidelberg, Germany
|
Posted: Tue Jul 08, 2008 11:58 am Post subject: [function] httpQuery GET and POST requests - update 0.3.3 |
|
|
Introduction
hi all
This is a small function for handling POST and GET queries natively in AHK so external utilities to handle it are not needed any more. It can be used where no additional utilities, such as cURL or wget are wanted.
The latest version handles following features:
- ports in URL
- Username:Password@domain style URLs (see this answer to avoid common problems)
- SSL aka https
- HeaderInformations / Dumping / Storing
- DownloadSizes / CurrentSize (for download progressbars)
- Flags for InternetConnection handling (auto follow feature etc)
- Referrer for Queries
- Accepttypes for Queries
- Proxy handling
- timeout handling
- custom UserAgent
Usage
Usage is pretty simple:
You need to define a Variable which will receive the returned databuffer.
When only Strings are expected as returnvalues, autohotkey needs to learn the new size, so a VarSetCapacity(buffer,-1) is needed. This has been modified to support binary downloads aswell.
httpQUERY(buffer:="",URL) will result the length of the standard HTML which is usually retrieved. The html itself is stored in the variable buffer. The query method is GET as the length of Postparams is zero.
httpQUERY(buffer:="",URL,POSTDATA) will work as the GET above just with POSTDATA being transmitted. The query method is POST as the length of Postparams is non zero.
The URL now supports following scheme:
http://username:password@server.tld:80/path/to/target.ext?queryops#fragment
since httpQuery has been updated to use InternetCrackURL from winINet, all essential parts will be recognized. so there is no need to set up any additional parameters. Attention: When u need to authetificate in the Website the username / password attempt will not work. u have to submit those parameters via POST or GET method.
Additional Parameters:
To see a dump of the received httpHeaders, there is buildIn support for a global Variable named httpQueryOps. It may consist of one or more Verbs. For now "showHeader", "storeHeader", and "updateSize" verbs are supported. If You use storeHeader the complete Header will be saved in a variable named HttpQueryHeader which will be made global at runtime. The verb updateSize will make two variables globally Available: httpQueryFullSize and httpQueryCurrentSize. An usage example to show a download status indicator is included
Following Variables are evaluated globally:
httpAgent: can hold an individual UserAgent Code. The default is AutoHotkeyScript
httpProxy: the name of a proxy server to use. default = 0
httpProxyByPass: list of domains which will not be used with the proxy. default = 0
httpQueryReferer: an URL which is recognized by QueriedServer, as the location where the Request was generated mind though its "Referrer" the name of the variable is httpQueryReferer [sic]
httpQueryAcceptType: this variable lets u specify your accepted stream formats for the results of the query
httpQueryDwFlags: if in need for any special flags for the current connection this is the variable to set (example V shows an useCase for this feat)
I think these are pretty self explaining.
Examples
Example I: Showing the raw HTML result of a search for httpQuery in AutoHotkey's forum in an edit control
| Code: | ; exmpl.searchAHKforum.httpQuery.ahk
; Searches the forum for a given Phrase: in this case httpQuery
#noenv
html := ""
URL := "http://www.autohotkey.com/forum/search.php?mode=results"
POSTData := "search_keywords=httpQuery&search_terms=all&search_forum=-1&"
. "search_time=0&search_fields=all&show_results=topics&return_chars=500"
. "&sort_by=0&sort_dir=DESC"
length := httpQuery(html,URL,POSTdata)
varSetCapacity(html,-1)
Gui,Add,Edit,w600 +Wrap r25,% html
Gui,Show
Return
GuiClose:
GuiEscape:
ExitApp
#include httpQuery-0-3-3.ahk |
Remarks: To make a function work with POST parameters, it is essential to know which parameters are actually sent. These can be obtained by studying the HTML source of the target URL or by using analyzing tools such as the HttpFox Addon for Firefox. You might also want to use the following example to learn the nature of the form in a HTML document
Example II: Showing a dump of a HTML form to help making valid POST parameters
| Code: | ; exmpl.formdump.httpquery.ahk
; Form Dumper v0.1b (w) 9th July 2008 by derRaphael
#NoEnv
InputBox
,URL ; OutputVariable
,Enter URL to analyze ; Title of box
,Please enter the complete URL starting with http:// to be analysed ; Descriptive text
,,,,,,,,http://www.autohotkey.com/forum/search.php ; default value
html := ""
htmlLength := httpQuery(html,URL)
VarSetCapacity(html,-1)
; The Complete Form Node from given URL's HTML
RegExMatch(html,"i)<form.+?</form>",formNode)
; Just the formtag for further analyzing
RegExMatch(formNode,"i)<form[^>]+?>",formTag)
; The name of the form
RegExMatch(formTag,"i)NAME=""?(?P<Name>.+?)""?>",form)
; The method used to process Data
RegExMatch(formTag,"i)METHOD=""?(?P<Method>.+?)""?\s",form)
; The complete address used to send data to
RegExMatch(formTag,"i)ACTION=""?(?P<Action>.+?)""?\s",form)
; just the url
RegExMatch(formAction,"i)(?P<URL>[^\?]+)",formA_)
; any existing GET parameters
RegExMatch(formAction,"i)\?(?P<GET>.*)",formA_)
; Fix & to & as delimiter
StringReplace,formA_GET,formA_GET,&`;,&,All
startPosI := startPosS := startPosT := 0
inpCount := selCount := txtCount := 0
Loop,
{
If (startPosI:=RegExMatch(formNode,"i)<input[^>]+>",inputTag,startPosI+1)) {
inpCount++
formInput%InpCount% := inputTag
} else {
noMoreInput := 1
}
If (startPosS:=RegExMatch(formNode,"i)<select.*?</select>",selectNode,startPosS+1)) {
selCount++
formSelect%selCount% := selectNode
} else {
noMoreSelectOptions := 1
}
If (startPosT:=RegExMatch(formNode,"i)<textarea.*?</textarea>",textareaNode,startPosT+1)) {
txtCount++
formTextarea%txtCount% := textareaNode
} else {
noMoreTextArea := 1
}
if (NoMoreInput) && (NoMoreTextarea) && (NoMoreSelectOptions) {
break
}
}
Loop,% inpCount
Inputs .= formInput%A_Index% "`n"
StringReplace,inputs,inputs,<,%A_Tab%<,All
Loop,% selCount
Selects .= formSelect%A_Index% "`n"
StringReplace,Selects,Selects,</option>,</option>`n,All
StringReplace,Selects,Selects,<option,%A_Tab%<option,All
Selects := RegExReplace(selects,"i)<select[^>]+>","$0`n")
Loop,% txtCount
txtAreas .= formTextArea%A_Index% "`n"
StringReplace,txtAreas,txtAreas,</textarea>,</textarea>`n,All
StringReplace,txtAreas,txtAreas,<textarea,%A_Tab%<textarea,All
; dump results to Gui
Gui,Add,Tab2, w800 h20,Analyzed Content|found Form|raw HTML
Gui,Tab,1
Gui,Add,Edit,w800 h600 yp+20 xp , % ""
. "formTag:`n" formTag "`n`n"
. "formMethod:`t" formMethod "`n"
. "formAction:`t" formAction "`n"
. "formActionURL:`t" formA_URL "`n"
. "formActionGET:`t" formA_GET "`n`n"
. "Total Inputs:`t" inpCount "`n"
. inputs
. "`nTotal Selects:`t" selCount "`n"
. selects "`n"
. "`nTotal TextAreas:`t" txtCount "`n"
. txtAreas "`n"
Gui,Tab,2
Gui,Add,Edit,w800 h600 yp xp , % formNode
Gui,Tab,3
Gui,Add,Edit,w800 h600 yp xp , % html
Gui,Show,,Statistics for Form: "%formName%" (%URL%)
return
GuiClose:
GuiEscape:
ExitApp
#include httpQuery-0-3-3.ahk
|
Remarks: Using this function provides informations of which elements are used in a form which will be submitted. It neither catches modded JavaScript content nor more than one form. It has been tested with google.com, autohotkey.com's searchform and autohotkey.net's pastebin
Example III: Downloading binary data and saving it to a local file.
| Code: | ; exmpl.downloadBinary.httpQuery.ahk
; This example downloads the latest AHK environment and stores
; the received binary data to a file.
#noenv
data := ""
URL := "http://www.autohotkey.net/programs/AutoHotkey104706.zip"
httpQueryOps := "updateSize"
SetTimer,showSize,10
length := httpQuery(data,URL)
Tooltip
if (write_bin(data,"ahk.exe",length)!=1)
MsgBox "There was an Error!"
else
MsgBox AHK Source downloaded and saved as "ahk.zip"!
Return
showSize:
Tooltip,% HttpQueryCurrentSize "/" HttpQueryFullSize
return
GuiClose:
GuiEscape:
ExitApp
write_bin(byref bin,filename,size){
h := DllCall("CreateFile","str",filename,"Uint",0x40000000
,"Uint",0,"UInt",0,"UInt",4,"Uint",0,"UInt",0)
IfEqual h,-1, SetEnv, ErrorLevel, -1
IfNotEqual ErrorLevel,0,ExitApp ; couldn't create the file
r := DllCall("SetFilePointerEx","Uint",h,"Int64",0,"UInt *",p,"Int",0)
IfEqual r,0, SetEnv, ErrorLevel, -3
IfNotEqual ErrorLevel,0, {
t = %ErrorLevel% ; save ErrorLevel to be returned
DllCall("CloseHandle", "Uint", h)
ErrorLevel = %t% ; return seek error
}
result := DllCall("WriteFile","UInt",h,"Str",bin,"UInt"
,size,"UInt *",Written,"UInt",0)
h := DllCall("CloseHandle", "Uint", h)
return, 1
}
#include httpQuery-0-3-3.ahk |
Remarks: This example downloads the current Autohotkey Source and saves it to local disk as ahk.zip in the same path as the script. While it does so, it shows the current progress of the download
Example IV: Uploading an image to Imageshack using the official (free) API
| Code: | ; exmpl.imageshack.httpQuery.ahk
; This example uploads an image and constructs a multipart/form-data Type
; for fileuploading and returns the XML which is returned to show the stored Imagepath
FileSelectFile,image
FileGetSize,size,%image%
SplitPath,image,OFN
FileRead,img,%image%
VarSetCapacity(placeholder,size,32)
boundary := makeProperBoundary()
post:="--" boundary "`ncontent-disposition: form-data; name=""MAX_FILE_SIZE""`n`n"
. "1048576`n--" boundary "`ncontent-disposition: form-data; name=""xml""`n`nyes`n--"
. boundary "`ncontent-disposition: form-data; name=""fileupload""; filename="""
. ofn """`nContent-type: " MimeType(img) "`nContent-Transfer-Encoding: binary`n`n"
. placeholder "`n--" boundary "--"
headers:="Content-type: multipart/form-data, boundary=" boundary "`nContent-Length: " strlen(post)
DllCall("RtlMoveMemory","uInt",(offset:=&post+strlen(post)-strlen(Boundary)-size-5)
,"uInt",&img,"uInt",size)
size := httpQuery(result:="","http://www.imageshack.us/index.php",post,headers)
VarSetCapacity(result,-1)
Gui,Add,Edit,w800 h600, % result
Gui,Show
return
GuiClose:
GuiEscape:
ExitApp
makeProperBoundary(){
Loop,26
n .= chr(64+a_index)
n .= "0123456789"
Loop,% StrLen(A_Now) {
Random,rnd,1,% StrLen(n)
Random,UL,0,1
b .= RegExReplace(SubStr(n,rnd,1),".$","$" (round(UL)? "U":"L") "0")
}
Return b
}
MimeType(ByRef Binary) {
MimeTypes:="424d image/bmp|4749463 image/gif|ffd8ffe image/jpeg|89504e4 image/png|4657530"
. " application/x-shockwave-flash|49492a0 image/tiff"
@:="0123456789abcdef"
Loop,8
hex .= substr(@,(*(a:=&Binary-1+a_index)>>4)+1,1) substr(@,((*a)&15)+1,1)
Loop,Parse,MimeTypes,|
if ((substr(hex,1,strlen(n:=RegExReplace(A_Loopfield,"\s.*"))))=n)
Mime := RegExReplace(A_LoopField,".*?\s")
Return (Mime!="") ? Mime : "application/octet-stream"
}
#include httpQuery-0-3-3.ahk |
Remarks: Uploading binary data where an encodingtype "multipart/form-data" was needed, didnt work with classic POST usage. this example shows how to build the complete POST request in a different manner and to include raw binary data with no encoding such as base64 or ascii85
Example V: Modifying dwFlags for special useCases where a 302 is the returncode such as with dynamic created content urls
| Code: | #NoEnv
Gui,add,edit,w800 h400 vPasteBin, this is a testtext!
Gui,Add,Button,gPaste wp, Upload this text to http://pastebin.com
Gui,Show
return
Paste:
Gui,Submit,Nohide
URL := "http://pastebin.com/pastebin.php"
POSTDATA := "parent_pid=&format=text&code2=" uriEncode(PasteBin) "&"
. "poster=&paste=Send&expiry=d&email=&"
httpQueryOps := "showHeader storeHeader"
httpQueryDwFlags := (INTERNET_FLAG_NO_AUTO_REDIRECT:= 0x00200000)
length := httpQuery(HTML:="",URL,POSTDATA)
VarSetCapacity(HTML,-1)
gui,destroy
Gui,Add,Edit,w800 h600,% HttpQueryHeader "`n`n" html
Gui,Show
Return
GuiClose:
GuiEscape:
ExitApp
uriEncode(str)
{ ; v 0.3 / (w) 24.06.2008 by derRaphael / zLib-Style release
b_Format := A_FormatInteger
data := ""
SetFormat,Integer,H
Loop,Parse,str
if ((Asc(A_LoopField)>0x7f) || (Asc(A_LoopField)<0x30) || (asc(A_LoopField)=0x3d))
data .= "%" . ((StrLen(c:=SubStr(ASC(A_LoopField),3))<2) ? "0" . c : c)
Else
data .= A_LoopField
SetFormat,Integer,%b_format%
return data
}
#include httpQUERY-0-3-3.ahk |
Remarks: Some WebServices such as PasteBin.com generate Dynamic Location Urls in order to show processed content. Using httpQuery in a default manner, results in a 404 Not found error when the script tries to access the location to fast. Specifying the NO_FOLLOW dwFlag solves this problem but needs a 2nd request to grab the fresh received Location Header.
httpQuery 0.3.3
| Code: | ; httpQuery-0-3-3.ahk
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{ ; v0.3.3 (w) 27.07.2008 by Heresy & derRaphael / zLib-Style release
; currently the verbs showHeader, storeHeader, and updateSize are supported in httpQueryOps
; in case u need a different UserAgent, Proxy, ProxyByPass, Referrer, and AcceptType just
; specify them as global variables - mind the varname for referrer is httpQueryReferer [sic].
; Also if any special dwFlags are needed such as INTERNET_FLAG_NO_AUTO_REDIRECT or cache
; handling this might be set using the httpQueryDwFlags variable as global
global httpQueryOps, httpAgent, httpProxy, httpProxyByPass, httpQueryReferer, httpQueryAcceptType
, httpQueryDwFlags
; Get any missing default Values
defaultOps =
(LTrim Join|
httpAgent=AutoHotkeyScript|httpProxy=0|httpProxyByPass=0|INTERNET_FLAG_SECURE=0x00800000
SECURITY_FLAG_IGNORE_UNKNOWN_CA=0x00000100|SECURITY_FLAG_IGNORE_CERT_CN_INVALID=0x00001000
SECURITY_FLAG_IGNORE_CERT_DATE_INVALID=0x00002000|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE=0x00000200
INTERNET_OPEN_TYPE_PROXY=3|INTERNET_OPEN_TYPE_DIRECT=1|INTERNET_SERVICE_HTTP=3
)
Loop,Parse,defaultOps,|
{
RegExMatch(A_LoopField,"(?P<Option>[^=]+)=(?P<Default>.*)",http)
if StrLen(%httpOption%)=0
%httpOption% := httpDefault
}
; Load Library
hModule := DllCall("LoadLibrary", "Str", "WinINet.Dll")
; SetUpStructures for URL_COMPONENTS / needed for InternetCrackURL
; http://msdn.microsoft.com/en-us/library/aa385420(VS.85).aspx
offset_name_length:= "4-lpszScheme-255|16-lpszHostName-1024|28-lpszUserName-1024|"
. "36-lpszPassword-1024|44-lpszUrlPath-1024|52-lpszExtrainfo-1024"
VarSetCapacity(URL_COMPONENTS,60,0)
; Struc Size ; Scheme Size ; Max Port Number
NumPut(60,URL_COMPONENTS,0), NumPut(255,URL_COMPONENTS,12), NumPut(0xffff,URL_COMPONENTS,24)
Loop,Parse,offset_name_length,|
{
RegExMatch(A_LoopField,"(?P<Offset>\d+)-(?P<Name>[a-zA-Z]+)-(?P<Size>\d+)",iCU_)
VarSetCapacity(%iCU_Name%,iCU_Size,0)
NumPut(&%iCU_Name%,URL_COMPONENTS,iCU_Offset)
NumPut(iCU_Size,URL_COMPONENTS,iCU_Offset+4)
}
; Split the given URL; extract scheme, user, pass, authotity (host), port, path, and query (extrainfo)
; http://msdn.microsoft.com/en-us/library/aa384376(VS.85).aspx
DllCall("WinINet\InternetCrackUrlA","Str",lpszUrl,"uInt",StrLen(lpszUrl),"uInt",0,"uInt",&URL_COMPONENTS)
; Update variables to retrieve results
Loop,Parse,offset_name_length,|
{
RegExMatch(A_LoopField,"-(?P<Name>[a-zA-Z]+)-",iCU_)
VarSetCapacity(%iCU_Name%,-1)
}
nPort:=NumGet(URL_COMPONENTS,24,"uInt")
; Import any set dwFlags
dwFlags := httpQueryDwFlags
; For some reasons using a selfsigned https certificates doesnt work
; such as an own webmin service - even though every security is turned off
; https with valid certificates works when
if (lpszScheme = "https")
dwFlags |= (INTERNET_FLAG_SECURE|SECURITY_FLAG_IGNORE_CERT_CN_INVALID
|SECURITY_FLAG_IGNORE_CERT_DATE_INVALID|SECURITY_FLAG_IGNORE_UNKNOWN_CA
|SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE)
; Check for Header and drop exception if unknown or invalid URL
if (lpszScheme="unknown") {
Result := "ERR: No Valid URL supplied."
Return StrLen(Result)
}
; Initialise httpQuery's use of the WinINet functions.
; http://msdn.microsoft.com/en-us/library/aa385096(VS.85).aspx
hInternet := DllCall("WinINet\InternetOpenA"
,"Str",httpAgent,"UInt"
,(httpProxy != 0 ? INTERNET_OPEN_TYPE_PROXY : INTERNET_OPEN_TYPE_DIRECT )
,"Str",httpProxy,"Str",httpProxyBypass,"Uint",0)
; Open HTTP session for the given URL
; http://msdn.microsoft.com/en-us/library/aa384363(VS.85).aspx
hConnect := DllCall("WinINet\InternetConnectA"
,"uInt",hInternet,"Str",lpszHostname, "Int",nPort
,"Str",lpszUserName, "Str",lpszPassword,"uInt",INTERNET_SERVICE_HTTP
,"uInt",0,"uInt*",0)
; Do we POST? If so, check for header handling and set default
if (Strlen(POSTDATA)>0) {
HTTPVerb:="POST"
if StrLen(Headers)=0
Headers:="Content-Type: application/x-www-form-urlencoded"
} else ; otherwise mode must be GET - no header defaults needed
HTTPVerb:="GET"
; Form the request with proper HTTP protocol version and create the request handle
; http://msdn.microsoft.com/en-us/library/aa384233(VS.85).aspx
hRequest := DllCall("WinINet\HttpOpenRequestA"
,"uInt",hConnect,"Str",HTTPVerb,"Str",lpszUrlPath . lpszExtrainfo
,"Str",ProVer := "HTTP/1.1", "Str",httpQueryReferer,"Str",httpQueryAcceptTypes
,"uInt",dwFlags,"uInt",Context:=0 )
; Send the specified request to the server
; http://msdn.microsoft.com/en-us/library/aa384247(VS.85).aspx
sRequest := DllCall("WinINet\HttpSendRequestA"
, "uInt",hRequest,"Str",Headers, "uInt",Strlen(Headers)
, "Str",POSTData,"uInt",Strlen(POSTData))
VarSetCapacity(header, 2048, 0) ; max 2K header data for httpResponseHeader
VarSetCapacity(header_len, 4, 0)
; Check for returned server response-header (works only _after_ request been sent)
; http://msdn.microsoft.com/en-us/library/aa384238.aspx
Loop, 5
if ((headerRequest:=DllCall("WinINet\HttpQueryInfoA","uint",hRequest
,"uint",21,"uint",&header,"uint",&header_len,"uint",0))=1)
break
If (headerRequest=1) {
VarSetCapacity(res,headerLength:=NumGet(header_len),32)
DllCall("RtlMoveMemory","uInt",&res,"uInt",&header,"uInt",headerLength)
Loop,% headerLength
if (*(&res-1+a_index)=0) ; Change binary zero to linefeed
NumPut(Asc("`n"),res,a_index-1,"uChar")
VarSetCapacity(res,-1)
} else
res := "timeout"
; Get 1st Line of Full Response
Loop,Parse,res,`n,`r
{
RetValue := A_LoopField
break
}
; No Connection established - drop exception
If (RetValue="timeout") {
html := "Error: timeout"
return -1
}
; Strip protocol version from return value
RetValue := RegExReplace(RetValue,"\Q" ProVer "\E\s+")
; List taken from http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
HttpRetCodes := "100=Continue|101=Switching Protocols|102=Processing (WebDAV) (RFC 2518)|"
. "200=OK|201=Created|202=Accepted|203=Non-Authoritative Information|204=No"
. " Content|205=Reset Content|206=Partial Content|207=Multi-Status (WebDAV)"
. "|300=Multiple Choices|301=Moved Permanently|302=Found|303=See Other|304="
. "Not Modified|305=Use Proxy|306=Switch Proxy|307=Temporary Redirect|400=B"
. "ad Request|401=Unauthorized|402=Payment Required|403=Forbidden|404=Not F"
. "ound|405=Method Not Allowed|406=Not Acceptable|407=Proxy Authentication "
. "Required|408=Request Timeout|409=Conflict|410=Gone|411=Length Required|4"
. "12=Precondition Failed|413=Request Entity Too Large|414=Request-URI Too "
. "Long|415=Unsupported Media Type|416=Requested Range Not Satisfiable|417="
. "Expectation Failed|418=I'm a teapot (RFC 2324)|422=Unprocessable Entity "
. "(WebDAV) (RFC 4918)|423=Locked (WebDAV) (RFC 4918)|424=Failed Dependency"
. " (WebDAV) (RFC 4918)|425=Unordered Collection (RFC 3648)|426=Upgrade Req"
. "uired (RFC 2817)|449=Retry With|500=Internal Server Error|501=Not Implem"
. "ented|502=Bad Gateway|503=Service Unavailable|504=Gateway Timeout|505=HT"
. "TP Version Not Supported|506=Variant Also Negotiates (RFC 2295)|507=Insu"
. "fficient Storage (WebDAV) (RFC 4918)|509=Bandwidth Limit Exceeded|510=No"
. "t Extended (RFC 2774)"
; Gather numeric response value
RetValue := SubStr(RetValue,1,3)
; Parse through return codes and set according informations
Loop,Parse,HttpRetCodes,|
{
HttpReturnCode := SubStr(A_LoopField,1,3) ; Numeric return value see above
HttpReturnMsg := SubStr(A_LoopField,5) ; link for additional information
if (RetValue=HttpReturnCode) {
RetMsg := HttpReturnMsg
break
}
}
; Global HttpQueryOps handling
if strlen(HTTPQueryOps)>0 {
; Show full Header response (usefull for debugging)
if (instr(HTTPQueryOps,"showHeader"))
MsgBox % res
; Save the full Header response in a global Variable
if (instr(HTTPQueryOps,"storeHeader"))
global HttpQueryHeader := res
; Check for size updates to export to a global Var
if (instr(HTTPQueryOps,"updateSize")) {
Loop,Parse,res,`n
If RegExMatch(A_LoopField,"Content-Length:\s+?(?P<Size>\d+)",full) {
global HttpQueryFullSize := fullSize
break
}
if (fullSize+0=0)
HttpQueryFullSize := "size unavailable"
}
}
; Check for valid codes and drop exception if suspicious
if !(InStr("100 200 201 202 302",RetValue)) {
Result := RetValue " " RetMsg
return StrLen(Result)
}
VarSetCapacity(BytesRead,4,0)
fsize := 0
Loop ; the receiver loop - rewritten in the need to enable
{ ; support for larger file downloads
bc := A_Index
VarSetCapacity(buffer%bc%,1024,0) ; setup new chunk for this receive round
ReadFile := DllCall("wininet\InternetReadFile"
,"uInt",hRequest,"uInt",&buffer%bc%,"uInt",1024,"uInt",&BytesRead)
ReadBytes := NumGet(BytesRead) ; how many bytes were received?
If ((ReadFile!=0)&&(!ReadBytes)) ; we have had no error yet and received no more bytes
break ; we must be done! so lets break the receiver loop
Else {
fsize += ReadBytes ; sum up all chunk sizes for correct return size
sizeArray .= ReadBytes "|"
}
if (instr(HTTPQueryOps,"updateSize"))
Global HttpQueryCurrentSize := fsize
}
sizeArray := SubStr(sizeArray,1,-1) ; trim last PipeChar
VarSetCapacity(result,fSize) ; reconstruct the result from above generated chunkblocks
Dest := &result ; to a our ByRef result variable
Loop,Parse,SizeArray,|
DllCall("RtlMoveMemory","uInt",Dest,"uInt",&buffer%A_Index%,"uInt",A_LoopField)
, Dest += A_LoopField
DllCall("WinINet\InternetCloseHandle", "uInt", hRequest) ; close all opened
DllCall("WinINet\InternetCloseHandle", "uInt", hInternet)
DllCall("WinINet\InternetCloseHandle", "uInt", hConnect)
DllCall("FreeLibrary", "UInt", hModule) ; unload the library
return fSize ; return the size - strings need update via VarSetCapacity(res,-1)
} |
Additional Informations / Needs to be done
- For now the SSL functionality (aka https or secured communication) works but just when using trusted SSL servers. Using servers with self-signed certificates such as WebMin wont work and needs further research.
Version History
v0.1 - Initial Version (Tue Jul 08, 2008)
v0.1b - Fixed Typo, thx (Thu Jul 10, 2008)
v0.3.1 - added Features such as InternetCrackURL, SSL, header Return Values, changed function call.
v0.3.2 - added new httpQueryOps Verb to update full and current download Size
v0.3.3 - added new global Vars to handle 302 answers properly by using custom dwFlags (thx skrommel for pointing this out).
Remarks: Thanks to Heresy who helped a big deal in collecting all needed winINet calls to make httpPOST happen, and to ahklerner, who collected the neccessary SSL parameters in his version of this script.
Also a big thanks to all who took their time to review this lil function
greets
DerRaphael _________________
Last edited by DerRaphael on Sun Jul 27, 2008 1:16 pm; edited 9 times in total |
|
| Back to top |
|
 |
Guest
|
Posted: Tue Jul 08, 2008 12:06 pm Post subject: |
|
|
| Thanks god we have native http function now |
|
| Back to top |
|
 |
Joy2DWorld
Joined: 04 Dec 2006 Posts: 422 Location: Galil, Israel
|
Posted: Tue Jul 08, 2008 3:41 pm Post subject: |
|
|
clean!
the IF structure
| Code: | if ((InStr(Server,"http://")) or (InStr(Server,"ftp://"))) {
Protocoll := (RegExMatch(Server,"(^http://|^ftp://)",pr)) ? pr1 : "unknown"
} |
seems redundant...
(although have not tested time to do two INSTR vs. one regexmatch)
and would suggested | Code: | if !instr(url,":")
url = http://%url% | or such.
cf. http://www.autohotkey.com/forum/topic10393.html
and http://www.autohotkey.com/forum/viewtopic.php?t=10466&start=30 (which Sean, using COM, shows how to have url request work as 'callback' so you don't have to wait for page//script continues even if download does not!) _________________ Joyce Jamce
Last edited by Joy2DWorld on Wed Jul 09, 2008 1:33 am; edited 1 time in total |
|
| Back to top |
|
 |
engunneer
Joined: 30 Aug 2005 Posts: 6772 Location: Pacific Northwest, US
|
|
| Back to top |
|
 |
Joy2DWorld
Joined: 04 Dec 2006 Posts: 422 Location: Galil, Israel
|
Posted: Tue Jul 08, 2008 6:51 pm Post subject: |
|
|
and the "IF" structure is needed for ? _________________ Joyce Jamce |
|
| Back to top |
|
 |
neXt
Joined: 19 Mar 2007 Posts: 461
|
Posted: Wed Jul 09, 2008 12:29 am Post subject: |
|
|
so this function can be used in auto login scripts? Can someone post an example if yes please.
P.S. Nice work DerRaphael! _________________ simplified csv - easy way to handle csv files. |
|
| Back to top |
|
 |
heresy
Joined: 11 Mar 2008 Posts: 269
|
Posted: Wed Jul 09, 2008 10:28 am Post subject: |
|
|
random xkcd comics getter
LButton : Drag GUI
LButton Double Click : change image
RButton : Exitapp
| Code: | ;WinXP+ & needs HttpQuery()
GetPic()
OnMessage(0x203, "GetPic")
OnMessage(0x200, "ShowAlt")
Return
GuiContextMenu:
Exitapp
Move:
PostMessage,0xA1,2,,,A
Return
GetPic(){
WinGetPos, X, Y,,, RandXKCD
Gui, Destroy
HttpQuery(raw, "http://dynamic.xkcd.com/comic/random/")
VarSetCapacity(raw, -1)
RegExMatch(raw,"Sm)<img src=""(?P<pic>http://imgs.xkcd.com/comics.+?)"" title=""(?P<title>.+?)"" alt=(?P<alt>"".+?)"" /><br/>", _)
Passer(_Alt,_Title)
SplitPath, _pic, PicFile
picloc := A_Temp "\" PicFile
If !FileExist(picloc)
URLDownloadToFile, %_pic%, %picloc%
Gui, Margin, 0,0
Gui, +ToolWindow -Caption
Gui, Add, Picture, gMove h-1, %picloc%
Gui, Show, NA AutoSize x%x% y%y%, RandXKCD
}
ShowAlt(){
Global a, b
If A_Gui=1
ToolTip, %a% "`n" %b%
Else
ToolTip
}
Passer(Byref _Alt, Byref _Title){
Global a:=_Alt, b:=_Title
} |
_________________ Easy WinAPI - Dive into Windows API World
Benchmark your AutoHotkey skills at PlayAHK.com
Last edited by heresy on Fri Jul 18, 2008 5:53 pm; edited 4 times in total |
|
| Back to top |
|
 |
DerRaphael
Joined: 23 Nov 2007 Posts: 445 Location: Heidelberg, Germany
|
Posted: Wed Jul 09, 2008 10:50 am Post subject: |
|
|
updated example section for a form dumper
derRaphael |
|
| Back to top |
|
 |
n-l-i-d Guest
|
Posted: Wed Jul 09, 2008 12:02 pm Post subject: |
|
|
Nice!
But isn't there a typo in your code?
| Code: | | httpQUERY(URL, POSTDATA="", AGENT="AutohotkeyScript", HTTP_POST=80) |
| Code: | | , "uInt",hInternet,"Str",ServerName, "Int",HTTP_PORT |
|
|
| Back to top |
|
 |
heresy
Joined: 11 Mar 2008 Posts: 269
|
Posted: Wed Jul 09, 2008 4:21 pm Post subject: |
|
|
Getting first hit from google search.
looping the regex part will get the whole result
| Code: | MsgBox % Google("AutoHotkey")
Google(Query){
Fp=1
StringReplace, Query, Query, ", `%22, All
StringReplace, Query, Query,%A_Space%,+, All
URL := "http://www.google.com/search?q=" . Query
httpQUERY(raw, URL)
VarSetCapacity(raw, -1)
StringReplace, Raw, Raw,<b>,, All
StringReplace, Raw, Raw,</b>,, All
Fp := RegExMatch(Raw,"Sm)<div class=g><a href=""(?P<Link>.+?)""", _, Fp)
Fp := RegExMatch(Raw,"Sm)class=l>(?P<Title>.+?)</a>", _, Fp)
Fp := RegExMatch(Raw,"Sm)<div class=std>(?P<description>.+?)<br><span", _, Fp)
Return _Link "`n" _Title "`n" _Description
} |
_________________ Easy WinAPI - Dive into Windows API World
Benchmark your AutoHotkey skills at PlayAHK.com
Last edited by heresy on Fri Jul 18, 2008 5:56 pm; edited 1 time in total |
|
| Back to top |
|
 |
tank
Joined: 21 Dec 2007 Posts: 616
|
Posted: Wed Jul 09, 2008 9:33 pm Post subject: |
|
|
Very interesting DerRaphael
very very interesting _________________ Read this
Com
Automate IE7 with Tabs |
|
| Back to top |
|
 |
tank
Joined: 21 Dec 2007 Posts: 616
|
Posted: Wed Jul 09, 2008 9:43 pm Post subject: |
|
|
wow been toying a bit here might end up canning my own library this is impressive _________________ Read this
Com
Automate IE7 with Tabs |
|
| Back to top |
|
 |
DerRaphael
Joined: 23 Nov 2007 Posts: 445 Location: Heidelberg, Germany
|
Posted: Thu Jul 10, 2008 3:25 am Post subject: |
|
|
fixed typo, thx n-l-i-d _________________
|
|
| Back to top |
|
 |
ahklerner
Joined: 26 Jun 2006 Posts: 1202 Location: USA
|
Posted: Thu Jul 10, 2008 5:46 pm Post subject: |
|
|
I wrapped the WinInet.dll functions to be StdLib compatible. It can be found here:
http://www.autohotkey.com/forum/viewtopic.php?t=33561
The WININET lib includes the functionality above as well as the FTP functions that were spread across the forum. This allows a single include that can be utilized for HTTP and FTP communication. _________________
 |
|
| Back to top |
|
 |
bigsalgo911
Joined: 02 Jul 2008 Posts: 21
|
Posted: Sat Jul 12, 2008 2:37 am Post subject: |
|
|
So this works great...however. I am having a problem getting this to work with any variables in the POSTdata field.
Here is the code I'm using:
| Code: | URL := "http://ecase.xxx.yyy.net/login.asp"
POSTdata := "logonid=%login%&password=%pass%"
httpQUERY(URL,POSTDATA) |
the variables are defined earlier. I have tried many combinations of using and not using quotes and I cannot figure it out. It works perfectly when I enter my login information, but it can't seem to read the variable. What am I doing wrong? |
|
| Back to top |
|
 |
|
|
You can post new topics in this forum You can reply to topics in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|