download urls to vars, partially/fully, via WinHttpRequest Topic is solved
download urls to vars, partially/fully, via WinHttpRequest
tl;dr
- download file (binary) to variable (via object)
- download file (binary) to variable (via object) (initial n bytes only)
I know that UrlDownloadToFile uses
InternetReadFile to download webpages,
and that with this you can download
files (binary data) and webpages to variables, and if desired,
only the start of the webpage.
One of the reasons to download only
the start of webpage, is to retrieve
the webpage titles for a list of urls.
Is it possible to download all or only the start
of a file (binary data) or webpage via WinHttpRequest,
to a variable?
Note: WinHttpRequest is preferable to
UrlDownloadToFile/InternetReadFile
because its faster.
[EDIT: 100th post! That happened quite quickly.]
- download file (binary) to variable (via object)
- download file (binary) to variable (via object) (initial n bytes only)
I know that UrlDownloadToFile uses
InternetReadFile to download webpages,
and that with this you can download
files (binary data) and webpages to variables, and if desired,
only the start of the webpage.
One of the reasons to download only
the start of webpage, is to retrieve
the webpage titles for a list of urls.
Is it possible to download all or only the start
of a file (binary data) or webpage via WinHttpRequest,
to a variable?
Note: WinHttpRequest is preferable to
UrlDownloadToFile/InternetReadFile
because its faster.
[EDIT: 100th post! That happened quite quickly.]
Last edited by jeeswg on 01 Sep 2017, 20:10, edited 3 times in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fullly, via WinHttpRequest
you mean , you search for url's ?
Code: Select all
;- idea search for begin=http and end=extension
;- laatste nieuws NL
modified=20150510
f1=http://nos.nl/uitzending/nos-journaal
httpQuery(aacc,f1)
A:="http" ;- begin
Extensions1 := "mp3,m4a,wav,wma,flv,vob,mp4,mpg,wmv,avi" ;- end
T=
Loop,parse,aacc,`n,`r
{
stringlen,L1,A
T=%A_LoopField%
;Loop Parse,t,`,` `"`>`=`;?`!`<
Loop Parse,t,`" ;- split for `"
{
StringLeft r,A_LoopField,L1
If (r=A)
{
SplitPath,A_LoopField, name, dir, ext, name_no_ext, drive
if ext in %Extensions1%
{
e .= A_LoopField "`n"
Last=%A_LoopField%
}
}
}
}
;------------------------------------
stringsplit,h,e,`n
msgbox,%e%`n`nH1=%h1%`nH2=%h2%`nH3=%h3%`nLAST=`n%last%
e=
return
;----------------------------
httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{
WebRequest := ComObjCreate("WinHttp.WinHttpRequest.5.1")
WebRequest.Open("GET", lpszUrl)
WebRequest.setRequestHeader("Content-Type", "application/x-www-form-urlencoded")
WebRequest.Send(POSTDATA)
Result := WebRequest.ResponseText
WebRequest := ""
}
return
;============================================================================================
Re: download urls to vars, partially/fullly, via WinHttpRequest
My two common scenarios are:
- The url is a webpage (e.g. http://ahkscript.org/),
I want to download the html as UrlDownloadToFile does,
but only the start of it, in order to retrieve the webpage title.
- The url is an image (e.g. http://ahkscript.org/static/ahk_logo.png),
I want to download the binary data for the image to a variable,
but not save it as a file. (I compare the data against an existing file,
and only if they differ do I then write the binary to a file.
E.g. YouTube thumbnails, same url, but image data can change.)
E.g. this downloads a file and saves it as a file.
Can it be amended to store it in a variable instead.
Also can it be set to download only the first n bytes.
- The url is a webpage (e.g. http://ahkscript.org/),
I want to download the html as UrlDownloadToFile does,
but only the start of it, in order to retrieve the webpage title.
- The url is an image (e.g. http://ahkscript.org/static/ahk_logo.png),
I want to download the binary data for the image to a variable,
but not save it as a file. (I compare the data against an existing file,
and only if they differ do I then write the binary to a file.
E.g. YouTube thumbnails, same url, but image data can change.)
E.g. this downloads a file and saves it as a file.
Can it be amended to store it in a variable instead.
Also can it be set to download only the first n bytes.
Code: Select all
JEE_UrlDownloadToFile(vUrl, vPath)
{
vOverwrite := True
oHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
oHTTP.Open("GET", vUrl)
oHTTP.Send()
oADODB := ComObjCreate("ADODB.Stream")
oADODB.Type := 1
oADODB.Open()
oADODB.Write(oHTTP.ResponseBody)
oADODB.SaveToFile(vPath, vOverwrite ? 2:1)
oADODB.Close()
oHTTP := ""
oADODB := ""
Return
}
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fullly, via WinHttpRequest
Please if anyone can help,
I've been going round in circles trying to use
ComObjCreate("WinHttp.WinHttpRequest.5.1")
and ComObjCreate("ADODB.Stream")
to download a file e.g. an image,
and put the data from ResponseBody
into a variable.
(Btw I'm trying to get binary data from ResponseBody,
and not text from ResponseText.)
(I show an example for downloading binary to a file,
above, but I want to download it to a variable.)
Thank you.
Some relevant links I found:
Stream Object (ADO) | Microsoft Docs
https://docs.microsoft.com/en-us/sql/ad ... object-ado
Read Method | Microsoft Docs
https://docs.microsoft.com/en-us/sql/ad ... ead-method
Convert Byte Array to String - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=9807
Help with HTTPRequest and BinRead please - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=5716
I've been going round in circles trying to use
ComObjCreate("WinHttp.WinHttpRequest.5.1")
and ComObjCreate("ADODB.Stream")
to download a file e.g. an image,
and put the data from ResponseBody
into a variable.
(Btw I'm trying to get binary data from ResponseBody,
and not text from ResponseText.)
(I show an example for downloading binary to a file,
above, but I want to download it to a variable.)
Thank you.
Some relevant links I found:
Stream Object (ADO) | Microsoft Docs
https://docs.microsoft.com/en-us/sql/ad ... object-ado
Read Method | Microsoft Docs
https://docs.microsoft.com/en-us/sql/ad ... ead-method
Convert Byte Array to String - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=9807
Help with HTTPRequest and BinRead please - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=5716
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fullly, via WinHttpRequest
[AHK] v2.0.5 | [WIN] 11 Pro (Version 22H2) | [GitHub] Profile
Re: download urls to vars, partially/fullly, via WinHttpRequest
Thank you jNizM, excellent functions btw.
I have functions already that use InternetReadFile,
but I'm trying to investigate this object method which appears to download much faster
(and to improve my knowledge of objects for doing AHK v1/v2 script conversions,
and possibly write a tutorial on objects).
However, I don't know if the object method allows a partial download of a file
which is something that the InternetReadFile method can do.
Appreciated.
I have functions already that use InternetReadFile,
but I'm trying to investigate this object method which appears to download much faster
(and to improve my knowledge of objects for doing AHK v1/v2 script conversions,
and possibly write a tutorial on objects).
However, I don't know if the object method allows a partial download of a file
which is something that the InternetReadFile method can do.
Appreciated.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fullly, via WinHttpRequest
[AHK] v2.0.5 | [WIN] 11 Pro (Version 22H2) | [GitHub] Profile
Re: download urls to vars, partially/fullly, via WinHttpRequest
Thanks, that's the frustrating thing, the method is fairly clear, from various sources,
it's actually the mechanics of getting binary data out of ResponseBody,
into a variable, possibly via an object, that is the problem.
To know the specifics of how to move the data within AutoHotkey.
To work out the lines I need to modify in JEE_UrlDownloadToFile above.
I've read things like VARIANT and binary array etc.
it's actually the mechanics of getting binary data out of ResponseBody,
into a variable, possibly via an object, that is the problem.
To know the specifics of how to move the data within AutoHotkey.
To work out the lines I need to modify in JEE_UrlDownloadToFile above.
I've read things like VARIANT and binary array etc.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fullly, via WinHttpRequest
Waitasec it might be here I'll just check.
[solved] UrlDownloadToFile slows down GUI response - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=9338
Btw is there a search engine that would let you search for:
AutoHotkey ADODB Read(
Specifically, to handle the bracket.
[EDIT:]
I found a good result when I searched for:
AutoHotkey "ADODB.Stream" ResponseBody
[solved] UrlDownloadToFile slows down GUI response - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=9338
Btw is there a search engine that would let you search for:
AutoHotkey ADODB Read(
Specifically, to handle the bracket.
[EDIT:]
I found a good result when I searched for:
AutoHotkey "ADODB.Stream" ResponseBody
Last edited by jeeswg on 22 Feb 2017, 03:37, edited 1 time in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fullly, via WinHttpRequest Topic is solved
This looks like it works:
- Btw is memcpy the best dll function for memory move?
- Also, it's a C function, can it handle Ptr? It appeared to work on 64-bit when I tested it.
- Also can 'Ptr,&vIndex' be replaced with 'PtrP,0' or something similar?
[EDIT:]
- I don't know if there is a way to specify to only download the first n bytes of a file.
- I've had problems when trying to use ResponseText from webpages, to get a webpage's title,
so I can now do it by using the binary ResponseBody and converting that to text,
like I did with InternetReadFile.
[EDIT 2:]
Btw I've seen a fair number of AHK libraries with functions for WinHttpRequest,
so feel free to copy this function, changing it as much or as little as you want, to match your other functions, if you would like to add this in to your library.
Code: Select all
JEE_UrlDownloadToVar(vUrl, ByRef vData, ByRef vSize)
{
oHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
oHTTP.Open("GET", vUrl)
oHTTP.Send()
oHTTP.WaitForResponse()
;vSize := oHTTP.ResponseBody.MaxIndex()
vSize := IsObject(oHTTP.ResponseBody) ? 1+oHTTP.ResponseBody.MaxIndex() : 0
VarSetCapacity(vData, vSize, 0)
VarSetCapacity(vIndex, 4, 0)
if !DllCall("OleAut32.dll\SafeArrayPtrOfIndex", Ptr,ComObjValue(oHTTP.ResponseBody), Ptr,&vIndex, PtrP,vPtrOfIndex)
DllCall("msvcrt\memcpy", UPtr,&vData, UPtr,vPtrOfIndex, UPtr,vSize)
oHTTP := ""
return
}
- Also, it's a C function, can it handle Ptr? It appeared to work on 64-bit when I tested it.
- Also can 'Ptr,&vIndex' be replaced with 'PtrP,0' or something similar?
[EDIT:]
- I don't know if there is a way to specify to only download the first n bytes of a file.
- I've had problems when trying to use ResponseText from webpages, to get a webpage's title,
so I can now do it by using the binary ResponseBody and converting that to text,
like I did with InternetReadFile.
[EDIT 2:]
Btw I've seen a fair number of AHK libraries with functions for WinHttpRequest,
so feel free to copy this function, changing it as much or as little as you want, to match your other functions, if you would like to add this in to your library.
Last edited by jeeswg on 28 Jul 2018, 16:56, edited 3 times in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fullly, via WinHttpRequest
You can use Range header. However not all sites support this.jeeswg wrote: I don't know if there is a way to specify to only download the first n bytes of a file.
Example:
Code: Select all
whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", "http://www.nirsoft.net/utils/nircmd.zip", true)
whr.SetRequestHeader("Range", "bytes=0-10")
whr.Send()
whr.WaitForResponse()
MsgBox % whr.ResponseBody.MaxIndex()
MsgBox % whr.Status " " whr.StatusText "`n`n" whr.GetAllResponseHeaders()
Re: download urls to vars, partially/fullly, via WinHttpRequest
Thanks so much, that's a great response.
It's not the end of the world if a site doesn't support it,
but it's a great help. One thing I like to do is
download the start to get the webpage title.
[EDIT:] Haha good call with the util url btw.
It's not the end of the world if a site doesn't support it,
but it's a great help. One thing I like to do is
download the start to get the webpage title.
[EDIT:] Haha good call with the util url btw.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fully, via WinHttpRequest
- download htm to variable
- download file to variable e.g. image/icon
- download htm to variable (partial) - get webpage title
- download file to variable e.g. image/icon
- download htm to variable (partial) - get webpage title
Code: Select all
;q:: ;download htm to variable
vUrl := "https://autohotkey.com/download/"
JEE_UrlDownloadToVar(vUrl, vData, vSize)
MsgBox, % Clipboard := StrGet(&vData, vSize, "CP0")
return
;==================================================
;[Gdip functions]
;GDI+ standard library 1.45 by tic - AutoHotkey Community
;https://autohotkey.com/boards/viewtopic.php?f=6&t=6517
;q:: ;download file to variable e.g. image/icon
pToken := Gdip_Startup()
Loop, 2
{
vUrl1 := "https://autohotkey.com/static/ahk_logo.png"
vUrl2 := "https://autohotkey.com/favicon.ico"
vUrl := vUrl%A_Index%
JEE_UrlDownloadToVar(vUrl, vData, vSize)
hData := DllCall("GlobalAlloc", UInt,2, UPtr,vSize, Ptr)
pData := DllCall("GlobalLock", Ptr,hData, Ptr)
DllCall("RtlMoveMemory", Ptr,pData, Ptr,&vData, UPtr,vSize)
DllCall("GlobalUnlock", Ptr,hData)
DllCall("ole32\CreateStreamOnHGlobal", Ptr,hData, Int,1, UIntP,pStream)
DllCall("gdiplus\GdipCreateBitmapFromStream", Ptr,pStream, UIntP,pBitmap)
hBitmap := Gdip_CreateHBITMAPFromBitmap(pBitmap)
SplashImage, % "HBITMAP:" hBitmap, B ;B: borderless
Sleep 2000
SplashImage, Off
DeleteObject(hBitmap)
Gdip_DisposeImage(pBitmap)
}
Gdip_Shutdown(pToken)
return
;==================================================
;q:: ;download htm to variable (partial) - get webpage title
;vUrl := "https://autohotkey.com/download/"
vUrl := "https://en.wikipedia.org/wiki/AutoHotkey"
JEE_UrlDownloadToVar(vUrl, vData, vSize)
oHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
oHTTP.Open("GET", vUrl)
oHTTP.SetRequestHeader("Range", "bytes=0-1000")
oHTTP.Send()
oHTTP.WaitForResponse()
;vSize := oHTTP.ResponseBody.MaxIndex()
vSize := IsObject(oHTTP.ResponseBody) ? 1+oHTTP.ResponseBody.MaxIndex() : 0
VarSetCapacity(vData, vSize, 0)
VarSetCapacity(vIndex, 4, 0)
if !DllCall("oleaut32\SafeArrayPtrOfIndex", Ptr,ComObjValue(oHTTP.ResponseBody), Ptr,&vIndex, PtrP,vPtrOfIndex)
DllCall("kernel32\RtlMoveMemory", Ptr,&vData, Ptr,vPtrOfIndex, UPtr,vSize)
MsgBox, % oHTTP.Status " " oHTTP.StatusText
MsgBox, % oHTTP.GetAllResponseHeaders()
oHTTP := ""
vText := StrGet(&vData, vSize, "CP0")
;MsgBox, % vText
if !(vPos1 := InStr(vText, "<title>"))
|| !(vPos2 := InStr(vText, "</title>", 0, vPos1+7))
return
MsgBox, % SubStr(vText, vPos1+7, vPos2-vPos1-7)
return
;==================================================
JEE_UrlDownloadToVar(vUrl, ByRef vData, ByRef vSize)
{
oHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
oHTTP.Open("GET", vUrl)
oHTTP.Send()
oHTTP.WaitForResponse()
;vSize := oHTTP.ResponseBody.MaxIndex()
vSize := IsObject(oHTTP.ResponseBody) ? 1+oHTTP.ResponseBody.MaxIndex() : 0
VarSetCapacity(vData, vSize, 0)
VarSetCapacity(vIndex, 4, 0)
if !DllCall("oleaut32\SafeArrayPtrOfIndex", Ptr,ComObjValue(oHTTP.ResponseBody), Ptr,&vIndex, PtrP,vPtrOfIndex)
DllCall("kernel32\RtlMoveMemory", Ptr,&vData, Ptr,vPtrOfIndex, UPtr,vSize)
oHTTP := ""
}
;==================================================
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Re: download urls to vars, partially/fully, via WinHttpRequest
- I've made an important change to the scripts above:
The scripts were missing the final byte when downloading files.
- For this:GeekDude mentions this shorter alternative, here:
GeekDude's Tips, Tricks, and Standalones - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=74&t=7190
Code: Select all
;before:
vSize := oHTTP.ResponseBody.MaxIndex()
;after:
vSize := IsObject(oHTTP.ResponseBody) ? 1+oHTTP.ResponseBody.MaxIndex() : 0
- For this:
Code: Select all
VarSetCapacity(vIndex, 4, 0)
DllCall("oleaut32\SafeArrayPtrOfIndex", Ptr,ComObjValue(oHTTP.ResponseBody), Ptr,&vIndex, PtrP,pData)
GeekDude's Tips, Tricks, and Standalones - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=74&t=7190
Code: Select all
pData := NumGet(ComObjValue(oHTTP.ResponseBody)+8+A_PtrSize)
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
- CheshireCat
- Posts: 11
- Joined: 07 Mar 2016, 15:58
Re: download urls to vars, partially/fully, via WinHttpRequest
Hi all, I've been trying to figure this out for days and I just cant seem to get it.
I'm trying to download a png / jpg using WinHttpRequest. I don't want to save it as an image file, instead I want to pass it to a base64 function, encode it, write and save the resulting encoding to a text file.
I found a function that does that in the old board - DownloadBin(), the result of which I then put into a b64enc() function, but the DownloadBin function uses the "wininet" dll calls rather than the WinHttpRequest object.
The reason I am asking mainly is because depending on the size of the image I download, the DownloadBin function seems to take a while to process.
I was wondering if anyone can lend a hand, or make any suggestions.
I'm trying to download a png / jpg using WinHttpRequest. I don't want to save it as an image file, instead I want to pass it to a base64 function, encode it, write and save the resulting encoding to a text file.
I found a function that does that in the old board - DownloadBin(), the result of which I then put into a b64enc() function, but the DownloadBin function uses the "wininet" dll calls rather than the WinHttpRequest object.
The reason I am asking mainly is because depending on the size of the image I download, the DownloadBin function seems to take a while to process.
I was wondering if anyone can lend a hand, or make any suggestions.
Code: Select all
URL := "http i.imgur.com /dS56Ewu.png" ;Broken Link for safety
vSize := httpGetFileSize(URL)
DownloadBin(URL, vData)
B64Data := Base64Enc(vData, vSize)
File := FileOpen(A_Desktop "\b64EncFile.txt", "w")
File.Write(B64Data)
File.Close()
Return
;----- Functions -----;
httpGetFileSize(URL) {
WebRequest := ComObjCreate("WinHttp.WinHttpRequest.5.1")
WebRequest.Open("HEAD", URL)
WebRequest.Send()
return ByteSize := WebRequest.GetResponseHeader("Content-Length")
}
DownloadBin(url, byref buf) {
static a := "AutoHotkey/" A_AhkVersion
if (!DllCall("LoadLibrary", "str", "wininet") || !(h := DllCall("wininet\InternetOpen", "str", a, "uint", 1, "ptr", 0, "ptr", 0, "uint", 0, "ptr")))
return 0
c := s := 0
if (f := DllCall("wininet\InternetOpenUrl", "ptr", h, "str", url, "ptr", 0, "uint", 0, "uint", 0x80003000, "ptr", 0, "ptr"))
{
while (DllCall("wininet\InternetQueryDataAvailable", "ptr", f, "uint*", s, "uint", 0, "ptr", 0) && s>0)
{
VarSetCapacity(b, c+s, 0)
if (c>0)
DllCall("RtlMoveMemory", "ptr", &b, "ptr", &buf, "ptr", c)
DllCall("wininet\InternetReadFile", "ptr", f, "ptr", &b+c, "uint", s, "uint*", r)
c += r
VarSetCapacity(buf, c, 0)
if (c>0)
DllCall("RtlMoveMemory", "ptr", &buf, "ptr", &b, "ptr", c)
}
DllCall("wininet\InternetCloseHandle", "ptr", f)
}
DllCall("wininet\InternetCloseHandle", "ptr", h)
return c
}
Base64Enc( ByRef Bin, nBytes, LineLength := 64, LeadingSpaces := 0 ) { ; By SKAN / 18-Aug-2017
Local Rqd := 0, B64, B := "", N := 0 - LineLength + 1 ; CRYPT_STRING_BASE64 := 0x1
CRYPT_STRING_BASE64 := 0x00000001
CRYPT_STRING_NOCRLF := 0x40000000
DllCall( "Crypt32.dll\CryptBinaryToString", "Ptr",&Bin ,"UInt",nBytes, "UInt",(CRYPT_STRING_BASE64), "Ptr",0, "UIntP",Rqd )
VarSetCapacity( B64, Rqd * ( A_Isunicode ? 2 : 1 ), 0 )
DllCall( "Crypt32.dll\CryptBinaryToString", "Ptr",&Bin, "UInt",nBytes, "UInt",(CRYPT_STRING_BASE64), "Str",B64, "UIntP",Rqd )
If ( LineLength = 64 and ! LeadingSpaces )
Return B64
B64 := StrReplace( B64, "`r`n" )
Loop % Ceil( StrLen(B64) / LineLength )
B .= Format("{1:" LeadingSpaces "s}","" ) . SubStr( B64, N += LineLength, LineLength ) . "`n"
Return RTrim( B,"`n" )
}
-
- Posts: 4358
- Joined: 29 Mar 2015, 09:41
- Contact:
Re: download urls to vars, partially/fully, via WinHttpRequest
Hi, @CheshireCat
An example:
An example:
Code: Select all
URL := "http://i.imgur.com/dS56Ewu.png"
whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", URL, true)
whr.Send()
whr.WaitForResponse
status := whr.status
if (status != 200)
throw "Failed to download data. Status: " . status
arr := whr.responseBody
pData := NumGet(ComObjValue(arr) + 8 + A_PtrSize)
length := arr.MaxIndex() + 1
MsgBox, % base64 := CryptBinaryToString(pData, length)
CryptBinaryToString(pData, size, formatName := "CRYPT_STRING_BASE64", NOCRLF := true)
{
static formats := { CRYPT_STRING_BASE64: 0x1
, CRYPT_STRING_HEX: 0x4
, CRYPT_STRING_HEXRAW: 0xC }
, CRYPT_STRING_NOCRLF := 0x40000000
fmt := formats[formatName] | (NOCRLF ? CRYPT_STRING_NOCRLF : 0)
if !DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Ptr", 0, "UIntP", chars)
throw "CryptBinaryToString failed. LastError: " . A_LastError
VarSetCapacity(outData, chars << !!A_IsUnicode)
DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Str", outData, "UIntP", chars)
Return outData
}
- CheshireCat
- Posts: 11
- Joined: 07 Mar 2016, 15:58
Re: download urls to vars, partially/fully, via WinHttpRequest
Hi @teadrinker
Thanks for your help, that worked perfectly, I would have never figured it out on my own.
However, though it works fine using the image in the example I gave, I'm still having difficulty if, lets say, the image doesn't have an extension.
For example:
Why is that? Is there a difference in the type of image, or because its missing the ext? I thought perhaps the image is already base64 encoded, and it needs no further encoding once its in a variable, but if so, then how would I get to that data to save as a text file?
Any insight would be great.
Thank you again
Thanks for your help, that worked perfectly, I would have never figured it out on my own.
However, though it works fine using the image in the example I gave, I'm still having difficulty if, lets say, the image doesn't have an extension.
For example:
Code: Select all
URL := "http i.imgur.com /dS56Ewu.png" ; this works fine - Broken Link for safety
URL := "https i.scdn.co /image/ab67616d0000b273b81c0f531cd7a188852ad5a4" ; this throws an error - Broken Link for safety
Any insight would be great.
Thank you again
-
- Posts: 4358
- Joined: 29 Mar 2015, 09:41
- Contact:
Re: download urls to vars, partially/fully, via WinHttpRequest
For me the code works without errors with this URL:
To find out what type of data the URL refers to, in most cases it is enough to read the "Content-Type" header:
According to this it's an image with the "jpeg" extension.
Code: Select all
URL := "https://i.scdn.co/image/ab67616d0000b273b81c0f531cd7a188852ad5a4"
whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", URL, true)
whr.Send()
whr.WaitForResponse
status := whr.status
if (status != 200)
throw "Failed to download data. Status: " . status
arr := whr.responseBody
pData := NumGet(ComObjValue(arr) + 8 + A_PtrSize)
length := arr.MaxIndex() + 1
MsgBox, % base64 := CryptBinaryToString(pData, length)
CryptBinaryToString(pData, size, formatName := "CRYPT_STRING_BASE64", NOCRLF := true)
{
static formats := { CRYPT_STRING_BASE64: 0x1
, CRYPT_STRING_HEX: 0x4
, CRYPT_STRING_HEXRAW: 0xC }
, CRYPT_STRING_NOCRLF := 0x40000000
fmt := formats[formatName] | (NOCRLF ? CRYPT_STRING_NOCRLF : 0)
if !DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Ptr", 0, "UIntP", chars)
throw "CryptBinaryToString failed. LastError: " . A_LastError
VarSetCapacity(outData, chars << !!A_IsUnicode)
DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Str", outData, "UIntP", chars)
Return outData
}
Code: Select all
URL := "https://i.scdn.co/image/ab67616d0000b273b81c0f531cd7a188852ad5a4"
whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", URL, false)
whr.Send()
MsgBox, % whr.getResponseHeader("Content-Type") ; image/jpeg
- CheshireCat
- Posts: 11
- Joined: 07 Mar 2016, 15:58
Re: download urls to vars, partially/fully, via WinHttpRequest
Hi @teadrinker
Your right, the code is working fine, the problem was me.
I had created a function that would return the pointer as a byref variable and the size as a return value. I guess the address of the var changes when returned that way.
Your right, the code is working fine, the problem was me.
I had created a function that would return the pointer as a byref variable and the size as a return value. I guess the address of the var changes when returned that way.
Code: Select all
URL := "https i.scdn.co /image/ab67616d0000b273b81c0f531cd7a188852ad5a4" Broken Link for safety
length := GetFileBin(URL, pData)
MsgBox, % base64 := CryptBinaryToString(pData, length)
Return
GetFileBin(URL, ByRef pData) {
whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", URL, true)
whr.Send()
whr.WaitForResponse
if (whr.status != 200)
throw "Failed to download data. Status: " . status
arr := whr.responseBody
pData := NumGet(ComObjValue(arr) + 8 + A_PtrSize)
return arr.MaxIndex() + 1
}
CryptBinaryToString(pData, size, formatName := "CRYPT_STRING_BASE64", NOCRLF := true)
{
static formats := { CRYPT_STRING_BASE64: 0x1
, CRYPT_STRING_HEX: 0x4
, CRYPT_STRING_HEXRAW: 0xC }
, CRYPT_STRING_NOCRLF := 0x40000000
fmt := formats[formatName] | (NOCRLF ? CRYPT_STRING_NOCRLF : 0)
if !DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Ptr", 0, "UIntP", chars)
throw "CryptBinaryToString failed. LastError: " . A_LastError
VarSetCapacity(outData, chars << !!A_IsUnicode)
DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Str", outData, "UIntP", chars)
Return outData
}
-
- Posts: 4358
- Joined: 29 Mar 2015, 09:41
- Contact:
Re: download urls to vars, partially/fully, via WinHttpRequest
Inside a function it is local data, when the function ends, data by it's pointer can be corrupted. Use this way:
Code: Select all
URL := "https://i.scdn.co/image/ab67616d0000b273b81c0f531cd7a188852ad5a4"
length := GetFileBin(URL, data)
MsgBox, % base64 := CryptBinaryToString(&data, length)
Return
GetFileBin(URL, ByRef data) {
whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", URL, true)
whr.Send()
whr.WaitForResponse
if (whr.status != 200)
throw "Failed to download data. Status: " . status
arr := whr.responseBody
pData := NumGet(ComObjValue(arr) + 8 + A_PtrSize)
VarSetCapacity(data, length := arr.MaxIndex() + 1, 0)
DllCall("RtlMoveMemory", "Ptr", &data, "Ptr", pData, "Ptr", length)
return length
}
CryptBinaryToString(pData, size, formatName := "CRYPT_STRING_BASE64", NOCRLF := true)
{
static formats := { CRYPT_STRING_BASE64: 0x1
, CRYPT_STRING_HEX: 0x4
, CRYPT_STRING_HEXRAW: 0xC }
, CRYPT_STRING_NOCRLF := 0x40000000
fmt := formats[formatName] | (NOCRLF ? CRYPT_STRING_NOCRLF : 0)
if !DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Ptr", 0, "UIntP", chars)
throw "CryptBinaryToString failed. LastError: " . A_LastError
VarSetCapacity(outData, chars << !!A_IsUnicode)
DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Str", outData, "UIntP", chars)
Return outData
}
Who is online
Users browsing this forum: Chunjee, DRS, slishnevsky, Tio_oddish and 81 guests