download urls to vars, partially/fully, via WinHttpRequest Topic is solved

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

download urls to vars, partially/fully, via WinHttpRequest

05 Jan 2017, 14:18

tl;dr
- download file (binary) to variable (via object)
- download file (binary) to variable (via object) (initial n bytes only)

I know that UrlDownloadToFile uses
InternetReadFile to download webpages,
and that with this you can download
files (binary data) and webpages to variables, and if desired,
only the start of the webpage.

One of the reasons to download only
the start of webpage, is to retrieve
the webpage titles for a list of urls.

Is it possible to download all or only the start
of a file (binary data) or webpage via WinHttpRequest,
to a variable?

Note: WinHttpRequest is preferable to
UrlDownloadToFile/InternetReadFile
because its faster.

[EDIT: 100th post! That happened quite quickly.]
Last edited by jeeswg on 01 Sep 2017, 20:10, edited 3 times in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
garry
Posts: 3740
Joined: 22 Dec 2013, 12:50

Re: download urls to vars, partially/fullly, via WinHttpRequest

05 Jan 2017, 16:55

you mean , you search for url's ?

Code: Select all

;- idea search for begin=http and end=extension
;- laatste nieuws NL
modified=20150510
f1=http://nos.nl/uitzending/nos-journaal
httpQuery(aacc,f1)

A:="http"                                                  ;- begin
Extensions1 := "mp3,m4a,wav,wma,flv,vob,mp4,mpg,wmv,avi"   ;- end

T=
Loop,parse,aacc,`n,`r
   {
   stringlen,L1,A
   T=%A_LoopField%
   ;Loop Parse,t,`,` `"`>`=`;?`!`<
   Loop Parse,t,`"      ;- split for `"
     {
     StringLeft r,A_LoopField,L1
     If (r=A)
        {
        SplitPath,A_LoopField, name, dir, ext, name_no_ext, drive
        if ext in %Extensions1%
           {
           e .= A_LoopField "`n"
           Last=%A_LoopField%
           }
        }
     }
   }
;------------------------------------
stringsplit,h,e,`n
msgbox,%e%`n`nH1=%h1%`nH2=%h2%`nH3=%h3%`nLAST=`n%last%
e=
return
;----------------------------

httpQuery(byref Result, lpszUrl, POSTDATA="", HEADERS="")
{
   WebRequest := ComObjCreate("WinHttp.WinHttpRequest.5.1")
   WebRequest.Open("GET", lpszUrl)
   WebRequest.setRequestHeader("Content-Type", "application/x-www-form-urlencoded")
   WebRequest.Send(POSTDATA)
   Result := WebRequest.ResponseText
   WebRequest := ""
}
return
;============================================================================================
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fullly, via WinHttpRequest

05 Jan 2017, 17:25

My two common scenarios are:
- The url is a webpage (e.g. http://ahkscript.org/),
I want to download the html as UrlDownloadToFile does,
but only the start of it, in order to retrieve the webpage title.
- The url is an image (e.g. http://ahkscript.org/static/ahk_logo.png),
I want to download the binary data for the image to a variable,
but not save it as a file. (I compare the data against an existing file,
and only if they differ do I then write the binary to a file.
E.g. YouTube thumbnails, same url, but image data can change.)

E.g. this downloads a file and saves it as a file.
Can it be amended to store it in a variable instead.
Also can it be set to download only the first n bytes.

Code: Select all

JEE_UrlDownloadToFile(vUrl, vPath)
{
vOverwrite := True
oHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
oHTTP.Open("GET", vUrl)
oHTTP.Send()

oADODB := ComObjCreate("ADODB.Stream")
oADODB.Type := 1
oADODB.Open()
oADODB.Write(oHTTP.ResponseBody)
oADODB.SaveToFile(vPath, vOverwrite ? 2:1)
oADODB.Close()

oHTTP := ""
oADODB := ""
Return
}
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fullly, via WinHttpRequest

22 Feb 2017, 02:30

Please if anyone can help,
I've been going round in circles trying to use
ComObjCreate("WinHttp.WinHttpRequest.5.1")
and ComObjCreate("ADODB.Stream")
to download a file e.g. an image,
and put the data from ResponseBody
into a variable.

(Btw I'm trying to get binary data from ResponseBody,
and not text from ResponseText.)
(I show an example for downloading binary to a file,
above, but I want to download it to a variable.)
Thank you.

Some relevant links I found:
Stream Object (ADO) | Microsoft Docs
https://docs.microsoft.com/en-us/sql/ad ... object-ado
Read Method | Microsoft Docs
https://docs.microsoft.com/en-us/sql/ad ... ead-method
Convert Byte Array to String - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=9807
Help with HTTPRequest and BinRead please - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=5716
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fullly, via WinHttpRequest

22 Feb 2017, 02:38

Thank you jNizM, excellent functions btw.
I have functions already that use InternetReadFile,
but I'm trying to investigate this object method which appears to download much faster
(and to improve my knowledge of objects for doing AHK v1/v2 script conversions,
and possibly write a tutorial on objects).

However, I don't know if the object method allows a partial download of a file
which is something that the InternetReadFile method can do.
Appreciated.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fullly, via WinHttpRequest

22 Feb 2017, 02:45

Thanks, that's the frustrating thing, the method is fairly clear, from various sources,
it's actually the mechanics of getting binary data out of ResponseBody,
into a variable, possibly via an object, that is the problem.
To know the specifics of how to move the data within AutoHotkey.

To work out the lines I need to modify in JEE_UrlDownloadToFile above.
I've read things like VARIANT and binary array etc.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fullly, via WinHttpRequest

22 Feb 2017, 02:49

Waitasec it might be here I'll just check.

[solved] UrlDownloadToFile slows down GUI response - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?t=9338

Btw is there a search engine that would let you search for:
AutoHotkey ADODB Read(

Specifically, to handle the bracket.

[EDIT:]
I found a good result when I searched for:
AutoHotkey "ADODB.Stream" ResponseBody
Last edited by jeeswg on 22 Feb 2017, 03:37, edited 1 time in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fullly, via WinHttpRequest  Topic is solved

22 Feb 2017, 03:36

This looks like it works:

Code: Select all

JEE_UrlDownloadToVar(vUrl, ByRef vData, ByRef vSize)
{
	oHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
	oHTTP.Open("GET", vUrl)
	oHTTP.Send()

	oHTTP.WaitForResponse()
	;vSize := oHTTP.ResponseBody.MaxIndex()
	vSize := IsObject(oHTTP.ResponseBody) ? 1+oHTTP.ResponseBody.MaxIndex() : 0
	VarSetCapacity(vData, vSize, 0)

	VarSetCapacity(vIndex, 4, 0)
	if !DllCall("OleAut32.dll\SafeArrayPtrOfIndex", Ptr,ComObjValue(oHTTP.ResponseBody), Ptr,&vIndex, PtrP,vPtrOfIndex)
		DllCall("msvcrt\memcpy", UPtr,&vData, UPtr,vPtrOfIndex, UPtr,vSize)

	oHTTP := ""
	return
}
- Btw is memcpy the best dll function for memory move?
- Also, it's a C function, can it handle Ptr? It appeared to work on 64-bit when I tested it.
- Also can 'Ptr,&vIndex' be replaced with 'PtrP,0' or something similar?

[EDIT:]
- I don't know if there is a way to specify to only download the first n bytes of a file.
- I've had problems when trying to use ResponseText from webpages, to get a webpage's title,
so I can now do it by using the binary ResponseBody and converting that to text,
like I did with InternetReadFile.

[EDIT 2:]
Btw I've seen a fair number of AHK libraries with functions for WinHttpRequest,
so feel free to copy this function, changing it as much or as little as you want, to match your other functions, if you would like to add this in to your library.
Last edited by jeeswg on 28 Jul 2018, 16:56, edited 3 times in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

Re: download urls to vars, partially/fullly, via WinHttpRequest

22 Feb 2017, 05:28

jeeswg wrote: I don't know if there is a way to specify to only download the first n bytes of a file.
You can use Range header. However not all sites support this.

Example:

Code: Select all

whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", "http://www.nirsoft.net/utils/nircmd.zip", true)
whr.SetRequestHeader("Range", "bytes=0-10")
whr.Send()
whr.WaitForResponse()
MsgBox % whr.ResponseBody.MaxIndex()
MsgBox % whr.Status " " whr.StatusText "`n`n" whr.GetAllResponseHeaders()
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fullly, via WinHttpRequest

22 Feb 2017, 05:42

Thanks so much, that's a great response.
It's not the end of the world if a site doesn't support it,
but it's a great help. One thing I like to do is
download the start to get the webpage title.

[EDIT:] Haha good call with the util url btw.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fully, via WinHttpRequest

01 Sep 2017, 22:40

- download htm to variable
- download file to variable e.g. image/icon
- download htm to variable (partial) - get webpage title

Code: Select all

;q:: ;download htm to variable
vUrl := "https://autohotkey.com/download/"
JEE_UrlDownloadToVar(vUrl, vData, vSize)
MsgBox, % Clipboard := StrGet(&vData, vSize, "CP0")
return

;==================================================

;[Gdip functions]
;GDI+ standard library 1.45 by tic - AutoHotkey Community
;https://autohotkey.com/boards/viewtopic.php?f=6&t=6517

;q:: ;download file to variable e.g. image/icon
pToken := Gdip_Startup()
Loop, 2
{
	vUrl1 := "https://autohotkey.com/static/ahk_logo.png"
	vUrl2 := "https://autohotkey.com/favicon.ico"
	vUrl := vUrl%A_Index%
	JEE_UrlDownloadToVar(vUrl, vData, vSize)
	hData := DllCall("GlobalAlloc", UInt,2, UPtr,vSize, Ptr)
	pData := DllCall("GlobalLock", Ptr,hData, Ptr)
	DllCall("RtlMoveMemory", Ptr,pData, Ptr,&vData, UPtr,vSize)
	DllCall("GlobalUnlock", Ptr,hData)
	DllCall("ole32\CreateStreamOnHGlobal", Ptr,hData, Int,1, UIntP,pStream)
	DllCall("gdiplus\GdipCreateBitmapFromStream", Ptr,pStream, UIntP,pBitmap)
	hBitmap := Gdip_CreateHBITMAPFromBitmap(pBitmap)
	SplashImage, % "HBITMAP:" hBitmap, B ;B: borderless
	Sleep 2000
	SplashImage, Off
	DeleteObject(hBitmap)
	Gdip_DisposeImage(pBitmap)
}
Gdip_Shutdown(pToken)
return

;==================================================

;q:: ;download htm to variable (partial) - get webpage title
;vUrl := "https://autohotkey.com/download/"
vUrl := "https://en.wikipedia.org/wiki/AutoHotkey"
JEE_UrlDownloadToVar(vUrl, vData, vSize)
oHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
oHTTP.Open("GET", vUrl)
oHTTP.SetRequestHeader("Range", "bytes=0-1000")
oHTTP.Send()
oHTTP.WaitForResponse()
;vSize := oHTTP.ResponseBody.MaxIndex()
vSize := IsObject(oHTTP.ResponseBody) ? 1+oHTTP.ResponseBody.MaxIndex() : 0
VarSetCapacity(vData, vSize, 0)
VarSetCapacity(vIndex, 4, 0)
if !DllCall("oleaut32\SafeArrayPtrOfIndex", Ptr,ComObjValue(oHTTP.ResponseBody), Ptr,&vIndex, PtrP,vPtrOfIndex)
	DllCall("kernel32\RtlMoveMemory", Ptr,&vData, Ptr,vPtrOfIndex, UPtr,vSize)
MsgBox, % oHTTP.Status " " oHTTP.StatusText
MsgBox, % oHTTP.GetAllResponseHeaders()
oHTTP := ""
vText := StrGet(&vData, vSize, "CP0")
;MsgBox, % vText
if !(vPos1 := InStr(vText, "<title>"))
|| !(vPos2 := InStr(vText, "</title>", 0, vPos1+7))
	return
MsgBox, % SubStr(vText, vPos1+7, vPos2-vPos1-7)
return

;==================================================

JEE_UrlDownloadToVar(vUrl, ByRef vData, ByRef vSize)
{
	oHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
	oHTTP.Open("GET", vUrl)
	oHTTP.Send()

	oHTTP.WaitForResponse()
	;vSize := oHTTP.ResponseBody.MaxIndex()
	vSize := IsObject(oHTTP.ResponseBody) ? 1+oHTTP.ResponseBody.MaxIndex() : 0
	VarSetCapacity(vData, vSize, 0)

	VarSetCapacity(vIndex, 4, 0)
	if !DllCall("oleaut32\SafeArrayPtrOfIndex", Ptr,ComObjValue(oHTTP.ResponseBody), Ptr,&vIndex, PtrP,vPtrOfIndex)
		DllCall("kernel32\RtlMoveMemory", Ptr,&vData, Ptr,vPtrOfIndex, UPtr,vSize)
	oHTTP := ""
}

;==================================================
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: download urls to vars, partially/fully, via WinHttpRequest

28 Jul 2018, 17:07

- I've made an important change to the scripts above:

Code: Select all

;before:
vSize := oHTTP.ResponseBody.MaxIndex()
;after:
vSize := IsObject(oHTTP.ResponseBody) ? 1+oHTTP.ResponseBody.MaxIndex() : 0
The scripts were missing the final byte when downloading files.
- For this:

Code: Select all

VarSetCapacity(vIndex, 4, 0)
DllCall("oleaut32\SafeArrayPtrOfIndex", Ptr,ComObjValue(oHTTP.ResponseBody), Ptr,&vIndex, PtrP,pData)
GeekDude mentions this shorter alternative, here:
GeekDude's Tips, Tricks, and Standalones - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=74&t=7190

Code: Select all

pData := NumGet(ComObjValue(oHTTP.ResponseBody)+8+A_PtrSize)
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
CheshireCat
Posts: 11
Joined: 07 Mar 2016, 15:58

Re: download urls to vars, partially/fully, via WinHttpRequest

21 Jul 2020, 19:53

Hi all, I've been trying to figure this out for days and I just cant seem to get it. :crazy:

I'm trying to download a png / jpg using WinHttpRequest. I don't want to save it as an image file, instead I want to pass it to a base64 function, encode it, write and save the resulting encoding to a text file.

I found a function that does that in the old board - DownloadBin(), the result of which I then put into a b64enc() function, but the DownloadBin function uses the "wininet" dll calls rather than the WinHttpRequest object.

The reason I am asking mainly is because depending on the size of the image I download, the DownloadBin function seems to take a while to process.

I was wondering if anyone can lend a hand, or make any suggestions.

Code: Select all

	URL := "http i.imgur.com /dS56Ewu.png"  ;Broken Link for safety

	vSize := httpGetFileSize(URL)
	DownloadBin(URL, vData)

	B64Data := Base64Enc(vData, vSize)
	
	File := FileOpen(A_Desktop "\b64EncFile.txt", "w")
	File.Write(B64Data)
	File.Close()
	
	Return
	
;----- Functions -----;	
	
	httpGetFileSize(URL) {
		WebRequest := ComObjCreate("WinHttp.WinHttpRequest.5.1")
		WebRequest.Open("HEAD", URL)
		WebRequest.Send()
		return ByteSize := WebRequest.GetResponseHeader("Content-Length")
	}

	DownloadBin(url, byref buf) {
		static a := "AutoHotkey/" A_AhkVersion
		if (!DllCall("LoadLibrary", "str", "wininet") || !(h := DllCall("wininet\InternetOpen", "str", a, "uint", 1, "ptr", 0, "ptr", 0, "uint", 0, "ptr")))
			return 0
		c := s := 0
		if (f := DllCall("wininet\InternetOpenUrl", "ptr", h, "str", url, "ptr", 0, "uint", 0, "uint", 0x80003000, "ptr", 0, "ptr"))
		{
			while (DllCall("wininet\InternetQueryDataAvailable", "ptr", f, "uint*", s, "uint", 0, "ptr", 0) && s>0)
			{
				VarSetCapacity(b, c+s, 0)
				if (c>0)
					DllCall("RtlMoveMemory", "ptr", &b, "ptr", &buf, "ptr", c)
				DllCall("wininet\InternetReadFile", "ptr", f, "ptr", &b+c, "uint", s, "uint*", r)
				c += r
				VarSetCapacity(buf, c, 0)
				if (c>0)
					DllCall("RtlMoveMemory", "ptr", &buf, "ptr", &b, "ptr", c)
			}
			DllCall("wininet\InternetCloseHandle", "ptr", f)
		}
		DllCall("wininet\InternetCloseHandle", "ptr", h)
		return c
	}

	Base64Enc( ByRef Bin, nBytes, LineLength := 64, LeadingSpaces := 0 ) { ; By SKAN / 18-Aug-2017
		Local Rqd := 0, B64, B := "", N := 0 - LineLength + 1  ; CRYPT_STRING_BASE64 := 0x1

		CRYPT_STRING_BASE64 := 0x00000001
		CRYPT_STRING_NOCRLF := 0x40000000

		DllCall( "Crypt32.dll\CryptBinaryToString", "Ptr",&Bin ,"UInt",nBytes, "UInt",(CRYPT_STRING_BASE64), "Ptr",0,   "UIntP",Rqd )
		VarSetCapacity( B64, Rqd * ( A_Isunicode ? 2 : 1 ), 0 )
		DllCall( "Crypt32.dll\CryptBinaryToString", "Ptr",&Bin, "UInt",nBytes, "UInt",(CRYPT_STRING_BASE64), "Str",B64, "UIntP",Rqd )
		If ( LineLength = 64 and ! LeadingSpaces )
			Return B64
		B64 := StrReplace( B64, "`r`n" )        
		Loop % Ceil( StrLen(B64) / LineLength )
			B .= Format("{1:" LeadingSpaces "s}","" ) . SubStr( B64, N += LineLength, LineLength ) . "`n" 
		Return RTrim( B,"`n" )    
	}
teadrinker
Posts: 4309
Joined: 29 Mar 2015, 09:41
Contact:

Re: download urls to vars, partially/fully, via WinHttpRequest

22 Jul 2020, 06:08

Hi, @CheshireCat
An example:

Code: Select all

URL := "http://i.imgur.com/dS56Ewu.png"

whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", URL, true)
whr.Send()
whr.WaitForResponse
status := whr.status
if (status != 200)
   throw "Failed to download data. Status: " . status

arr := whr.responseBody
pData := NumGet(ComObjValue(arr) + 8 + A_PtrSize)
length := arr.MaxIndex() + 1
MsgBox, % base64 := CryptBinaryToString(pData, length)

CryptBinaryToString(pData, size, formatName := "CRYPT_STRING_BASE64", NOCRLF := true)
{
   static formats := { CRYPT_STRING_BASE64: 0x1
                     , CRYPT_STRING_HEX:    0x4
                     , CRYPT_STRING_HEXRAW: 0xC }
        , CRYPT_STRING_NOCRLF := 0x40000000
   fmt := formats[formatName] | (NOCRLF ? CRYPT_STRING_NOCRLF : 0)
   if !DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Ptr", 0, "UIntP", chars)
      throw "CryptBinaryToString failed. LastError: " . A_LastError
   VarSetCapacity(outData, chars << !!A_IsUnicode)
   DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Str", outData, "UIntP", chars)
   Return outData
}
User avatar
CheshireCat
Posts: 11
Joined: 07 Mar 2016, 15:58

Re: download urls to vars, partially/fully, via WinHttpRequest

26 Jul 2020, 20:49

Hi @teadrinker

Thanks for your help, that worked perfectly, I would have never figured it out on my own. :bravo:

However, though it works fine using the image in the example I gave, I'm still having difficulty if, lets say, the image doesn't have an extension.

For example:

Code: Select all

URL := "http i.imgur.com /dS56Ewu.png" ; this works fine - Broken Link for safety 

URL := "https i.scdn.co /image/ab67616d0000b273b81c0f531cd7a188852ad5a4"  ; this throws an error - Broken Link for safety 
Why is that? Is there a difference in the type of image, or because its missing the ext? I thought perhaps the image is already base64 encoded, and it needs no further encoding once its in a variable, but if so, then how would I get to that data to save as a text file?

Any insight would be great.

Thank you again
teadrinker
Posts: 4309
Joined: 29 Mar 2015, 09:41
Contact:

Re: download urls to vars, partially/fully, via WinHttpRequest

27 Jul 2020, 07:56

For me the code works without errors with this URL:

Code: Select all

URL := "https://i.scdn.co/image/ab67616d0000b273b81c0f531cd7a188852ad5a4"

whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", URL, true)
whr.Send()
whr.WaitForResponse
status := whr.status
if (status != 200)
   throw "Failed to download data. Status: " . status

arr := whr.responseBody
pData := NumGet(ComObjValue(arr) + 8 + A_PtrSize)
length := arr.MaxIndex() + 1
MsgBox, % base64 := CryptBinaryToString(pData, length)

CryptBinaryToString(pData, size, formatName := "CRYPT_STRING_BASE64", NOCRLF := true)
{
   static formats := { CRYPT_STRING_BASE64: 0x1
                     , CRYPT_STRING_HEX:    0x4
                     , CRYPT_STRING_HEXRAW: 0xC }
        , CRYPT_STRING_NOCRLF := 0x40000000
   fmt := formats[formatName] | (NOCRLF ? CRYPT_STRING_NOCRLF : 0)
   if !DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Ptr", 0, "UIntP", chars)
      throw "CryptBinaryToString failed. LastError: " . A_LastError
   VarSetCapacity(outData, chars << !!A_IsUnicode)
   DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Str", outData, "UIntP", chars)
   Return outData
}
To find out what type of data the URL refers to, in most cases it is enough to read the "Content-Type" header:

Code: Select all

URL := "https://i.scdn.co/image/ab67616d0000b273b81c0f531cd7a188852ad5a4"
whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("GET", URL, false)
whr.Send()
MsgBox, % whr.getResponseHeader("Content-Type") ; image/jpeg
According to this it's an image with the "jpeg" extension.
User avatar
CheshireCat
Posts: 11
Joined: 07 Mar 2016, 15:58

Re: download urls to vars, partially/fully, via WinHttpRequest

27 Jul 2020, 09:37

Hi @teadrinker

Your right, the code is working fine, the problem was me.
I had created a function that would return the pointer as a byref variable and the size as a return value. I guess the address of the var changes when returned that way.

Code: Select all

	URL := "https i.scdn.co /image/ab67616d0000b273b81c0f531cd7a188852ad5a4"  Broken Link for safety

	length := GetFileBin(URL, pData)
	MsgBox, % base64 := CryptBinaryToString(pData, length)
Return 

GetFileBin(URL, ByRef pData) {
	whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
	whr.Open("GET", URL, true)
 	whr.Send()
	whr.WaitForResponse

	if (whr.status != 200)
		throw "Failed to download data. Status: " . status

	arr := whr.responseBody
	pData := NumGet(ComObjValue(arr) + 8 + A_PtrSize)

	return arr.MaxIndex() + 1
}

CryptBinaryToString(pData, size, formatName := "CRYPT_STRING_BASE64", NOCRLF := true)
{
   static formats := { CRYPT_STRING_BASE64: 0x1
                     , CRYPT_STRING_HEX:    0x4
                     , CRYPT_STRING_HEXRAW: 0xC }
        , CRYPT_STRING_NOCRLF := 0x40000000
   fmt := formats[formatName] | (NOCRLF ? CRYPT_STRING_NOCRLF : 0)
   if !DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Ptr", 0, "UIntP", chars)
      throw "CryptBinaryToString failed. LastError: " . A_LastError
   VarSetCapacity(outData, chars << !!A_IsUnicode)
   DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Str", outData, "UIntP", chars)
   Return outData
}
teadrinker
Posts: 4309
Joined: 29 Mar 2015, 09:41
Contact:

Re: download urls to vars, partially/fully, via WinHttpRequest

27 Jul 2020, 09:56

Inside a function it is local data, when the function ends, data by it's pointer can be corrupted. Use this way:

Code: Select all

URL := "https://i.scdn.co/image/ab67616d0000b273b81c0f531cd7a188852ad5a4"

length := GetFileBin(URL, data)
MsgBox, % base64 := CryptBinaryToString(&data, length)
Return 

GetFileBin(URL, ByRef data) {
   whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
   whr.Open("GET", URL, true)
   whr.Send()
   whr.WaitForResponse

   if (whr.status != 200)
      throw "Failed to download data. Status: " . status

   arr := whr.responseBody
   pData := NumGet(ComObjValue(arr) + 8 + A_PtrSize)
   VarSetCapacity(data, length := arr.MaxIndex() + 1, 0)
   DllCall("RtlMoveMemory", "Ptr", &data, "Ptr", pData, "Ptr", length)
   return length
}

CryptBinaryToString(pData, size, formatName := "CRYPT_STRING_BASE64", NOCRLF := true)
{
   static formats := { CRYPT_STRING_BASE64: 0x1
                     , CRYPT_STRING_HEX:    0x4
                     , CRYPT_STRING_HEXRAW: 0xC }
        , CRYPT_STRING_NOCRLF := 0x40000000
   fmt := formats[formatName] | (NOCRLF ? CRYPT_STRING_NOCRLF : 0)
   if !DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Ptr", 0, "UIntP", chars)
      throw "CryptBinaryToString failed. LastError: " . A_LastError
   VarSetCapacity(outData, chars << !!A_IsUnicode)
   DllCall("Crypt32\CryptBinaryToString", "Ptr", pData, "UInt", size, "UInt", fmt, "Str", outData, "UIntP", chars)
   Return outData
}

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: Google [Bot], imstupidpleshelp, jaka1 and 173 guests