CreateFormData - Create "multipart/form-data" for http post (Updated:2019-01-13)

Post your working scripts, libraries and tools for AHK v1.1 and older
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

CreateFormData - Create "multipart/form-data" for http post (Updated:2019-01-13)

13 May 2015, 16:22

CreateFormData v1.30 (2019-01-13)

Source
Modified version by SKAN (doesn't require BinArr.ahk)

Functions:
  • CreateFormData(ByRef retData, ByRef retHeader, objParam)
    • retData - (out) Data used for HTTP POST.
    • retHeader - (out) Content-Type header used for HTTP POST.
    • objParam - (in) An object defines the form parameters. To specify files, use array as the value.
      Example
  • CreateFormData_WinInet(ByRef retData, ByRef retHeader, objParam)
    Used for VxE's HTTPRequest().

Example:
Upload multiple images to postimage.org
ChangeLog:
1.30 / 2019-01-13 - The file parameters are now placed at the end of the retData
1.20 / 2016-06-17 - Added CreateFormData_WinInet(), which can be used for VxE's HTTPRequest().
1.10 / 2015-06-23 - Fixed a bug
1.00 / 2015-05-14
Last edited by tmplinshi on 26 Aug 2019, 12:33, edited 11 times in total.
User avatar
Pulover
Posts: 612
Joined: 29 Sep 2013, 19:51
Location: Brazil
Contact:

Re: CreateFormData - Creates "multipart/form-data" for http post

17 Jan 2016, 22:54

Thank you for this, tmplinshi! I've been searching for hours to find a way to upload a text file to my server and was almost giving up. Very nice! ;)
Rodolfo U. Batista
Pulover's Macro Creator - Automation Tool (Recorder & Script Writer)
Suresh
Posts: 35
Joined: 03 May 2016, 18:58

Re: CreateFormData - Creates "multipart/form-data" for http post

03 May 2016, 19:25

@tmplinshi

Wonderful code! Thank you very much for sharing this.
I've adapted your code into a standalone function for my flickr class.

I have one doubt. The following part of code will accept multiple images and post them under the same parameter.

Code: Select all

If IsObject(v) {
     For i, FileName in v
Is this loop intended? Wouldn't FileName := v[1] be sufficient?

Thanks again.
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

Re: CreateFormData - Creates "multipart/form-data" for http post

04 May 2016, 00:39

@Suresh

Yes, it was intended to support multiple files, e.g. "upload[]" : ["1.png", "2.png"]
I've added description and example to the #1 post.
Suresh
Posts: 35
Joined: 03 May 2016, 18:58

Re: CreateFormData - Creates "multipart/form-data" for http post

04 May 2016, 05:51

Fantastic! Thanks for the clarification and the updated example.
Now I need to check if flickr supports multiple images per POST.
carno
Posts: 265
Joined: 20 Jun 2014, 16:48

Re: CreateFormData - Creates "multipart/form-data" for http post

06 May 2016, 08:31

Could you please explain in newbie terms what this code does?
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

Re: CreateFormData - Creates "multipart/form-data" for http post

06 May 2016, 09:23

@carno

If you want to post string "key=val" using WinHttp or XMLHttp, it's easy:

Code: Select all

whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("POST", URL)
whr.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
whr.Send("key=val")
But if you want to post a binary file, such as uploading an image, there's no direct way like whr.Send("file=c:\test.png"). CreateFormData() can be used to create the PostData, which can be used by whr.Send(PostData).
Suresh
Posts: 35
Joined: 03 May 2016, 18:58

Re: CreateFormData - Creates "multipart/form-data" for http post

06 May 2016, 10:06

@carno

Take a look at flickr example:
https://www.flickr.com/services/api/upload.example.html

You will see {RAW JFIF DATA} at the end of that POST, which in actual should be the image file contents in binary....
whereas the rest of the POST has to be in UTF-8.

CreateFormData() accepts parameters and creates the HTTP POST mixing UTF-8 text and binary together.

Hope that helps.
Suresh
Posts: 35
Joined: 03 May 2016, 18:58

Re: CreateFormData - Creates "multipart/form-data" for http post

08 May 2016, 21:06

Ref: ByteArray() by Coco
Here is an alternate version that builds formdata ( UTF-8 + Binary ) on Global memory and then copies it to a COM bytearray.
Spoiler
Two copies of formdata exists on memory which was what Coco was trying to avoid with his ByteArray() function.
The simpler, straight forward, memory conservative method would be to build formdata as a file object and then load it to a COM bytearray.
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

Re: CreateFormData - Creates "multipart/form-data" for http post

08 May 2016, 22:34

Thank you Suresh! I did tried the ByteArray() before, but I can't figure out how to use it to combine string and binary data. The code you provide works, thanks.
Bruttosozialprodukt
Posts: 463
Joined: 24 Jan 2014, 22:28

Re: CreateFormData - Creates "multipart/form-data" for http post

11 May 2016, 02:27

Did you actually get the winhttprequest object to send binary files? Or only text-based files?
I've done something similar before, but I was not able to do it with the internal com object.
https://autohotkey.com/boards/viewtopic ... 986#p21986
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

Re: CreateFormData - Creates "multipart/form-data" for http post

11 May 2016, 09:36

Bruttosozialprodukt wrote:Did you actually get the winhttprequest object to send binary files?
Yes, as you can see in the first post example or SKAN's modified version or https://autohotkey.com/boards/viewtopic ... 731#p41731.
Bruttosozialprodukt
Posts: 463
Joined: 24 Jan 2014, 22:28

Re: CreateFormData - Creates "multipart/form-data" for http post

11 May 2016, 12:55

That's sick, I never thought of this approach.
Doode

Re: CreateFormData - Creates "multipart/form-data" for http post

07 Jun 2016, 07:26

Is it possible to make CreateFormData work with VxE's HttpRequest ?
The reason I need this is because it's a very simple matter to show upload progress with HTTPRequest. But it's apparently impossible to show upload progress with this COM WinHttpRequest.5.1 used in your example. I've tried to use the postdata that CreateFormData makes with HttpRequest, but it somehow gets garbled when it's sent to the sever and isn't interpreted correctly. Does anyone know why this is happening?
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

Re: CreateFormData - Creates "multipart/form-data" for http post

17 Jun 2016, 03:25

Update on version 1.20 / 2016-6-17 - Added CreateFormData_WinInet(), which can be used for VxE's HTTPRequest().

Hi Doode, use CreateFormData_WinInet().
arcticir
Posts: 693
Joined: 17 Nov 2013, 11:32

Re: CreateFormData - Creates "multipart/form-data" for http post

14 Aug 2016, 22:02

V2 HTTPRequest()

Code: Select all

FormData(ByRef body, Param){
	Boundary := SubStr(StrReplace(Sort("0|1|2|3|4|5|6|7|8|9|a|b|c|d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z","D| Random"), "|"), 1, 12)
	crlf := "`r`n",line := "--" . Boundary
	VarSetCapacity(body,0),offset := 0
	For k, v in Param
		If IsObject(v) 
		{
			For i, FileName in v
			{
				file:=FileOpen(FileName, "r"),n := file.ReadUInt()
				,buffer := line crlf "Content-Disposition: form-data; name=`"" . k . "`"; filename=`"" . FileName . "`"" . crlf
					. "Content-Type: " . ((n        = 0x474E5089) ? "image/png"
					: (n        = 0x38464947) ? "image/gif"
					: (n&0xFFFF = 0x4D42    ) ? "image/bmp"
					: (n&0xFFFF = 0xD8FF    ) ? "image/jpeg"
					: (n&0xFFFF = 0x4949    ) ? "image/tiff"
					: (n&0xFFFF = 0x4D4D    ) ? "image/tiff"
					: "application/octet-stream") . crlf . crlf
				,FormData_Memory(body,buffer,offset,StrLen(buffer),1)
				,bufferSize := VarSetCapacity(buffer,file.Length)
				,File.Tell(0),file.RawRead(&buffer, bufferSize)
				,FormData_Memory(body,buffer,offset,bufferSize,0)
				,FormData_Memory(body,crlf,offset,2,1)
			}
		} 
		Else
		{
			buffer := line . crlf . "Content-Disposition: form-data; name=`"" . k "`"" . crlf . crlf . v . crlf
			FormData_Memory(body,buffer,offset,StrLen(buffer),1)
		}
	FormData_Memory(body,buffer:= line "--" crlf,offset,StrLen(buffer),1)
	return {"Content-Length":offset,"Content-Type":"multipart/form-data; boundary=" . Boundary}
}

FormData_Memory(ByRef f,ByRef k,ByRef p,ByRef s,t){
		VarSetCapacity(w,r:=p,0),DllCall("RtlMoveMemory", "Ptr", &w, "Ptr", &f, "UInt", r),VarSetCapacity(f,p+= s,0),DllCall("RtlMoveMemory", "Ptr", &f, "Ptr", &w, "UInt", r)
		,t?StrPut(k, (&f)+r, s, "CP0"):DllCall("RtlMoveMemory", "Ptr", &f+r, "Ptr", &k, "UInt", s)
}
DanielToward13
Posts: 74
Joined: 18 May 2017, 10:56

Re: CreateFormData - Creates "multipart/form-data" for http post

30 May 2017, 15:14

I need to upload a file (photo, audio, etc) to my API using multipart/form-data. I am using your code and it uploads the photo successfully but at the end I get the below error:


What are the settings that needs to be changed? How can I get the response from my API after uploading the file? (The response should return in JSON)

Code: Select all

#Include CreateFormData.ahk
#Include BinArr.ahk

objParam := { "photo": ["1.png"] }
CreateFormData(postData, hdr_ContentType, objParam)

whr := ComObjCreate("WinHttp.WinHttpRequest.5.1")
whr.Open("POST", "https://api.myapi.org/bot*******************/*******", true)
whr.SetRequestHeader("Content-Type", hdr_ContentType)
whr.SetRequestHeader("Referer", "https://api.myapi.org/*******")
whr.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko")
whr.Option(6) := False ; No auto redirect
whr.Send(postData)
whr.WaitForResponse()
Run, % whr.GetResponseHeader("Location")
return
Last edited by DanielToward13 on 31 May 2017, 03:19, edited 2 times in total.
tmplinshi
Posts: 1604
Joined: 01 Oct 2013, 14:57

Re: CreateFormData - Creates "multipart/form-data" for http post

30 May 2017, 22:22

That means there is no Location header in the response headers. So just remove that line.
To retrieve the response content, use whr.ResponseText, e.g. MsgBox % whr.ResponseText.
User avatar
Joe Glines
Posts: 770
Joined: 30 Sep 2013, 20:49
Location: Dallas
Contact:

Re: CreateFormData - Creates "multipart/form-data" for http post

20 Jun 2017, 13:05

BTW- I'd seen this before (and even used it) but I was unclear on what "multipart/form-data" meant and why I would need to use it.

I found this post on Stackoverflow which gives a pretty good explanation. :)
Sign-up for the 🅰️HK Newsletter

ImageImageImageImage:clap:
AHK Tutorials:Web Scraping | | Webservice APIs | AHK and Excel | Chrome | RegEx | Functions
Training: AHK Webinars Courses on AutoHotkey :ugeek:
YouTube

:thumbup: Quick Access Popup, the powerful Windows folders, apps and documents launcher!
egocarib
Posts: 100
Joined: 21 May 2015, 18:21

Re: CreateFormData - Creates "multipart/form-data" for http post

15 Oct 2017, 12:42

Anyone know what I need to do to get this to work in AHKv2? (64-bit)

Specifically, I am trying to use this to send a file over HTTP, based on BlackHolyMan's example in this thread.

Running into this error (in BinArr.ahk)

Code: Select all

Error in #include file "...\BinArr.ahk":
     0x800A0BB9 - 
Source:		ADODB.Stream
Description:	Arguments are of the wrong type, are out of acceptable range, or are in conflict with one another.
HelpFile:		C:\WINDOWS\HELP\ADO270.CHM
HelpContext:	1240641

Specifically: Write

	Line#
	024: }
	026: {
	027: oADO := ComObjCreate("ADODB.Stream")
	029: oADO.Type := 1
	030: oADO.Mode := 3
	031: oADO.Open
	032: For i,arr in Arrays
--->	033: oADO.Write(arr)
	034: oADO.Position := 0
	035: Return,oADO.Read, oADO.Close
	036: }
	038: {
	039: oADO := ComObjCreate("ADODB.Stream")
	041: oADO.Type := 1
	042: oADO.Mode := 3

Continue running the script?

I spent some time debugging but can't really figure out the problem. The BinArr_ functions seem to be working okay but I don't fully understand it (the ADO stream is working correctly I think: I can read size, position, etc, and it returns an object, which I assume is a binary object)

Seems like BinArr also causes problems for some other people. See here and here. Not sure if they had the same problem. Maybe I should try the alternative they suggest?


My code (BlackHolyMan's code with double-quote escaping updated to work for AHKv2)

Code: Select all

#Include %A_ScriptDir%\CreateFormData.ahk ; https://gist.github.com/tmplinshi/8428a280bba58d25ef0b
#Include %A_ScriptDir%\BinArr.ahk ; https://gist.github.com/tmplinshi/a97d9a99b9aa5a65fd20
#Include %A_ScriptDir%\Jxon.ahk ; https://github.com/cocobelgica/AutoHotkey-JSON/blob/master/JSON.ahk

image_path := "C:\Xerxes.png"

PB_Token   := "<will need to use your own PushBullet token to test this>"
PB_Image_Name   := "Xerxes.png"

JSON_Response := PB_PushUpload_request(PB_Token, PB_Image_Name)

JSON_Response_object := Jxon_Load( JSON_Response )

objParam := { "file": [image_path]}
CreateFormData(postData, hdr_ContentType, objParam)

Status := PB_PushUpload_formData(JSON_Response_object.upload_url, postData, hdr_ContentType)
if (Status != 204)
{
	msgbox % "Upload Error!"
	return
}

msgbox % PB_PushFile(PB_Token, "This is an image test", JSON_Response_object.file_name, JSON_Response_object.file_type, JSON_Response_object.file_url)
return

PB_PushUpload_request(PB_Token, PB_Image_Name)
{
	WinHTTP := ComObjCreate("WinHTTP.WinHttpRequest.5.1")
	WinHTTP.SetProxy(0)
	WinHTTP.Open("POST", "https://api.pushbullet.com/v2/upload-request", 0)
	WinHTTP.SetCredentials(PB_Token, "", 0)
	WinHTTP.SetRequestHeader("Content-Type", "application/json")
	PB_Body := "{`"file_name`": `"" PB_Image_Name "`", `"file_type`": `"image/png`"}"
	WinHTTP.Send(PB_Body)
	Result := WinHTTP.ResponseText
	Status := WinHTTP.Status
	return Result
}

PB_PushUpload_formData(PB_upload_url, PB_postData, PB_ContentType)
{
	WinHTTP := ComObjCreate("WinHttp.WinHttpRequest.5.1")
	WinHTTP.Open("POST", PB_upload_url, true)
	WinHTTP.SetRequestHeader("Content-Type", PB_ContentType)
	WinHTTP.Option(6) := False ; No auto redirect
	WinHTTP.Send(PB_postData)
	WinHTTP.WaitForResponse()
	Result := WinHTTP.ResponseText
	Status := WinHTTP.Status
	return Status
}

PB_PushFile(PB_Token, PB_Body, PB_file_name, PB_file_type, PB_file_url)
{
	WinHTTP := ComObjCreate("WinHTTP.WinHttpRequest.5.1")
	WinHTTP.SetProxy(0)
	WinHTTP.Open("POST", "https://api.pushbullet.com/v2/pushes", 0)
	WinHTTP.SetCredentials(PB_Token, "", 0)
	WinHTTP.SetRequestHeader("Content-Type", "application/json")
	PB_Body := "{`"type`": `"file`", `"body`": `"" PB_Body "`", `"file_name`": `"" PB_file_name "`", `"file_type`": `"" PB_file_type "`", `"file_url`": `"" PB_file_url "`"}"
	WinHTTP.Send(PB_Body)
	Result := WinHTTP.ResponseText
	Status := WinHTTP.Status
	return Status
}
CreateFormData.ahk (only updated double-quote escaping for AHKv2 compatibility)

Code: Select all

/*
	CreateFormData - Creates "multipart/form-data" for http post

	Usage: CreateFormData(ByRef retData, ByRef retHeader, objParam)

		retData   - (out) Data used for HTTP POST.
		retHeader - (out) Content-Type header used for HTTP POST.
		objParam  - (in)  An object defines the form parameters.

		            To specify files, use array as the value. Example:
		                objParam := { "key1": "value1"
		                            , "upload[]": ["1.png", "2.png"] }

	Requirement: BinArr.ahk -- https://gist.github.com/tmplinshi/a97d9a99b9aa5a65fd20
	Version    : 1.20 / 2016-6-17 - Added CreateFormData_WinInet(), which can be used for VxE's HTTPRequest().
	             1.10 / 2015-6-23 - Fixed a bug
	             1.00 / 2015-5-14
*/

; Used for WinHttp.WinHttpRequest.5.1, Msxml2.XMLHTTP ...
CreateFormData(ByRef retData, ByRef retHeader, objParam) {
	New CreateFormData(retData, retHeader, objParam)
}

; Used for WinInet
CreateFormData_WinInet(ByRef retData, ByRef retHeader, objParam) {
	New CreateFormData(safeArr, retHeader, objParam)

	size := safeArr.MaxIndex() + 1
	VarSetCapacity(retData, size, 1)
	DllCall("oleaut32\SafeArrayAccessData", "ptr", ComObjValue(safeArr), "ptr*", pdata)
	DllCall("RtlMoveMemory", "ptr", &retData, "ptr", pdata, "ptr", size)
	DllCall("oleaut32\SafeArrayUnaccessData", "ptr", ComObjValue(safeArr))
}

Class CreateFormData {

	__New(ByRef retData, ByRef retHeader, objParam) {

		CRLF := "`r`n"

		Boundary := this.RandomBoundary()
		BoundaryLine := "------------------------------" . Boundary

		; Loop input paramters
		binArrs := []
		For k, v in objParam
		{
			If IsObject(v) {
				For i, FileName in v
				{
					str := BoundaryLine . CRLF
					     . "Content-Disposition: form-data; name=`"" . k . "`"; filename=`"" . FileName . "`"" . CRLF
					     . "Content-Type: " . this.MimeType(FileName) . CRLF . CRLF
					binArrs.Push( BinArr_FromString(str) )
					binArrs.Push( BinArr_FromFile(FileName) )
					binArrs.Push( BinArr_FromString(CRLF) )
				}
			} Else {
				str := BoundaryLine . CRLF
				     . "Content-Disposition: form-data; name=`"" . k "`"" . CRLF . CRLF
				     . v . CRLF
				binArrs.Push( BinArr_FromString(str) )
			}
		}

		str := BoundaryLine . "--" . CRLF
		binArrs.Push( BinArr_FromString(str) )

		retData := BinArr_Join(binArrs*)
		retHeader := "multipart/form-data; boundary=----------------------------" . Boundary
	}

	RandomBoundary() {
		str := "0|1|2|3|4|5|6|7|8|9|a|b|c|d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z"
		Sort, str, D| Random
		str := StrReplace(str, "|")
		Return SubStr(str, 1, 12)
	}

	MimeType(FileName) {
		n := FileOpen(FileName, "r").ReadUInt()
		Return (n        = 0x474E5089) ? "image/png"
		     : (n        = 0x38464947) ? "image/gif"
		     : (n&0xFFFF = 0x4D42    ) ? "image/bmp"
		     : (n&0xFFFF = 0xD8FF    ) ? "image/jpeg"
		     : (n&0xFFFF = 0x4949    ) ? "image/tiff"
		     : (n&0xFFFF = 0x4D4D    ) ? "image/tiff"
		     : "application/octet-stream"
	}

}
BinArr.ahk (unchanged)

Code: Select all

; Update: 2015-6-4 - Added BinArr_ToFile()

BinArr_FromString(str) {
	oADO := ComObjCreate("ADODB.Stream")

	oADO.Type := 2 ; adTypeText
	oADO.Mode := 3 ; adModeReadWrite
	oADO.Open()
	oADO.Charset := "UTF-8"
	oADO.WriteText(str)
	oADO.Position := 0
	oADO.Type := 1 ; adTypeBinary
	oADO.Position := 3 ; Skip UTF-8 BOM
	return oADO.Read, oADO.Close
}

BinArr_FromFile(FileName) {
	oADO := ComObjCreate("ADODB.Stream")

	oADO.Type := 1 ; adTypeBinary
	oADO.Open
	oADO.LoadFromFile(FileName)
	return oADO.Read, oADO.Close
}

BinArr_Join(Arrays*) {
	oADO := ComObjCreate("ADODB.Stream")

	oADO.Type := 1 ; adTypeBinary
	oADO.Mode := 3 ; adModeReadWrite
	oADO.Open
	For i, arr in Arrays
		oADO.Write(arr)
	oADO.Position := 0
	return oADO.Read, oADO.Close
}

BinArr_ToString(BinArr, Encoding := "UTF-8") {
	oADO := ComObjCreate("ADODB.Stream")

	oADO.Type := 1 ; adTypeBinary
	oADO.Mode := 3 ; adModeReadWrite
	oADO.Open
	oADO.Write(BinArr)

	oADO.Position := 0
	oADO.Type := 2 ; adTypeText
	oADO.Charset  := Encoding 
	return oADO.ReadText, oADO.Close
}

BinArr_ToFile(BinArr, FileName) {
	oADO := ComObjCreate("ADODB.Stream")

	oADO.Type := 1 ; adTypeBinary
	oADO.Open
	oADO.Write(BinArr)
	oADO.SaveToFile(FileName, 2)
	oADO.Close
}


(Working on a general PushBullet class for AHKv2 but the file sending here is the last wrench holding me up from supporting the core elements of the PushBullet API)

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: No registered users and 143 guests