MemoryFileIO

Post your working scripts, libraries and tools for AHK v1.1 and older
Sam_
Posts: 146
Joined: 20 Mar 2014, 20:24

MemoryFileIO

15 Feb 2019, 19:17

MemoryFileIO is a class that wraps NumGet, NumPut, StrGet, and StrPut to provide File Object-like syntax for in-memory files and structures. It is available on GitHub. This is a dependency of several other classes/libraries that I hope to release in the semi-near future. The early versions had a lot of error checking, but were too slow for byte-by-byte reads/writes of large files. One of the next iterations had virtually no error checking, but I found that too many errors were slipping through the cracks only to cause issues later. The current version is relatively quick and (in most cases) throws an exception on error. As I work towards releasing other classes/libraries, the exceptions thrown by this class will likely be updated to incorporate more intelligent error handling.

Here is the code from GitHub as of 20190215:

Code: Select all

;////////////////////////////////////////////////
;///////////// Class MemoryFileIO ///////////////
;////////////////////////////////////////////////
;;; Provides Input/Output File Object syntax for in-memory files.
;;; Usage is very similar to the File Object syntax: 
;;; 	https://autohotkey.com/docs/objects/File.htm
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;; New MemoryFileIO() ;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Creates a new instance of the class.
;;; Syntax:  Instance := New MemoryFileIO(InputVar, [VarSize, DefaultEncoding])
;;; Pass it the variable (directly or by address) you will be performing
;;; 	file I/O operations on.  DefaultEncoding defaults to A_FileEncoding.
;;; The MemoryFileIO class will bound all reads/writes to the
;;; 	memory already allocated to the variable upon creating
;;; 	the new instance of the class.  Use VarSetCapacity()
;;; 	to initialize enough memory for your needs BEFORE creating
;;; 	a new MemoryFileIO instance with your variable.
;;; This class will raise an exception on error.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;; Read ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Reads a string of characters from the file and advances the file pointer.
;;; Syntax:  String := Instance.Read([Characters, Encoding = None])
;;; Characters:	The maximum number of characters to read. If omitted, 
;;;		the rest of the file is read and returned as one string.
;;; Encoding: The source encoding; for example, "UTF-8", "UTF-16" or "CP936".
;;;		Specify an empty string or "CP0" to use the system default ANSI code page.
;;; Returns: A string.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;; Write ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Writes a string of characters to the file and advances the file pointer.
;;; Syntax:  NumWritten := Instance.Write(String [, Characters, Encoding])
;;; String: A string.
;;; Characters: The maximum number of characters to write. If omitted, 
;;;		all of String is written.
;;; Encoding: The target encoding; for example, "UTF-8", "UTF-16" or "CP936".
;;;		Specify an empty string or "CP0" to use the system default ANSI code page.
;;; Returns: The number of bytes (not characters) that were written.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;; ReadNum ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Reads a number from the file and advances the file pointer.
;;; Syntax:  Num := Instance.ReadNumType()
;;; NumType: One of the following specified directly as part of the method name:
;;; 	UInt, Int, Int64, Short, UShort, Char, UChar, Double, Float, Ptr, UPtr
;;; 	DWORD, Long, WORD, or BYTE
;;; Returns: A number if successful.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;; WriteNum ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Writes a number to the file and advances the file pointer.
;;; Syntax:  Instance.WriteNumType(Num)
;;; NumType: One of the following specified directly as part of the method name:
;;; 	UInt, Int, Int64, Short, UShort, Char, UChar, Double, Float, Ptr, UPtr
;;; 	DWORD, Long, WORD, or BYTE
;;; Num: A number.
;;; Returns: The number of bytes that were written. For instance, WriteUInt 
;;; 	returns 4 if successful.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;; RawRead ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Copies raw binary data from the file to another memory address or variable.
;;; 	If a var is specified, it is expanded automatically when necessary.
;;; 	If a var is specified that contains only an integer, that integer is 
;;; 	considered the address.
;;; Syntax:  Instance.RawRead(VarOrAddress, Bytes)
;;; VarOrAddress: A variable or memory address to which the data will be copied.
;;; Bytes: The maximum number of bytes to read.
;;; Returns: The number of bytes that were read.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;; RawWrite ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Write raw binary data to the file from another memory address or variable.
;;; 	If a var is specified that contains only an integer, that integer is 
;;; 	considered the address.  If VarOrAddress is a variable and Bytes is
;;; 	greater than the capacity of VarOrAddress, Bytes is reduced to the capacity
;;; 	of VarOrAddress.
;;; Syntax:  Instance.RawWrite(VarOrAddress, Bytes)
;;; VarOrAddress: A variable containing the data or the address of the data 
;;; 	in memory.
;;; Bytes: The number of bytes to write.
;;; Returns: The number of bytes that were written.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;; Seek ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Moves the file pointer.
;;; Syntax:	Instance.Seek(Distance [, Origin])
;;; 	Instance.Position := Distance
;;; 	Instance.Pos := Distance
;;; Distance: Distance to move, in bytes. Lower values are closer to the 
;;; 	beginning of the file.
;;; Origin: Starting point for the file pointer move. Must be one of the following:
;;; 	0 (SEEK_SET): Beginning of the file. Distance must be zero or greater.
;;; 	1 (SEEK_CUR): Current position of the file pointer.
;;; 	2 (SEEK_END): End of the file. Distance should usually be negative.
;;; 	If omitted, Origin defaults to SEEK_END when Distance is negative 
;;; 	and SEEK_SET otherwise.
;;; Returns one of the following values:
;;; 	-1 : Pointer was instructed to move before beginning of file.
;;; 		 Automatically moved to beginning of file instead.
;;; 	1  : Pointer is still in bounds or if EoF was reached.
;;; 	2  : Pointer was instructed to move past EoF.
;;; 		 Automatically moved to EoF instead.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;; Tell ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Syntax:	Pos := Instance.Tell()
;;; 	Pos := Instance.Position
;;; 	Pos := Instance.Pos
;;; Returns: The current position of the file pointer, where 0 is the 
;;; 	beginning of the file.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;; Length ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Retrieves the size of the file.  Setting a new size is not supported.
;;; Syntax:	FileSize := Instance.Length
;;; Returns: 	The size of the file, in bytes.
;;; 
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;; AtEOF ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Syntax:	IsAtEOF := Instance.AtEOF
;;; Returns: A non-zero value if the file pointer has reached the 
;;; 	end of the file, otherwise zero.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;; Encoding ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Retrieves or sets the text encoding used by this method.
;;; Encoding defaults to A_FileEncoding which defaults to the system 
;;; 	default ANSI code page ("CP0") if not specified.
;;; Syntax:	Encoding := Instance.Encoding
;;; 		Instance.Encoding := Encoding
;;; Encoding: 	A numeric code page identifier (see MSDN) or 
;;; 	one of the following strings:
;;; 	UTF-8: Unicode UTF-8, equivalent to CP65001.
;;; 	UTF-16: Unicode UTF-16 with little endian byte order, equivalent to CP1200.
;;; 	CPnnn: a code page with numeric identifier nnn.
;;; Setting Encoding never causes a BOM to be added or removed.
;;; 
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;; NumTypes ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;; Name      Size    Alias ;;;;;;;;;;;;
;;;;;;;;;;;; UInt      4       DWORD ;;;;;;;;;;;;
;;;;;;;;;;;; Int       4       Long  ;;;;;;;;;;;;
;;;;;;;;;;;; Int64     8             ;;;;;;;;;;;;
;;;;;;;;;;;; Short     2             ;;;;;;;;;;;;
;;;;;;;;;;;; UShort    2       WORD  ;;;;;;;;;;;;
;;;;;;;;;;;; Char      1             ;;;;;;;;;;;;
;;;;;;;;;;;; UChar     1       BYTE  ;;;;;;;;;;;;
;;;;;;;;;;;; Double    8             ;;;;;;;;;;;;
;;;;;;;;;;;; Float     4             ;;;;;;;;;;;;
;;;;;;;;;;;; Ptr       A_PtrSize     ;;;;;;;;;;;;
;;;;;;;;;;;; UPtr      A_PtrSize     ;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Declare class:
Class MemoryFileIO{
	__New(ByRef InputVar, VarSize:="", DefaultEncoding:=""){
		If InputVar is not integer ; Is a Variable not an Address
			this.Address:=&InputVar
		Else
			this.Address:=InputVar
		this.Position:=0
		this.Length:=(VarSize=""?VarSetCapacity(InputVar):VarSize)
		this.AtEOF:=0
		this.Encoding:=(DefaultEncoding=""?A_FileEncoding:DefaultEncoding)
		}
	ReadUInt(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UInt"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadDWORD(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UInt"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadInt(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Int"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadLong(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Int"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadInt64(){
		If (this.Position+8>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Int64"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=8
		Return Num
	}
	ReadShort(){
		If (this.Position+2>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Short"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=2
		Return Num
	}
	ReadUShort(){
		If (this.Position+2>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UShort"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=2
		Return Num
	}
	ReadWORD(){
		If (this.Position+2>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UShort"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=2
		Return Num
	}
	ReadChar(){
		If (this.Position+1>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Char"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position++
		Return Num
	}
	ReadUChar(){
		If (this.Position+1>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UChar"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position++
		Return Num
	}
	ReadBYTE(){
		If (this.Position+1>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UChar"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position++
		Return Num
	}
	ReadDouble(){
		If (this.Position+8>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Double"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=8
		Return Num
	}
	ReadFloat(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Float"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadPtr(){
		If (this.Position+A_PtrSize>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Ptr"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=A_PtrSize
		Return Num
	}
	ReadUPtr(){
		If (this.Position+A_PtrSize>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UPtr"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=A_PtrSize
		Return Num
	}
	WriteUInt(Number){
		If (Number="") OR (this.Position+4>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "UInt"), this.Position+=4
		Return 4
	}
	WriteDWORD(Number){
		If (Number="") OR (this.Position+4>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "UInt"), this.Position+=4
		Return 4
	}
	WriteInt(Number){
		If (Number="") OR (this.Position+4>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "Int"), this.Position+=4
		Return 4
	}
	WriteLong(Number){
		If (Number="") OR (this.Position+4>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "Int"), this.Position+=4
		Return 4
	}
	WriteInt64(Number){
		If (Number="") OR (this.Position+8>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "Int64"), this.Position+=8
		Return 8
	}
	WriteShort(Number){
		If (Number="") OR (this.Position+2>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "Short"), this.Position+=2
		Return 2
	}
	WriteUShort(Number){
		If (Number="") OR (this.Position+2>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "UShort"), this.Position+=2
		Return 2
	}
	WriteWORD(Number){
		If (Number="") OR (this.Position+2>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "UShort"), this.Position+=2
		Return 2
	}
	WriteChar(Number){
		If (Number="") OR (this.Position+1>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "Char"), this.Position++
		Return 1
	}
	WriteUChar(Number){
		If (Number="") OR (this.Position+1>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "UChar"), this.Position++
		Return 1
	}
	WriteBYTE(Number){
		If (Number="") OR (this.Position+1>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "UChar"), this.Position++
		Return 1
	}
	WriteDouble(Number){
		If (Number="") OR (this.Position+8>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "Double"), this.Position+=8
		Return 8
	}
	WriteFloat(Number){
		If (Number="") OR (this.Position+4>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "Float"), this.Position+=4
		Return 4
	}
	WritePtr(Number){
		If (Number="") OR (this.Position+A_PtrSize>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "Ptr"), this.Position+=A_PtrSize
		Return A_PtrSize
	}
	WriteUPtr(Number){
		If (Number="") OR (this.Position+A_PtrSize>this.Length)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		NumPut(Number, this.Address+0, this.Position, "UPtr"), this.Position+=A_PtrSize
		Return A_PtrSize
	}
	Tell(){
		Return this.Position
	}
	Position(){
		Return this.Position
	}
	Pos(){
		Return this.Position
	}
	Length(){
		Return this.Length
	}
	Read(Length:="", Encoding:=""){
		If (Encoding="")
			Encoding:=this.Encoding
		CharLen:=((encoding="utf-16"||encoding="cp1200")?2:1) ; calculate length of each character in bytes
		If (Length="")
			Length:=this.Length/CharLen ; convert length of Length from bytes to chars
		Length:=(this.Position+Length*CharLen>this.Length?(this.Length-this.Position)/CharLen:Length)
		Length:=(Length<0?0:Length)
		Str:=StrGet(this.Address+this.Position, Length, Encoding)
		this.Seek(Length*CharLen,1)
		Return Str
	}
	Write(String, Length:="", Encoding:=""){
		If (Encoding="")
			Encoding:=this.Encoding
		CharLen:=((encoding="utf-16"||encoding="cp1200")?2:1) ; calculate length of each character in bytes
		If (Length="")
			Length:=StrLen(String)
		Length:=(this.Position+Length*CharLen>this.Length?(this.Length-this.Position)/CharLen:Length)
		Length:=(Length<0?0:Length)
		NumWritten:=StrPut(SubStr(String,1,Length),this.Address+this.Position,Length,Encoding)
		this.Seek(NumWritten*CharLen,1)
		Return NumWritten*CharLen
	}
	RawRead(ByRef VarOrAddress,Bytes){
		Bytes:=(this.Position+Bytes>this.Length?this.Length-this.Position:Bytes)
		If VarOrAddress is not integer ; Is a Variable not an Address
			{
			If (VarSetCapacity(VarOrAddress)<Bytes)
				VarSetCapacity(VarOrAddress,Bytes)
			this._BCopy(this.Address+this.Position,&VarOrAddress,Bytes)
			}
		Else ; Is an Address not a Variable
			this._BCopy(this.Address+this.Position,VarOrAddress,Bytes)
		this.Seek(Bytes,1)
		Return Bytes
	}
	RawWrite(ByRef VarOrAddress,Bytes){
		Bytes:=(this.Position+Bytes>this.Length?this.Length-this.Position:Bytes)
		If VarOrAddress is not integer ; Is a Variable not an Address
			{
			If (VarSetCapacity(VarOrAddress)<Bytes)
				Bytes:=VarSetCapacity(VarOrAddress) ; Ensures Bytes is not greater than the size of VarOrAddress
			this._BCopy(&VarOrAddress,this.Address+this.Position,Bytes)
			}
		Else ; Is an Address not a Variable
			this._BCopy(VarOrAddress,this.Address+this.Position,Bytes)
		this.Seek(Bytes,1)
		Return Bytes
	}
	Seek(Distance, Origin:=""){
		If (Origin="")
			Origin:=(Distance<1?2:0)
		If (Origin=0)
			this.Position:=Distance
		Else If (Origin=1)
			this.Position+=Distance
		Else If (Origin=2)
			this.Position:=this.Length+Distance
		If (this.Position<1)
			{
			this.Position:=0
			this.AtEOF:=0
			Return -1 ; Returns -1 if Position was moved before beginning of file
			}
		Else If (this.Position>=this.Length)
			{
			this.Position:=this.Length
			this.AtEOF:=1
			If (this.Position>this.Length)
				Return 2 ; Returns 2 if EoF was passed
			Else
				Return 1 ; Returns 1 if EoF was reached
			}
		Else
			{
			this.AtEOF:=0
			Return 1 ; Returns 1 if Position is still in bounds
			}
	}
	_BCopy(Source,Destination,Length){
		DllCall("RtlMoveMemory","Ptr",Destination,"Ptr",Source,"UInt",Length) ;https://msdn.microsoft.com/en-us/library/windows/hardware/ff562030(v=vs.85).aspx
	}
}
I can provide an example of the usage if anyone is interested.

Improvements, suggestions, bug fixes, speed improvements, etc. are of course welcome.
burque505
Posts: 1736
Joined: 22 Jan 2017, 19:37

Re: MemoryFileIO

17 Feb 2019, 14:42

@Sam_, thank you for this class. I would appreciate an example, if you don't mind.
I recall nnnik just wrote about why not to use FileAppend. Will this offer advantages over opening the file for reading, doing all the processes you need to, and then closing the file, as he describes? (I'm sure it must, otherwise you wouldn't have written it, but no use cases are leaping out at me.)
And thanks again!
Regards,
burque505
Sam_
Posts: 146
Joined: 20 Mar 2014, 20:24

Re: MemoryFileIO

17 Feb 2019, 16:15

burque505 wrote:
17 Feb 2019, 14:42
@Sam_, thank you for this class. I would appreciate an example, if you don't mind.
I recall nnnik just wrote about why not to use FileAppend. Will this offer advantages over opening the file for reading, doing all the processes you need to, and then closing the file, as he describes? (I'm sure it must, otherwise you wouldn't have written it, but no use cases are leaping out at me.)
And thanks again!
Regards,
burque505
That's a good topic, and nnnik makes a good point about usage of FileAppend.

In general, repeated reads/writes to memory are much quicker than repeated reads/writes to disk. If you're just appending a bit of text to the end of a file once, just write it to disk. If you are reading kilobytes or megabytes of data from a binary file a byte at a time, you will notice a substantial increase in performance if you read the entire file into memory and then perform in-memory reads/writes. In the example you linked to, you'll get the best performance simply by using

Code: Select all

VarSetCapacity(var,2*99-9,0)
Loop, 99
	var.=A_Index

file:=FileOpen("numbers.txt","w-d")
	file.write(var)
file.close()
You could do the same thing with MemoryFileIO

Code: Select all

fs:=2*99-9
VarSetCapacity(var,fs,0)
DataMem:=New MemoryFileIO(&var,fs)
Loop, 99
	DataMem.Write(A_Index)
DataMem:=""

file:=FileOpen("numbers.txt","w-d")
	file.write(var)
file.close()
however, in this specific case the added overhead will make the process slower but with no benefit. This is a bad example.

Here is some quick and dirty code I threw together a while back to create a 3 color progress bar in memory (by writing a paletted 8-bit windows bitmap in memory) and then show it in a GUI. I'm sure the same result could be achieved using only GDIP, but maybe it gives you an example of what can be done. If there is something specific you'd like me to demonstrate, feel free to ask.

Code: Select all

Progress1:=New PS_Progress(755,30)
Progress1.AddSection(255,0,0,0,9)
Progress1.AddSection(255,255,0,0,1)
Progress1.AddSection(0,128,0,0,90)
Progress:=Progress1.GethProgress()
Progress1:=""

;~ ;file:=FileOpen(A_ScriptDir "\tmpbmp.bmp","w-d")
	;~ ;file.RawWrite(Progress1.GetAddress("Raw"),Progress1.FileSize)
;~ ;file.Close()

Gui, Add, Picture, x10 y10 w755 h30, % "hBitmap:*" Progress
Gui, Show, w800 h100
Return

GuiClose:
	Progress1:=""
	DllCall("DeleteObject","ptr",Progress)
ExitApp


class PS_Progress{
	__New(Width,Height){
		this.Width:=Width, this.Height:=Height
		this.ScanLinePadding:=0, this.Palette:={}, this.Section:={}, this.pToken:=Gdip_Startup()
		While (Mod(this.Width+this.ScanLinePadding,4)>0)
			this.ScanLinePadding++
		this.FileSize:=1078+this.Width*this.Height+this.ScanLinePadding*this.Height
		this.Raw:=" ", this.SetCapacity("Raw",this.FileSize), DllCall("RtlFillMemory","Ptr",this.GetAddress("Raw"),"UInt",this.FileSize,"UChar",0)
		this.DataMem:=New MemoryFileIO(this.GetAddress("Raw"),this.FileSize)
		this.DataMem.Seek(0,0)
		this.DataMem.Write("BM",2)
		this.DataMem.WriteUInt(this.FileSize)
		this.DataMem.WriteUShort(0)
		this.DataMem.WriteUShort(0)
		this.DataMem.WriteUInt(1078)
		this.DataMem.WriteUInt(40)
		this.DataMem.WriteInt(this.Width)
		this.DataMem.WriteInt(this.Height)
		this.DataMem.WriteUShort(1)
		this.DataMem.WriteUShort(8)
		this.DataMem.WriteUInt(0)
		this.DataMem.WriteUInt(0)
		this.DataMem.WriteInt(0)
		this.DataMem.WriteInt(0)
		this.DataMem.WriteUInt(NumPal:=256)
		this.DataMem.WriteUInt(NumPal)
	}
	SetSectionColor(Idx,RR,GG,BB,AA:=0){ ; Idx should be 0-based
		this.Palette[Idx,"RR"]:=RR
		this.Palette[Idx,"GG"]:=GG
		this.Palette[Idx,"BB"]:=BB
		this.Palette[Idx,"AA"]:=AA
	}
	AddSection(RR,GG,BB,AA:=0,Percent:=0){
		this.SetSectionColor((Idx:=this.Palette.Count()),RR,GG,BB,AA)
		this.Section[Idx]:=Percent
	}
	SetSectionPercent(Idx,Percent){
		this.Section[Idx]:=Percent
	}
	GethProgress(){
		; Write Palette
		this.DataMem.Seek(54,0)
		Loop, % this.Palette.Count()
			{
			Index:=A_Index-1
			this.DataMem.WriteUChar(this.Palette[Index,"BB"])
			this.DataMem.WriteUChar(this.Palette[Index,"GG"])
			this.DataMem.WriteUChar(this.Palette[Index,"RR"])
			this.DataMem.WriteUChar(this.Palette[Index,"AA"])
			}
		; Write Frame Data
		this.DataMem.Seek(1078,0)
		Index:=0
		Loop, % this.Height ; For each row in Height of Progress
			{
			For Idx,Percent in this.Section	; For each section of Progress
				{
				Loop, % (Cnt:=Round(Percent*this.Width/100))	; Sets pixels for this section
					{
					this.DataMem.WriteUChar(Idx)
					Index++
					If (Index=this.Width)
						Break, 2
					}
				}
			Loop, % (this.ScanLinePadding)
				this.DataMem.WriteUChar(0)
			Index:=0
			}
		Return GDIPlus_hBitmapFromBuffer("",this.FileSize,this.GetAddress("Raw"))
	}
	__Delete(){
		this.Raw:="", this.Delete("Raw"), this.DataMem:=""
		Gdip_Shutdown(this.pToken)
	}
}

;;;;;;;;;;;;;;;;;;;;;;;;;	Gdip	;;;;;;;;;;;;;;;;;;;;;;;;;
GDIPlus_hBitmapFromBuffer( ByRef Buffer, nSize , BufferAddress) { ;  Last Modifed : 21-Jun-2011
; https://autohotkey.com/boards/viewtopic.php?p=178626#p178626
; Adapted version by SKAN www.autohotkey.com/forum/viewtopic.php?p=383863#383863
; Original code   by Sean www.autohotkey.com/forum/viewtopic.php?p=147029#147029
 pStream:=pBitmap:=hBitmap:=""
 hData := DllCall("GlobalAlloc", UInt,2, UInt,nSize )
 pData := DllCall("GlobalLock",  UInt,hData )
 DllCall( "RtlMoveMemory", UInt,pData, UInt,(BufferAddress?BufferAddress:&Buffer), UInt,nSize )
 DllCall( "GlobalUnlock" , UInt,hData )
 DllCall( "ole32\CreateStreamOnHGlobal", UInt,hData, Int,True, UIntP,pStream )
 DllCall( "gdiplus\GdipCreateBitmapFromStream",  UInt,pStream, UIntP,pBitmap )
 DllCall( "gdiplus\GdipCreateHBITMAPFromBitmap", UInt,pBitmap, UIntP,hBitmap, UInt,RtlUlongByteSwap64(DllCall( "GetSysColor", Int,15 )<<8) | 0xFF000000 )
 DllCall( "gdiplus\GdipDisposeImage", UInt,pBitmap )
 DllCall( NumGet( NumGet(1*pStream)+8 ), UInt,pStream ) ; IStream::Release
Return hBitmap
; Also See:
;   GdiPlus_SaveImageToBuffer() - Scripts and Functions - AutoHotkey Community
;   https://autohotkey.com/board/topic/85523-gdiplus-saveimagetobuffer/
}
Gdip_Startup()
	{
	Ptr := A_PtrSize ? "UPtr" : "UInt"
	pToken := 0

	if !DllCall("GetModuleHandle", "str", "gdiplus", Ptr)
		DllCall("LoadLibrary", "str", "gdiplus")
	VarSetCapacity(si, A_PtrSize = 8 ? 24 : 16, 0), si := Chr(1)
	DllCall("gdiplus\GdiplusStartup", A_PtrSize ? "UPtr*" : "uint*", pToken, Ptr, &si, Ptr, 0)
	return pToken
	}
Gdip_Shutdown(pToken)
	{
	Ptr := A_PtrSize ? "UPtr" : "UInt"

	DllCall("gdiplus\GdiplusShutdown", Ptr, pToken)
	if hModule := DllCall("GetModuleHandle", "str", "gdiplus", Ptr)
		DllCall("FreeLibrary", Ptr, hModule)
	return 0
	}
RtlUlongByteSwap64(num){
	; Url: https://autohotkey.com/boards/viewtopic.php?p=181437&sid=ed1119d85377e323927b569b0281d9cd#p181437
	;	- https://msdn.microsoft.com/en-us/library/windows/hardware/ff562886(v=vs.85).aspx (RtlUlongByteSwap routine)
	;	- https://msdn.microsoft.com/en-us/library/e8cxb8tk.aspx (_swab function)
	; For example, if the Source parameter value is 0x12345678, the routine returns 0x78563412.
	; works on both 32 and 64 bit.
	; v1 version
	static dest, src
	static i := varsetcapacity(dest,4) + varsetcapacity(src,4)
	numput(num,src,"uint")
	,DllCall("MSVCRT.dll\_swab", "ptr", &src, "ptr", &dest+2, "int", 2, "cdecl")
	,DllCall("MSVCRT.dll\_swab", "ptr", &src+2, "ptr", &dest, "int", 2, "cdecl")
	return numget(dest,"uint")
}

#Include, MemoryFileIO.ahk
burque505
Posts: 1736
Joined: 22 Jan 2017, 19:37

Re: MemoryFileIO

18 Feb 2019, 14:21

@Sam_, thank you very much for the example.

Might MemoryIO.ahk be used for inter-script communication? It seems like it would speed it up and avoid disk writes entirely.

I'll bet the RPA people would like what you've come up with, especially UiPath which already has AutoHotkey Activities. If software robots could write to memory instead of to disk, it seems like it would be an advantage both in speed and wear and tear on disks. Since they work with large files, like huge Excel spreadsheets and BLOBs of all kinds, it would make sense to me at least.

I look forward very much to seeing your other libraries also.

Regards,
burque505
Sam_
Posts: 146
Joined: 20 Mar 2014, 20:24

Re: MemoryFileIO

26 Feb 2019, 10:11

@burque505
Inter-script communication is not a topic I have ever looked into. Intuitively, I'd say MemoryFileIO might help, but it isn't a solution in and of itself as the other script still won't know where in memory to look for the information it needs. RHCP's classMemory may be of interest as it is designed to read from the memory of another process. It can search the memory of another process for a unique byte string, which might could be used to find where in memory to find the data to "transfer" (if you prepended it with a unique string that can be searched for).
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: MemoryFileIO

26 Feb 2019, 10:56

The advantage of using this script and the file object is the fact that it doesn't matter wether you write to a file or buffer it in the memory first.
The syntax and code remains exactly the same:

Code: Select all

writeSomeData(streamObject) {
	streamObject.writeUInt(42)
	streamObject.writeUInt(32)
	streamObject.write("This Program cannot be run in DOS mode")
	;...
}
writeSomeData(fileOpen("test.exe","w")) ;write the data to a file
var := ""
writeSomeData(New MemoryFileIO(var)) ;write the data to the memory
This will help creating more poweful and reuseable code.
I will reccomend this library when I see the chance :)


That being said I find it strange that you expect a variable in the constructor.
To me it would make more sense if you would expect a variable, a pointer + size or nothing.
I rarely use variables as storage and it kind of defeats the point if I have to keep a superglobal/global or similar variable around for each and every MemoryFileIO Object.
You can even store binary data inside objects using ObjGetAddress and ObjSetCapacity - I think this would be my prefered method of interacting with the class.
The name is a bit awkward too :P


@Offtopic - keeping the file in RAM vs writing bit by bit.
I do say something about writing every iteration vs. keeping the file completely in RAM in my post.
It's under additional results - the batchsizing problem. The result is essentially that how you do it depends on the problem.
For small files (a few dozen MBits or less) you can easily buffer the entire file in RAM but once you get larger files you do want to write to the disk to keep the RAM clean.
Recommends AHK Studio
Sam_
Posts: 146
Joined: 20 Mar 2014, 20:24

Re: MemoryFileIO

26 Feb 2019, 11:31

nnnik wrote:
26 Feb 2019, 10:56
That being said I find it strange that you expect a variable in the constructor.
To me it would make more sense if you would expect a variable, a pointer + size or nothing.
I rarely use variables as storage and it kind of defeats the point if I have to keep a superglobal/global or similar variable around for each and every MemoryFileIO Object.
Both should be supported. If InputVar is an integer, it is treated as an address; otherwise a variable. VarSize should be specified if you give it an address. If you give it a variable but not VarSize, VarSetCapacity will be used to retrieve the size of the variable (but may be 1 byte longer than intended due to the terminating null).
nnnik wrote:
26 Feb 2019, 10:56
You can even store binary data inside objects using ObjGetAddress and ObjSetCapacity - I think this would be my prefered method of interacting with the class.
If I understand you correctly, I did exactly this in my second example above. I wanted the ability to read/write to structures outside of the calss itself, so I didn't make the buffer internal (just the pointer to it).
nnnik wrote:
26 Feb 2019, 10:56
The name is a bit awkward too :P
It's not released under any particular license. Feel free to assume WTFPL, so you're welcome to do what you want with it, including rename it.
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: MemoryFileIO

26 Feb 2019, 15:43

Both should be supported. If InputVar is an integer, it is treated as an address; otherwise a variable. VarSize should be specified if you give it an address. If you give it a variable but not VarSize, VarSetCapacity will be used to retrieve the size of the variable (but may be 1 byte longer than intended due to the terminating null).
Ah yeah didn't see that in the documentation.
If I understand you correctly, I did exactly this in my second example above. I wanted the ability to read/write to structures outside of the calss itself, so I didn't make the buffer internal (just the pointer to it).
Yes one can modify your object to do exactly what I want - but as it will be the default usage for pretty much all the users (at least the ones I know) of this library it's kind of strange that you don't offer it directly.
For me it would make more sense to offer a way to have the buffer internally by default and potentially offer a way to pass a pointer and a size. I probably wouldn't offer this kind of byref passing tbh since thats covered with ptr+size.

Also when using the file object the code doesn't need any facilities to tell the size of the result beforehand. It's safe to assume that the code is not structured in a way to allow this to happen easily.
Adding an option to automatically making the binary extend and making it default might be a good idea.
Recommends AHK Studio
Sam_
Posts: 146
Joined: 20 Mar 2014, 20:24

Re: MemoryFileIO

26 Feb 2019, 17:38

nnnik wrote:
26 Feb 2019, 15:43
If I understand you correctly, I did exactly this in my second example above. I wanted the ability to read/write to structures outside of the class itself, so I didn't make the buffer internal (just the pointer to it).
Yes one can modify your object to do exactly what I want - but as it will be the default usage for pretty much all the users (at least the ones I know) of this library it's kind of strange that you don't offer it directly.
For me it would make more sense to offer a way to have the buffer internally by default and potentially offer a way to pass a pointer and a size. I probably wouldn't offer this kind of byref passing tbh since thats covered with ptr+size.
I see what you're saying, it's just completely contrary to how I've been using this class for years. I'll think on it.
nnnik wrote:
26 Feb 2019, 15:43
Also when using the file object the code doesn't need any facilities to tell the size of the result beforehand. It's safe to assume that the code is not structured in a way to allow this to happen easily.
Adding an option to automatically making the binary extend and making it default might be a good idea.
I see the upside of making the code as idiot-proof dynamic as possible. I started down that road before (although I never got to the point of dynamically increasing the buffer), but turned around because performance was taking too big of a hit.

Thinking out loud: let's assume for a minute that the buffer isn't stored internally in the class, and that an address and size were used to initialize it. How would you increase the size? I suppose you could allocate a new variable of appropriate size, RtlMoveMemory the old buffer into the new, and then what? Have separate methods to return the new address and new size? What if the initial buffer was stored in a particular place on purpose, like in another class or object? Lastly, I don't know of a way to increase a variable's size without losing the contents (as VarSetCapacity does in AHK_L). The end result being that you initialize a new, larger buffer and copy the old contents into the new buffer, then free the old buffer. If allowed to, many people will end up extending the buffer a byte at a time (possibly inadvertently) which IMO is really not a good coding practice. Object.SetCapacity(Key, ByteSize) claims it preserves existing data when growing the capacity, but I imagine it has to be doing something similar internally? FWIW I see this class mostly of interest to people working with binary files and buffers, the nature of which means you generally have an idea of how big your buffer/file is or needs to be. If you're mostly dealing with reading/writing strings or lines, you may have no idea about how much space you need reserved, but then AHK has many built-in commands and functions better suited to this task. Am I totally off the mark here?
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: MemoryFileIO

26 Feb 2019, 21:03

If someone passed along a pointer and a size you would throw an error - since there is no method available for resizing.
Additionally you could allow people to pass along a function that increases size:

Code: Select all

class .. {
	__New( .., resizeMethod := "") {
		if (isExternalPtr + Size) {
			if (resizeMethod != "") {
				this.resizeMethod := resizeMethod
			} else {
				this.noResize := true
			}
		} else {
			this.resizeMethod := this.internalResizeMethod.bind(this)
		}
	}
	
	resize(newSize) {
		if (this.noResize) {
			throw exception("Attempted to resize a binary which cannot be resized",-2) ;since the resize method will be called internally and the error is outside of that we need -2 
		}
		this.length := this.resizeMethod.call(newSize)
	}
}
Recommends AHK Studio
Sam_
Posts: 146
Joined: 20 Mar 2014, 20:24

Re: MemoryFileIO

28 Feb 2019, 16:28

New version on GitHub. It adds the option to store the buffer internally, and hopefully doesn't break backwards compatibility. I've done some basic testing, and it seems to work as intended. Note that if the buffer is resized, any stored address or pointer to it will be invalidated (but variable references should still be valid). I've implemented some new methods to compensate.

Code: Select all

;
; AutoHotkey Version: 1.1.30.01
; Language:       English
; Platform:       Optimized for Windows 10
; Author:         Sam.
;

;;;;;	Reference Documents	;;;;;
; https://github.com/Sampsca/MemoryFileIO
; https://www.autohotkey.com/boards/viewtopic.php?f=6&t=62067
; https://autohotkey.com/docs/objects/File.htm
; https://docs.microsoft.com/en-us/windows/desktop/devnotes/rtlmovememory
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;          MemoryFileIO          ;;;;;
;;;;;  Copyright (c) 2016-2019 Sam.  ;;;;;
;;;;;     Last Updated 20190228      ;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;


;////////////////////////////////////////////////
;///////////// Class MemoryFileIO ///////////////
;////////////////////////////////////////////////
;;; Provides Input/Output File Object syntax for in-memory files and buffers.
;;; Usage is very similar to the File Object syntax: 
;;; 	https://autohotkey.com/docs/objects/File.htm
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;; New MemoryFileIO() ;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Creates a new instance of the class.
;;; Syntax:  Instance := New MemoryFileIO(VarOrAddress [, Size, DefaultEncoding, StoreBufferInternally])
;;; Pass a buffer (by variable name or memory address) to VarOrAddress that you will be performing read/write
;;; 	operations on.  If the buffer is passed by address, VarSize should be the size of the buffer in bytes.  If
;;; 	VarOrAddress is passed a variable and VarSize is not given, the size will be determined by VarSetCapacity()
;;; 	which may return a size 1 byte larger than intended due to an ending null terminator in the buffer.
;;; This class will bound all reads/writes to the memory already allocated to the input variable or address+Size
;;; 	upon creating the instance of the class, UNLESS StoreBufferInternally=1.  If StoreBufferInternally=0, you
;;; 	will need to use VarSetCapacity() to initialize enough memory for your needs BEFORE creating a new
;;; 	instance with your variable.
;;; If StoreBufferInternally=1, the memory buffer will be stored internally in the class and will be dynamically
;;; 	resized as required.  If VarOrAddress was passed a valid variable or memory address, the internal buffer
;;; 	will be initialized to a copy of this buffer.  Otherwise, an empty internal buffer will be initialized of Size
;;; 	bytes, unless Size is also not specified in which case a blank internal buffer will be initialized.  Note
;;; 	that changing the buffer size will invalidate any previously saved pointers to it.  See .GetBufferAddress().
;;; DefaultEncoding defaults to A_FileEncoding if not specified.
;;; This class will raise an exception on error.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;; Read ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Reads a string of characters from the file and advances the file pointer.
;;; Syntax:  String := Instance.Read([Characters, Encoding = None])
;;; Characters:	The maximum number of characters to read. If omitted, 
;;;		the rest of the file is read and returned as one string.
;;; Encoding: The source encoding; for example, "UTF-8", "UTF-16" or "CP936".
;;;		Specify an empty string or "CP0" to use the system default ANSI code page.
;;; Returns: A string.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;; Write ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Writes a string of characters to the file and advances the file pointer.
;;; Syntax:  NumWritten := Instance.Write(String [, Characters, Encoding])
;;; String: A string.
;;; Characters: The maximum number of characters to write. If omitted, 
;;;		all of String is written.
;;; Encoding: The target encoding; for example, "UTF-8", "UTF-16" or "CP936".
;;;		Specify an empty string or "CP0" to use the system default ANSI code page.
;;; Returns: The number of bytes (not characters) that were written.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;; ReadNum ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Reads a number from the file and advances the file pointer.
;;; Syntax:  Num := Instance.ReadNumType()
;;; NumType: One of the following specified directly as part of the method name:
;;; 	UInt, Int, Int64, Short, UShort, Char, UChar, Double, Float, Ptr, UPtr
;;; 	DWORD, Long, WORD, or BYTE
;;; Returns: A number if successful.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;; WriteNum ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Writes a number to the file and advances the file pointer.
;;; Syntax:  Instance.WriteNumType(Num)
;;; NumType: One of the following specified directly as part of the method name:
;;; 	UInt, Int, Int64, Short, UShort, Char, UChar, Double, Float, Ptr, UPtr
;;; 	DWORD, Long, WORD, or BYTE
;;; Num: A number.
;;; Returns: The number of bytes that were written. For instance, WriteUInt 
;;; 	returns 4 if successful.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;; RawRead ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Copies raw binary data from the file to another memory address or variable.
;;; 	If a var is specified, it is expanded automatically when necessary.
;;; 	If a var is specified that contains only an integer, that integer is 
;;; 	considered the address.
;;; Syntax:  Instance.RawRead(VarOrAddress, Bytes)
;;; VarOrAddress: A variable or memory address to which the data will be copied.
;;; Bytes: The maximum number of bytes to read.
;;; Returns: The number of bytes that were read.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;; RawWrite ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Write raw binary data to the file from another memory address or variable.
;;; 	If a var is specified that contains only an integer, that integer is 
;;; 	considered the address.  If VarOrAddress is a variable and Bytes is
;;; 	greater than the capacity of VarOrAddress, Bytes is reduced to the capacity
;;; 	of VarOrAddress (unless the buffer is being stored internally in the class).
;;; Syntax:  Instance.RawWrite(VarOrAddress, Bytes)
;;; VarOrAddress: A variable containing the data or the address of the data in memory.
;;; Bytes: The number of bytes to write.
;;; Returns: The number of bytes that were written.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;; Seek ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Moves the file pointer.
;;; Syntax:	Instance.Seek(Distance [, Origin])
;;; 	Instance.Position := Distance
;;; 	Instance.Pos := Distance
;;; Distance: Distance to move, in bytes. Lower values are closer to the 
;;; 	beginning of the file.
;;; Origin: Starting point for the file pointer move. Must be one of the following:
;;; 	0 (SEEK_SET): Beginning of the file. Distance must be zero or greater.
;;; 	1 (SEEK_CUR): Current position of the file pointer.
;;; 	2 (SEEK_END): End of the file. Distance should usually be negative.
;;; 	If omitted, Origin defaults to SEEK_END when Distance is negative 
;;; 	and SEEK_SET otherwise.
;;; Returns one of the following values:
;;; 	-1 : Pointer was instructed to move before beginning of file.
;;; 		 Automatically moved to beginning of file instead.
;;; 	1  : Pointer is still in bounds or if EoF was reached.
;;; 	2  : Pointer was instructed to move past EoF.
;;; 		 Automatically moved to EoF instead.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;; Tell ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Syntax:	Pos := Instance.Tell()
;;; 	Pos := Instance.Position
;;; 	Pos := Instance.Pos
;;; Returns: The current position of the file pointer, where 0 is the 
;;; 	beginning of the file.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;; Length ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Retrieves the size of the file.  Setting a new size is not directly supported.
;;; Syntax:	FileSize := Instance.Length
;;; Returns: 	The size of the file, in bytes.
;;; 
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;; AtEoF ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Syntax:	IsAtEoF := Instance.AtEoF
;;; Returns: A non-zero value if the file pointer has reached the 
;;; 	end of the file, otherwise zero.
;;; Note that in the current implementation, only Seek() adjusts the value of AtEoF.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;; Encoding ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Retrieves or sets the text encoding used by this method.
;;; Encoding defaults to A_FileEncoding which defaults to the system 
;;; 	default ANSI code page ("CP0") if not specified.
;;; Syntax:	Encoding := Instance.Encoding
;;; 		Instance.Encoding := Encoding
;;; Encoding: 	A numeric code page identifier (see MSDN) or 
;;; 	one of the following strings:
;;; 	UTF-8: Unicode UTF-8, equivalent to CP65001.
;;; 	UTF-16: Unicode UTF-16 with little endian byte order, equivalent to CP1200.
;;; 	CPnnn: a code page with numeric identifier nnn.
;;; Setting Encoding never causes a BOM to be added or removed.
;;;
;////////////////////////////////////////////////
;//////////// Supplemental Methods //////////////
;////////////////////////////////////////////////
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;; GetBufferAddress ;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; If the buffer has been resized, any previously saved pointers to it will be
;;; 	invalidated.  This method can be used to find the current address.
;;; Syntax:  Address := Instance.GetBufferAddress()
;;; Returns: The current memory address of the buffer.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;; GetBufferSize ;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Syntax:  Size := Instance.GetBufferSize()
;;; Returns: The current size (in bytes) of the memory buffer.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;; CopyBufferToVar ;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Copies the entire current buffer into the given variable, resizing it if necessary.
;;; Syntax:  Size := Instance.CopyBufferToVar(Var)
;;; Returns: The new size (in bytes) of Var.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;; CopyBufferToAddress ;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Copies the current buffer to the given memory address.  There must already be enough space reserved
;;; 	at the given address to hold the entire buffer.
;;; Syntax:  NumWritten := Instance.CopyBufferToAddress(DestinationAddress)
;;; Returns: The number of bytes written to DestinationAddress.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;; ResizeInternalBuffer ;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Adjusts the capacity of the current buffer to RequestedCapacity.  Only the capacity of an internal
;;; 	buffer can be resized in this way.
;;; Syntax:  GrantedCapacity := Instance.ResizeInternalBuffer(RequestedCapacity)
;;; Returns: The new capacity of the buffer.
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;; ResizeExternalVarContainingBuffer ;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Resizes the non-object variable containing the external memory buffer.  The current contents are preserved.
;;; Syntax:  NumWritten := Instance.ResizeExternalVarContainingBuffer(Var,CurrentCapacity,RequestedCapacity)
;;; Var: The variable containing the external buffer.
;;; CurrentCapacity:  The current capacity of this variable.
;;; RequestedCapacity:  The desired capacity of this variable.
;;; Returns: The new address referenced by Var.
;;;
;////////////////////////////////////////////////
;/////////////// Internal Methods ///////////////
;////////////////////////////////////////////////
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;; _BCopy ;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Copies binary data from one address to another.
;;; Syntax:  Instance._BCopy(Source,Destination,Length)
;;; Source:  Address of buffer to be copied.
;;; Destination:  Address of where the buffer will be copied to.
;;; Length:  Length of the binary buffer to copy.
;;; Returns: N/A
;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;; NumTypes ;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;; Name      Size    Alias ;;;;;;;;;;;;
;;;;;;;;;;;; UInt      4       DWORD ;;;;;;;;;;;;
;;;;;;;;;;;; Int       4       Long  ;;;;;;;;;;;;
;;;;;;;;;;;; Int64     8             ;;;;;;;;;;;;
;;;;;;;;;;;; Short     2             ;;;;;;;;;;;;
;;;;;;;;;;;; UShort    2       WORD  ;;;;;;;;;;;;
;;;;;;;;;;;; Char      1             ;;;;;;;;;;;;
;;;;;;;;;;;; UChar     1       BYTE  ;;;;;;;;;;;;
;;;;;;;;;;;; Double    8             ;;;;;;;;;;;;
;;;;;;;;;;;; Float     4             ;;;;;;;;;;;;
;;;;;;;;;;;; Ptr       A_PtrSize     ;;;;;;;;;;;;
;;;;;;;;;;;; UPtr      A_PtrSize     ;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Declare class:
Class MemoryFileIO{
	__New(ByRef VarOrAddress, Size:="", DefaultEncoding:="", StoreBufferInternally:=0){
		If VarOrAddress is not integer ; Is a Variable not an Address
			this.Address:=&VarOrAddress
		Else
			this.Address:=VarOrAddress
		this.Position:=0
		this.Length:=(Size=""?VarSetCapacity(VarOrAddress):Size)
		this.AtEoF:=0
		this.Encoding:=(DefaultEncoding=""?A_FileEncoding:DefaultEncoding)
		If (this.Internal:=((!VarOrAddress AND !Size)?1:StoreBufferInternally))
			{
			this.Raw:=" ", this.SetCapacity("Raw",this.Length)
			If (this.Address>0) AND (VarOrAddress<>"") ; Copy an existing buffer into the internal one
				this._BCopy(this.Address,this.GetAddress("Raw"),this.Length)
			Else ; Initialize an empty internal write buffer
				DllCall("RtlFillMemory","Ptr",this.GetAddress("Raw"),"UInt",this.Length,"UChar",0)
			this.Address:=this.GetAddress("Raw")
			}
	}
	ReadUInt(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UInt"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadDWORD(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UInt"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadInt(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Int"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadLong(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Int"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadInt64(){
		If (this.Position+8>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Int64"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=8
		Return Num
	}
	ReadShort(){
		If (this.Position+2>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Short"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=2
		Return Num
	}
	ReadUShort(){
		If (this.Position+2>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UShort"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=2
		Return Num
	}
	ReadWORD(){
		If (this.Position+2>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UShort"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=2
		Return Num
	}
	ReadChar(){
		If (this.Position+1>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Char"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position++
		Return Num
	}
	ReadUChar(){
		If (this.Position+1>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UChar"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position++
		Return Num
	}
	ReadBYTE(){
		If (this.Position+1>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UChar"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position++
		Return Num
	}
	ReadDouble(){
		If (this.Position+8>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Double"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=8
		Return Num
	}
	ReadFloat(){
		If (this.Position+4>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Float"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=4
		Return Num
	}
	ReadPtr(){
		If (this.Position+A_PtrSize>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "Ptr"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=A_PtrSize
		Return Num
	}
	ReadUPtr(){
		If (this.Position+A_PtrSize>this.Length) OR ((Num:=NumGet(this.Address+0, this.Position, "UPtr"))="")
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory read.", extra: "Val='" Num "' at offset " this.Position " of " this.Length-1}
		this.Position+=A_PtrSize
		Return Num
	}
	WriteUInt(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+4)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "UInt"), this.Position+=4
		Return 4
	}
	WriteDWORD(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+4)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "UInt"), this.Position+=4
		Return 4
	}
	WriteInt(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+4)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "Int"), this.Position+=4
		Return 4
	}
	WriteLong(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+4)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "Int"), this.Position+=4
		Return 4
	}
	WriteInt64(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+8)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "Int64"), this.Position+=8
		Return 8
	}
	WriteShort(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+2)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "Short"), this.Position+=2
		Return 2
	}
	WriteUShort(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+2)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "UShort"), this.Position+=2
		Return 2
	}
	WriteWORD(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+2)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "UShort"), this.Position+=2
		Return 2
	}
	WriteChar(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+1)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "Char"), this.Position++
		Return 1
	}
	WriteUChar(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+1)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz) ;(this.Position>=this.Length?1:0)
		NumPut(Number, this.Address+0, this.Position, "UChar"), this.Position++
		Return 1
	}
	WriteBYTE(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+1)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "UChar"), this.Position++
		Return 1
	}
	WriteDouble(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+8)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "Double"), this.Position+=8
		Return 8
	}
	WriteFloat(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+4)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "Float"), this.Position+=4
		Return 4
	}
	WritePtr(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+A_PtrSize)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "Ptr"), this.Position+=A_PtrSize
		Return A_PtrSize
	}
	WriteUPtr(Number){
		If (Number="") OR ((Expand:=((NewSz:=this.Position+A_PtrSize)>this.Length)) AND !this.Internal)
			throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "Invalid memory write.", extra: "Number='" Number "' at offset " this.Position " of " this.Length-1}
		If Expand
			this.ResizeInternalBuffer(NewSz)
		NumPut(Number, this.Address+0, this.Position, "UPtr"), this.Position+=A_PtrSize
		Return A_PtrSize
	}
	Tell(){
		Return this.Position
	}
	Position(){ ; Does not work as a method since it is overwritten by the property with the same name.
		Return this.Position
	}
	Pos(){
		Return this.Position
	}
	Length(){ ; Does not work as a method since it is overwritten by the property with the same name.
		Return this.Length
	}
	Read(Length:="", Encoding:=""){
		Encoding:=(Encoding=""?this.Encoding:Encoding) ; Set default encoding if not specified
		CharLen:=((Encoding="utf-16"||Encoding="cp1200")?2:1) ; calculate length of each character in bytes
		If (Length="")
			Length:=this.Length/CharLen ; convert length of Length from bytes to chars
		Length:=(this.Position+Length*CharLen>this.Length?(this.Length-this.Position)/CharLen:Length)
		Length:=(Length<0?0:Length)
		Str:=StrGet(this.Address+this.Position, Length, Encoding)
		this.Seek(Length*CharLen,1)
		Return Str
	}
	Write(String, Length:="", Encoding:=""){
		Encoding:=(Encoding=""?this.Encoding:Encoding) ; Set default encoding if not specified
		CharLen:=((Encoding="utf-16"||Encoding="cp1200")?2:1) ; Calculate length of each character in bytes
		Length:=(Length=""?StrLen(String):(Length<0?0:Length)) ; Length of string, in chars
		If ((NewSz:=(this.Position+Length*CharLen))>this.Length) ; String is too long to fit in buffer
			{
			If this.Internal ; We can expand buffer
				this.ResizeInternalBuffer(NewSz) ; Expand the buffer
			Else ; We can't expand, so truncate String to number of chars that can fit into the buffer
				Length:=(this.Length-this.Position)/CharLen ; Length (in characters) that will fit into the buffer
			}
		NumWritten:=StrPut(SubStr(String,1,Length),this.Address+this.Position,Length,Encoding)
		this.Seek(Bytes:=NumWritten*CharLen,1)
		Return Bytes
	}
	RawRead(ByRef VarOrAddress,Bytes){
		Bytes:=(this.Position+Bytes>this.Length?this.Length-this.Position:Bytes)
		If VarOrAddress is not integer ; Is a Variable not an Address
			{
			If (VarSetCapacity(VarOrAddress)<Bytes)
				VarSetCapacity(VarOrAddress,Bytes)
			this._BCopy(this.Address+this.Position,&VarOrAddress,Bytes)
			}
		Else ; Is an Address not a Variable
			this._BCopy(this.Address+this.Position,VarOrAddress,Bytes)
		this.Seek(Bytes,1)
		Return Bytes
	}
	RawWrite(ByRef VarOrAddress,Bytes){
		If (this.Position+Bytes>this.Length) ; Too many bytes to fit in buffer
			{
			If this.Internal ; We can expand the buffer
				this.ResizeInternalBuffer(this.Position+Bytes) ; Expand the buffer
			Else ; We can't expand, so truncate number of bytes to what can fit in the buffer
				Bytes:=this.Length-this.Position ; Number of bytes that can fit in the buffer
			}
		If VarOrAddress is not integer ; Is a Variable not an Address
			{
			Bytes:=((Szz:=VarSetCapacity(VarOrAddress))<Bytes?Szz:Bytes) ; Ensures Bytes is not greater than the size of VarOrAddress
			this._BCopy(&VarOrAddress,this.Address+this.Position,Bytes)
			}
		Else ; Is an Address not a Variable
			this._BCopy(VarOrAddress,this.Address+this.Position,Bytes)
		this.Seek(Bytes,1)
		Return Bytes
	}
	Seek(Distance, Origin:=""){
		If (Origin="")
			Origin:=(Distance<1?2:0)
		If (Origin=0)
			this.Position:=Distance
		Else If (Origin=1)
			this.Position+=Distance
		Else If (Origin=2)
			this.Position:=this.Length+Distance
		If (this.Position<1)
			{
			this.Position:=0
			this.AtEoF:=0
			Return -1 ; Returns -1 if Position was moved before beginning of file
			}
		Else If (this.Position>=this.Length)
			{
			this.Position:=this.Length
			this.AtEoF:=1
			If (this.Position>this.Length)
				Return 2 ; Returns 2 if EoF was passed
			Else
				Return 1 ; Returns 1 if EoF was reached
			}
		Else
			{
			this.AtEoF:=0
			Return 1 ; Returns 1 if Position is still in bounds
			}
	}
	GetBufferAddress(){
		Return this.Address
	}
	GetBufferSize(){
		Return this.Length
	}
	CopyBufferToVar(ByRef Var){
		VarSetCapacity(Var,this.Length)
		this._BCopy(this.Address,&Var,this.Length)
		Return this.Length ; The new length of Var
	}
	CopyBufferToAddress(DestinationAddress){
		this._BCopy(this.Address,DestinationAddress,this.Length)
		Return this.Length ; The number of bytes written to DestinationAddress
	}
	ResizeInternalBuffer(RequestedCapacity){
		If this.Internal ; Buffer if stored internally
			{
			NewCapacity:=this.SetCapacity("Raw",RequestedCapacity), this.Address:=this.GetAddress("Raw")
			Return this.Length:=(NewCapacity<>""?NewCapacity:this.Length) ; Returns the new capacity
			}
		If (RequestedCapacity=this.Length) ; It's okay to not change the capacity
			Return this.Length
		throw { what: (IsFunc(A_ThisFunc)?"function: " A_ThisFunc "()":"") A_Tab (IsLabel(A_ThisLabel)?"label: " A_ThisLabel:""), file: A_LineFile, line: A_LineNumber, message: "The capacity of external buffers passed by address cannot be updated.", extra: "RequestedCapacity='" Num "', CurrentCapacity='" this.Length "'"}
	}
	ResizeExternalVarContainingBuffer(ByRef Var,CurrentCapacity,RequestedCapacity){
		VarSetCapacity(tmp,CurrentCapacity)
		this._BCopy(&Var,&tmp,CurrentCapacity)
		VarSetCapacity(Var,RequestedCapacity,0)
		this._BCopy(&tmp,&Var,CurrentCapacity)
		this.Address:=&Var, this.Length:=RequestedCapacity
		this.Seek(0,1)
		Return this.Address ; Returns the new address referenced by Var
	}
	_BCopy(Source,Destination,Length){
		DllCall("RtlMoveMemory","Ptr",Destination,"Ptr",Source,"UInt",Length) ;https://msdn.microsoft.com/en-us/library/windows/hardware/ff562030(v=vs.85).aspx
	}
}
SOTE
Posts: 1426
Joined: 15 Jun 2015, 06:21

Re: MemoryFileIO

09 Jan 2020, 15:08

Sam_ wrote:
26 Feb 2019, 10:11
@burque505
Inter-script communication is not a topic I have ever looked into. Intuitively, I'd say MemoryFileIO might help, but it isn't a solution in and of itself as the other script still won't know where in memory to look for the information it needs. RHCP's classMemory may be of interest as it is designed to read from the memory of another process. It can search the memory of another process for a unique byte string, which might could be used to find where in memory to find the data to "transfer" (if you prepended it with a unique string that can be searched for).
I think the direction that this sounds like is a quasi or temporary RAM Disk. There are several real RAM Disk applications out there (open source and proprietary), but for programming purposes, they are all kind of crappy because they require some form of installation, reboot, work only from a GUI, or are geared towards being large (in the gigabytes). In most cases, a single application doesn't require a massive percentage of total RAM, so can be quite small without it being an issue.

Seems like it would be great if there were a temporary RAM Disk utility with no installation, where you could work with files as if they were touching Hard Disk. To include those files having a destination and handled like they were on a Hard Drive. And when your application shuts down, so does the temp Ram Disk.
Sam_ wrote:
26 Feb 2019, 11:31
It's not released under any particular license. Feel free to assume WTFPL, so you're welcome to do what you want with it, including rename it.
I believe that legally, if the author doesn't specify a license, than the most restrictive rules on copyright apply. If the author wants to allow for liberal or public use, they must/should choose a license that explicitly allows such. Like MIT, WTFPL, etc... Something can't be assumed to be Public Domain, unless the author gives explicit written permission that it's so. Otherwise, whether it is or isn't can only be decided in a court of law, which may go with the most restrictive interpretation. To avoid confusion, it's arguably better to choose a license, probably even more so if on GitHub.
Sam_
Posts: 146
Joined: 20 Mar 2014, 20:24

Re: MemoryFileIO

09 Jan 2020, 15:26

SOTE wrote:
09 Jan 2020, 15:08
Sam_ wrote:
26 Feb 2019, 11:31
It's not released under any particular license. Feel free to assume WTFPL, so you're welcome to do what you want with it, including rename it.
I believe that legally, if the author doesn't specify a license, than the most restrictive rules on copyright apply. If the author wants to allow for liberal or public use, they must/should choose a license that explicitly allows such. Like MIT, WTFPL, etc... Something can't be assumed to be Public Domain, unless the author gives explicit written permission that it's so. Otherwise, whether it is or isn't can only be decided in a court of law, which will usually go with the most restrictive interpretation.
It may not be legally binding, but my impression is that this community operates under the assumption that code posted to the forums is there to be used, shared, and modified by anyone and everyone without restriction, unless the author explicitly states otherwise. If that is not the case...
SOTE
Posts: 1426
Joined: 15 Jun 2015, 06:21

Re: MemoryFileIO

09 Jan 2020, 15:42

Public Forums are an exception, where this courtesy is extended to the website. Many websites will have a notice, which basically states that if you post on their site, they are not subject to any copyright violations. This exception to copyright extends to the website, not to the users/viewers of it. As the creator of a work, you retain the copyright to it. Others can only use such works, based on your permission and to the extent in which "Fair Use" laws allow. Fair Use laws primarily allow for only personal use, but where the line begins and ends depends on many factors. Thus if you want to avoid confusion, it's best to give explicit permission, if you would like a creation to be public domain.

With that stated, copyright is only as good as the creator can defend it in court. Another point, is how unique the item created is, but that is a technical detail that courts decide. This is why you will see various cases of exploitation, because the creator doesn't have a lawyer, can't sue, or doesn't want to waste the time to get involved with legal matters. But, this doesn't mean there can't be legal problems or fights at some point, as many individuals and companies have found out (sometimes many years after the fact). Thus to avoid such issues and headaches, it's arguably better for the creator to be explicit about permissions and if something is or isn't public domain. You can do a Google search about copyright and public domain, to see how involved and tangled it can get.
User avatar
rommmcek
Posts: 1478
Joined: 15 Aug 2014, 15:18

Re: MemoryFileIO

10 Jan 2020, 21:09

SOTE wrote:Seems like it would be great if there were a temporary RAM Disk utility with no installation, where you could work with files as if they were touching Hard Disk. To include those files having a destination and handled like they were on a Hard Drive. And when your application shuts down, so does the temp Ram Disk.
Maybe RamDrive would help.
SOTE
Posts: 1426
Joined: 15 Jun 2015, 06:21

Re: MemoryFileIO

11 Jan 2020, 20:06

rommmcek wrote:
10 Jan 2020, 21:09
SOTE wrote:Seems like it would be great if there were a temporary RAM Disk utility with no installation, where you could work with files as if they were touching Hard Disk. To include those files having a destination and handled like they were on a Hard Drive. And when your application shuts down, so does the temp Ram Disk.
Maybe RamDrive would help.
Yes, this is a usable solution and close to what I was referring to. However, it relies on 3rd party files, which creates trust and security issues. An entirely AutoHotkey solution, like MemoryFileIO is, would be preferable and more trustworthy.
Qriist
Posts: 82
Joined: 11 Sep 2016, 04:02

Re: MemoryFileIO

27 Mar 2020, 14:44

Hey this is great!

In terms of usage I only need to change one line, right?

Example:
(Full example function: https://github.com/Qriist/Digest/blob/master/lib/Digest_Bin_110f_levels.ahk )

Code: Select all

Digest_Bin_110f_levels(BinToDecompile)	;BinToDecompile = complete filepath
{
	global
	Bin := FileOpen(BinToDecompile,"r")	;current code
	Bin.Read(4)
	RecordSize := 544
	
	loop, % (Bin.Length  - 4) / RecordSize
	{
		;Record size: 544
		RecordID := a_index 
		Record := Digest[ModFullName,"Decompile",Module,RecordID] := {}
		Record["Id"] := Bin.ReadUShort() 
		Record["Pal"] := Bin.ReadUChar() 
		Record["Act"] := Bin.ReadUChar() 
		Record["Teleport"] := Bin.ReadUChar() 
		Record["Rain"] := Bin.ReadUChar()
		===truncated=== 
So I should use this as my new line?

Code: Select all

Bin := New MemoryFileIO(FileOpen(BinToDecompile,"r").ReadRaw())	
Sam_
Posts: 146
Joined: 20 Mar 2014, 20:24

Re: MemoryFileIO

27 Mar 2020, 15:39

Qriist wrote:
27 Mar 2020, 14:44
So I should use this as my new line?

Code: Select all

Bin := New MemoryFileIO(FileOpen(BinToDecompile,"r").ReadRaw())	
I'm not familiar with .ReadRaw(). In general, you'll want to read a file into memory, and then initialize MemoryFIleIO to point to that buffer. Alternatively, you can initialize MemoryFileIO with a blank internal buffer, and then read the file directly into that buffer. Either way, you can then treat the object returned by MemoryFileIO as a File Object.

Untested, but here is an example of the second option mentioned above:

Code: Select all

file:=FileOpen(BinToDecompile,"r")
	Bin:=New MemoryFileIO("",Sz:=file.Length,"",1)
	file.RawRead(Bin.GetBufferAddress()+0,Sz)
file.Close()

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: No registered users and 108 guests