19 replies to this topic

[TheGood]

Similar in essence to Laszlo's BinRead and BinWrite, or Sean's File functions, but I needed something with a bit more flexibility regarding file pointers and keeping the file open for operations (instead of re-opening and closing for every operation). Requires Windows 2000 minimum.

Download or copy/paste:

/* TheGood
Simple file functions

File_Open: Opens a file for subsequent operations
File_Read: Reads bytes from a file at the current file pointer and moves the file pointer forward by the number of bytes read
File_Write: Writes bytes to a file at the current file pointer and moves the file pointer forward by the number of bytes written
File_Pointer: Moves the file pointer to a specified position
File_Size: Returns the size of a file
File_Close: Closes a file
*/

/* sType = One of the following strings:
"READ" : Opens the file for reading
"WRITE" : Opens the file for writing
"READSEQ" : Opens the file for sequential reading (same as READ but offers better performance if reading will mainly be sequential)
sFile = File name to open
On success, returns the file handle
On failure, returns -1 with ErrorLevel = error code
*/
File_Open(sType, sFile) {

bRead := InStr(sType, "READ")
bSeq := sType = "READSEQ"

;Open the file for writing with GENERIC_WRITE/GENERIC_READ, NO SHARING/FILE_SHARE_READ & FILE_SHARE_WRITE, and OPEN_ALWAYS/OPEN_EXISTING, and FILE_FLAG_SEQUENTIAL_SCAN
hFile := DllCall("CreateFile", "str", sFile, "uint", bRead ? 0x80000000 : 0x40000000, "uint", bRead ? 3 : 0, "uint", 0, "uint", bRead ? 3 : 4, "uint", bSeq ? 0x08000000 : 0, "uint", 0)
If (hFile = -1 Or ErrorLevel) { ;Check for any error other than ERROR_FILE_EXISTS
ErrorLevel := ErrorLevel ? ErrorLevel : A_LastError
Return -1 ;Return INVALID_HANDLE_VALUE
} Else Return hFile
}

/* hFile = File handle
bData = Variable which will hold data read
iLength = Number of bytes to read; set to 0 to read to the end
On success, returns the number of bytes actually read
On failure, returns -1 with ErrorLevel = error code
*/
File_Read(hFile, ByRef bData, iLength = 0) {

;Check if we're reading up to the rest of the file
If Not iLength ;Set the length equal to the remaining part of the file
iLength := File_Size(hFile) - File_Pointer(hFile)

;Prep the variable
VarSetCapacity(bData, iLength, 0)

;Read the file
r := DllCall("ReadFile", "uint", hFile, "uint", &bData, "uint", iLength, "uint*", iLengthRead, "uint", 0)
If (Not r Or ErrorLevel) {
ErrorLevel := ErrorLevel ? ErrorLevel : A_LastError
Return -1
} Else Return iLengthRead
}

/* hFile = File handle
ptrData = Pointer to the data to write to the file
iLength = Number of bytes to write
On success, returns the number of bytes actually written
On failure, returns -1 with ErrorLevel = error code
*/
File_Write(hFile, ptrData, iLength) {

;Write to the file
r := DllCall("WriteFile", "uint", hFile, "uint", ptrData, "uint", iLength, "uint*", iLengthWritten, "uint", 0)
If (Not r Or ErrorLevel) {
ErrorLevel := ErrorLevel ? ErrorLevel : A_LastError
Return -1
} Else Return iLengthWritten
}

/* hFile = File handle
iOffset = Number of bytes to move the file pointer. If iMethod = -1, then
A positive iOffset moves the pointer forward from the beginning of the file.
A negative iOffset moves the pointer backwards from the end of the file.
Leave as 0 to return the current file pointer.
If iMethod <> -1, iOffset represents the number of bytes to move the file pointer using the selected method
iMethod = Leave as -1 for the movement to behave as described in iOffset's description
Otherwise, explicitly set to either "BEGINNING" (or 0), "CURRENT" (or 1), or "END" (or 2):
"BEGINNING" : Move starts from the beginning of the file. iOffset must be greater than or equal to 0.
"CURRENT" : Move starts from the current file pointer.
"END" : Move starts from the end of the file. iOffset must be less than or equal to 0.
On success, returns the new file pointer
On failure, returns -1 with ErrorLevel = error code
*/
File_Pointer(hFile, iOffset = 0, iMethod = -1) {

;Check if we're on auto
If (iMethod = -1) {

;Check if we should use FILE_BEGIN, FILE_CURRENT, or FILE_END
If (iOffset = 0)
iMethod := 1 ;We're just retrieving the current pointer. FILE_CURRENT
Else If (iOffset > 0)
iMethod := 0 ;We're moving from the beginning. FILE_BEGIN
Else If (iOffset < 0)
iMethod := 2 ;We're moving from the end. FILE_END
} Else If iMethod Is Not Integer
iMethod := (iMethod = "BEGINNING" ? 0 : (iMethod = "CURRENT" ? 1 : (iMethod = "END" ? 2 : 0)))

r := DllCall("SetFilePointerEx", "uint", hFile, "int64", iOffset, "int64*", iNewPointer, "uint", iMethod)
If (Not r Or ErrorLevel) {
ErrorLevel := ErrorLevel ? ErrorLevel : A_LastError
Return -1
} Else Return iNewPointer
}

/* hFile = File handle
On success, returns the file size
On failure, returns -1 with ErrorLevel = error code
*/
File_Size(hFile) {
r := DllCall("GetFileSizeEx", "uint", hFile, "int64*", iFileSize)
If (Not r Or ErrorLevel) {
ErrorLevel := ErrorLevel ? ErrorLevel : A_LastError
Return -1
} Else Return iFileSize
}

/* hFile = File handle
On success, returns True
On failure, returns False
*/
File_Close(hFile) {
If Not (r := DllCall("CloseHandle", "uint", hFile)) {
ErrorLevel := ErrorLevel ? ErrorLevel : A_LastError
Return False
} Return True
}

Download for AHK_L x86/x64 Unicode

Examples:

Writing to a file:

;Make sure the file doesn't exist
FileDelete, %A_Temp%\File.dat

;Open the file for writing
hFile := File_Open("Write", A_Temp "\File.dat")
If (hFile = -1) {
MsgBox, % "Could not open the file! Error code = " ErrorLevel
ExitApp
}

;Write 3 bytes of 0xFF to the file
VarSetCapacity(bData, 3, 0xFF)
File_Write(hFile, &bData, 3)

;The file now contains FF FF FF

;Append 3 more bytes of 0xFE
VarSetCapacity(bData, 3, 0xFE)
File_Write(hFile, &bData, 3)

;The file now contains FF FF FF FE FE FE

;Put pointer back to the beginning
File_Pointer(hFile, 0, "Beginning")

;(Over)write the first two bytes with 0xFD
VarSetCapacity(bData, 2, 0xFD)
File_Write(hFile, &bData, 2)

;The file now contains FD FD FF FE FE FE

;Put pointer at the end of the file
File_Pointer(hFile, 0, "End")

;Append 3 bytes of 0xFC
VarSetCapacity(bData, 3, 0xFC)
File_Write(hFile, &bData, 3)

;The file now contains FD FD FF FE FE FE FC FC FC

;Close the file now
File_Close(hFile)

Reading from a file (run this after running the first example):

;Open the file for reading
hFile := File_Open("Read", A_Temp "\File.dat")
If (hFile = -1) {
MsgBox, % "Could not open the file! Error code = " ErrorLevel
ExitApp
}

;Get file size
iSize := File_Size(hFile)
MsgBox, % "File size is " iSize " bytes."

;Read the whole file
iRead := File_Read(hFile, bData)
MsgBox, % "File data is: `n" Bin2Hex(&bData, iRead)

;Go back to the beginning of the file
File_Pointer(hFile, 0, "Beginning")

;Read the first 3 bytes only
iRead := File_Read(hFile, bData, 3)
MsgBox, % "First 3 bytes are: `n" Bin2Hex(&bData, iRead)

;Now read the rest of the file (i.e. from the 3rd byte to the end of the file)
iRead := File_Read(hFile, bData)
MsgBox, % "Remaining " iRead " bytes are: `n" Bin2Hex(&bData, iRead)

;We can close the file now
File_Close(hFile)

Return

;By Laszlo
;http://www.autohotkey.com/forum/viewtopic.php?p=135402#135402
Bin2Hex(addr, len) {
Static fun
If (fun = "") {
h=8B54240C85D2568B7424087E3A53578B7C24148A07478AC8C0E90480F9090F97C3F6DB80E30702D980C330240F881E463C090F97C1F6D980E10702C880C130880E464A75CE5F5BC606005EC3
VarSetCapacity(fun, 76)
Loop 76
NumPut("0x" . SubStr(h, 2 * A_Index - 1, 2), fun, A_Index - 1, "Char")
}
VarSetCapacity(hex, 2 * len + 1)
DllCall(&fun, "uint", &hex, "uint", addr, "uint", len, "cdecl")
VarSetCapacity(hex, -1) ;update StrLen
Return hex
}

Also, when manipulating file pointers, make sure you allow for 8 bytes and not only 4 (they are 64-bit integers), or you'll get a wrap-around if manipulating a file with size greater than 4.3 GB.

Enjoy!

[TheGood]

Hi Tuncay,
I will not be renaming the functions, but you can do so if you wish.
And yes, the code is public domain.

[TheGood]

Nice cleanly written code, but you should add a requirements statement to your top post. Some of the system functions (for example GetFileSizeEx) you call are not supported in win9x/me (not sure about w2k).

Good point. I updated the post.

I also updated the code to return strictly -1 on error, to allow 0 as a valid value.

[TheGood]

Not sure how but my examples had disappeared. They're back now.
I also made a few modifications to make the functions more readable (for example, the sType parameter of File_Open and the iMethod parameter of File_Pointer) by using strings instead of numbers/booleans.

[flashkid]

Hi, that's a great code, thanks for sharing it.

Do you think there is a possibility to not overwrite the byte, but paste something at the current pointer?
Example: my text file is "Helo I'm the test" and I want to add a "l" so it becomes "Hello I'm the test".

That's a simple example, but when you do this with large files, you won't have to rewrite the complete file, only because you add a single letter at the beginning.

I hope you know a solution
Greetings, flashkid

[SKAN]

There's no API function to insert bytes.

True.

You would have to write a temporary file where you place all the bytes up to the insertion point, then write your inserted bytes and finally appending the remaining bytes.

You can then replace the temp file with the original.

That would apply only to small files. For large database files, the operation should be done without temporary file.

For eg: To insert "hello" in the beginning of a 10MB file, the following steps would be involved:
1) Open the file
2) Move the pointer to EOF
3) * Write ( Append ) 5 null bytes ( StrLen( "hello" ) = 5 bytes ).
4) Shift 1 MB chunks by 5 bytes from its original postion starting from EOF. ( Read & Write looping backwards )
5) Overwrite first 5 bytes with "Hello"
6) Done. Close the File

* Step 3 is redundant

This is the principle behind for 'record insertion' in a xbase database ( .dbf )

[flashkid]

Thanks for your replies

The way SKAN has described is how I'm doing it at the moment.
That means, if I would delete the 6. byte in a 2gb file, nearly 2gb of data must be written to the hdd?
Why isn't there a solution to delete this byte and the rest of the data is shifted 1 byte to the beginning?

I hope somebody knows a solution, or can explain why it can't be done.
Greetings, flashkid

[SKAN]

That means, if I would delete the 6. byte in a 2gb file, nearly 2gb of data must be written to the hdd?

Why do you want to delete 1 byte in 2gb of data? What kind of file?

Let me give you a classic example:
When ID3 V1 tag was invented, it was decided to append it to the tail of MP3 file so that the MP3 data would be left untouched when the tag data is edited.

Time passed and ID3 V2 dawned. To support backward compatibility with ID3 V1, it was designated to be on the beginning of the file.
So, what many of the good audio-rippers do is that: They accept a few tags before ripping, but actually create a leading space of 4096 bytes and then overwrite that tags in that space.

The unused space left by the ripping software is the delight of Tag editing software. Most of the time, the tag editors will update the tags without moving MP3 data, i.e., as long as the data fits within the 4096 byte range. If not, it will create extra space for the current+future updates. For eg., if tag data is 4097 bytes, it will insert/allocate an additional space of 1024 bytes instead of 1 byte. Hope you get the idea!?

The numbers mentioned will vary between different software

When you insert/update a few tags in a MP3, you will hardly see the HDD light blink. But when you try to insert a large APIC ( Album art ) and you will see it happening... i.e., the data being shifted.

Data management is all about proper design and planning.

[flashkid]

Thanks for that idea, I understand what you mean.

The purpose why I'm trying to insert or delete a byte is, that I want to create a script, which syncs files in binary mode and only the changed data has to be copied and not the complete file.
So a sync between a 1gb and the new 2gb file could be done double as fast as before, because only the next 1gb has to be appended and it's not necessary to delete the first 1gb file and then copy the 2gb file.
Maybe the example is better, if I want to sync a new 1gb with an old 2gb file. So only the first 1gb of the 2gb has to be deleted and not the 2 gb deleted and then the 1 gb appended.

Of course this only works, if the remaining part of the files is matching.
Do you get what I want to do?

Kindly regards, flashkid

[mikennb]

Hi,

 

I am looking for the "https://ahknet.autoh.../AHK_L/File.ahk" the AHK_L x86/x64 Unicode of FILE.AHK but the link seems to be dead. Is there an alternative link to download it or matbe someone would be kind to upload it and post the link.

 

Thank you

[faqbot]

@mikennb: as you can read in the first post: you can Download or copy/paste the code

[mikennb]

Try to open the link https://ahknet.autoh.../AHK_L/File.ahk and it is broken: "

The webpage cannot be found

". If someone has this files I would appreciate if he could upload it and post the link. 

 

Thank you