 |
AutoHotkey Community Let's help each other out
|
| View previous topic :: View next topic |
| Author |
Message |
Laszlo
Joined: 14 Feb 2005 Posts: 4078 Location: Pittsburgh
|
Posted: Tue Jul 19, 2005 6:33 pm Post subject: simple binary file read/write functions |
|
|
Chris keeps saying that someone has to write simple binary file read/write functions based on the example he provided. Here is mine. They open a file of the given name, read/write hex data and finally close the file. The returned value is the actual number of bytes processed. If there was an error, Errorlevel is set. The data is a long stream of 2-digit hex numbers, without the 0x prefixes.
The script below starts with calling examples, followed by the functions which could be included in other scripts. | Code: | file = c:\z.dat
FileDelete %file%
IfNotEqual ErrorLevel,0, MsgBox Can't delete file "%file%"`nErrorLevel = "%ErrorLevel%"
res := BinWrite(file,"000102030405060708090a0b0c0d0e0f00")
MsgBox ErrorLevel = %ErrorLevel%`nBytes Written = %res%
res := BinRead(file,data)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"
res := BinWrite(file,"aa00bb",0,2)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Written = %res%
res := BinRead(file,data)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"
res := BinRead(file,data,3,-2)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"
ExitApp
/* ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; BinWrite ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
| - Open binary file
| - (Over)Write n bytes (n = 0: all)
| - From offset (offset < 0: counted from end)
| - Close file
| data -> file[offset + 0..n-1], rest of file unchanged
| Return #bytes actually written
*/ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
BinWrite(file, data, n=0, offset=0)
{
; Open file for WRITE (0x40..), OPEN_ALWAYS (4): creates only if it does not exists
h := DllCall("CreateFile","str",file,"Uint",0x40000000,"Uint",0,"UInt",0,"UInt",4,"Uint",0,"UInt",0)
IfEqual h,-1, SetEnv, ErrorLevel, -1
IfNotEqual ErrorLevel,0,Return,0 ; couldn't create the file
m = 0 ; seek to offset
IfLess offset,0, SetEnv,m,2
r := DllCall("SetFilePointerEx","Uint",h,"Int64",offset,"UInt *",p,"Int",m)
IfEqual r,0, SetEnv, ErrorLevel, -3
IfNotEqual ErrorLevel,0, {
t = %ErrorLevel% ; save ErrorLevel to be returned
DllCall("CloseHandle", "Uint", h)
ErrorLevel = %t% ; return seek error
Return 0
}
TotalWritten = 0
m := Ceil(StrLen(data)/2)
If (n <= 0 or n > m)
n := m
Loop %n%
{
StringLeft c, data, 2 ; extract next byte
StringTrimLeft data, data, 2 ; remove used byte
c = 0x%c% ; make it number
result := DllCall("WriteFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Written,"UInt",0)
TotalWritten += Written ; count written
if (!result or Written < 1 or ErrorLevel)
break
}
IfNotEqual ErrorLevel,0, SetEnv,t,%ErrorLevel%
h := DllCall("CloseHandle", "Uint", h)
IfEqual h,-1, SetEnv, ErrorLevel, -2
IfNotEqual t,,SetEnv, ErrorLevel, %t%
Return TotalWritten
}
/* ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; BinRead ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
| - Open binary file
| - Read n bytes (n = 0: all)
| - From offset (offset < 0: counted from end)
| - Close file
| data (replaced) <- file[offset + 0..n-1]
| Return #bytes actually read
*/ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
BinRead(file, ByRef data, n=0, offset=0)
{
h := DllCall("CreateFile","Str",file,"Uint",0x80000000,"Uint",3,"UInt",0,"UInt",3,"Uint",0,"UInt",0)
IfEqual h,-1, SetEnv, ErrorLevel, -1
IfNotEqual ErrorLevel,0,Return,0 ; couldn't open the file
m = 0 ; seek to offset
IfLess offset,0, SetEnv,m,2
r := DllCall("SetFilePointerEx","Uint",h,"Int64",offset,"UInt *",p,"Int",m)
IfEqual r,0, SetEnv, ErrorLevel, -3
IfNotEqual ErrorLevel,0, {
t = %ErrorLevel% ; save ErrorLevel to be returned
DllCall("CloseHandle", "Uint", h)
ErrorLevel = %t% ; return seek error
Return 0
}
TotalRead = 0
data =
IfEqual n,0, SetEnv n,0xffffffff ; almost infinite
format = %A_FormatInteger% ; save original integer format
SetFormat Integer, Hex ; for converting bytes to hex
Loop %n%
{
result := DllCall("ReadFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Read,"UInt",0)
if (!result or Read < 1 or ErrorLevel)
break
TotalRead += Read ; count read
c += 0 ; convert to hex
StringTrimLeft c, c, 2 ; remove 0x
c = 0%c% ; pad left with 0
StringRight c, c, 2 ; always 2 digits
data = %data%%c% ; append 2 hex digits
}
IfNotEqual ErrorLevel,0, SetEnv,t,%ErrorLevel%
h := DllCall("CloseHandle", "Uint", h)
IfEqual h,-1, SetEnv, ErrorLevel, -2
IfNotEqual t,,SetEnv, ErrorLevel, %t%
SetFormat Integer, %format% ; restore original format
Totalread += 0 ; convert to original format
Return TotalRead
} | Edit 2005.07.20: With Chris’ tip I was able to update the functions. Now BinRead can read n characters (all if n = 0) starting from an offset ( offset < 0: counting from the end of the file). It allows, for example, getting MP3 tags or icons from a file, without any external SW.
BinWrite now overwrites n bytes of the file (if it exists) starting from the offset (offset < 0 counting from the end of the file) taken from the beginning of data. If n = 0 all of data is written. It does not change the length of the file unless the written data expands over the original end of file. If the file does not exist, it gets created anew. Un-initialized data could be left in the beginning of a newly created file when non-zero offset is specified. Also, writing at an offset beyond the end of the file also leaves un-initialized data between the old end and the offset.
Edit 2005.07.21: Minor clarifications
Last edited by Laszlo on Thu Jul 21, 2005 2:49 pm; edited 3 times in total |
|
| Back to top |
|
 |
Rajat
Joined: 28 Mar 2004 Posts: 1718
|
Posted: Tue Jul 19, 2005 10:39 pm Post subject: |
|
|
aw crap! i was going to post those! for binread i was tinkering to see if there's a way to not read 1 byte in a loop and maybe read the whole file or atleast 1kb a time.
thanx for posting 'em!  _________________
 |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4078 Location: Pittsburgh
|
Posted: Tue Jul 19, 2005 11:02 pm Post subject: |
|
|
Rajat, I was thinking about the larger block read speedup, too. Windows probably keeps a whole block of data (512 bytes at ATA drives) in a buffer, and the DLL-calls only access this buffer, not fetching the disk blocks repeatedly. It means, the speedup only eliminates some subroutine calls and returns. They are not drastic.
For me the other issues look more interesting: to read/write in the middle of the file. I have no clue... |
|
| Back to top |
|
 |
Rajat
Joined: 28 Mar 2004 Posts: 1718
|
Posted: Tue Jul 19, 2005 11:59 pm Post subject: |
|
|
| Quote: | | Windows probably keeps a whole block of data (512 bytes at ATA drives) in a buffer, and the DLL-calls only access this buffer, not fetching the disk blocks repeatedly. It means, the speedup only eliminates some subroutine calls and returns. They are not drastic. |
i didn't know that... so it probably already runs at a near optimal rate. _________________
 |
|
| Back to top |
|
 |
Rajat
Joined: 28 Mar 2004 Posts: 1718
|
Posted: Wed Jul 20, 2005 12:10 am Post subject: |
|
|
i tried the following, but the file wasn't recreated.
file = e:\stuff\water.exe
data =
res := BinRead(file,data)
BinWrite("new.exe",res)
the result was just a 2 byte file... where did i mess up? _________________
 |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4078 Location: Pittsburgh
|
Posted: Wed Jul 20, 2005 12:26 am Post subject: |
|
|
| BinWrite() expects as the second parameter a sequence of hex digits. res contains only the number of bytes read by the preceeding Binread(), data is containing the read bytes. |
|
| Back to top |
|
 |
Chris Site Admin
Joined: 02 Mar 2004 Posts: 10480
|
Posted: Wed Jul 20, 2005 12:59 am Post subject: Re: simple binary file read/write functions |
|
|
Thanks for writing these functions. I feel a little guilty for pressuring you to do it because your time would probably have been better spent proving that P != NP.
| Laszlo wrote: | | It would be nice to be able to read from a specified offset in a file, or overwrite certain bytes, insert new ones, delete a few. | I Googled it and it seems SetFilePointer is most likely the API call used for random access of a file's data. |
|
| Back to top |
|
 |
Rajat
Joined: 28 Mar 2004 Posts: 1718
|
Posted: Wed Jul 20, 2005 1:28 am Post subject: |
|
|
doh!
thanx laszlo! _________________
 |
|
| Back to top |
|
 |
Rajat
Joined: 28 Mar 2004 Posts: 1718
|
Posted: Wed Jul 20, 2005 1:59 am Post subject: |
|
|
i made a few minor changes to suit me:
- binread returns the read data
- errorlevel contains bytes read/written for both functions
if function didn't work out smoothly then errorlevel will still have error code
| Code: |
BinRead(file) ; return #bytes read
{
format = %A_FormatInteger% ; save original integer format
SetFormat Integer, Hex ; need hex in here
h := DllCall("CreateFile","str",file,"Uint",0x80000000,"Uint",3,"UInt",0,"UInt",3,"Uint",0,"UInt", 0)
IfEqual h,-1, SetEnv, ErrorLevel, -1
IfNotEqual ErrorLevel,0,Return,0 ; couldn't open the file
TotalRead = 0
data =
Loop
{
result := DllCall("ReadFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Read,"UInt",0)
if (!result or Read < 1 or ErrorLevel)
break
TotalRead += Read ; count read
c += 0 ; convert to hex
StringTrimLeft c, c, 2 ; remove 0x
c = 0%c% ; pad left with 0
StringRight c, c, 2 ; always 2 digits
StringUpper, c, c
data = %data%%c%
}
SetFormat Integer, %format% ; restore original format
Totalread += 0 ; convert to original format
ErrorLevel = %TotalRead%
h := DllCall("CloseHandle", "Uint", h)
IfEqual h,-1, SetEnv, ErrorLevel, -2
Return Data
}
BinWrite(file,data) ; return #bytes written
{
h := DllCall("CreateFile","str",file,"Uint",0x40000000,"Uint",0,"UInt",0,"UInt",2,"Uint",0,"UInt",0)
IfEqual h,-1, SetEnv, ErrorLevel, -1
IfNotEqual ErrorLevel,0,Return,0 ; couldn't create the file
TotalWritten = 0
Loop % Ceil(StrLen(data)/2)
{
StringLeft c, data, 2 ; extract next byte
StringTrimLeft data, data, 2 ; remove used byte
c = 0x%c% ; make it number
result := DllCall("WriteFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Written,"UInt",0)
TotalWritten += Written ; count written
if (!result or Written < 1 or ErrorLevel)
break
}
ErrorLevel = %TotalWritten%
h := DllCall("CloseHandle", "Uint", h)
IfEqual h,-1, SetEnv, ErrorLevel, -2
}
|
_________________
 |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4078 Location: Pittsburgh
|
Posted: Wed Jul 20, 2005 2:45 am Post subject: |
|
|
| The original functions had the same calling syntax result:=func(file, data), which I like for their symmetry. Also, if something went wrong ErrorLevel used to tell a lot about what had happened. Now you see less information. The last dll calls might clear ErrorLevel, so you should move the assignment to it after the dll calls, under an Else, but still it is not a clean solution. You have a different design, so some more changes are necessary (besides updating the comments). |
|
| Back to top |
|
 |
Thalon
Joined: 12 Jul 2005 Posts: 640
|
Posted: Wed Jul 20, 2005 8:04 am Post subject: |
|
|
I'll have a look over it, maybe I will overtake it in some piece of software (with little changes)!
Thx Laszlo! |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4078 Location: Pittsburgh
|
Posted: Wed Jul 20, 2005 11:42 pm Post subject: |
|
|
With Chris’ tip I was able to update the functions (see edit in the original post). Now BinRead can read n characters (all if n = 0) starting from an offset ( < 0 counting from the end of the file). It allows, for example, getting MP3 tags or icons from a file, without any external SW.
BinWrite now overwrites n bytes of the file (if it exists) starting from the offset (< 0 counting from the end of the file) taken from the beginning of data. It does not change the length of the file. If the file does not exist, it gets created anew. In this case un-initialized data could be left in the beginning of the file when non-zero offset is specified.
There could be a further generalization of the BinWrite function with somewhat different syntax: BinWrite(file, data, insert_offset, continue_offset), meaning that the file remains unchanged before the insert_offset, the new data comes after it, but the bytes until continue_offset are dropped. The end of the file remains unchanged.
Changing the length of a binary file looks dangerous, though. If it has code, we can easily mess it up. Still, using the posted functions this data insertion can be easily programmed: read the beginning of the file, concatenate the new data followed by the end of the file also read. The resulting data can be written to a new file. |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4078 Location: Pittsburgh
|
Posted: Thu Jul 21, 2005 3:03 pm Post subject: |
|
|
The original post is further edited with minor clarifications an cleanup.
These functions keep the data in hex form, such that the powerful string manipulating and search functions of AHK could be applied. If only pieces need to be cut and pasted, there is no need for hex conversions. Large chunk of data can be manipulated in binary buffers. The details are left as an exercise for the reader.  |
|
| Back to top |
|
 |
Rabiator
Joined: 17 Apr 2005 Posts: 271 Location: Sauerland
|
Posted: Sun Jul 24, 2005 5:56 pm Post subject: Re: simple binary file read/write functions |
|
|
I always get errorlevel -4 when I run the script on Win 98. The reason is, SetFilePointerEx doesn't exist in this OS.
Is there a similar function? |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4078 Location: Pittsburgh
|
Posted: Sun Jul 24, 2005 6:02 pm Post subject: |
|
|
| Look here |
|
| Back to top |
|
 |
|
|
You can post new topics in this forum You can reply to topics in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|