AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

simple binary file read/write functions
Goto page 1, 2, 3, 4, 5  Next
 
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Scripts & Functions
View previous topic :: View next topic  
Author Message
Laszlo



Joined: 14 Feb 2005
Posts: 4078
Location: Pittsburgh

PostPosted: Tue Jul 19, 2005 6:33 pm    Post subject: simple binary file read/write functions Reply with quote

Chris keeps saying that someone has to write simple binary file read/write functions based on the example he provided. Here is mine. They open a file of the given name, read/write hex data and finally close the file. The returned value is the actual number of bytes processed. If there was an error, Errorlevel is set. The data is a long stream of 2-digit hex numbers, without the 0x prefixes.

The script below starts with calling examples, followed by the functions which could be included in other scripts.
Code:
file = c:\z.dat
FileDelete %file%
IfNotEqual ErrorLevel,0, MsgBox Can't delete file "%file%"`nErrorLevel = "%ErrorLevel%"

res := BinWrite(file,"000102030405060708090a0b0c0d0e0f00")
MsgBox ErrorLevel = %ErrorLevel%`nBytes Written = %res%
res := BinRead(file,data)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"

res := BinWrite(file,"aa00bb",0,2)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Written = %res%
res := BinRead(file,data)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"

res := BinRead(file,data,3,-2)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"
ExitApp

/* ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; BinWrite ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|  - Open binary file
|  - (Over)Write n bytes (n = 0: all)
|  - From offset (offset < 0: counted from end)
|  - Close file
|  data -> file[offset + 0..n-1], rest of file unchanged
|  Return #bytes actually written
*/ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

BinWrite(file, data, n=0, offset=0)
{
   ; Open file for WRITE (0x40..), OPEN_ALWAYS (4): creates only if it does not exists
   h := DllCall("CreateFile","str",file,"Uint",0x40000000,"Uint",0,"UInt",0,"UInt",4,"Uint",0,"UInt",0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,Return,0 ; couldn't create the file

   m = 0                            ; seek to offset
   IfLess offset,0, SetEnv,m,2
   r := DllCall("SetFilePointerEx","Uint",h,"Int64",offset,"UInt *",p,"Int",m)
   IfEqual r,0, SetEnv, ErrorLevel, -3
   IfNotEqual ErrorLevel,0, {
      t = %ErrorLevel%              ; save ErrorLevel to be returned
      DllCall("CloseHandle", "Uint", h)
      ErrorLevel = %t%              ; return seek error
      Return 0
   }

   TotalWritten = 0
   m := Ceil(StrLen(data)/2)
   If (n <= 0 or n > m)
       n := m
   Loop %n%
   {
      StringLeft c, data, 2         ; extract next byte
      StringTrimLeft data, data, 2  ; remove  used byte
      c = 0x%c%                     ; make it number
      result := DllCall("WriteFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Written,"UInt",0)
      TotalWritten += Written       ; count written
      if (!result or Written < 1 or ErrorLevel)
         break
   }

   IfNotEqual ErrorLevel,0, SetEnv,t,%ErrorLevel%

   h := DllCall("CloseHandle", "Uint", h)
   IfEqual h,-1, SetEnv, ErrorLevel, -2
   IfNotEqual t,,SetEnv, ErrorLevel, %t%

   Return TotalWritten
}

/* ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; BinRead ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|  - Open binary file
|  - Read n bytes (n = 0: all)
|  - From offset (offset < 0: counted from end)
|  - Close file
|  data (replaced) <- file[offset + 0..n-1]
|  Return #bytes actually read
*/ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

BinRead(file, ByRef data, n=0, offset=0)
{
   h := DllCall("CreateFile","Str",file,"Uint",0x80000000,"Uint",3,"UInt",0,"UInt",3,"Uint",0,"UInt",0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,Return,0 ; couldn't open the file

   m = 0                            ; seek to offset
   IfLess offset,0, SetEnv,m,2
   r := DllCall("SetFilePointerEx","Uint",h,"Int64",offset,"UInt *",p,"Int",m)
   IfEqual r,0, SetEnv, ErrorLevel, -3
   IfNotEqual ErrorLevel,0, {
      t = %ErrorLevel%              ; save ErrorLevel to be returned
      DllCall("CloseHandle", "Uint", h)
      ErrorLevel = %t%              ; return seek error
      Return 0
   }

   TotalRead = 0
   data =
   IfEqual n,0, SetEnv n,0xffffffff ; almost infinite

   format = %A_FormatInteger%       ; save original integer format
   SetFormat Integer, Hex           ; for converting bytes to hex

   Loop %n%
   {
      result := DllCall("ReadFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Read,"UInt",0)
      if (!result or Read < 1 or ErrorLevel)
         break
      TotalRead += Read             ; count read
      c += 0                        ; convert to hex
      StringTrimLeft c, c, 2        ; remove 0x
      c = 0%c%                      ; pad left with 0
      StringRight c, c, 2           ; always 2 digits
      data = %data%%c%              ; append 2 hex digits
   }

   IfNotEqual ErrorLevel,0, SetEnv,t,%ErrorLevel%

   h := DllCall("CloseHandle", "Uint", h)
   IfEqual h,-1, SetEnv, ErrorLevel, -2
   IfNotEqual t,,SetEnv, ErrorLevel, %t%

   SetFormat Integer, %format%      ; restore original format
   Totalread += 0                   ; convert to original format
   Return TotalRead
}
Edit 2005.07.20: With Chris’ tip I was able to update the functions. Now BinRead can read n characters (all if n = 0) starting from an offset ( offset < 0: counting from the end of the file). It allows, for example, getting MP3 tags or icons from a file, without any external SW.

BinWrite now overwrites n bytes of the file (if it exists) starting from the offset (offset < 0 counting from the end of the file) taken from the beginning of data. If n = 0 all of data is written. It does not change the length of the file unless the written data expands over the original end of file. If the file does not exist, it gets created anew. Un-initialized data could be left in the beginning of a newly created file when non-zero offset is specified. Also, writing at an offset beyond the end of the file also leaves un-initialized data between the old end and the offset.

Edit 2005.07.21: Minor clarifications


Last edited by Laszlo on Thu Jul 21, 2005 2:49 pm; edited 3 times in total
Back to top
View user's profile Send private message
Rajat



Joined: 28 Mar 2004
Posts: 1718

PostPosted: Tue Jul 19, 2005 10:39 pm    Post subject: Reply with quote

aw crap! i was going to post those! Wink for binread i was tinkering to see if there's a way to not read 1 byte in a loop and maybe read the whole file or atleast 1kb a time.
thanx for posting 'em! Smile
_________________
Back to top
View user's profile Send private message
Laszlo



Joined: 14 Feb 2005
Posts: 4078
Location: Pittsburgh

PostPosted: Tue Jul 19, 2005 11:02 pm    Post subject: Reply with quote

Rajat, I was thinking about the larger block read speedup, too. Windows probably keeps a whole block of data (512 bytes at ATA drives) in a buffer, and the DLL-calls only access this buffer, not fetching the disk blocks repeatedly. It means, the speedup only eliminates some subroutine calls and returns. They are not drastic.

For me the other issues look more interesting: to read/write in the middle of the file. I have no clue...
Back to top
View user's profile Send private message
Rajat



Joined: 28 Mar 2004
Posts: 1718

PostPosted: Tue Jul 19, 2005 11:59 pm    Post subject: Reply with quote

Quote:
Windows probably keeps a whole block of data (512 bytes at ATA drives) in a buffer, and the DLL-calls only access this buffer, not fetching the disk blocks repeatedly. It means, the speedup only eliminates some subroutine calls and returns. They are not drastic.

i didn't know that... so it probably already runs at a near optimal rate.
_________________
Back to top
View user's profile Send private message
Rajat



Joined: 28 Mar 2004
Posts: 1718

PostPosted: Wed Jul 20, 2005 12:10 am    Post subject: Reply with quote

i tried the following, but the file wasn't recreated.

file = e:\stuff\water.exe

data =
res := BinRead(file,data)
BinWrite("new.exe",res)

the result was just a 2 byte file... where did i mess up?
_________________
Back to top
View user's profile Send private message
Laszlo



Joined: 14 Feb 2005
Posts: 4078
Location: Pittsburgh

PostPosted: Wed Jul 20, 2005 12:26 am    Post subject: Reply with quote

BinWrite() expects as the second parameter a sequence of hex digits. res contains only the number of bytes read by the preceeding Binread(), data is containing the read bytes.
Back to top
View user's profile Send private message
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10480

PostPosted: Wed Jul 20, 2005 12:59 am    Post subject: Re: simple binary file read/write functions Reply with quote

Thanks for writing these functions. I feel a little guilty for pressuring you to do it because your time would probably have been better spent proving that P != NP. Smile

Laszlo wrote:
It would be nice to be able to read from a specified offset in a file, or overwrite certain bytes, insert new ones, delete a few.
I Googled it and it seems SetFilePointer is most likely the API call used for random access of a file's data.
Back to top
View user's profile Send private message Send e-mail
Rajat



Joined: 28 Mar 2004
Posts: 1718

PostPosted: Wed Jul 20, 2005 1:28 am    Post subject: Reply with quote

doh!
thanx laszlo!
_________________
Back to top
View user's profile Send private message
Rajat



Joined: 28 Mar 2004
Posts: 1718

PostPosted: Wed Jul 20, 2005 1:59 am    Post subject: Reply with quote

i made a few minor changes to suit me:
- binread returns the read data
- errorlevel contains bytes read/written for both functions

if function didn't work out smoothly then errorlevel will still have error code

Code:

BinRead(file)            ; return #bytes read
{
   format = %A_FormatInteger%       ; save original integer format
   SetFormat Integer, Hex           ; need hex in here

   h := DllCall("CreateFile","str",file,"Uint",0x80000000,"Uint",3,"UInt",0,"UInt",3,"Uint",0,"UInt", 0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,Return,0 ; couldn't open the file
   TotalRead = 0
   data =
   Loop
   {
      result := DllCall("ReadFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Read,"UInt",0)
      if (!result or Read < 1 or ErrorLevel)
         break
      TotalRead += Read             ; count read
      c += 0                        ; convert to hex
      StringTrimLeft c, c, 2        ; remove 0x
      c = 0%c%                      ; pad left with 0
      StringRight c, c, 2           ; always 2 digits
      StringUpper, c, c
      data = %data%%c%
   }

   SetFormat Integer, %format%      ; restore original format
   Totalread += 0                   ; convert to original format
   ErrorLevel = %TotalRead%

   h := DllCall("CloseHandle", "Uint", h)
   IfEqual h,-1, SetEnv, ErrorLevel, -2

   Return Data
}


BinWrite(file,data)                 ; return #bytes written
{
   h := DllCall("CreateFile","str",file,"Uint",0x40000000,"Uint",0,"UInt",0,"UInt",2,"Uint",0,"UInt",0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,Return,0 ; couldn't create the file
   TotalWritten = 0

   Loop % Ceil(StrLen(data)/2)
   {
      StringLeft c, data, 2         ; extract next byte
      StringTrimLeft data, data, 2  ; remove  used byte
      c = 0x%c%                     ; make it number
      result := DllCall("WriteFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Written,"UInt",0)
      TotalWritten += Written       ; count written
      if (!result or Written < 1 or ErrorLevel)
         break
   }
 
   ErrorLevel = %TotalWritten%

   h := DllCall("CloseHandle", "Uint", h)
   IfEqual h,-1, SetEnv, ErrorLevel, -2
}

_________________
Back to top
View user's profile Send private message
Laszlo



Joined: 14 Feb 2005
Posts: 4078
Location: Pittsburgh

PostPosted: Wed Jul 20, 2005 2:45 am    Post subject: Reply with quote

The original functions had the same calling syntax result:=func(file, data), which I like for their symmetry. Also, if something went wrong ErrorLevel used to tell a lot about what had happened. Now you see less information. The last dll calls might clear ErrorLevel, so you should move the assignment to it after the dll calls, under an Else, but still it is not a clean solution. You have a different design, so some more changes are necessary (besides updating the comments).
Back to top
View user's profile Send private message
Thalon



Joined: 12 Jul 2005
Posts: 640

PostPosted: Wed Jul 20, 2005 8:04 am    Post subject: Reply with quote

I'll have a look over it, maybe I will overtake it in some piece of software (with little changes)!

Thx Laszlo!
Back to top
View user's profile Send private message
Laszlo



Joined: 14 Feb 2005
Posts: 4078
Location: Pittsburgh

PostPosted: Wed Jul 20, 2005 11:42 pm    Post subject: Reply with quote

With Chris’ tip I was able to update the functions (see edit in the original post). Now BinRead can read n characters (all if n = 0) starting from an offset ( < 0 counting from the end of the file). It allows, for example, getting MP3 tags or icons from a file, without any external SW.

BinWrite now overwrites n bytes of the file (if it exists) starting from the offset (< 0 counting from the end of the file) taken from the beginning of data. It does not change the length of the file. If the file does not exist, it gets created anew. In this case un-initialized data could be left in the beginning of the file when non-zero offset is specified.

There could be a further generalization of the BinWrite function with somewhat different syntax: BinWrite(file, data, insert_offset, continue_offset), meaning that the file remains unchanged before the insert_offset, the new data comes after it, but the bytes until continue_offset are dropped. The end of the file remains unchanged.

Changing the length of a binary file looks dangerous, though. If it has code, we can easily mess it up. Still, using the posted functions this data insertion can be easily programmed: read the beginning of the file, concatenate the new data followed by the end of the file also read. The resulting data can be written to a new file.
Back to top
View user's profile Send private message
Laszlo



Joined: 14 Feb 2005
Posts: 4078
Location: Pittsburgh

PostPosted: Thu Jul 21, 2005 3:03 pm    Post subject: Reply with quote

The original post is further edited with minor clarifications an cleanup.

These functions keep the data in hex form, such that the powerful string manipulating and search functions of AHK could be applied. If only pieces need to be cut and pasted, there is no need for hex conversions. Large chunk of data can be manipulated in binary buffers. The details are left as an exercise for the reader. Wink
Back to top
View user's profile Send private message
Rabiator



Joined: 17 Apr 2005
Posts: 271
Location: Sauerland

PostPosted: Sun Jul 24, 2005 5:56 pm    Post subject: Re: simple binary file read/write functions Reply with quote

I always get errorlevel -4 when I run the script on Win 98. The reason is, SetFilePointerEx doesn't exist in this OS.
Is there a similar function?
Back to top
View user's profile Send private message
Laszlo



Joined: 14 Feb 2005
Posts: 4078
Location: Pittsburgh

PostPosted: Sun Jul 24, 2005 6:02 pm    Post subject: Reply with quote

Look here
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Scripts & Functions All times are GMT
Goto page 1, 2, 3, 4, 5  Next
Page 1 of 5

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group