AutoHotkey Community

It is currently May 26th, 2012, 8:18 pm

All times are UTC [ DST ]




Post new topic Reply to topic  [ 62 posts ]  Go to page 1, 2, 3, 4, 5  Next
Author Message
PostPosted: July 19th, 2005, 7:33 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
Chris keeps saying that someone has to write simple binary file read/write functions based on the example he provided. Here is mine. They open a file of the given name, read/write hex data and finally close the file. The returned value is the actual number of bytes processed. If there was an error, Errorlevel is set. The data is a long stream of 2-digit hex numbers, without the 0x prefixes.

The script below starts with calling examples, followed by the functions which could be included in other scripts.
Code:
file = c:\z.dat
FileDelete %file%
IfNotEqual ErrorLevel,0, MsgBox Can't delete file "%file%"`nErrorLevel = "%ErrorLevel%"

res := BinWrite(file,"000102030405060708090a0b0c0d0e0f00")
MsgBox ErrorLevel = %ErrorLevel%`nBytes Written = %res%
res := BinRead(file,data)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"

res := BinWrite(file,"aa00bb",0,2)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Written = %res%
res := BinRead(file,data)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"

res := BinRead(file,data,3,-2)
MsgBox ErrorLevel = %ErrorLevel%`nBytes Read = %res%`nData = "%data%"
ExitApp

/* ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; BinWrite ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|  - Open binary file
|  - (Over)Write n bytes (n = 0: all)
|  - From offset (offset < 0: counted from end)
|  - Close file
|  data -> file[offset + 0..n-1], rest of file unchanged
|  Return #bytes actually written
*/ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

BinWrite(file, data, n=0, offset=0)
{
   ; Open file for WRITE (0x40..), OPEN_ALWAYS (4): creates only if it does not exists
   h := DllCall("CreateFile","str",file,"Uint",0x40000000,"Uint",0,"UInt",0,"UInt",4,"Uint",0,"UInt",0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,Return,0 ; couldn't create the file

   m = 0                            ; seek to offset
   IfLess offset,0, SetEnv,m,2
   r := DllCall("SetFilePointerEx","Uint",h,"Int64",offset,"UInt *",p,"Int",m)
   IfEqual r,0, SetEnv, ErrorLevel, -3
   IfNotEqual ErrorLevel,0, {
      t = %ErrorLevel%              ; save ErrorLevel to be returned
      DllCall("CloseHandle", "Uint", h)
      ErrorLevel = %t%              ; return seek error
      Return 0
   }

   TotalWritten = 0
   m := Ceil(StrLen(data)/2)
   If (n <= 0 or n > m)
       n := m
   Loop %n%
   {
      StringLeft c, data, 2         ; extract next byte
      StringTrimLeft data, data, 2  ; remove  used byte
      c = 0x%c%                     ; make it number
      result := DllCall("WriteFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Written,"UInt",0)
      TotalWritten += Written       ; count written
      if (!result or Written < 1 or ErrorLevel)
         break
   }

   IfNotEqual ErrorLevel,0, SetEnv,t,%ErrorLevel%

   h := DllCall("CloseHandle", "Uint", h)
   IfEqual h,-1, SetEnv, ErrorLevel, -2
   IfNotEqual t,,SetEnv, ErrorLevel, %t%

   Return TotalWritten
}

/* ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; BinRead ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|  - Open binary file
|  - Read n bytes (n = 0: all)
|  - From offset (offset < 0: counted from end)
|  - Close file
|  data (replaced) <- file[offset + 0..n-1]
|  Return #bytes actually read
*/ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

BinRead(file, ByRef data, n=0, offset=0)
{
   h := DllCall("CreateFile","Str",file,"Uint",0x80000000,"Uint",3,"UInt",0,"UInt",3,"Uint",0,"UInt",0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,Return,0 ; couldn't open the file

   m = 0                            ; seek to offset
   IfLess offset,0, SetEnv,m,2
   r := DllCall("SetFilePointerEx","Uint",h,"Int64",offset,"UInt *",p,"Int",m)
   IfEqual r,0, SetEnv, ErrorLevel, -3
   IfNotEqual ErrorLevel,0, {
      t = %ErrorLevel%              ; save ErrorLevel to be returned
      DllCall("CloseHandle", "Uint", h)
      ErrorLevel = %t%              ; return seek error
      Return 0
   }

   TotalRead = 0
   data =
   IfEqual n,0, SetEnv n,0xffffffff ; almost infinite

   format = %A_FormatInteger%       ; save original integer format
   SetFormat Integer, Hex           ; for converting bytes to hex

   Loop %n%
   {
      result := DllCall("ReadFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Read,"UInt",0)
      if (!result or Read < 1 or ErrorLevel)
         break
      TotalRead += Read             ; count read
      c += 0                        ; convert to hex
      StringTrimLeft c, c, 2        ; remove 0x
      c = 0%c%                      ; pad left with 0
      StringRight c, c, 2           ; always 2 digits
      data = %data%%c%              ; append 2 hex digits
   }

   IfNotEqual ErrorLevel,0, SetEnv,t,%ErrorLevel%

   h := DllCall("CloseHandle", "Uint", h)
   IfEqual h,-1, SetEnv, ErrorLevel, -2
   IfNotEqual t,,SetEnv, ErrorLevel, %t%

   SetFormat Integer, %format%      ; restore original format
   Totalread += 0                   ; convert to original format
   Return TotalRead
}
Edit 2005.07.20: With Chris’ tip I was able to update the functions. Now BinRead can read n characters (all if n = 0) starting from an offset ( offset < 0: counting from the end of the file). It allows, for example, getting MP3 tags or icons from a file, without any external SW.

BinWrite now overwrites n bytes of the file (if it exists) starting from the offset (offset < 0 counting from the end of the file) taken from the beginning of data. If n = 0 all of data is written. It does not change the length of the file unless the written data expands over the original end of file. If the file does not exist, it gets created anew. Un-initialized data could be left in the beginning of a newly created file when non-zero offset is specified. Also, writing at an offset beyond the end of the file also leaves un-initialized data between the old end and the offset.

Edit 2005.07.21: Minor clarifications


Last edited by Laszlo on July 21st, 2005, 3:49 pm, edited 3 times in total.

Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 19th, 2005, 11:39 pm 
Offline

Joined: March 28th, 2004, 3:53 pm
Posts: 1870
aw crap! i was going to post those! ;) for binread i was tinkering to see if there's a way to not read 1 byte in a loop and maybe read the whole file or atleast 1kb a time.
thanx for posting 'em! :)

_________________
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 20th, 2005, 12:02 am 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
Rajat, I was thinking about the larger block read speedup, too. Windows probably keeps a whole block of data (512 bytes at ATA drives) in a buffer, and the DLL-calls only access this buffer, not fetching the disk blocks repeatedly. It means, the speedup only eliminates some subroutine calls and returns. They are not drastic.

For me the other issues look more interesting: to read/write in the middle of the file. I have no clue...


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 20th, 2005, 12:59 am 
Offline

Joined: March 28th, 2004, 3:53 pm
Posts: 1870
Quote:
Windows probably keeps a whole block of data (512 bytes at ATA drives) in a buffer, and the DLL-calls only access this buffer, not fetching the disk blocks repeatedly. It means, the speedup only eliminates some subroutine calls and returns. They are not drastic.

i didn't know that... so it probably already runs at a near optimal rate.

_________________
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 20th, 2005, 1:10 am 
Offline

Joined: March 28th, 2004, 3:53 pm
Posts: 1870
i tried the following, but the file wasn't recreated.

file = e:\stuff\water.exe

data =
res := BinRead(file,data)
BinWrite("new.exe",res)

the result was just a 2 byte file... where did i mess up?

_________________
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 20th, 2005, 1:26 am 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
BinWrite() expects as the second parameter a sequence of hex digits. res contains only the number of bytes read by the preceeding Binread(), data is containing the read bytes.


Report this post
Top
 Profile  
Reply with quote  
PostPosted: July 20th, 2005, 1:59 am 
Offline

Joined: March 2nd, 2004, 3:36 pm
Posts: 10720
Thanks for writing these functions. I feel a little guilty for pressuring you to do it because your time would probably have been better spent proving that P != NP. :)

Laszlo wrote:
It would be nice to be able to read from a specified offset in a file, or overwrite certain bytes, insert new ones, delete a few.
I Googled it and it seems SetFilePointer is most likely the API call used for random access of a file's data.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 20th, 2005, 2:28 am 
Offline

Joined: March 28th, 2004, 3:53 pm
Posts: 1870
doh!
thanx laszlo!

_________________
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 20th, 2005, 2:59 am 
Offline

Joined: March 28th, 2004, 3:53 pm
Posts: 1870
i made a few minor changes to suit me:
- binread returns the read data
- errorlevel contains bytes read/written for both functions

if function didn't work out smoothly then errorlevel will still have error code

Code:
BinRead(file)            ; return #bytes read
{
   format = %A_FormatInteger%       ; save original integer format
   SetFormat Integer, Hex           ; need hex in here

   h := DllCall("CreateFile","str",file,"Uint",0x80000000,"Uint",3,"UInt",0,"UInt",3,"Uint",0,"UInt", 0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,Return,0 ; couldn't open the file
   TotalRead = 0
   data =
   Loop
   {
      result := DllCall("ReadFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Read,"UInt",0)
      if (!result or Read < 1 or ErrorLevel)
         break
      TotalRead += Read             ; count read
      c += 0                        ; convert to hex
      StringTrimLeft c, c, 2        ; remove 0x
      c = 0%c%                      ; pad left with 0
      StringRight c, c, 2           ; always 2 digits
      StringUpper, c, c
      data = %data%%c%
   }

   SetFormat Integer, %format%      ; restore original format
   Totalread += 0                   ; convert to original format
   ErrorLevel = %TotalRead%

   h := DllCall("CloseHandle", "Uint", h)
   IfEqual h,-1, SetEnv, ErrorLevel, -2

   Return Data
}


BinWrite(file,data)                 ; return #bytes written
{
   h := DllCall("CreateFile","str",file,"Uint",0x40000000,"Uint",0,"UInt",0,"UInt",2,"Uint",0,"UInt",0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,Return,0 ; couldn't create the file
   TotalWritten = 0

   Loop % Ceil(StrLen(data)/2)
   {
      StringLeft c, data, 2         ; extract next byte
      StringTrimLeft data, data, 2  ; remove  used byte
      c = 0x%c%                     ; make it number
      result := DllCall("WriteFile","UInt",h,"UChar *",c,"UInt",1,"UInt *",Written,"UInt",0)
      TotalWritten += Written       ; count written
      if (!result or Written < 1 or ErrorLevel)
         break
   }
 
   ErrorLevel = %TotalWritten%

   h := DllCall("CloseHandle", "Uint", h)
   IfEqual h,-1, SetEnv, ErrorLevel, -2
}

_________________
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 20th, 2005, 3:45 am 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
The original functions had the same calling syntax result:=func(file, data), which I like for their symmetry. Also, if something went wrong ErrorLevel used to tell a lot about what had happened. Now you see less information. The last dll calls might clear ErrorLevel, so you should move the assignment to it after the dll calls, under an Else, but still it is not a clean solution. You have a different design, so some more changes are necessary (besides updating the comments).


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 20th, 2005, 9:04 am 
Offline

Joined: July 12th, 2005, 1:21 pm
Posts: 633
I'll have a look over it, maybe I will overtake it in some piece of software (with little changes)!

Thx Laszlo!


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 21st, 2005, 12:42 am 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
With Chris’ tip I was able to update the functions (see edit in the original post). Now BinRead can read n characters (all if n = 0) starting from an offset ( < 0 counting from the end of the file). It allows, for example, getting MP3 tags or icons from a file, without any external SW.

BinWrite now overwrites n bytes of the file (if it exists) starting from the offset (< 0 counting from the end of the file) taken from the beginning of data. It does not change the length of the file. If the file does not exist, it gets created anew. In this case un-initialized data could be left in the beginning of the file when non-zero offset is specified.

There could be a further generalization of the BinWrite function with somewhat different syntax: BinWrite(file, data, insert_offset, continue_offset), meaning that the file remains unchanged before the insert_offset, the new data comes after it, but the bytes until continue_offset are dropped. The end of the file remains unchanged.

Changing the length of a binary file looks dangerous, though. If it has code, we can easily mess it up. Still, using the posted functions this data insertion can be easily programmed: read the beginning of the file, concatenate the new data followed by the end of the file also read. The resulting data can be written to a new file.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 21st, 2005, 4:03 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
The original post is further edited with minor clarifications an cleanup.

These functions keep the data in hex form, such that the powerful string manipulating and search functions of AHK could be applied. If only pieces need to be cut and pasted, there is no need for hex conversions. Large chunk of data can be manipulated in binary buffers. The details are left as an exercise for the reader. ;-)


Report this post
Top
 Profile  
Reply with quote  
PostPosted: July 24th, 2005, 6:56 pm 
Offline

Joined: April 17th, 2005, 7:47 pm
Posts: 289
Location: Sauerland
I always get errorlevel -4 when I run the script on Win 98. The reason is, SetFilePointerEx doesn't exist in this OS.
Is there a similar function?


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: July 24th, 2005, 7:02 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
Look here


Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 62 posts ]  Go to page 1, 2, 3, 4, 5  Next

All times are UTC [ DST ]


Who is online

Users browsing this forum: Exabot [Bot], notsoobvious and 20 guests


You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group