AutoHotkey Community

It is currently May 27th, 2012, 4:08 am

All times are UTC [ DST ]




Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: January 23rd, 2006, 3:39 pm 
Offline

Joined: December 27th, 2005, 1:46 pm
Posts: 6837
Location: France (near Paris)
Laszlo already provided some nice code to do this, and actually I "stole" (with due credit) a lot of code from his work, thanks!

So, why does I publish this? I plan to parse some binary headers, so I need to do several operations on the same file. Laszlo's routines open the file, perform the operation and close the file. Mine open the file, keep the file handle around for various operations, and close the file.

Laszlo's routines are perfectly in the spirit of most AHK commands, like FileAppend: you don't have to remember to open or close the file.
My routines seek a little more performance, even if I am not sure if this is really noticeable.

Well, at least, I improved my DllCall knowledge, and I have some little bonus like allowing relative moves, calling SetFilePointer only if requested (great for sequential reading/writing), no call to GetFileSize if not needed, etc. Plus I had fun making some tricks like improving the decimal to 2-digit hex conversion... :-)

Caveat: by design, these functions may malfunction with files larger than 2GB... (videos, DVD ISO images, server logs...)
Remark 1: it can still work if you only want to read the first few bytes to parse the header.
Remark 2: when dealing with such files, you should better switch to a compiled language, or perhaps to Perl...

You can download BinReadWrite.ahk and TestBinReadWrite.ahk to avoid selecting the code, copying it and pasting it in an editor, loosing tabs...

BinReadWrite.ahk
Code:
/*
BinReadWrite.ahk

Routines to read and write binary data from/to files.
Based on original functions written by Laszlo
http://www.autohotkey.com/forum/viewtopic.php?t=4604

TODO: Perhaps set a lastError variable to explicit the errors.

// by Philippe Lhoste <PhiLho(a)GMX.net> http://Phi.Lho.free.fr
// File/Project history:
 1.03.000 -- 2006/02/15 (PL) -- Moved Bin2Hex & Hex2Bin to DllCallStruct, apply code rules.
 1.02.000 -- 2006/01/24 (PL) -- Slight change of the API: all functions return -1 if error.
           Integrated Laszlo suggestions on improving Bin2Hex and Hex2Bin.
 1.01.000 -- 2006/01/23 (PL) -- Declaration of the local variables, to get access to global
          WinAPI constants, and for consistency.
 1.00.000 -- 2006/01/19 (PL) -- Rewrote the functions to separate opening and closing,
          allowing efficient multiple operations.
*/

; WinAPI constants
INVALID_HANDLE_VALUE = -1
INVALID_FILE_SIZE = 0xFFFFFFFF
FILE_BEGIN = 0
FILE_CURRENT = 1
FILE_END = 2

/*
// Open the file for reading.
// Return the file handle to provide in further read operations and in the final close operation,
// or INVALID_HANDLE_VALUE if an error was found.
*/
OpenFileForRead(_filename)
{
   local handle

   handle := DllCall("CreateFile"
         , "Str", _filename      ; lpFileName
         , "UInt", 0x80000000      ; dwDesiredAccess (GENERIC_READ)
         , "UInt", 3      ; dwShareMode (FILE_SHARE_READ|FILE_SHARE_WRITE)
         , "UInt", 0      ; lpSecurityAttributes
         , "UInt", 3      ; dwCreationDisposition (OPEN_EXISTING)
         , "UInt", 0      ; dwFlagsAndAttributes
         , "UInt", 0)   ; hTemplateFile
   If (handle = INVALID_HANDLE_VALUE or handle = 0)
   {
      ErrorLevel = -1
   }
   IfNotEqual ErrorLevel, 0, Return INVALID_HANDLE_VALUE   ; Couldn't open the file
   Return handle
}

/*
// Open the file for writing.
// Return the file handle to provide in further write operations and in the final close operation,
// or INVALID_HANDLE_VALUE if an error was found.
*/
OpenFileForWrite(_filename)
{
   local handle

   handle := DllCall("CreateFile"
         , "Str", _filename      ; lpFileName
         , "UInt", 0x40000000   ; dwDesiredAccess (GENERIC_WRITE)
         , "UInt", 3      ; dwShareMode (FILE_SHARE_READ|FILE_SHARE_WRITE)
         , "UInt", 0      ; lpSecurityAttributes
         , "UInt", 4      ; dwCreationDisposition (OPEN_ALWAYS: create if not exists)
         , "UInt", 0      ; dwFlagsAndAttributes
         , "UInt", 0)   ; hTemplateFile
   If (handle = INVALID_HANDLE_VALUE or handle = 0)
   {
      ErrorLevel = -1
   }
   IfNotEqual ErrorLevel, 0, Return INVALID_HANDLE_VALUE   ; Couldn't open the file
   Return handle
}

/*
// Close the file.
*/
CloseFile(_handle)
{
   local result

   result := DllCall("CloseHandle"
         , "UInt", _handle)
   If (result = 0)
   {
      ErrorLevel = -1
   }
}

/*
// Get the size of the opened file, in bytes.
// Limited to 4GB, so it is more limited that AHK's FileGetSize.
// It is here for consistency, and because it accepts a file handle instead of a path.
//
// Return the size in bytes, -1 if there was an error.
*/
GetFileSize(_handle)
{
   local fileSize

   fileSize := DllCall("GetFileSize"
         , "UInt", _handle
         , "UInt", 0)
   If (fileSize = INVALID_FILE_SIZE)
   {
      ErrorLevel = -1
   }
   IfNotEqual ErrorLevel, 0, Return -1
   Return fileSize
}

/*
// Move the file pointer in the file to the given offset relative to moveMethod.
//
// moveMethod can be FILE_BEGIN, FILE_CURRENT or FILE_END.
// If moveMethod is -1, nothing is done (default, for operations at current position).
// To get the current position, call this function with just FILE_CURRENT (null offset).
// offset can be positive (toward end of the file) or negative (toward start of the file).
//
// Return -1 if there was an error, the new file pointer position if OK.
// Note: Currently it doesn't work for files larger than 2GB...
*/
MoveInFile(_handle, _moveMethod=-1, _offset=0)
{
   local result

   result = %INVALID_FILE_SIZE%
   if (_moveMethod != -1)
   {
      result := DllCall("SetFilePointer"
            , "UInt", _handle      ; hFile
            , "Int", _offset      ; lDistanceToMove
            , "UInt", 0         ; lpDistanceToMoveHigh
            , "UInt", _moveMethod)   ; dwMoveMethod
      if (result = -1)   ; INVALID_SET_FILE_POINTER
      {
         ErrorLevel = -1
      }
      IfNotEqual ErrorLevel, 0, Return -1   ; Couldn't make the move
   }
   Return result
}

/*
// Write in a file opened for writing.
//
// Move to position given by moveMethod and offset
// (by default stand at current position) and
// write byteNb bytes from data (all data if byteNb = 0;
// data contains binary bytes that can be a string or
// raw bytes generated from hexa data with the Hex2Bin routine).
//
// moveMethod, defaulting to -1 (no move, write at current position),
// can also be FILE_BEGIN, FILE_CURRENT or FILE_END.
// offset can be positive (toward end of file) or negative (toward beginning of file).
//
// Return the number of bytes written (-1 if there was an error).
*/
WriteInFile(_handle, ByRef @data, _byteNb=0, _moveMethod=-1, _offset=0)
{
   local dataSize, result, written

   _offset := MoveInFile(_handle, _moveMethod, _offset)
   IfNotEqual ErrorLevel, 0, Return -1   ; Couldn't make the move

   dataSize := VarSetCapacity(@data)        ; Get the capacity (>= used length!)
   If (_byteNb < 1 or _byteNb > dataSize)
   {
      byteNb := dataSize
   }
   result := DllCall("WriteFile"
         , "UInt", _handle   ; hFile
         , "Str", @data      ; lpBuffer
         , "UInt", _byteNb   ; nNumberOfBytesToWrite
         , "UInt *", written   ; lpNumberOfBytesWritten
         , "UInt", 0)      ; lpOverlapped
   if (result = 0 or written < _byteNb)
   {
      ErrorLevel = -2
   }
   IfNotEqual ErrorLevel, 0, Return -1   ; Couldn't write in the file
   Return written
}

/*
// Read from a file opened for reading.
//
// Move to position given by moveMethod and offset
// (by default stand at current position) and
// read byteNb bytes in data (the whole file if byteNb = 0;
// data contains binary bytes that can be a string or
// raw bytes that can be converted to hex digits with the Bin2Hex routine).
//
// moveMethod, defaulting to -1 (no move, read at current position),
// can also be FILE_BEGIN, FILE_CURRENT or FILE_END.
// offset can be positive (toward end of file) or negative (toward beginning of file).
//
// Return the number of bytes read (-1 if there was an error), which can be less
// than requested if end-of-file is meet.
*/
ReadFromFile(_handle, ByRef @data, _byteNb=0, _moveMethod=-1, _offset=0)
{
   local fileSize, granted, result, read

   _offset := MoveInFile(_handle, _moveMethod, _offset)
   IfNotEqual ErrorLevel, 0, Return -1   ; Couldn't make the move

   if (_byteNb = 0)
   {
      ; Read whole file (or less if file pointer isn't at start)
      fileSize := GetFileSize(_handle)
      IfNotEqual ErrorLevel, 0, Return -1   ; Couldn't get the file size
      _byteNb := fileSize
   }
   granted := VarSetCapacity(@data, _byteNb, 0)
   if (granted < _byteNb)
   {
      ; Cannot allocate enough memory
      ErrorLevel = Mem=%granted%
      Return -1
   }

   result := DllCall("ReadFile"
         , "UInt", _handle   ; hFile
         , "Str", @data      ; lpBuffer
         , "UInt", _byteNb   ; nNumberOfBytesToRead
         , "UInt *", read   ; lpNumberOfBytesRead
         , "UInt", 0)      ; lpOverlapped
   if (result = 0)
   {
      ErrorLevel = -2
   }
;~ MsgBox fileSize: %fileSize% - offset: %_offset% - byteNb: %_byteNb% - granted: %granted% - read: %read%
   IfNotEqual ErrorLevel, 0, Return -1   ; Couldn't read the file

   ; Note that we can have read less data than requested,
   ; eg. if end of file has been meet
   Return read
}


Note (obsolete, see links above): providing large sources on this forum is inconvenient, I would prefer to offer a file. I can, on my site, but it would need to set up a AHK page, to avoid direct download, as host doesn't like much them...
I know the inconveniences of allowing to upload files to a forum (lot of dead files, spam, virus...), so I won't ask for that.

[Edit 1: 1.01.000] I updated the file, as I was bitten by the particular variable scoping in AHK (a bit like PHP): I was trying to access FILE_CURRENT from a function and I was wondering why it didn't work... Instead of declaring the global variables as global, I prefered to declare my local variables as such. Seemed cleaner and more Lua-like... For consistency, I did that even in functions not accessing global WinAPI constants.

[Edit 2: 1.02.000] Slight change of API, some more comments, integration of Laszlo suggestions.
I uploaded the files in AutoHotkey.net for easier grabbing. I left them there, as I like to take a look at code before downloading it :-)

[Edit 3: 1.03.000] Moved Bin2Hex & Hex2Bin to DllCallStruct.ahk.
I also applied my new coding conventions on variable prefixes.

DllCallStruct.ahk (relevant extract, real file also has Set/GetInteger and DumpDWORD...)
Code:
/*
// Convert raw bytes stored in a variable to a string of hexa digit pairs.
// Convert either byteNb bytes or, if null, the whole content of the variable.
//
// Return the number of converted bytes, or -1 if error (memory allocation)
*/
Bin2Hex(ByRef @hex, ByRef @bin, _byteNb=0)
{
   local intFormat, dataSize, dataAddress, granted, x

   ; Save original integer format
   intFormat = %A_FormatInteger%
   ; For converting bytes to hex
   SetFormat Integer, Hex

   ; Get size of data
   dataSize := VarSetCapacity(@bin)
   If (_byteNb < 1 or _byteNb > dataSize)
   {
      _byteNb := dataSize
   }
   dataAddress := &@bin
   ; Make enough room (faster)
   granted := VarSetCapacity(@hex, _byteNb * 2)
   if (granted < _byteNb * 2)
   {
      ; Cannot allocate enough memory
      ErrorLevel = Mem=%granted%
      Return -1
   }
   Loop %_byteNb%
   {
      ; Get byte in hexa
      x := *dataAddress + 0x100
      StringRight x, x, 2   ; 2 hex digits
      StringUpper x, x
      @hex = %@hex%%x%
      dataAddress++   ; Next byte
   }
   ; Restore original integer format
   SetFormat Integer, %intFormat%

   Return _byteNb
}

/*
// Convert a string of hexa digit pairs to raw bytes stored in a variable.
// Convert either byteNb bytes or, if null, the whole content of the variable.
//
// Return the number of converted bytes, or -1 if error (memory allocation)
*/
Hex2Bin(ByRef @bin, _hex, _byteNb=0)
{
   local dataSize, granted, dataAddress, x

   ; Get size of data
   x := StrLen(_hex)
   dataSize := Ceil(x / 2)
   if (x = 0 or dataSize * 2 != x)
   {
      ; Invalid string, empty or odd number of digits
      ErrorLevel = Param
      Return -1
   }
   If (_byteNb < 1 or _byteNb > dataSize)
   {
      _byteNb := dataSize
   }
   ; Make enough room
   granted := VarSetCapacity(@bin, _byteNb, 0)
   if (granted < _byteNb)
   {
      ; Cannot allocate enough memory
      ErrorLevel = Mem=%granted%
      Return -1
   }
   dataAddress := &@bin

   Loop Parse, _hex
   {
      if (A_Index & 1)   ; Odd
      {
         x = %A_LoopField%   ; Odd digit
      }
      else
      {
         ; Concatenate previous x and even digit, converted to hex
         x := "0x" . x . A_LoopField
         ; Store integer in memory
         DllCall("RtlFillMemory"
               , "UInt", dataAddress
               , "UInt", 1
               , "UChar", x)
         dataAddress++
      }
   }

   Return _byteNb
}

_________________
Image vPhiLho := RegExReplace("Philippe Lhoste", "^(\w{3})\w*\s+\b(\w{3})\w*$", "$1$2")


Last edited by PhiLho on March 17th, 2006, 11:25 am, edited 7 times in total.

Report this post
Top
 Profile  
Reply with quote  
PostPosted: January 23rd, 2006, 3:48 pm 
Offline

Joined: March 2nd, 2004, 3:36 pm
Posts: 10720
Looks great. Thanks for posting it.

PhiLho wrote:
I would prefer to offer a file.
If you ever need to host a zip file, etc., try www.autohotkey.net


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 23rd, 2006, 10:51 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
Impressive! I am sure it will prove to be useful.
PhiLho wrote:
...improving the decimal to 2-digit hex conversion...

Well, in Bin2Hex you can save one more line with
Code:
x := *dataAddress + 0x100
StringRight x, x, 2
With huge hex strings Hex2Bin could be made faster with a Parse loop, which does not create many copies of "hex", each 2 characters shorter. Something like this (untested)
Code:
Loop Parse, hex
{
   If (A_Index & 1)
      x = %A_LoopField%
   else {
      DllCall("RtlFillMemory"
            , "UInt", dataAddress
            , "UInt", 1
            , "UChar", "0x" x A_LoopField)
      dataAddress++
   }
}


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 24th, 2006, 11:04 am 
Offline

Joined: December 27th, 2005, 1:46 pm
Posts: 6837
Location: France (near Paris)
@Chris: thanks, I didn't understood the related sticky post...
I have uploaded the files so they can be retreived in an easier way, less prone to error than select/copy/paste.

@Laszlo: thanks again, you beat me on the clever ideas, I integrated them in the new release...
I didn't tried the parse loop before, you make a smart use of it.


Report this post
Top
 Profile  
Reply with quote  
 Post subject: Return in functions
PostPosted: January 25th, 2006, 5:26 pm 
Offline

Joined: December 27th, 2005, 1:46 pm
Posts: 6837
Location: France (near Paris)
I had to update my script when I saw that the opening functions returned a string in case of error instead of a numerical value.
The trick is that if a raw Return in a function does interpret its parameter as an expression, a Return in a 'classical' If is still using the 'classical' syntax:
Code:
TestFunction()
{
[snip]
    IfNotEqual ErrorLevel, 0, Return %ERROR_CODE%
    Return functionResult
}

Perhaps I missed a bit of information somewhere. AHK documentation is very rich and detailled, and it is easy to skip or miss some parts.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 25th, 2006, 6:01 pm 
Offline

Joined: March 2nd, 2004, 3:36 pm
Posts: 10720
PhiLho wrote:
a Return in a 'classical' If is still using the 'classical' [%Var%] syntax
Return supports the classical syntax as a side-effect of allowing commands like "Sleep X" to be treated the same as "Sleep %X%". Thus, I believe the following produce identical behavior:
return Var
return %Var%

However, I don't believe return's behavior varies just because you use it on the same line as another command (I tested it and couldn't reproduce it). If you can post a short test script that proves otherwise, I'll definitely try to fix it. Thanks.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 26th, 2006, 2:51 pm 
Offline

Joined: December 27th, 2005, 1:46 pm
Posts: 6837
Location: France (near Paris)
Doh, I tried to reproduce it, with no result... Whatever combination I tried. So I am totally confused (I really can remember having seen the variable name instead of its value), but you can see this as a false alert...

Oh, so if I have:
bar = 1000
foo = bar

and I write:
x := %foo%
x will be 1000.
But if I write:
Sleep %foo%
the command will be ignored because Sleep gets "bar" as value.
The problem is the versatility of Sleep (and Return) which accepts either a variable name (classical or naked) or an expression (or a value).
Gosh, this is still confusing...
Well, once I know that, I suppose I can get used to it...


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 27th, 2006, 11:16 am 
Offline

Joined: February 7th, 2005, 11:11 am
Posts: 192
Location: Munich, Germany
PhiLho wrote:
bar = 1000
foo = bar

and I write:
x := %foo%

With this line ( x := %foo%) you process the content of the content. Content of "foo" are the letters B, A and R = "bar". This letters are processed again with ":=". So you now get the content of "bar" which is "1000".
This is nothing special to AHK, but hardly to see through it.

_________________
Peter

Wisenheiming for beginners: KaPeGe (German only, sorry)


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 27th, 2006, 11:49 am 
Offline

Joined: July 12th, 2005, 1:21 pm
Posts: 633
PhiLho wrote:
bar = 1000
foo = bar
and I write:
x := %foo%
To do what you want you have 2 ways:
Code:
 bar = 1000
foo = bar
x = %foo%
or
Code:
 bar = 1000
foo = bar
x := foo
I prefer the first method, even if it is more to write, because I have a clear difference to strings..

Thalon

_________________
AHK-Icon-Changer
AHK-IRC
deutsches Forum


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 27th, 2006, 1:28 pm 
Offline

Joined: December 27th, 2005, 1:46 pm
Posts: 6837
Location: France (near Paris)
I fear Thalon and kapege.de didn't understood what I meant (but thanks for wanting to help).

I understand the x := %foo% mechanism, which is powerful.

I wanted to point out that if Sleep was like MsgBox, it would accept only %bar% to get the value of the variable. Sleep foo would be illegal.
If Sleep accepted only expressions, Sleep bar would sleep 1000ms and so would do Sleep %foo%.

Sleep is more flexible than that and accepts both variables and expressions, hence the ambiguity:
Sleep bar ; Wait 1s
Sleep %bar% ; Also wait 1s
Sleep %foo% ; Don't wait (non numerical)
Sleep %foo% + 1 ; Wait 1s
Sleep %bar% + 1 ; Don't wait (invalid expression)


As often in AHK, context is paramount in interpreting the code...


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 29th, 2006, 11:50 am 
Offline

Joined: December 18th, 2005, 5:40 pm
Posts: 234
Location: Italy - Galatro(RC)
I'm trying to write an image contained in clipboard with this code, but it doesn't work

Code:
file = e:\temp3.bmp
IfExist, %file%
FileDelete, %file%
fh := OpenFileForWrite(file)
l := Hex2Bin(data, ClipboardAll)
WriteInFile(fh,data )
CloseFile(fh)


What's wrong?

_________________
____________________
______________________
kiu - www.romeosa.com


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 29th, 2006, 1:22 pm 
Offline

Joined: March 2nd, 2004, 3:36 pm
Posts: 10720
The format of ClipboardAll is probably not suitable for becoming an on-disk BMP file. Its data format is described in the documentation. By contrast, the format of a BMP file is something I can't tell you much about.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 30th, 2006, 12:05 pm 
Offline

Joined: December 27th, 2005, 1:46 pm
Posts: 6837
Location: France (near Paris)
Perhaps running IrfanView (for example) and pasting the content of the clipboard, then saving the file may be easier than transforming the content of the clipboard.

You can also use Paint for this, but I remember a bug in the early versions of Paint where pasting only painted to the visible size of the image. I can't remember if pasting twice was a workaround or not...

You can also use ImageMagick: convert clipboard: foo.bmp.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 30th, 2006, 12:24 pm 
Offline

Joined: January 31st, 2005, 9:50 am
Posts: 3910
Location: Bremen, Germany
You can use the command line options of IrfanView to convert the clipboard to any image file. You do not have to start IrfanView for it. Which is much better, since you do not have the flickering of windows poping up and disappearing.

_________________
Ciao
toralf
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 30th, 2006, 6:15 pm 
Offline

Joined: December 18th, 2005, 5:40 pm
Posts: 234
Location: Italy - Galatro(RC)
thanks

_________________
____________________
______________________
kiu - www.romeosa.com


Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next

All times are UTC [ DST ]


Who is online

Users browsing this forum: Stigg and 22 guests


You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group