Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate

AxC : Pack and Unpack Binary files


  • Please log in to reply
29 replies to this topic
SKAN!
  • Guests
  • Last active:
  • Joined: --
** Newer version has been posted **: AxC² - AutoHotkey Cabinet v1.00


AxC : AutoHotkey Cabinet

Introduction: It all started in this topic: how does one create an image file library? posted by sublimation9 where this idea of creating a structured file for binding binary files originated. AxC cabinet is similar ( though not exactly ) to a ZIP file but does not use any kind of compression. I was mainly interested in this project so I could create Slideshow presention of my large library of family photos with a single data file containing all images ( .jpg ) descriptions (.txt) some music file ( .wav ) and use AnimateWindow() to produce the Slideshow.

This file format and the main functions uses CRC32 hash ( binary code ) to check the file integerity and is embedded directly into the script as well as gets embedded in every AxC file created.
If I had provided a link, PSPad users would have never been able to open it owing to the raw binary code in it. So, I decided to post the wrapper in [ code ] tags instead.

The CRC32 binary code was developed by his kindness, Laszlo Hars in this topic : Machine code functions: Bit Wizardry ( Many thanks again, Sir. )

If you opt to FileInstall in your script, all the files included will be stored inside using standard compression techniques, so that applies to a an AxC file also. There is a upper file size limit for includables at 16MB / file as I chose not to increase and complicate the code to handle data in chunks. AxC_Pack() is sturdy, but I did not have enough time to make the other functions and so please help to fix any bugs that might exist. The AxC file will be useful for the project that requires many small binary resources like BMP,WAV,ICO and TXT. I recommend a setting of SetBatchLines, 1ms. If this project interests you and need to see a few examples, post your query and I will try to reproduce an example.

As of now, the code looks very obfuscated.. I will try to post a cleaner version soon.

The following is the library which you may save as axc.ahk

axc_pack(packfile,files) {                ; take care of the following binary line SKAN .. 
  crc:="U‹ì‹E…À~8‹USV‰E‹EWŠ" chr(10) "jB^‹ø¶ÙÁï3ûÀ÷Ç€ÿÿÿt3EÉNuäÿMuÙ_^[]ËE]Ã"
  ls:="_llseek",lr:="_lread",,lc:="_lclose",lt:="_lcreat",lw:="_lwrite",u:="uint",i:="int"
  s:="str",varsetcapacity(b,16,0),numput(0x20437841,b),numput(0x3f666666,b,4),f:=0
  h:=dllcall(lt,s,packfile,i,0),dllcall(lw,u,h,s,b,u,16),q:="""",c:=",",lf:="`n"
  z:=chr(strlen(crc)),dllcall(lw,u,h,s,z,i,1),dllcall(lw,u,h,s,crc,u,strlen(crc))
  loop,parse,files,|
{ filegetsize,sz,%a_loopfield%
  if (sz<1||sz>(16*1024*1024))
     continue
  filegettime,mt,%a_loopfield%,m
  filegetattrib,at,%a_loopfield%
  fileread,bin,%a_loopfield%
  splitpath,a_loopfield,fil,dir,ext,filne
  crv:=dllcall(&CRC,str,bin,uint,sz,int,-1,uint,0x04C11DB7,"cdecl uint"),nl:=strlen(fil)+1
  varsetcapacity(x,4,0),numput(nl+sz,x),dllcall(lw,u,h,s,x,u,4),dllcall(lw,u,h,s,fil,u,nl)
  f:=f+1,cp:=dllcall(ls,u,h,i,0,i,1),dllcall(lw,u,h,s,bin,u,sz)
  csv.=q fil q c cp c sz c crv c mt c at lf    
} stringtrimright,csv,csv,1
  numput((cl:=strlen(csv)),x),dllcall(lw,u,h,s,csv,u,cl),dllcall(lw,u,h,s,x,u,4)
  dllcall(ls,u,h,i,8,i,0),numput(f,x),dllcall(lw,u,h,s,x,u,4),dllcall(lc,u,h)
return csv
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_unpackall(packfile,dir="",progress="axm_progress") {
  ifequal,dir,,setenv,dir,%a_workingdir%
  ifnotexist,%dir%,filecreatedir,%dir%
  ls:="_llseek",lr:="_lread",lo:="_lopen",lc:="_lclose",u:="uint",i:="int",s:="str"
  lt:="_lcreat",lw:="_lwrite",h:=dllcall(lo,s,packfile,i,0),varsetcapacity(x,4,0)
  varsetcapacity(hdr,16),dllcall(lr,u,h,s,hdr,i,16),fls=numget(hdr,8),f:=0
  if (h<1||(numget(hdr)<>0x20437841)||(numget(hdr,4)<>0x3f666666)||fls<1)+(axc:=0) 
{ dllcall(lc,u,h)
  return -1
} dllcall(lr,u,h,s,hdr,i,1),z:=asc(hdr),varsetcapacity(crc,z),dllcall(lr,u,h,s,crc,i,z)
  dllcall(ls,u,h,u,-0,i,2),dllcall(ls,u,h,u,-0,i,2),dllcall(ls,u,h,i,0,i,1)
  dllcall(ls,u,h,i,-4,i,2),dllcall(lr,u,h,s,x,i,4),z:=numget(x),dllcall(ls,u,h,i,-z-4,i,2)
  varsetcapacity(csv,z),dllcall(lr,u,h,s,csv,i,z)
  loop,parse,csv,`n
{ loop, parse, a_loopfield, csv
  f%A_Index%:=a_loopfield
  dllcall(ls,u,h,i,f2,i,0),varsetcapacity(bin,f3),dllcall(lr,u,h,s,bin,i,f3)
  If ((e:=dllcall(&CRC,s,bin,u,f3,i,-1,u,0x04C11DB7,"cdecl uint")+0)<>f4)
{ msgbox,16,Checksum Error!,CRC32 failed on %f1% %e%
  continue
} tf:=dir "\" f1,h2:=dllcall(lt,s,tf,i,0),f:=f+1
  ;ifnotequal,progress,,setenv,dummy,% %progress%(tf,a_index,fls)
  dllcall(lw,u,h2,s,bin,u,f3),dllcall(lc,u,h2)
  filesettime,%f5%,%tf%,c
  filesettime,%f5%,%tf%,m  
  filesetattrib,+%f6%,%tf%
} Return f+dllcall(lc,u,h)
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_unpack(packfile,dest="") {
  ls:="_llseek",lr:="_lread",lo:="_lopen",lc:="_lclose",u:="uint",i:="int",s:="str"
  lt:="_lcreat",lw:="_lwrite",h:=dllcall(lo,s,packfile,i,0),varsetcapacity(x,4,0)
  varsetcapacity(hdr,16),dllcall(lr,u,h,s,hdr,i,16),fls=numget(hdr,8)
  if (h<1||(numget(hdr)<>0x20437841)||(numget(hdr,4)<>0x3f666666)||fls<1)+(axc:=0) 
{ dllcall(lc,u,h)
  return -1
} dllcall(lr,u,h,s,hdr,i,1),z:=asc(hdr),varsetcapacity(crc,z),dllcall(lr,u,h,s,crc,i,z)
  dllcall(ls,u,h,u,-0,i,2),dllcall(ls,u,h,u,-0,i,2),dllcall(ls,u,h,i,0,i,1)
  dllcall(ls,u,h,i,-4,i,2),dllcall(lr,u,h,s,x,i,4),z:=numget(x),dllcall(ls,u,h,i,-z-4,i,2)
  varsetcapacity(csv,z),dllcall(lr,u,h,s,csv,i,z)
  splitpath,dest,fil,dir
  loop,parse,csv,`n
{ if (subStr(a_loopfield,2,instr(a_loopfield,"""",0,2)-2)=fil)
{ loop,parse,a_loopfield,csv
  f%a_index%:=a_loopfield  
  dllcall(ls,u,h,i,f2,i,0),varsetcapacity(bin,f3),dllcall(lr,u,h,s,bin,i,f3)
  If ((e:=dllcall(&CRC,s,bin,u,f3,i,-1,u,0x04C11DB7,"cdecl uint")+0)<>f4)
{ msgbox,16,Checksum Error!,CRC32 failed on %f1% %e%
  dllcall(lc,u,h)
  return -3
} h2:=dllcall(lt,s,dest,i,0),dllcall(lw,u,h2,s,bin,u,f3),dllcall(lc,u,h2),dllcall(lc,u,h)
  filesettime,%f5%,%tf%,c
  filesettime,%f5%,%tf%,m  
  filesetattrib,+%f6%,%tf%
  return dest
}} return dllcall(lc,u,h)
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_unpacktomem(packfile,fil,byref bin) {
  ls:="_llseek",lr:="_lread",lo:="_lopen",lc:="_lclose",u:="uint",i:="int",s:="str"
  lt:="_lcreat",lw:="_lwrite",h:=dllcall(lo,s,packfile,i,0),varsetcapacity(x,4,0)
  varsetcapacity(hdr,16),dllcall(lr,u,h,s,hdr,i,16),fls=numget(hdr,8)
  if (h<1||(numget(hdr)<>0x20437841)||(numget(hdr,4)<>0x3f666666)||fls<1)+(axc:=0) 
{ dllcall(lc,u,h)
  return -1
} dllcall(lr,u,h,s,hdr,i,1),z:=asc(hdr),varsetcapacity(crc,z),dllcall(lr,u,h,s,crc,i,z)
  dllcall(ls,u,h,u,-0,i,2),dllcall(ls,u,h,u,-0,i,2),dllcall(ls,u,h,i,0,i,1)
  dllcall(ls,u,h,i,-4,i,2),dllcall(lr,u,h,s,x,i,4),z:=numget(x),dllcall(ls,u,h,i,-z-4,i,2)
  varsetcapacity(csv,z),dllcall(lr,u,h,s,csv,i,z)
  loop,parse,csv,`n
{ if ( subStr(a_loopfield,2,instr(a_loopfield,"""",0,2)-2)=fil)
{ loop,parse,a_loopfield,csv
  f%a_index%:=a_loopfield  
  dllcall(ls,u,h,i,f2,i,0),varsetcapacity(bin,f3),dllcall(lr,u,h,s,bin,i,f3)
  if ((e:=dllcall(&CRC,s,bin,u,f3,i,-1,u,0x04C11DB7,"cdecl uint")+0)<>f4)
{ msgbox,16,Checksum Error!,CRC32 failed on %f1% %e%
  varsetcapacity(bin,0),dllcall(lc,u,h)
  return -3
} dllcall(lc,u,h)
   return f3
}} return -2
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_extractall(packfile,dir="") {
  ls:="_llseek",lr:="_lread",lo:="_lopen",lc:="_lclose",u:="uint",i:="int",s:="str"
  lt:="_lcreat",lw:="_lwrite",varsetcapacity(x,4,0),varsetcapacity(tfil,256,0)
  h:=dllcall(lo,s,packfile,i,0),dllcall(ls,u,h,u,8,i,0),dllcall(lr,u,h,s,x,i,4)
  f:=numget(x),dllcall(ls,u,h,u,4,i,1),dllcall(lr,u,h,s,x,i,1)
  csz:=numget(x,0,"uchar"),dllcall(ls,u,h,u,csz,i,1)
  ifequal,dir,,setenv,dir,%a_workingdir%
  ifnotexist,%dir%,filecreatedir,%dir%
  loop %f%
{ ix:=a_index,dllcall(lr,u,h,s,x,i,4),varsetcapacity(tfil,256)
  dllcall(lr,u,h,s,tfil,u,256),nl:=strlen(tfil)+1,sz:=numget(x)-nl,tfil:=substr(tfil,1,nl)
  varsetcapacity(bin,sz,0),  dllcall(ls,u,h,u,-256+nl,i,1),dllcall(lr,u,h,s,bin,u,sz)
  h2:=dllcall(lt,s,dir "\" tfil,i,0),dllcall(lw,u,h2,s,bin,u,sz),dllcall(lc,u,h2)
} return ix-f
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_extract(packfile,dest="") { 
  ls:="_llseek",lr:="_lread",lo:="_lopen",lc:="_lclose",u:="uint",i:="int",s:="str"
  lt:="_lcreat",lw:="_lwrite",varsetcapacity(x,4,0),varsetcapacity(tfil,256,0)
  h:=dllcall(lo,s,packfile,i,0),dllcall(ls,u,h,u,8,i,0),dllcall(lr,u,h,s,x,i,4)
  f:=numget(x),dllcall(ls,u,h,u,4,i,1),dllcall(lr,u,h,s,x,i,1)
  csz:=numget(x,0,"uchar"),dllcall(ls,u,h,u,csz,i,1)
  splitpath,dest,fil,dir
  loop %f%
{ dllcall(lr,u,h,s,x,i,4),dllcall(lr,u,h,s,tfil,u,256)
  nl:=strlen(tfil)+1,sz:=numget(x)-nl,tfil:=substr(tfil,1,nl)
  if (tfil!=fil)
{ dllcall(ls,u,h,u,-256+numget(x),i,1)
  continue
} dllcall(ls,u,h,u,-256+nl,i,1),varsetcapacity(bin,sz,0),dllcall(lr,u,h,s,bin,u,sz)
  h2:=dllcall(lt,s,dest,i,0),dllcall(lw,u,h2,s,bin,u,sz),dllcall(lc,u,h2),fnd:=1
  break
} dllcall(lc,u,h)
return fnd ? dest : ""    
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_extracttomem(packfile,fil,byref bin) { 
  ls:="_llseek",lr:="_lread",lo:="_lopen",lc:="_lclose",u:="uint",i:="int",s:="str"
  lt:="_lcreat",lw:="_lwrite",varsetcapacity(x,4,0),varsetcapacity(tfil,256,0)
  h:=dllcall(lo,s,packfile,i,0),dllcall(ls,u,h,u,8,i,0),dllcall(lr,u,h,s,x,i,4)
  f:=numget(x),dllcall(ls,u,h,u,4,i,1),dllcall(lr,u,h,s,x,i,1)
  csz:=numget(x,0,"uchar"),dllcall(ls,u,h,u,csz,i,1)
  loop %f%
{ dllcall(lr,u,h,s,x,i,4),dllcall(lr,u,h,s,tfil,u,256)
  nl:=strlen(tfil)+1,sz:=numget(x)-nl,tfil:=substr(tfil,1,nl)
  if (tfil!=fil)
{ dllcall(ls,u,h,u,-256+numget(x),i,1)
  continue
} dllcall(ls,u,h,u,-256+nl,i,1),varsetcapacity(bin,sz,0)
  Return dllcall(lr,u,h,s,bin,u,sz)+dllcall(lc,u,h2)+dllcall(lc,u,h)
} return dllcall(lc,u,h)    
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_getcsv(packfile) {
  ls:="_llseek",lr:="_lread",lo:="_lopen",lc:="_lclose",u:="uint",i:="int",s:="str"
  lt:="_lcreat",lw:="_lwrite",h:=dllcall(lo,s,packfile,i,0),varsetcapacity(x,4,0)
  varsetcapacity(hdr,16),dllcall(lr,u,h,s,hdr,i,16),fls=numget(hdr,8)
  if (h<1||(numget(hdr)<>0x20437841)||(numget(hdr,4)<>0x3f666666)||fls<1)+(axc:=0) 
{ dllcall(lc,u,h)
  return -1
} dllcall(ls,u,h,i,-0,i,2),dllcall(ls,u,h,i,-0,i,2),dllcall(ls,u,h,i,0,i,1)
dllcall(ls,u,h,i,-4,i,2),dllcall(lr,u,h,s,x,i,4),sz:=numget(x),dllcall(ls,u,h,i,-sz-4,i,2)
  varsetcapacity(csv,sz),dllcall(lr,u,h,s,csv,i,sz),dllcall(lc,u,h)
return csv  
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_offsettofile(packfile,offset,sz,dest) {
  h:=dllcall("_lopen",str,packfile,int,0),varsetcapacity(x,4,0),varsetcapacity(hdr,16)
  dllcall("_lread",uint,h,str,hdr,int,16),fls=numget(hdr,8),varsetcapacity(bin,sz)
  if (h<1||(numget(hdr)<>0x20437841)||(numget(hdr,4)<>0x3f666666)||fls<1)+(axc:=0) 
{ dllcall("_lclose",uint,h)
  return -1
} dllcall("_llseek",uint,h,uint,offset,int,0),dllcall("_lread",uint,h,str,bin,int,sz)
  h2:=dllcall("_lcreat",str,dest,int,0),bytes:=dllcall("_lwrite",uint,h2,str,bin,uint,sz)
return dllcall("_lclose",uint,h) + dllcall("_lclose",uint,h2) + bytes 
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
axc_offsettomem(packfile,offset,sz,byref bin) {
  h:=dllcall("_lopen",str,packfile,int,0),varsetcapacity(x,4,0),varsetcapacity(hdr,16)
  dllcall("_lread",uint,h,str,hdr,int,16),fls=numget(hdr,8),varsetcapacity(bin,sz)
  if (h<1||(numget(hdr)<>0x20437841)||(numget(hdr,4)<>0x3f666666)||fls<1)+(axc:=0) 
{ dllcall("_lclose",uint,h)
  return -1
} dllcall("_llseek",uint,h,uint,offset,int,0)
return dllcall("_lread",uint,h,str,bin,int,sz) + dllcall("_lclose",uint,h) 
} ;  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

List of functions in AxC Library


AxC_Pack( packfile, files )
The main function - creates an AxC file.

packfile : the fullpath and name of the packfile to be created, eg. A_ScriptDir . "\MyCab.AxC"
files : a pipe delimited list of files ( with fullpath ) to be packed into the AxC file

hint: you may build a AxC file with three lines of code as shown is the example code below

Loop, c:\project1\projectfiles\*.* 
   files .= ( A_Index=1 ? "" : "|" ) . A_LoopFileLongPath 
AxC_Pack( "pngs.axc" , files  )

Return value: CSV based lookup table that was appended to the end of AxC file. You might want to save it for later reference like shown in the example below:

Loop, c:\project1\projectfiles\*.* 
   files .= ( A_Index=1 ? "" : "|" ) . A_LoopFileLongPath 
FileAppend, % AxC_Pack( "myaxc.axc", files ), myaxc.csv

Note: This library has AxC_GetCSV() function that will fetch the CSV lookup table from a valid AxC file.




AxC_UnPackAll( packfile, folder )
Unpacks all the files available in an AxC file.

packfile - the fullpath and name of the packfile to be unpacked.
folder - the fullpath to target folder where files would be unpacked. the target folder would be created if neccessary.
progress - the dynamic function name that will display a progress bar ( requires AutoHotkey 1.0.47.06 ). You will have to remove comment marker ( ; ) on line 22.

The default progress function name is axm_progress and you may insert the following function in your code:

axm_progress( file,f,fls ) {
progress,% (f/fls*100),%file%,unpacking %f%/%fls% files,AxC Unpacker,tahoma fs8
ifequal,f,%fls%,progress,off
}

Return value: the number of files unpacked.

[*:25dtn2pf]Variant: AxC_ExtractAll() does not use the Lookup table, does not check the header and no CRC32 hash.



AxC_UnPack( packfile, destination )
Unpacks a single file available in an AxC file.

packfile - the fullpath and name of the packfile to be unpacked.
destination - the fullpath to the destination file. The calling script should make sure that the target folder exists before calling this function.


[*:25dtn2pf]Variant: AxC_Extract() does not use the Lookup table, does not check the header and no CRC32 hash.
[*:25dtn2pf]Variant: AxC_OffsetToFile() is same as above but uses offset instead of filename



AxC_UnPackToMem( packfile, filename, byref bin )
loads a single file available in an AxC file into AutoHotkey memory.

packfile - the fullpath and name of the packfile to be unpacked.
filename - the filename (no path) as displayed in a the CSV lookup table.
bin - variable to be passed to recieve the binary contents


[*:25dtn2pf]Variant: AxC_ExtractToMem() does not use the Lookup table, does not check the header and no CRC32 hash.
[*:25dtn2pf]Variant: AxC_OffsetToMem() is same as above but uses offset instead of filename



AxC_GetCSV( packfile )
Fetches a copy of CSV lookup table embedded at the tail of AxC file

packfile - the fullpath and name of the packfile.

A sample of CSV lookup table Data:

{

CSV Structure:
[*:25dtn2pf]field1 : filename
[*:25dtn2pf]field2 : data starting offset ( in AxC file )
[*:25dtn2pf]field3 : data size ( number of bytes )
[*:25dtn2pf]field4 : crc32 uint hash value
[*:25dtn2pf]field5 : file datetime stamp - modified
[*:25dtn2pf]field6 : file attributes


}
"Dashcode Doc.png",[color=red]109[/color],[color=red]25075[/color],3576921079,20080309073216,A ; refer the FileReadEx example given below
"Dictionary.png",25203,27862,4213588085,20080309073216,A
"Directory.png",53083,28752,579127752,20080309073216,A
"Document.png",81852,13703,2207712199,20080309073216,A
"DVD+RW.png",95570,32016,324081652,20080309073216,A
"DVD-R.png",127600,31805,328609434,20080309073216,A
"File Vault.png",159424,20525,3042359571,20080309073216,A
"FireWire HD.png",179969,19137,497968045,20080309073216,A
"Folder Actions Setup.png",199135,21304,1450761774,20080309073216,A
"Frontrow.png",220456,25110,2497983650,20080309073216,RA
"Generic Document Icon.png",245596,7302,398548922,20080309073216,RA
"Install Mac OS X.png",252923,25923,1250554920,20080309073216,RA
"Instruments.png",278866,32550,2457350283,20080309073216,RA
"Interface Builder.png",311442,30041,246708242,20080309073216,A
"Internal.png",341500,21545,4026779188,20080309073216,A
"Java Plugin Settings.png",363074,17600,108513278,20080309073216,A
"Keynote.png",380690,26446,752800427,20080309073216,A
"Library Icon.png",407157,35527,194375412,20080309073216,A
"Mbox.png",442697,16755,3877834847,20080309073216,A
"MemoryStickProDuo.png",459478,8794,1377418520,20080309073216,A
"Metapackage.png",468292,24588,618142894,20080309073216,A
"Package.png",492896,22704,545643370,20080309073216,A
"Podcast Capture.png",515624,34851,2511701275,20080309073216,A
"Prefs Camera.png",550496,12832,2565885115,20080309073216,A
"QuickTime Player Pro.png",563357,26265,4204980344,20080309073216,A
"QuickTime Updater.png",589648,20595,4278385354,20080309073216,A
"RAID Volume.png",610263,15109,1431467625,20080309073216,A
"Removable.png",625390,11659,233858604,20080309073216,A
"Saved X File.png",637070,16350,2764842288,20080309073218,A
"SD.png",653431,9369,1694722836,20080309073218,A
"SEScriptEditorX.png",662824,20905,3700267845,20080309073218,A
"SmartMedia.png",683748,16090,673368522,20080309073218,A
"Software Update.png",699862,37179,4176144940,20080309073218,AH
"Spotlight.png",737059,16795,410370576,20080309073218,AH
"Untitled.png",753871,35501,3753089342,20080309073218,AH
"WebStart.png",789389,28647,3547914634,20080309073218,AH
"Xcode3 Doc.png",818055,23886,2354383255,20080309073218,AH
"AirPort Utility.png",841965,20123,1027538490,20080309073218,A
"Apple iPhone.png",862109,23512,976380892,20080309073218,A
"AppleScript Utility.png",885649,28476,2686549699,20080309073218,A
"Application Icon.png",914150,37218,2141067462,20080309073218,A
"Bonjour.png",951384,22401,3485349766,20080309073218,A
"CD-RW.png",973799,31555,1245953688,20080309073218,A
"Chess.png",1005368,21505,2615656425,20080309073218,A
"CompactFlash.png",1026894,9684,1136258240,20080309073218,A
[/list]

Generic Binary Functions

Further to some queries in this topic: how does one create an image file library? posted by tic, I hereby present a tiny wrapper that should help anyone to create their own file structure:

; V = Var , F = File , B = Number of Bytes, O = Starting Offset

FileReadEx( ByRef V,F,B,O ) {
return varsetcapacity(v,b,0)-o+(h:=dllcall("_lopen",str,f,int,0))-h+dllcall("_llseek",uint
 ,h,uint,o,int,0)-dllcall("_lread",uint,h,str,v,int,b)+dllcall("_lclose",uint,h) ? -1 : &v     
}

FileAppendEx( ByRef V,F,B,O=0 ) {
 h:=(fileexist(f)="") ? dllcall("_lcreat",str,f,int,0) : dllcall("_lopen",str,f,int,1)
return dllcall("_llseek",uint,h,int,0,int,2)+dllcall("_lwrite",uint,h,str,v,uint,B)
 +dllcall("_lclose",uint,h)
}

FileWriteEx( ByRef V,F,B,O=0 ) {
 h:=(fileexist(f)="") ? dllcall("_lcreat",str,f,int,0) : dllcall("_lopen",str,f,int,1)
return dllcall("_llseek",uint,h,int,O,int,0)+dllcall("_lwrite",uint,h,str,v,uint,B)
 +dllcall("_lclose",uint,h)
}

FileReadEx Example:
FileReadEx() can be used on an AxC file like the following example code which extracts and loads a file into AutoHotkey memory

FileReadEx( ImgData, "png.axc", [color=red]25075, 109[/color] )

FileReadEx( ByRef V,F,B,O ) {
return varsetcapacity(v,b,0)-o+(h:=dllcall("_lopen",str,f,int,0))-h+dllcall("_llseek",uint
 ,h,uint,o,int,0)-dllcall("_lread",uint,h,str,v,int,b)+dllcall("_lclose",uint,h) ? -1 : &v     
}

for the bytes and offset ( in Red ) refer the CSV lookup sample data given above.
Generally speaking, I would not recommend hardcoding the offsets in code .. but the above is the best possible short kode you can use in the calling script.


Thanks :)

tic
  • Members
  • 1934 posts
  • Last active: May 30 2018 08:13 PM
  • Joined: 22 Apr 2007
Heh. I posted one as well :) I'll give yours a try.....

MiamiGuy
  • Members
  • 61 posts
  • Last active: Nov 25 2011 07:44 AM
  • Joined: 03 Nov 2006
Hey guys,

This code is awesome, I've been wondering how I could create a fully skinned GUI which uses a lot of bitmaps and not have to have a huge folder full of little bitmaps and pngs.

This AxC file is perfect for what I'm trying to accomplish.
I want to use it like an image only dll, to house all the small bitmaps. I wanted to use the Unpack to Memory function to load the bitmaps first then create the GUI.

However, I cant seem to get the UnpacktoMem function to work, can someone help?

I'm not sure if I should have something like this:

Pack = %A_ScriptDir%\Test.axc  ; Path to the AxC file
Dfile = left_Corner.bmp  ;bitmap I neet to load into memory


AxC_UnPackToMem( Pack, Dfile, byref PictTest ) ;<==I'm confused here

Gui, 1: Add, Picture, x0 y0 vPic, %PictTest% ; PictTest would be the picture loaded into memory?  Thats what I cant figure out.

Gui, 1: Show, AutoSize
Return


I keep receiving the message "Caller must pass a variable to this ByRef parameter"

I had also tried something like this:

Pack = %A_ScriptDir%\Test.axc  ; Path to the AxC file
Dfile = left_Corner.bmp  ;bitmap I neet to load into memory
PictTest = AxC_UnPackToMem( Pack, Dfile, byref ":PictTest" )

Gui, 1: Add, Picture, x0 y0 vPic, %PictTest%
Gui, 1: Show, AutoSize
Return

The issue I am having, and can't seem to understan is the ByRef variable. I read the help file and Im not following either. I know my mistake is like AHK 101, but....

What is it I'm doing wrong?
I got all the other functions to work ok, (AxC_Pack, AxC_UnPackAll, etc.) but I can't figure out how to work with the one I need. :cry:
Any help would be great.
Thanks.
:roll:

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005
AxC_UnPackToMem( Pack, Dfile, byref PictTest ) ;<==I'm confused here

Gui, 1: Add, Picture, x0 y0 vPic, %PictTest% ; PictTest would be the picture loaded into memory?  Thats what I cant figure out.

When you load image data directly into memory it is no more an image and but a mere byte array of binary content. You need to process it before you can show it in a picture control.

The simplest way is to let AHK handle the picture .. and AHK can handle only if it can load a picture off the disk.
So you may use axc_unpack() to unpack it to the temp directory and load it from there:

SetWorkingDir, %A_ScriptDir%
Gui, 1: Add, Picture, x0 y0 vPic, % axc_unpack( "[color=red]test.axc[/color]",  [color=red]A_Temp "\left_corner.bmp"[/color] )

If you are interested only in loading the bitmaps directly, let me know and I will put a demo.

:)

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005

Using AxC file and FileReadEx() to create a Pseudo-skinned GUI
#SingleInstance, Force
SetWorkingDir, %A_ScriptDir%
IfNotExist,skin.axc, URLDownloadToFile,https://ahknet.autohotkey.com/~goyyah/skin.axc,skin.axc

Gui 1:+LastFound -Caption ; +0x400000
Gui, Color, EFEFEF
AppTitle := "Skan's GUI skinning Demo  [   using AxC file and FileReadEx()   ]" 
GUI1:=WinExist(), hDC:=DllCall("GetDC",UInt,Gui1)
Gui,Margin,0,0

Gui, Add, Text, x0    y0     w6   h22 0xE hwndPic1
Gui, Add, Text, x+0          w1   h22 0xE hwndPic2  vCaptionBG
Gui, Add, Text, x+579        w16  h22 0xE hwndPic3  gMinimize
Gui, Add, Text, x+0          w16  h22 0xE hwndPic4
Gui, Add, Text, x+0          w16  h22 0xE hwndPic5  gGuiEscape
Gui, Add, Text, x+0          w6   h22 0xE hwndPic6
Gui, Add, Text, x0    y+0    w3   h1  0xE hwndPic7  vFrameVertical1
Gui, Add, Text, x+634        w3   h1  0xE hwndPic8  vFrameVertical2
Gui, Add, Text, x0    yp+455 w3   h3  0xE hwndPic9                      
Gui, Add, Text, x+0          w1   h3  0xE hwndPic10 vFrameHorizontal
Gui, Add, Text, x+633        w3   h3  0xE hwndPic11 

FileReadEx( FRTL, "skin.axc", 494,4420 )                 ; frame_top_left.bmp 
SetImage( FRTL, Pic1, Gui1, hDC ) 

FileReadEx( CAPT, "skin.axc", 142,4930 )                 ; caption.bmp
SetImage( CAPT, Pic2, Gui1, hDC ) 
GuiControl, MoveDraw, CaptionBG, w580                    ; Stretch 1x22 to fit area 526x22 

FileReadEx( MINI, "skin.axc", 1110,105 )                 ; minimize.bmp 
SetImage( MINI, Pic3, Gui1, hDC ) 

FileReadEx( MAXI, "skin.axc", 1110,1232 )                ; maximize.bmp 
SetImage( MAXI, Pic4, Gui1, hDC ) 

FileReadEx( CLOSE, "skin.axc", 1110,2356 )               ; close.bmp 
SetImage( CLOSE, Pic5, Gui1, hDC ) 

FileReadEx( FRTR, "skin.axc", 494,3903 )                 ; frame_top_right.bmp 
SetImage( FRTR, Pic6, Gui1, hDC ) 

FileReadEx( FRV, "skin.axc", 66,3489 )                   ; frame_vertical.bmp 
SetImage( FRV, Pic7, Gui1, hDC )
GuiControl, MoveDraw, FrameVertical1, h455               ; Stretch 3x1 to fit area 3x455
SetImage( FRV, Pic8, Gui1, hDC )
GuiControl, MoveDraw, FrameVertical2, h455               ; Stretch 3x1 to fit area 3x455

FileReadEx( FRBL, "skin.axc", 90,3789 )                  ; frame_bottom_left.bmp 
SetImage( FRBL, Pic9, Gui1, hDC )

FileReadEx( FRH, "skin.axc", 66,3580  )                  ; frame_horizontal.bmp 
SetImage( FRH, Pic10, Gui1, hDC )
GuiControl, MoveDraw, FrameHorizontal, w634              ; Stretch 1x3 to fit area 1x3

FileReadEx( FRBR, "skin.axc", 90,3673 )                  ; frame_bottom_right.bmp 
SetImage( FRBR, Pic11, Gui1, hDC )

; Frame loading over, now to set a nice icon and caption text

Gui, Add, Text, x6 y4 w14 h14 0xE hwndIcon gTrayMenu
FileReadEx( FRBR, "skin.axc", 670,5086 )                 ; hicon.bmp 
SetImage( FRBR, Icon, Gui1, hDC )
Gui, Font, S9, Arial
Gui, Add, Text, x24 y2 w560 h18 0x201 BackgroundTrans cEEEEEE GuiMove, %AppTitle%
Gui, Font
Gui,Show, w640 h480, %AppTitle%

Return                                             ;;   Phew!! End of Auto-Execute Section

GuiEscape:
  ExitApp
Return  

uiMove: 
  PostMessage, 0xA1, 2,,, A 
Return 

Minimize:
  Winminimize, A
Return

TrayMenu:
 Menu, Tray, Show
Return 

SetImage( ByRef BMP, hPic, Gui1, hDC ) {            
 hBMP := DllCall( "CreateDIBitmap", UInt,hDC, UInt,(bmiHAddr:=&BMP+14)
  ,UInt,(CBM_INIT:=4), UInt,&BMP+NumGet(BMP,10), UInt,&BMP+14, UInt,(DIB_PAL_COLORS:=1) )       
 SendMessage, (STM_SETIMAGE:=0x172), (IMAGE_BITMAP:=0x0), hBMP,, ahk_id %hPic%
}

FileReadEx( ByRef V,F,B,O ) { 
return varsetcapacity(v,b,0)-o+(h:=dllcall("_lopen",str,f,int,0))-h+dllcall("_llseek",uint 
 ,h,uint,o,int,0)-dllcall("_lread",uint,h,str,v,int,b)+dllcall("_lclose",uint,h) ? -1 : &v      
}


/*

------------------------------------------------------------------------------------------
   Author     : Suresh Kumar A N [ arian dot suresh at gmail dot com ]
   Title      : using AxC file and FileReadEx() to create a Pseudo-skinned GUI
   Version    : None, 20-Mar-2007 
   Topic      : AxC : Pack and Unpack Binary files
                http://www.autohotkey.com/forum/viewtopic.php?t=29593
   This Script: http://www.autohotkey.com/forum/viewtopic.php?p=185668#185668
------------------------------------------------------------------------------------------

Important note:
Since we are hardcoding the file offsets, I created the AxC file with following code:

------------------------------------------------------------------------------------------
[color=blue]SetWorkingDir, %A_ScriptDir%
filelist=
( join
minimize.bmp|
maximize.bmp|
close.bmp|
frame_vertical.bmp|
frame_horizontal.bmp|
frame_bottom_right.bmp|
frame_bottom_left.bmp|
frame_top_right.bmp|
frame_top_left.bmp|
caption.bmp|
hicon.bmp
)
FileAppend, % AxC_Pack( "skin.axc", filelist ), skin.csv[/color]------------------------------------------------------------------------------------------

.. so we can be sure that the offsets never change when we want to add a new file


The output CSV lookup table for skin.axc follows :
------------------------------------------------------------------------------------------
[color=brown]"minimize.bmp",105,1110,1939603633,20080320132356,A
"maximize.bmp",1232,1110,1939603633,20080320132442,A
"close.bmp",2356,1110,1939603633,20080320132558,A
"frame_vertical.bmp",3489,66,904776919,20080320132722,A
"frame_horizontal.bmp",3580,66,904776919,20080320132800,A
"frame_bottom_right.bmp",3673,90,2952008854,20080320132850,A
"frame_bottom_left.bmp",3789,90,2952008854,20080320132918,A
"frame_top_right.bmp",3903,494,1550802213,20080320133612,A
"frame_top_left.bmp",4420,494,1550802213,20080320133630,A
"caption.bmp",4930,142,961223416,20080320133736,A
"hicon.bmp",5086,670,546880133,20080320160814,A[/color]
------------------------------------------------------------------------------------------

Note: The second field in above table is Offset and third field is Size ( in bytes ),
But for FileReadEx() Size is followed by Offset.
This was desgined as such to be consistent with FileWriteEx() and FileAppendEx()



Credits:

[color=indigo]                    SetImage() was adapted from PhiLho's code 
            Philippe Lhoste <PhiLho(a)GMX.net> http://Phi.Lho.free.fr

                         from the following topic 
                      Include bitmaps in your scripts!
             http://www.autohotkey.com/forum/viewtopic.php?t=14941[/color]
             
That topic was very detailed and took me few hours to compress it into a 5 line function.
One problem PhiLho faced and did not set right was that a picture control could not be 
initialised with "Gui, Add, Picture" unless it was passed with an existing image file name
So he opted to search for an image and use it temporarily to.

I found the trick to workaround it: Create a text control with SS_BITMAP (0xE) in options

                       Example:    [color=blue]Gui, Add, Text, +0xE[/color]

Note: You may use 0x20E if you need to center a smaller image within a bigger control.
 
Additional Tip: 

FileReadEx() returns the pointer to variable buffer.
If you can alter SetImage() to accept a pointer instead of a variable as by ref, then you
could simplify the call like the following example:

  [color=blue]SetImage( FileReadEx( MINI,"skin.axc",1110,105 ) , Pic3, Gui1, hDC )[/color] ; minimize.bmp 

I would love to use nested calls, but have avoided here to maintain simplicity.
              
*/

Screenshot of the GUI
Posted Image[/list]

8)

Sarah
  • Members
  • 103 posts
  • Last active: Oct 12 2009 05:42 PM
  • Joined: 12 Jul 2007
You did a great job putting this together with complete documentation, well structured and in-depth it seems. Skan, I know we conferred a little on this, and just wanted to follow-up with my vision.

Is there any way on Earth to do something like this, with this or maybe some different system? :roll:


AXC_UnPackToMem("hello_best.axc","hello_best.exe", byref bin) 


Best Wishes - Sarah.

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005

Is there any way on Earth to do something like this, with this or maybe some different system? :roll:

AXC_UnPackToMem("hello_best.axc","hello_best.exe", byref bin)


That would would work in the sense that the PE will get loaded into memory as a byte array. But there is no method I know of to execute it directly from memory.. :(
You will have to write it back to disk to execute it.

:)

Sarah
  • Members
  • 103 posts
  • Last active: Oct 12 2009 05:42 PM
  • Joined: 12 Jul 2007
I kinda started a similar line of thought on the ASCII 85 thread you were commended at S..... but I am wondering, what about .WAV or sound files -- can I load/play those from memory only.... similarity like execution of images, without actually generating a show file..... using your technique? :)

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005

what about .WAV or sound files -- can I load/play those from memory only....


100% yes for WAV .. for other sound files like MP3 you will need an external DLL.

Edit: Example, something like follows:

axc_unpacktomem( "my.axc","my.wav", [color=red]WavBuffer[/color] )
DllCall("winmm.dll\sndPlaySoundA", UInt,&[color=red]WavBuffer[/color], UINT, ( (SND_MEMORY:=0x4) | (SND_NODEFAULT:=0x2)) )


Sarah
  • Members
  • 103 posts
  • Last active: Oct 12 2009 05:42 PM
  • Joined: 12 Jul 2007
That looks great SKAN, awesome work :D

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005
On a related note, here is how we can directly load an image from a .ZIP using UnZip.DLL ( 19k )

#SingleInstance, Force
SetWorkingDir %A_ScriptDir%  

IfNotExist, UnZip.DLL
 UrlDownloadToFile, http://arian.suresh.googlepages.com/UnZip.DLL, UnZip.dll    ; 19k only

IfNotExist, bmp.zip
 UrlDownloadToFile,http://arian.suresh.googlepages.com/bmp.zip, bmp.zip       ; 5.28k only

hmod := DllCall( "LoadLibrary", str,"UnZip.dll" )
varsetcapacity( buffer,8,0 ), varsetcapacity( nsize,8,0 )
errorLevel := DllCall( "UnZip\ZipLoadFile", Str,"bmp.zip",Str,"256x256.bmp"
                      , UintP,Buffer, UintP,nSize, "CDecl" )

Gui 1:+LastFound -Sysmenu 
Gui, Margin, 0, 0
GUI1:=WinExist() , hDC:=DllCall( "GetDC",UInt,Gui1 ) 
Gui, Add, Text, w256 h256 0xE hwndhPic ; SS_BITMAP = 0xE 

hBMP := DllCall( "CreateDIBitmap", UInt,hDC, UInt,(bmiHAddr:=Buffer+14) 
                , UInt,(CBM_INIT:=4), UInt,Buffer+NumGet(Buffer+0,10), UInt,Buffer+14
                , UInt,(DIB_PAL_COLORS:=1) )        
VarSetCapacity( Buffer,0 )
SendMessage, (STM_SETIMAGE:=0x172), (IMAGE_BITMAP:=0x0), hBMP,, ahk_id %hPic%

Gui, Show, , 256x256.bmp loaded directly from ZIP 

Return                                                 ; // end of auto-execute section //

GuiEscape: 
  ExitApp


Note: 256x256.bmp weighs 192k and has been compressed to 5.28k as .zip !!

:)

MiamiGuy
  • Members
  • 61 posts
  • Last active: Nov 25 2011 07:44 AM
  • Joined: 03 Nov 2006
Quick question,

I think I got this down, and it works great.
So far I have been using bmp images inside the axc file and they show up perfectly. (Thanks for the Skinned GUI example, that was exactly what I was trying to do myself) However, just today I was trying to add a png and no matter what I do, it doesnt seem to appear in the GUI.

Its packed into the axc file, and it shows on csv file, but no luck.

Should I be doing something differently to handle png images?

WankaUSR
  • Members
  • 87 posts
  • Last active: Jul 14 2013 09:59 AM
  • Joined: 14 Aug 2007
thanks all for this scripts it realy ,ade some project work nicely :D
And SKAN do you know how i can skin the status bar too because it seems to out of my understanding

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005

Should I be doing something differently to handle png images?


Yes! For JPG/GIF/PNG you would have to use GDI+ to obtain bitmap.
Initialise GDI+ In auto-execute section, like:

DllCall( "LoadLibrary", Str,"gdiplus" ) 
VarSetCapacity(si, 16, 0), si := Chr(1) 
DllCall( "gdiplus\GdiplusStartup", UIntP,pToken, UInt,&si, UInt,0 )

Use FileReadEx() to load the image data and then call gdiBitmap() to obtain the bitmap handle:

gdiBitmap( byref buffer, nsize, byref wxh ) { 
; Sean : http://www.autohotkey.com/forum/viewtopic.php?p=147029#147029 
  hData := DllCall("GlobalAlloc", UInt,2, UInt, nSize ) 
  pData := DllCall("GlobalLock",  UInt,hData ) 
  DllCall( "RtlMoveMemory", UInt,pData, UInt,&Buffer, UInt,nSize ) 
  DllCall( "GlobalUnlock", UInt,hData ) 
  DllCall( "ole32\CreateStreamOnHGlobal", UInt,hData, Int,True, UIntP,pStream ) 

  DllCall( "gdiplus\GdipCreateBitmapFromStream", UInt,pStream, UIntP,pBitmap ) 
  DllCall( "gdiplus\GdipCreateHBITMAPFromBitmap", UInt,pBitmap, UIntP,hBitmap, UInt,8 ) 
  DllCall("gdiplus\GdipGetImageWidth" , "Uint", pBitmap, "UintP", nW) 
  DllCall("gdiplus\GdipGetImageHeight", "Uint", pBitmap, "UintP", nH), wxh := nW "x" nH 
  DllCall( "gdiplus\GdipDisposeImage", UInt,pBitmap ) 
return hbitmap 
}

then, use SendMessage, STM_SetImage to apply the hBitmap
and then shutdown GDI+ when it is no more needed, like:

DllCall( "gdiplus\GdiplusShutdown", UInt,pToken ) 
 DllCall( NumGet(NumGet(1*pStream)+8 ), UInt,pStream )

:)

Edit: gdiBitmap() has been replaced

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005

how i can skin the status bar


I do not know. If I had to - I would put a nice gradient and would use a number of transparent Text controls precisely placed and styled/ex-styled to simulate a pseudo-status bar.

The advantage : You cannot control the foreground text color in a sb control, but with text control, you can.

:)