InBin() = InStr() for searching binary buffer.

Post your working scripts, libraries and tools for AHK v1.1 and older
User avatar
SKAN
Posts: 1551
Joined: 29 Sep 2013, 16:58

InBin() = InStr() for searching binary buffer.

08 May 2021, 13:36

InBin( pHaystack, HaystackBytes, pNeedle, NeedleBytes:=0, CaseSensitive:=1, StartingPos:=1, Occurence:=1 )
Searches binary data and returns a pointer when a needle is found or returns 0 when search fails.
ErrorLevel is set with an error message on failure.
This function was modeled after my favorite InStr() function. This function employs machine code for a faster search.

Reference:
I had wanted to write this one for a while, but recently got motivated by @teadrinker`s SearchInBuff()
There is useful info and InBuf() by @Helgef here
 
Quick example calls: InBin() works very similar to InStr()
In following example calls, search is made for single byte so that it works both in Ansi and Unicode AHK.

Code: Select all

#NoEnv
#Warn
#SingleInstance, Force

Hay    := "The Quick Brown Fox Jumps Over the Lazy Dog"
HayLen := VarSetCapacity(Hay)
Ndl    := "o"
NdlLen := 1

FoundPtr := InBin(&Hay, HayLen, &Ndl, NdlLen)          ; Normal search
MsgBox % StrGet( FoundPtr )

FoundPtr := InBin(&Hay, HayLen, &Ndl, NdlLen,, 0)      ; Search from right
MsgBox % StrGet( FoundPtr )

FoundPtr := InBin(&Hay, HayLen, &Ndl, NdlLen,, 0, 2)   ; Search from right the 2nd occurence
MsgBox % StrGet( FoundPtr )

FoundPtr := InBin(&Hay, HayLen, &Ndl, NdlLen, 0, 0, 2) ; Search from right the 2nd occurence, case-insensitive
MsgBox % StrGet( FoundPtr )
 
Normally, you should prepare a needle and pass its pointer to pNeedle parameter and the data length to NeedleBytes parameter.
The valid data length range for NeedleBytes parameter is 1 to 256.
InBin() will automatically prepare the needle when NeedleBytes is not a valid data length value.
The default value of NeedleBytes is 0 and when 0 it will expect pNeedle to contain hex data in which case the function will convert
Hex-to-Bin automatically, as seen in following example:

Code: Select all

#NoEnv
#Warn
#SingleInstance, Force

VarSetCapacity(Hay, 128, 0)
StrPut("The Quick Brown Fox Jumps Over the Lazy Dog", &Hay, 128, "cp0")

FoundPtr := InBin(&Hay, 128, "4C 61 7A 79") ; 'L' 'a' 'z' 'y'
MsgBox % StrGet( FoundPtr, "cp0" )
Note: I have omitted 4th parameter as the default value of NeedleBytes is 0.
 
NeedleBytes parameter may also contain a string, in which case, InBin() will use NumPut() / StrPut()
to prepare the Needle. Here follows two different ways to prepare the same needle:

Code: Select all

#NoEnv
#Warn
#SingleInstance, Force

VarSetCapacity(Hay, 128, 0)
StrPut("The Quick Brown Fox Jumps Over the Lazy Dog", &Hay, 128, "cp0")

FoundPtr := InBin(&Hay, 128, 0x797A614C, "UInt") ; uses NumPut()
MsgBox % StrGet( FoundPtr, "cp0" )
FoundPtr := InBin(&Hay, 128, "Lazy", "cp0")      ; uses StrPut()
MsgBox % StrGet( FoundPtr, "cp0" )
 
One limitation of NumPut() is that there is no support for Int24, Int40, Int48 Int56 types.
You may specify the an Int type's bytes size as a negative number and get the conversion done.
Note: While 0 is for Hex-to-bin, you may use -1 -2 -3 -4 -5 -6 -7 -8 to encode a custom integer.
Sample calls:

Code: Select all

#NoEnv
#Warn
#SingleInstance, Force

VarSetCapacity(Hay, 128, 0)
StrPut("The Quick Brown Fox Jumps Over the Lazy Dog", &Hay, 128, "cp0")

FoundPtr := InBin(&Hay, 128, 0x6B63697551, -5) ; Hex of 'Quick' in reverse = 'kciuQ'
MsgBox % StrGet( FoundPtr, "cp0" )
FoundPtr := InBin(&Hay, 128, 74, -1) ; Ord("J") = 74
MsgBox % StrGet( FoundPtr, "cp0" )
 
One final test. Haystack size is not known, but I know the needle exists:

Code: Select all

#NoEnv
#Warn
#SingleInstance, Force

hMod     := DllCall("GetModuleHandle", "Str",A_AhkPath, "Ptr")
FoundPtr := InBin(hMod, 0xffffff, "FileDescription", "utf-16")
If ( FoundPtr )
   MsgBox % StrGet(FoundPtr + 34, "utf-16")
 
2 more working examples here: https://www.autohotkey.com/boards/viewtopic.php?t=90383
 
 
Speed test
 
 
C source for machine code
 
 
The function
 

Code: Select all

InBin( pHaystack, HaystackBytes, pNeedle, NeedleBytes:=0, CaseSensitive:=1, StartingPos:=1, Occurence:=1 )  {
Local                                                      ; InBin v0.60 by SKAN on D456/D459 @ tiny.cc/inbin
Static InBinMcode  := InBin(0,0,0,0)                                
Static MemCpyLower := InBinMcode + (A_PtrSize=8 ? 204 : 184)

  If ( ! VarSetCapacity(InBinMcode) )
  {
     M1 := DllCall("Kernel32.dll\GlobalAlloc", "Int",0, "Ptr",Msz := A_PtrSize=8 ? 268 : 248, "UPtr")
     M2 := DllCall("Kernel32.dll\VirtualProtect", "Ptr",M1, "Ptr",Msz, "Int",0x40, "IntP",0)

     M3 := DllCall("Crypt32.dll\CryptStringToBinary", "Str",A_PtrSize=8
     ? "U1ZXQVSLRCRIRItcJFCJ00Qpy4XAvgEAAABBuv////9BD07yhcB+B2dEjVD/6wgBwkGJ0kUpykGD6QFFhdJyQzHSQTnadzxEidBB"
     . "ijhAODwBdShEichBigQAZ0ONPAqJ/zgEOXUVQYP5AnMbg8IB6wVEOc909kQ52nRDQQHyRYXSc78xwOs9vwEAAABBg/kBdt8PH0QA"
     . "AGYPH4QAAAAAAIn4QYoEAGdFjSQ6RYnkQjgEIXW9g8cBRDnPcuTrs0SJ0EgByEFcX15bwwAARYXAdjoxwEGJwUaKFAlBgPpAdhJB"
     . "icNCgDwZW3MIQbsgAAAA6wNFMdtFD7bSRQHaRYjSRogUCoPAAUQ5wHLIww"    :    "VYnlg+wQU1ZXi1UIi00Qi3UUi0UMKfC"
     . "JRfQxwIN9GAAPntD32IPg/kCJRfyDfRgAfgmLRRhIiUX46wuLRQwDRRgp8IlF+ItF+InHToX/cjvHRfAAAAAAO330dy+KBDo6AXU"
     . "hjQQ3igQCOgQxdRaD/gJzHP9F8OsEOfN094tF8DtFHHQnA338hf9zzDHA6x+7AQAAAIP+AXbfjQQfigQCOgQZddRDOfNy8OvNjQQ"
     . "6X15biexdwwAAAFNWV4tEJBCLVCQUi0wkGIXJdiYx9oocMID7QHYNgDwwW3MHvyAAAADrAjH/D7bbAfuIHDJGOc5y3F9eW8MAAAA"
     , "Int",A_PtrSize=8 ? 358 : 331, "Int",0x1, "Ptr",M1, "IntP",Msz, "Int",0, "Int",0)

     Return M1
  }

  If NeedleBytes is number
  {
       NeedleBytes := Max(-8, Format("{:d}", NeedleBytes))

       If ( NeedleBytes<0 && NeedleBytes>-9 )
       {
            VarSetCapacity(Bin,8)
            pNeedle := NumPut(pNeedle, Bin, "UInt64") - 8
            NeedleBytes := Abs(NeedleBytes)
       }

       Else
       If ( NeedleBytes=0 )
       {
            Src := StrReplace(pNeedle,A_Space)
            Len := StrLen(Src)
            NeedleBytes := Ceil(Len/2)
            VarSetCapacity(Bin, NeedleBytes, 0)
            If ! DllCall("Crypt32.dll\CryptStringToBinary", "Str",Src, "Int",Len, "Int",12
                     , "Ptr",pNeedle := &Bin, "UIntP",NeedleBytes, "Int",0, "Int",0)
                 Return (0, ErrorLevel := "Hex to Bin conversion failed.")
       }
  }

  Else
  If ( InStr(".double.float.uint64.int64.uint.int.ushort.short.uptr.ptr.uchar.char."
           , "." . RTrim(NeedleBytes, "*p") . ".") )
  {
       VarSetCapacity(Bin, 8, 0)
       NeedleBytes := NumPut(pNeedle, &Bin, NeedleBytes) - &Bin
       pNeedle := &Bin
  }
  Else
  {
       m := ( NeedleBytes="utf-16" || NeedleBytes="cp1200" ? 2 : 1 )
       If ! ( nBytes := StrPut(pNeedle, NeedleBytes) - 1 )
              Return (0, ErrorLevel := "String encode failed: '" . NeedleBytes . "'")
       VarSetCapacity(sStr, nBytes * m)
       nBytes := StrPut(pNeedle, &sStr, nBytes, NeedleBytes)
       pNeedle  := &sStr,   NeedleBytes := nBytes * m
  }

  NeedleBytes := Min(256, NeedleBytes)
  Occurence := Max(1, Format("{:d}", Occurence))
, CaseSensitive := !!CaseSensitive

  If ( CaseSensitive=0 && HaystackBytes>0x1000000 )
       Return (0, ErrorLevel := "Haystack too large for case-insensitve search. (Limit: 16 MiB)")

  If ( HaystackBytes - NeedleBytes - (StartingPos>0 ? StartingPos-1 : Abs(StartingPos)) < 0 )
       Return (0, ErrorLevel := "Haystack is too small to accomodate StartPosition and Needle")

  If ( CaseSensitive = False )
  {
       VarSetCapacity(Hay, HaystackBytes), pHaystack2 := &Hay
       DllCall(MemCpyLower, "Ptr",pHaystack, "Ptr",pHaystack2, "Int", HaystackBytes, "CDecl")
       VarSetCapacity(Ndl, NeedleBytes),   pNeedle2   := &Ndl
       DllCall(MemCpyLower, "Ptr",pNeedle,   "Ptr",pNeedle2,   "Int", NeedleBytes,   "CDecl")
  }

  FoundPtr := DllCall(InBinMcode, "Ptr",CaseSensitive ? pHaystack : pHaystack2,  "Int", HaystackBytes
                                , "Ptr",CaseSensitive ? pNeedle   : pNeedle2,  "Short", NeedleBytes
                                , "Int",StartingPos, "Int",Occurence, "CDecl Ptr")

  ErrorLevel := ( FoundPtr="" ? "Memory access violation" : FoundPtr=0 ? "Needle not found." : "" )
Return ( FoundPtr ? ( CaseSensitive ? FoundPtr : pHaystack + FoundPtr - pHaystack2 ) : FoundPtr )
}
My Scripts and Functions: V1  V2
hasantr
Posts: 933
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: InBin() = InStr() for searching binary buffer.

10 May 2021, 05:01

This has been a great job. I've always looked for a faster alternative to the instr. Will I be able to gain more speed in use?
Is there a Binreplace function too? :)
Thanks. All your things are very good.
User avatar
SKAN
Posts: 1551
Joined: 29 Sep 2013, 16:58

Re: Yd: InBin() = inStr() ikili arabellek aramak için.

10 May 2021, 06:03

hasantr wrote:
10 May 2021, 05:01
I've always looked for a faster alternative to the instr. Will I be able to gain more speed in use?
No. this will be slightly slower than InStr(), however, it can do much more than InStr()
For example: The following can't be done with InStr()
I already wrote:One final test. Haystack size is not known, but I know the needle exists:

Code: Select all

#NoEnv
#Warn
#SingleInstance, Force

hMod     := DllCall("GetModuleHandle", "Str",A_AhkPath, "Ptr")
FoundPtr := InBin(hMod, 0xffffff, "FileDescription", "utf-16")
If ( FoundPtr )
   MsgBox % StrGet(FoundPtr + 34, "utf-16")
 
With file object, you can open/read/write but cannot search.
I'm writing a MapFile() function which will be a bridge between File object and InBin().
I'll post it in a few hours. (Edit: Done!)
Is there a Binreplace function too? :)
I''ll write one for you, if you need it. :)
hasantr
Posts: 933
Joined: 05 Apr 2016, 14:18
Location: İstanbul

InBin()

10 May 2021, 06:10

SKAN wrote:
10 May 2021, 06:03
hasantr wrote:
10 May 2021, 05:01
I've always looked for a faster alternative to the instr. Will I be able to gain more speed in use?
No. this will be slightly slower than InStr(), however, it can do much more than InStr()
For example: The following can't be done with InStr()
I already wrote:One final test. Haystack size is not known, but I know the needle exists:

Code: Select all

#NoEnv
#Warn
#SingleInstance, Force

hMod     := DllCall("GetModuleHandle", "Str",A_AhkPath, "Ptr")
FoundPtr := InBin(hMod, 0xffffff, "FileDescription", "utf-16")
If ( FoundPtr )
   MsgBox % StrGet(FoundPtr + 34, "utf-16")
 
With file object, you can open/read/write but cannot search.
I'm writing a MapFile() function which will be a bridge between File object and InBin(). I'll post it in a few hours.
Is there a Binreplace function too? :)
I''ll write one for you, if you need it. :)
Thank you. It will be used in some places I need for my work. I have tried some winding roads because there is no such function. To get rid of UTF8 ansi clutter does the Inbin be good?

BinReplace will not be faster than StrReplace. Then it is not an important need. But maybe it can help others' work. I had been needed in a function like Binreplace. But I don't remember how I troubleshooted.
User avatar
SKAN
Posts: 1551
Joined: 29 Sep 2013, 16:58

Re: Yd: InBin() = inStr() ikili arabellek aramak için.

10 May 2021, 07:21

hasantr wrote:
10 May 2021, 06:10
To get rid of UTF8 ansi clutter does the Inbin be good?
Can you elaborate on this?
InBin() is a binary search function. There is no concept of text.
hasantr
Posts: 933
Joined: 05 Apr 2016, 14:18
Location: İstanbul

InBin()

10 May 2021, 09:05

SKAN wrote:
10 May 2021, 07:21
hasantr wrote:
10 May 2021, 06:10
To get rid of UTF8 ansi clutter does the Inbin be good?
Can you elaborate on this?
InBin() is a binary search function. There is no concept of text.
Yes I know. UTF-8 Ascii has no confusion because there is Binary. I meant that but it doesn't matter already. I can not speak English well. :)
User avatar
SKAN
Posts: 1551
Joined: 29 Sep 2013, 16:58

InBin()

10 May 2021, 09:28

hasantr wrote:
10 May 2021, 09:05
Yes I know. UTF-8 Ascii has no confusion because there is Binary.
Ahh! :) :thumbup:
 
I've posted my MapFile()
https://www.autohotkey.com/boards/viewtopic.php?t=90383

Please, do try the examples.
aldrinjohnom
Posts: 77
Joined: 18 Apr 2018, 08:49

Re: InBin() = InStr() for searching binary buffer.

08 Nov 2023, 07:33

@SKAN can you convert this for v2?

I prefer using InBin rather than InBuffer for DMA Purpose.
Developer of AJOM's DOTA2 MOD Master, Easiest way to MOD your DOTA2 without the use of internet :lol: . I created this program using Autohotkey thats why I love Autohotkey and the community!

GitHub:
https://github.com/Aldrin-John-Olaer-Manalansan

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: No registered users and 137 guests