mCrc() : Computes CRC *value for binary data. (*arbitrary-precision 8-64 bits)

Post your working scripts, libraries and tools for AHK v1.1 and older
User avatar
SKAN
Posts: 1551
Joined: 29 Sep 2013, 16:58

mCrc() : Computes CRC *value for binary data. (*arbitrary-precision 8-64 bits)

Post by SKAN » 22 Apr 2021, 13:57

mCrc( pPtr, nBytes, Preset* )
This is an improved machine code variant of aCrc()
 
Preset objects
 
Usage example:

Code: Select all

CRC_8            := [0x07, False, 8, 0x00, 0x00]                      ; CRC-8
CRC_32           := [0xedb88320, True, 32, 0xffffffff, 0xffffffff]    ; CRC-32
CRC_16_AUG_CCITT := [0x1021, False, 16, 0x1d0f, 0x0000]               ; CRC-16/AUG-CCITT
CRC_64_XZ        := ["0xc96c5795d7870f42", True,  64, "0xffffffffffffffff", "0xffffffffffffffff"] ; CRC-64/XZ


FileGetSize, nBytes, %A_AhkPath%
FileRead, Bin, *c %A_AhkPath%
MsgBox % mCrc(&Bin, nBytes, CRC_64_XZ*)

VarSetCapacity(sStr, 128)
nBytes := StrPut("AutoHotkey", &sStr, 128, "utf-8") - 1

MsgBox % mCrc(&sStr, nBytes, CRC_32*)
MsgBox % mCrc(&sStr, nBytes, CRC_16_AUG_CCITT*)
MsgBox % mCrc(&sStr, nBytes, CRC_8*)
 
 
The function:
 

Code: Select all

mCrc(pptr, nbytes, poly, refin, width, init, xorout) {   ; v0.33 by SKAN on D44N/D44N @ tiny.cc/mcrc
Local
Static mCrcMcode := mCrc(0,0,0,0,0,0,0)

  If (!VarSetCapacity(mCrcMcode))
  {
    M1 := DllCall("Kernel32.dll\GlobalAlloc", "Int",0, "Ptr",MSz:=(A_PtrSize=8 ? 531 : 917), "UPtr")
    M2 := DllCall("Kernel32.dll\VirtualProtect", "Ptr",M1, "Ptr",MSZ, "Int",0x40, "IntP",0)
    M3 := DllCall("Crypt32.dll\CryptStringToBinary", "Str",A_PtrSize=8 ? "
    (  LTrim Join
       U1ZXQVRIi0QkSEyLVCRQTItcJFhIi1wkYEiLdCRoSIt8JHCFyQ+ECwEAADHJ6fYAAABBicxOD7YkIk0x4Un3wQEAAAB0C
       02JzEnR7EkxxOsGTYnMSdHsTYnhSffBAQAAAHQLTYnMSdHsSTHE6wZNicxJ0exNieFJ98EBAAAAdAtNicxJ0exJMcTrBk
       2JzEnR7E2J4Un3wQEAAAB0C02JzEnR7EkxxOsGTYnMSdHsTYnhSffBAQAAAHQLTYnMSdHsSTHE6wZNicxJ0exNieFJ98E
       BAAAAdAtNicxJ0exJMcTrBk2JzEnR7E2J4Un3wQEAAAB0C02JzEnR7EkxxOsGTYnMSdHsTYnhSffBAQAAAHQLTYnMSdHs
       STHE6wZNicxJ0exNieGDwQFEOcEPggH////pzgAAAESJ0UUx0um6AAAARYnUTg+2JCJJ0+RNMeFNhdl0CU+NJAlJMcTrB
       E+NJAlNieFNhdl0CU+NJAlJMcTrBE+NJAlNieFNhdl0CU+NJAlJMcTrBE+NJAlNieFNhdl0CU+NJAlJMcTrBE+NJAlNie
       FNhdl0CU+NJAlJMcTrBE+NJAlNieFNhdl0CU+NJAlJMcTrBE+NJAlNieFNhdl0CU+NJAlJMcTrBE+NJAlNieFNhdl0CU+
       NJAlJMcTrBE+NJAlNieFBg8IBRTnCD4I9////SSHZSTHxTIkPQVxfXlvD
    )" : "
    (  LTrim Join
       VYnlg+wMVleLRRSLVRiDfQgAD4SaAQAAx0X8AAAAAOl9AQAAi00Mi3X8D7YMMYnOMf8x8DH6icaJ14PmATH/hfZ0EonGi
       dcPrP4B0e8zdRwzfSDrConGidcPrP4B0e+J8In6icaJ14PmATH/hfZ0EonGidcPrP4B0e8zdRwzfSDrConGidcPrP4B0e
       +J8In6icaJ14PmATH/hfZ0EonGidcPrP4B0e8zdRwzfSDrConGidcPrP4B0e+J8In6icaJ14PmATH/hfZ0EonGidcPrP4
       B0e8zdRwzfSDrConGidcPrP4B0e+J8In6icaJ14PmATH/hfZ0EonGidcPrP4B0e8zdRwzfSDrConGidcPrP4B0e+J8In6
       icaJ14PmATH/hfZ0EonGidcPrP4B0e8zdRwzfSDrConGidcPrP4B0e+J8In6icaJ14PmATH/hfZ0EonGidcPrP4B0e8zd
       RwzfSDrConGidcPrP4B0e+J8In6icaJ14PmATH/hfZ0EonGidcPrP4B0e8zdRwzfSDrConGidcPrP4B0e+J8In6/0X8i0
       38O00QD4J3/v//6ckBAACLdSSLfSiJ8YlN+MdF9AAAAADppgEAAItNDIt19A+2DDGJzjH/i034D6X30+b2wSB0BIn3MfY
       x8DH6i3Usi30wIcYh13UEhfZ0EonGidcPpPcBAfYzdRwzfSDrConGidcPpPcBAfaJ8In6i3Usi30wIcYh13UEhfZ0EonG
       idcPpPcBAfYzdRwzfSDrConGidcPpPcBAfaJ8In6i3Usi30wIcYh13UEhfZ0EonGidcPpPcBAfYzdRwzfSDrConGidcPp
       PcBAfaJ8In6i3Usi30wIcYh13UEhfZ0EonGidcPpPcBAfYzdRwzfSDrConGidcPpPcBAfaJ8In6i3Usi30wIcYh13UEhf
       Z0EonGidcPpPcBAfYzdRwzfSDrConGidcPpPcBAfaJ8In6i3Usi30wIcYh13UEhfZ0EonGidcPpPcBAfYzdRwzfSDrCon
       GidcPpPcBAfaJ8In6i3Usi30wIcYh13UEhfZ0EonGidcPpPcBAfYzdRwzfSDrConGidcPpPcBAfaJ8In6i3Usi30wIcYh
       13UEhfZ0EonGidcPpPcBAfYzdRwzfSDrConGidcPpPcBAfaJ8In6/0X0i030O00QD4JO/v//i01EI0U0I1U4M0U8M1VAi
       QGJUQRfXonsXcM=
    )", "Int",A_PtrSize=8 ? 708 : 1224, "Int",0x1, "Ptr",M1, "IntP",MSZ, "Int",0, "Int",0)

    Return M1
  }
  
  DllCall(mCrcMcode, "Int",refin, "Ptr",pptr, "Int",nbytes, "UInt64",init, "UInt64",poly
      , "UInt64",width-8, "UInt64",1<<width-1, "UInt64",Format("0x{:x}", width=64 ? -1 : 2**width-1)
      , "UInt64",xorout, "UInt64P",CRC:=0)

Return Format("0x{:X}", CRC)
}
 
C source for machine code
My Scripts and Functions: V1  V2
User avatar
SKAN
Posts: 1551
Joined: 29 Sep 2013, 16:58

mCrc() : CRC-40/SKAN-AUTOPASS

Post by SKAN » 03 May 2021, 08:07

The following CRC-40 preset is good for for small/medium size strings:

Code: Select all

CRC_40_SKAN_AUTOPASS := [0xb244000000, True,  40, 0xffffffffff, 0xffffffffff] ; CRC-40/SKAN-AUTOPASS
No collisions found in a words list with 466,546 items.
No collisions in 421,184 file-full-paths in my HDD.
 
 
The standard CRC-32 gives me 23 collisions in a words list with 466,546 items.

Code: Select all

0x87B6ADBA  Ilysa           Endamoebidae
0x122FB1DF  bermensch       Audras
0x5CAF9D11  demobilisation  defeudalize
0x3709DBCA  eminency        Kelcie
0xE468340E  gigmanism       dyn
0x177DF009  hyp             gigmaness
0x42133A56  intertwinements card-cutting
0x003C842A  lattermath      drift-netter
0x74B376B8  meny            menthols
0x12290422  morenosite      funerary
0x6C308CD4  platycarpous    envoyship
0x4DDB0C25  plumless        buckeroo
0xA981FFE1  precedential    fetishists
0xBF6A229E  prepituitary    penetration
0xB48262AE  rabbitoh        death-polluted
0x2A0E95EE  slagging        Bridget
0x78BB5CD9  superspecial    bimasty
0xA690B53B  thalloid        droopingness
0x72313FFA  twice-given     luminesce
0x257FFB3A  tythed          coach-box
0xD73A7E23  uncontorted     casewood
0xDDC3AF7B  unexigible      bronziest
0x1ED02BC9  wind-changing   pachadoms
Here is a sample:

Code: Select all

#NoEnv
#Warn
#SingleInstance, Force

crc1 := DllCall("ntdll.dll\RtlComputeCrc32", "Int", 0, "AStr", "slagging", "Int",8, "UInt")
crc2 := DllCall("ntdll.dll\RtlComputeCrc32", "Int", 0, "AStr", "Bridget",  "Int",7, "UInt")
MsgBox % Format( "0x{:X}`n0x{:X}", crc1, crc2)
User avatar
jNizM
Posts: 3183
Joined: 30 Sep 2013, 01:33
Contact:

Re: mCrc() : Computes CRC *value for binary data. (*arbitrary-precision 8-64 bits)

Post by jNizM » 04 May 2021, 01:57

Thanks for your mcode version
Here is an alternative for 32 / 16-CCITT (https://www.autohotkey.com/boards/viewtopic.php?t=63052)
[AHK] v2.0.5 | [WIN] 11 Pro (Version 22H2) | [GitHub] Profile
hasantr
Posts: 933
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: mCrc() : Computes CRC *value for binary data. (*arbitrary-precision 8-64 bits)

Post by hasantr » 11 May 2021, 16:21

jNizM wrote:
04 May 2021, 01:57
Thanks for your mcode version
Here is an alternative for 32 / 16-CCITT (https://www.autohotkey.com/boards/viewtopic.php?t=63052)
I have used the functions that you have provided for a long time. Will mcode be faster?
hasantr
Posts: 933
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: mCrc() : Computes CRC *value for binary data. (*arbitrary-precision 8-64 bits)

Post by hasantr » 23 Mar 2022, 16:03

Hello. In this way I want to get CRC data of all paths in C drive. But why not. Collisions are constantly occurring.

Code: Select all

CRC_8            		:= [0x07, False, 8, 0x00, 0x00]                      ; CRC-8
CRC_32           		:= [0xedb88320, True, 32, 0xffffffff, 0xffffffff]    ; CRC-32
CRC_16_AUG_CCITT 		:= [0x1021, False, 16, 0x1d0f, 0x0000]               ; CRC-16/AUG-CCITT
CRC_40_SKAN_AUTOPASS 	:= ["0xb244000000", True,  40, "0xffffffffff", "0xffffffffff"] ; CRC-40/SKAN-AUTOPASS
CRC_64_XZ        		:= ["0xc96c5795d7870f42", True,  64, "0xffffffffffffffff", "0xffffffffffffffff"] ; CRC-64/XZ

gui + Owner
gui + OwnDialogs
FilO := {}
sStr := 

Loop, Files, C:\*.*, R
{
	paths := A_LoopFileLongPath
	VarSetCapacity(sStr, 128)
	nBytes := StrPut(paths, &sStr, 128, "utf-8") - 1
	CvrcV := mCrc(&paths, nBytes, CRC_64_XZ*)
	if(FilO.HasKey(CvrcV))
		MsgBox,64,Clash , % CvrcV " " A_index "`r " A_LoopFileLongPath ""
	FilO[CvrcV] := "1"
}
For k, v in FilO
	MsgBox % k
MsgBox % A_LoopFileLongPath " " "Last"
return
Edit:
I've been trying to fix this for a long time. I finally guessed why. Keys starting with 0x are problematic. I made a small correction in the return part of the function. It works fine now. Keys starting with 0x were generating too many conflicts because I was trying to assign and check conflicts to the object.

Code: Select all

Return Format("{:X}", CRC)
Post Reply

Return to “Scripts and Functions (v1)”