AutoHotkey Community

It is currently May 24th, 2012, 9:09 pm

All times are UTC [ DST ]




Post new topic Reply to topic  [ 43 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: MD5
PostPosted: March 27th, 2007, 5:08 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
MD5 is a cryptographic hash, that is, an arbitrary long sequence of binary bits is transformed to 128 bits of "digests". Although it is not very secure (different inputs can be created, which are hashed to the same digest), it is still widely used to verify code integrity.

Here is a pure AHK function, which computes the MD5 hash of binary data of size < 512MB. The result is a sequence of 32 hex digits.
Code:
; GLOBAL CONSTANTS r[64], k[64]
r =  12, 17, 22,  7, 12, 17, 22,  7, 12, 17, 22,  7, 12, 17, 22
, 5,  9, 14, 20,  5,  9, 14, 20,  5,  9, 14, 20,  5,  9, 14, 20
, 4, 11, 16, 23,  4, 11, 16, 23,  4, 11, 16, 23,  4, 11, 16, 23
, 6, 10, 15, 21,  6, 10, 15, 21,  6, 10, 15, 21,  6, 10, 15, 21
StringSplit r, r, `,
r0 := 7
Loop 64
   i := A_Index-1, k%i% := floor(abs(sin(A_Index)) * 2**32)

; TEST CASES
MsgBox % MD5(x:="", 0)                                                    ; d41d8cd98f00b204e9800998ecf8427e
MsgBox % MD5(x:="a", StrLen(x))                                           ; 0cc175b9c0f1b6a831c399e269772661
MsgBox % MD5(x:="abc", StrLen(x))                                         ; 900150983cd24fb0d6963f7d28e17f72
MsgBox % MD5(x:="message digest", StrLen(x))                              ; f96b697d7cb7938d525a2f31aaf161d0
MsgBox % MD5(x:="abcdefghijklmnopqrstuvwxyz", StrLen(x))                  ; c3fcd3d76192e4007dfb496cca67e13b
MsgBox % MD5(x:="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789", StrLen(x))
                                                                          ; d174ab98d277d9f5a5611c2c9f419d9f
MsgBox % MD5(x:="12345678901234567890123456789012345678901234567890123456789012345678901234567890", StrLen(x))
                                                                          ; 57edf4a22be3c955ac49da2e2107b67a
MsgBox % MD5(x:="The quick brown fox jumps over the lazy dog", StrLen(x)) ; 9e107d9d372bb6826bd81d3542a419d6
MsgBox % MD5(x:="The quick brown fox jumps over the lazy cog", StrLen(x)) ; 1055d3e698d289f2af8663725127bd4b

MD5(ByRef Buf, L) {                                                   ; Binary buffer, Length in bytes
   Static P, Q, N, i, a,b,c,d, t, h0,h1,h2,h3, y = 0xFFFFFFFF

   h0 := 0x67452301, h1 := 0xEFCDAB89, h2 := 0x98BADCFE, h3 := 0x10325476

   N := ceil((L+9)/64)*64                                             ; padded length (100..separator, 8B length)
   VarSetCapacity(Q,N,0)                                              ; room for padded data
   P := &Q                                                            ; pointer
   DllCall("RtlMoveMemory", UInt,P, UInt,&Buf, UInt,L)                ; copy data
   DllCall("RtlFillMemory", UInt,P+L, UInt,1, UInt,0x80)              ; pad separator
   DllCall("ntdll.dll\RtlFillMemoryUlong",UInt,P+N-8,UInt,4,UInt,8*L) ; at end: length in bits < 512 MB

   Loop % N//64 {
      Loop 16
         i := A_Index-1, w%i% := *P | *(P+1)<<8 | *(P+2)<<16 | *(P+3)<<24, P += 4

      a := h0, b := h1, c := h2, d := h3

      Loop 64 {
         i := A_Index-1
         If i < 16
             f := (b & c) | (~b & d), g := i
         Else If i < 32
             f := (d & b) | (~d & c), g := 5*i+1 & 15
         Else If i < 48
             f := b ^ c ^ d,          g := 3*i+5 & 15
         Else
             f := c ^ (b | ~d),       g :=  7*i  & 15

         t := d, d := c, c := b
         b += rotate(a + f + k%i% + w%g%, r%i%) ; reduced to 32 bits later
         a := t
      }

      h0 := h0+a & y, h1 := h1+b & y, h2 := h2+c & y, h3 := h3+d & y
   }
   Return hex(h0) . hex(h1) . hex(h2) . hex(h3)
}

rotate(a,b) { ; 32-bit rotate a to left by b bits, bit32..63 garbage
   Return a << b | (a & 0xFFFFFFFF) >> (32-b)
}

hex(x) {      ; 32-bit little endian hex digits
   SetFormat Integer, HEX
   x += 0x100000000, x := SubStr(x,-1) . SubStr(x,8,2) . SubStr(x,6,2) . SubStr(x,4,2)
   SetFormat Integer, DECIMAL
   Return x
}


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 28th, 2007, 4:33 pm 
Offline

Joined: January 18th, 2006, 3:35 pm
Posts: 28
Can this be used to get the MD5 of a file? if so is faster than using the hashes.dll posted in the forums..

Thanxs


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 28th, 2007, 4:47 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
short wrote:
Can this be used to get the MD5 of a file?
If the file does not fit to an AHK variable, the script would need some modifications: in the outer loop you have to read the next 512 bytes from the file and process the padding separately.
short wrote:
is faster than using the hashes.dll posted in the forums?
Of course, not. AHK scripts are interpreted line-by-line. They are always slower than compiled code. The main use of this script is for hashing data in AHK variables. A small file (up to a few MB) can be read in a variable and processed, but files of hundreds of MB are better handled by third party programs.

My 2GHz Centrino laptop takes 20 seconds for hashing 1MB of data.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 28th, 2007, 5:45 pm 
Offline

Joined: January 18th, 2006, 3:35 pm
Posts: 28
ok, yeah the dll process them in ms.

But thanks alot


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 28th, 2007, 7:15 pm 
Offline

Joined: May 24th, 2006, 2:49 pm
Posts: 4511
Location: Belgrade
very nice. THank you.

_________________
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 4:23 am 
Offline

Joined: November 13th, 2004, 4:08 am
Posts: 2951
Location: Minnesota
Laszlo wrote:
short wrote:
is faster than using the hashes.dll posted in the forums?
Of course, not. AHK scripts are interpreted line-by-line. They are always slower than compiled code.


Actually, the code is mostly interpreted when the script starts up, making AHK more of a semi-compiled language. It still runs slower since the code has to be generic, but with the current model it's probably close to an optimal speed.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 6:52 am 
Offline

Joined: February 12th, 2007, 7:54 am
Posts: 2462
Why don't you take advantage of API's?
It took about 20 ~ 30 ms for a text file of size > 2MB in my machine.

Code:
#NoEnv
SetBatchLines, -1

;DllCall("QueryPerformanceFrequency", "int64P", perffreq)
;DllCall("QueryPerformanceCounter", "int64P", perfcount0)

sData := "12345678901234567890123456789012345678901234567890123456789012345678901234567890"
sHash := Hash(sData, StrLen(sData))

;DllCall("QueryPerformanceCounter", "int64P", perfcount1)
;MsgBox % (perfcount1 - perfcount0) / perffreq

MsgBox % sHash


Hash(ByRef sData, nLen, SID = 3)   ; SID: 3 for MD5, 4 for SHA
{
   DllCall("advapi32\CryptAcquireContextA", "UintP", hProv, "Uint", 0, "Uint", 0, "Uint", 1, "Uint", 0xF0000000)
   DllCall("advapi32\CryptCreateHash", "Uint", hProv, "Uint", 0x8000|0|SID , "Uint", 0, "Uint", 0, "UintP", hHash)

   DllCall("advapi32\CryptHashData", "Uint", hHash, "Uint", &sData, "Uint", nLen, "Uint", 0)

   DllCall("advapi32\CryptGetHashParam", "Uint", hHash, "Uint", 2, "Uint", 0, "UintP", nSize, "Uint", 0)
   VarSetCapacity(HashVal, nSize, 0)
   DllCall("advapi32\CryptGetHashParam", "Uint", hHash, "Uint", 2, "Uint", &HashVal, "UintP", nSize, "Uint", 0)

   DllCall("advapi32\CryptDestroyHash", "Uint", hHash)
   DllCall("advapi32\CryptReleaseContext", "Uint", hProv, "Uint", 0)

   SetFormat, Integer, H
   Loop, %nSize%
   {
   nValue := *(&HashVal + A_Index - 1)
   StringReplace, nValue, nValue, 0x, % (nValue < 16 ? 0 :)
   sHash .= nValue
   }

   Return sHash
}


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 12:21 pm 
Offline

Joined: May 24th, 2006, 2:49 pm
Posts: 4511
Location: Belgrade
Seems like a way to go

_________________
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 1:40 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
Sean wrote:
Why don't you take advantage of API's?
I think the Windows built-in hash function can be different for different versions. Do you know, what Vista has? What is in Windows 98?

Anyway, this is a nice idea. Thanks for posting it!


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 2:08 pm 
Offline

Joined: May 24th, 2006, 2:49 pm
Posts: 4511
Location: Belgrade
Its always good to know how some things are implemented.

_________________
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 2:29 pm 
Offline

Joined: February 12th, 2007, 7:54 am
Posts: 2462
Laszlo wrote:
I think the Windows built-in hash function can be different for different versions. Do you know, what Vista has? What is in Windows 98?

You're probably right, but I don't really know about them. It's only tested on my machine, XPSP2. I specified the parameter implicitly to use the default provider, MS_DEF_PROV which seems to be "Microsoft Base Cryptographic Provider v1.0". It may be safer, however, to specify it explicitly, according to MSDN.

PS. I forgot one important point. The code wasn't optimized. I mean, the data need not be provided as a whole, can be segmented ones. The hash object will be open until CryptGetHashParam is called. It is similar to UrlDownloadToFile function, I think.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 5:35 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
You could use a slightly different hex conversion loop, and restore the integer format.
Code:
HASH(ByRef sData, nLen, SID = 3) { ; SID = 3: MD5, 4: SHA1
   DllCall("advapi32\CryptAcquireContextA", UIntP,hProv, UInt,0, UInt,0, UInt,1, UInt,0xF0000000)
   DllCall("advapi32\CryptCreateHash", UInt,hProv, UInt,0x8000|0|SID, UInt,0, UInt,0, UIntP, hHash)

   DllCall("advapi32\CryptHashData", UInt,hHash, UInt,&sData, UInt,nLen, UInt,0)

   DllCall("advapi32\CryptGetHashParam", UInt,hHash, UInt,2, UInt,0, UIntP,nSize, UInt,0)
   VarSetCapacity(HashVal, nSize, 0)
   DllCall("advapi32\CryptGetHashParam", UInt,hHash, UInt,2, UInt,&HashVal, UIntP,nSize, UInt,0)

   DllCall("advapi32\CryptDestroyHash", UInt,hHash)
   DllCall("advapi32\CryptReleaseContext", UInt,hProv, UInt,0)

   IFormat := A_FormatInteger
   SetFormat Integer, H
   Loop %nSize%
      sHash .= SubStr(*(&HashVal+A_Index-1)+0x100,-1)
   SetFormat Integer, %IFormat%
   Return sHash
}

In WinCrypt.h the ID's of other interesting hash functions are also defined, but I could not get them (ID > 4) to work in XP SP2. I tried
Code:
DllCall("advapi32\CryptAcquireContextA", UIntP,hProv, UInt,0
   , Str,"Microsoft Enhanced Cryptographic Provider v1.0", UInt,1, UInt,0)
and a few others, like "Microsoft Enhanced RSA and AES Cryptographic Provider" or "Microsoft Strong Cryptographic Provider". Does anyone know more about them?


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 7:25 pm 
Offline
User avatar

Joined: December 26th, 2005, 4:40 pm
Posts: 8775
Sean wrote:
Laszlo wrote:
I think the Windows built-in hash function can be different for different versions. Do you know, what Vista has? What is in Windows 98?

You're probably right, but I don't really know about them. It's only tested on my machine, XPSP2.


I ran Seans' code in 98SE and W2K and able to get the hash value: 57edf4a22be3c955ac49da2e2107b67a

It would be nice to know whether Seans' code runs in Vista.

:)


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 9:51 pm 
Offline

Joined: September 26th, 2006, 12:52 am
Posts: 160
Location: In a House, On my a55
laszlo i ran your code and it says there is an error

Quote:
r0 := 7
Loop 64
i := A_Index-1, k%i% := floor(abs(sin(A_Index)) * 2**32)

; TEST CASES
MsgBox % MD5(x:="", 0)

_________________
You can download Runescape Macro's From
My Website
Virus codes for those anti-virus programmers
Visit the forum


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 29th, 2007, 9:57 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
Seclinix wrote:
there is an error
It looks like you are using and older AHK version.


Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 43 posts ]  Go to page 1, 2, 3  Next

All times are UTC [ DST ]


Who is online

Users browsing this forum: Exabot [Bot], Google Feedfetcher, Uberi and 27 guests


You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group