DllCall() - Help required with ExtractInteger()

AutoHotkey Community
Let's help each other out

FAQ

Memberlist

Profile

DllCall() - Help required with ExtractInteger()

AutoHotkey Community Forum Index -> Ask for Help

View previous topic :: View next topic

Author

Message

SKAN

Joined: 26 Dec 2005
Posts: 6223

Posted: Thu Jul 20, 2006 8:56 pm Post subject: DllCall() - Help required with ExtractInteger()

Dear Friends, Smile

The following is a part of my function which gives successful results!

Code:

Loop, %xtimes% {
Integer:=ExtractInteger(Variable, A_Index-1 ,False ,1 )
If Integer >= %SomeNumber%

But this does not work!

Code:

Loop, %xtimes% {
Integer = *(&Variable + A_Index-1)<< 8*(A_Index-1)
If Integer >= %SomeNumber%

I just want to loop a single byte integer extraction and so I want to import
ExtractInteger() 's functionality into my function.

Here is ExtractInteger() I am calling in the working code (comments stripped-off) :

Code:

ExtractInteger(ByRef pSource, pOffset=0, pIsSigned=false, pSize=4) {
Loop %pSize%
result += *(&pSource + pOffset + A_Index-1) << 8*(A_Index-1)
if (!pIsSigned OR pSize > 4 OR result < 0x80000000)
return result
return -(0xFFFFFFFF-result+1)
}

Here is ExtractInteger() given in DllCall() page of AHK Documentation:

Code:

ExtractInteger(ByRef pSource, pOffset = 0, pIsSigned = false, pSize = 4)
; pSource is a string (buffer) whose memory area contains a raw/binary integer at pOffset.
; The caller should pass true for pSigned to interpret the result as signed vs. unsigned.
; pSize is the size of PSource's integer in bytes (e.g. 4 bytes for a DWORD or Int).
; pSource must be ByRef to avoid corruption during the formal-to-actual copying process
; (since pSource might contain valid data beyond its first binary zero).
{
Loop %pSize% ; Build the integer by adding up its bytes.
result += *(&pSource + pOffset + A_Index-1) << 8*(A_Index-1)
if (!pIsSigned OR pSize > 4 OR result < 0x80000000)
return result ; Signed vs. unsigned doesn't matter in these cases.
; Otherwise, convert the value (now known to be 32-bit) to its signed counterpart:
return -(0xFFFFFFFF - result + 1)
}

I am completely lost and do not understand what ExtractInteger() is about.
I will be grateful if somebody can help me with this. Rolling Eyes

Regards,

_________________

corrupt

Joined: 29 Dec 2004
Posts: 2421

Posted: Fri Jul 21, 2006 12:06 am Post subject:

** Untested **

Code:

Loop, %xtimes% {
Integer := *(&Variable+Offset+A_Index-1) << 8*A_Index-1
If Integer >= %SomeNumber%

PhiLho

Joined: 27 Dec 2005
Posts: 6721
Location: France (near Paris)

Posted: Fri Jul 21, 2006 8:22 am Post subject: Re: DllCall() - Help required with ExtractInteger()

Goyyah wrote:

I just want to loop a single byte integer extraction

If that's so, no need for complex byte shifting...

Code:

p := &variable
Loop %xtimes%
{
integer := *(p + A_Index - 1)
If (integer >= someNumber)
; [...]
}

* returns one byte at a time.
_________________

vPhiLho := RegExReplace("Philippe Lhoste", "^(\w{3})\w*\s+\b(\w{3})\w*$", "$1$2")

ParanoidX

Joined: 16 Dec 2005
Posts: 149
Location: Australia

Posted: Fri Jul 21, 2006 9:14 am Post subject:

Goyyah wrote:

I just want to loop a single byte integer extraction and so I want to import
ExtractInteger() 's functionality into my function.

What integer source are you attempting to extract from? If its simply from within AHK, then use "PhiLho" method (I just saw it popped up).

Goyyah wrote:

I am completely lost and do not understand what ExtractInteger() is about.

The ExtractInteger function is created mainly for structure/array based on Little Endian method(how Intel CPU stores actual data in memory) data extraction from memory.

So say if we have the number = 0x987654321(hex) in a variable, the data is stored like: 21 43 65 87 09 in actual memory.
Therefore to extract the data from memory we reverse it back to get the original value 0x09 87 65 43 21.
The core of the function is this part:

Code:

result += *(&pSource + pOffset + A_Index-1) << 8*(A_Index-1)

it simply keep appending the byte(by left shifting it 8bit/1byte at a time) read from pOffset pointing to pSource to the left hand side accumulating to the "Result" var.
e.g.
--edited--

Code:

MyBuffer := 987654321 ;
ExVar := ExtractInteger(MyBuffer,0,false,4)
return
>>----append the ExtractInteger funtion here----<<

Loop1: Result = 57 = 0x39(hex) = "9"
Loop2: Result = 14393 = 0x3839(hex) = "89"
Loop3: Result = 3618873 = 0x373839(hex) = "789"
Loop4: Result = 909588537 = 0x36373839(hex) = "6789"

This obviously was not the intention of the function, but the example demonstrate clearly what exactly is happening to the data.

I hope this helps Wink

_________________

546F206C69766520
6973204368726973742C0D746F2064696520
6973206761696E2E0D285068696C20313A323129

Last edited by ParanoidX on Sat Jul 22, 2006 6:44 am; edited 1 time in total

SKAN

Joined: 26 Dec 2005
Posts: 6223

Posted: Fri Jul 21, 2006 4:05 pm Post subject:

corrupt wrote:

** Untested **

Code:

Loop, %xtimes% {
Integer := *(&Variable+Offset+A_Index-1) << 8*A_Index-1
If Integer >= %SomeNumber%

@Corrupt: That did not work .. but Thanks for trying to help me .. Smile

PhiLho wrote:

... no need for complex byte shifting...

Code:

p := &variable
Loop %xtimes%
{
integer := *(p + A_Index - 1)
If (integer >= someNumber)
; [...]
}

* returns one byte at a time.

@PhiLho: That does exactly what ExtractInteger() did .. Thanks a lot .. Very Happy

.. you saved me from many hours of confusion.
I also Thank you for your POST in Ask for Help topic: DllCall. Help, anyone?

ParanoidX wrote:

I hope this helps Wink

@ParanoidX: Great help! & many Thanks for the explanation! Very Happy

. I have book marked your post ..
I'm a bit dumb & have to read this explanation many times.
I will try a test code and check 0x987654321 <--> 21 43 65 87 09 to understand this better..

I thank in advance for your future replies to my doubts. Wink

Regards to all, Very Happy

_________________

ParanoidX

Joined: 16 Dec 2005
Posts: 149
Location: Australia

Posted: Sat Jul 22, 2006 6:56 am Post subject:

Quote:

I'm a bit dumb & have to read this explanation many times.

Not true! everyone who first encounter this has problems understanding as it is low level manipulation technique. And for those who is use to programming high level such as AHK it is not a easy concept to take in given that we are use to everything is done for us by the language.

When we say bit shift we are dealing with a totally diff number system called binary (base 2) our normal number system is base 10 in english basically its saying given any 1 char the highest number it can represent is 2 for base 2 hence 0 and 1. so the count goes like 0,1,10,11,100,101 <--notice something about the shifting?

and in normal system, since it is base 10. the highest number each char can represent is 10 i.e. 0,1,2,3,4,5,6,7,8,9 so it goes like 0..9,10,11,.....19,20,21,22.....29,30,31<--notice something about the shifting when the char representation maxes?

In hexadecimal(meaning base 16) each char can have a max of 16, so 0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F so 0..F,10,11..,1E,1F,20,21..2E,2F,30 <- again notice something about the shifting?

Basically any Base-X(base anything) number after it reaches the max i.e "X" it is shifted "Left".
so in decimal

Code:

\/now all number has the 1 carried
1,2,3,4,5,6,7,8,9,10,11....repeats until 99, so 2 char max hence carry to the next 100
^Max, carry forward(shift left)

There are many other Base-x variation system used esp. in encryption.
Going back to the topic of bit shifting, so what happens when we do a single bit shift to the left?

Code:

MyNum = 1
Loop 8
{
MyBitShift%A_Index% := MyNum << A_index
Disp := Disp A_Index ": " MyBitShift%A_Index% "`n"
}
msgbox % "Bitshift of 1`n`n" Disp

From the result we can learn that every single bit shift left, we see the number doubles.(p.s. this is how in we do quick multiplication)

result wrote:

Bitshift of 1

1: 2

2: 4

3: 8

4: 16

5: 32

6: 64

7: 128

8: 256

Here's a more obvious example based on the function:

Code:

SetFormat, integer, hex
x = 0x40
y = 0xFF
z = 0xCC
resultx := x << 8 ; left shift x by 8bits
resulty := resultx + y ; add the y to the left shifted x
resultz := (resulty << 8) + z ; left shift resulty by 8bits and add z
msgbox % "X = " x " | Y = " y " | Z = " z "`n`n X << 8`t`t: " resultx "`n+ Y`t`t: " resulty "`nResultY << 8 + Z`t: " resultz

Please don't hesitate to ask question as my explanation is not the clearest.
_________________

546F206C69766520
6973204368726973742C0D746F2064696520
6973206761696E2E0D285068696C20313A323129

SKAN

Joined: 26 Dec 2005
Posts: 6223

Posted: Sat Jul 22, 2006 9:39 am Post subject:

ParanoidX wrote:

.. everyone who first encounter this has problems understanding as it is low level manipulation technique. And for those who is use to programming high level such as AHK it is not a easy concept to take in given that we are use to everything is done for us by the language

Very true ... Your posts in the topic will be a boon for those aspiring to understand these concepts.

ParanoidX wrote:

Please don't hesitate to ask question

I have been reading your post over and over for the past 2 hours .. Sad

I think - I can learn better with real time examples

Re: Bit Shift

The following is one of my most used function (that does .. er.. right Bit-Shift ? )

Code:

; Written by PhiLho and posted @
; http://www.autohotkey.com/forum/viewtopic.php?p=58173#58173

Hex2RGB(_hexRGB, _delimiter="")
{
local color, r, g, b, decimalRGB

If _delimiter =
_delimiter = ,
color += "0x" . _hexRGB
b := color & 0xFF
g := (color & 0xFF00) >> 8
r := (color & 0xFF0000) >> 16
decimalRGB := r _delimiter g _delimiter b
Return decimalRGB
}

Hex2RGB("AC80CD"," ") returns "172 128 205"
Can you explain in plain english how "172 128 205" is derived from "AC80CD"

Please take your own time to reply. I am still trying to get a picture from your previous posts .. Very Happy

Many Regards, Smile

_________________

Titan

Joined: 11 Aug 2004
Posts: 5376
Location: /b/

Posted: Sat Jul 22, 2006 10:24 am Post subject:

Goyyah wrote:

Can you explain in plain english how "172 128 205" is derived from "AC80CD"

I don't know why PhiLho zero padded the bit mask and increased the shift proportionality because it's the same as:

Code:

Setformat, integer, h
MsgBox, % 172 & 0xff "`n" 128 & 0xff "`n" 205 & 0xff

Take a look at bit shift and bitwise and on the wikipedia.

Edit: I didn't look at the function, he does make sense.
_________________

Last edited by Titan on Sat Jul 22, 2006 10:27 am; edited 1 time in total

SKAN

Joined: 26 Dec 2005
Posts: 6223

Posted: Sat Jul 22, 2006 10:27 am Post subject:

Titan wrote:

Take a look at bit shift and bitwise and on the wikipedia.

Thanks for these links .. Regards, Smile

_________________

ParanoidX

Joined: 16 Dec 2005
Posts: 149
Location: Australia

Posted: Wed Jul 26, 2006 8:02 am Post subject:

Goyyah wrote:

The following is one of my most used function (that does .. er.. right Bit-Shift ? )

Okay before we go into this, basically the example is to demonstrate the difference in numbering systems and what Base-X (base anything) means. Since it is a numbering system, therefore it adds 1 everytime and left shifts everytime the Base is reached.

Quote:

Hex2RGB("AC80CD"," ") returns "172 128 205"
Can you explain in plain english how "172 128 205" is derived from "AC80CD"

This typical of how hex is pratically used to represent color codes. To write it properly, it should have a comma in between. So 172,128,205 = AC,80,CD because it is not AC80CD (11305165) hex but rather the numbers are 3 parts each with a maximum of 0xFF = 255.

Quote:

Hex Decimal
AC = 172 = Level of Red Component
80 = 128 = Level of Green Compenent
CD = 205 = Level of Blue Component

if in doubt try it on a calc or ahk script. So it simply using a different numbering systems i.e. base16 to represent the color code. The advantage is you need less chars to represent the same number in decimal. So to get white where R,B,G = 255,255,255 = FF,FF,FF hence only 6 chars(hex) saying the same thing as 9 chars(dec) so less typing/more readily accessible by the machine(binary and hex is native to computers).

Its like me saying to you what is C4 - 60 (hex)? you would have to grab a calc to get the result unless you remember hex readily. But on the other hand if I asked, what is 196 - 100? normally you can easily do that on the top of your head because it is a symbol we have used so often we recall it off by heart, and yes, I used the word "symbol". In the English world we use Arabic numbers to represent decimal i.e. 0,1,2,3,4,.. but Romans(ancient English)/Chinese and other culture would have a totally different symbolic system representing the same decimal numbers.

According to the Hex2RGB(_hexRGB, _delimiter="") function:

Code:

b := color & 0xFF
g := (color & 0xFF00) >> 8
r := (color & 0xFF0000) >> 16

There is masking(& 0xFF) technique and right shift(>> 8/16). I explain the simplist one first, i.e right shift. There are 2 variation of right shift according to asm or intel cpu i.e with carry and no carry. AHK only allows the latter variation.

Goyyah wrote:

I think - I can learn better with real time examples

I will demonstrate this in a script.

Given the hex number AC80CD. How do we split it out into the 3 parts i.e. AC,80,CD then to its decimal equivalent?
Obviously with AHK we can simply do a Stringmid, then convert it back to decimal using SetFormat function. But PhilHo is using a lower level technique using mask and right shift out which is suppose to be faster. Here is a little benchmark:

Code:

; Purpose: a small benchmark on string/bit shift manipulation

; make sure the format is RRGGBB i.e fixated 6 chars
; so 1,2,3 = 010203
xValue = AC80CD
nNumLoops = 100000 ; the higher the more accurate the result

;-------Mask and Right Bitshift method
nStart1 := A_TickCount
Loop %nNumLoops%
Hex2RGB(xValue)
nStop1 := A_TickCount - nStart1

;-------Stringmid Method
nStart2 := A_TickCount
Loop %nNumLoops%
Hex2RGB2(xValue)
nStop2 := A_TickCount - nStart2

msgbox % "Bitshift Method: " nStop1 "ms`nStringmid Method: " nStop2 "ms`nLooping " nNumLoops " times"
return

Hex2RGB(_hexRGB)
{
local c1,c2,c3, decimalRGB

color += "0x" . _hexRGB
b := color & 0xFF
g := (color & 0xFF00) >> 8
r := (color & 0xFF0000) >> 16
decimalRGB := r "," g "," b
Return decimalRGB
}

Hex2RGB2(_hexRGB)
{
local c1,c2,c3, decimalRGB

StringMid, c1, _hexRGB, 1, 2
StringMid, c2, _hexRGB, 3, 2
StringMid, c3, _hexRGB, 5, 2

Loop 3
c%A_Index% += "0x" . c%A_Index%

decimalRGB := c1 "," c2 "," c3
return decimalRGB
}

To my surprise there was barely a difference which goes to show the inefficiency of scripting languages. I pulled out the delimeter processing section from PhiLho's function to minimize the deviation in the bench result since we are trying to see which function is faster in AHK.

Right Bitshift and Masking Technique to split Hex value
====== Right Bitshift ======
All it does is that it shifts whatever bits to the right.
To illustrate this, lets say we have a "Right Decimal Shift" (you can easily use Stringleft to achieve this).

Let Num = 987654321, if I right shift decimal by 1 using the function Right Decimal Shift(RDS),
RDS (Num, 1) = 98765432.
RDS (Num, 2) = 9876543;
RDS (Num, 3) = 987654 etc..

Code:

; Right Decimal Shift
MyNum = 987654321
msgbox % rds(MyNum, 2)
return

rds(szStr, nNum)
{
nLen := strlen(szStr)
StringLeft, result, szStr, nLen - nNum
return result
}

So it works more or less like truncating the end.

In similar ways, right bit shift does this:

Code:

SetFormat, integer, hex

xNum := 0xAC80CD ;original value
xNum8B := xNum >> 8 ;shift the xNum 8bits right
xNum8B += 0
xNum16B := xNum >> 16 ;shift the xNum 16bits right
xNum16B += 0
msgbox % "Original Value (hex):`t" xNum "`nShift right 8 Bit:`t`t" xNum8B "`nShift Right 16 Bit:`t`t" xNum16B

You might be wondering, why right shift 8bit and 16bit? the answer is simple because:
8bit = 1 byte = (hex XX) = truncates 2 char from hex value
16bit = 2 byte = (Hex XXXX) = truncates 4 chars from hex value.
Result:

Code:

Original Value (hex): 0xAC80CD : we need it to be ONLY "CD"
Shift right 8 Bit: 0xAC80 ; we need it to be ONLY "80"
Shift Right 16 Bit: 0XAC ; This part we can use directly

From this result, we obviously can't use the numbers as is. Remember our purpose is to get the value input value (xNum) and split to 3 parts (R,G,B) decimal. This is where "Bit Masking" comes in.

====== BitMasking ======
According to the funtion,

Code:

xNum & 0xFF ; where xNum = HTML color code i.e. like FFFFFF

The masking is Logical AND 0xFF.

Lets see what this actually does.

Code:

SetFormat, integer, hex
xNum := 0xAC80CD

xNumMask1 := xNum & 0xFF
xNumMask2 := xNum & 0xFF00
xNumMask3 := xNum & 0xFF0000

msgbox % "xNum: " xNum "`n`nLogic & 0xFF:`t`t" xNumMask1 "`nLogic & 0xFF00:`t`t" xNumMask2 "`nLogic & 0xFF0000:`t`t" xNumMask3

Result:

Code:

xNum: 0xAC80CD

Logic & 0xFF: 0xcd
Logic & 0xFF00: 0x8000
Logic & 0xFF0000: 0xac0000

To write what it actually doing in a clearer fashion:

Code:

0xAC80CD & (Logical AND)
0x0000FF
--------
0x0000cd

Code:

0xAC80CD & (Logical AND)
0x00FF00
--------
0x008000

Code:

0xAC80CD & (Logical AND)
0xFF0000
--------
0xac0000

Can you see what is happening? Look carefully at the position.
Where ever I Logic AND using 0xF, the value is preserved.
To understand further about the logical AND use Titans like http://en.wikipedia.org/wiki/Bitwise#AND
It basically functions like a filter (masking).
Guess the result:

Code:

0xAC80CD & (Logical AND)
0x0FF0FF

Try not to cheat(so no nothing except brain and eye(s)), if you understood the above, you can easily do this on top of your head.
====== BitMasking FAQ Ends======
Putting the 2 components together, Bitmask and Right bitshift, the function Hex2RGB(_hexRGB) is born. Another point in regards to logic AND is that the order is not important, so 0xAC80CD & 0xFF0000 is the same the other way around.

In PhiLho Hex2RGB(_hexRGB) function, in the red component calc, the logic AND can be removed providing you check if its <FFFFFF (max 24bit):

Code:

r := color >> 16

since it is already the most signicant number i.e. AC another way of saying it, there is nothing else on the left of AC so we don't need to mask/filter anymore. But PhiLho wants to be safe in case some people bring in AABBCCDD = 32bit. In which it will still retain the value BB,CC,DD ignoring AA. But if that happens, I would consider:

Code:

if _hexRGB > 0xFFFFFF
msgbox error only 0xFFFFFF max is allowed.

Since I don't know if the programmer accidentally inputs the value or don't understand the function rather than continue to calculate ignoring AA.

In the benchmarking, on the Stringmid Method, if you remove the Loop, so:

Code:

Hex2RGB2(_hexRGB)
{
local c1,c2,c3, decimalRGB

StringMid, c1, _hexRGB, 1, 2
StringMid, c2, _hexRGB, 3, 2
StringMid, c3, _hexRGB, 5, 2

c1 += "0x" . c1
c2 += "0x" . c2
c3 += "0x" . c3

decimalRGB := c1 "," c2 "," c3
return decimalRGB
}

It actually becomes significantly faster than Bitshift method.
Congratulations!! Welcome to your computer's native language, if you understood these methods of manipulation. This is how computer functions, using bits/hex and logics to do EVERYTHING!!
So you can imagine how fast they must be to do all these processing real time. From sound, graphics, calculations, reponding to inputs(mouse, k/b), displaying colors on your 24bit color monitors refreshing average of 60hz+ etc.
Computers = Truely amazing advance calculator.
_________________

546F206C69766520
6973204368726973742C0D746F2064696520
6973206761696E2E0D285068696C20313A323129

SKAN

Joined: 26 Dec 2005
Posts: 6223

Posted: Wed Jul 26, 2006 10:01 am Post subject:

Dear ParanoidX, Thank you very much! Very Impressive Illustration! After reading this again & again (one hour) , I am starting to understand the concept. This is more than sufficient. I will post again with doubts, if any! I thank you again for your kindness. Regards, _________________

PhiLho

Joined: 27 Dec 2005
Posts: 6721
Location: France (near Paris)

Posted: Wed Jul 26, 2006 10:36 am Post subject:

ParanoidX wrote:

Code:

Hex2RGB(_hexRGB)
{
local c1,c2,c3, decimalRGB

color += "0x" . _hexRGB
b := color & 0xFF
g := (color & 0xFF00) >> 8
r := (color & 0xFF0000) >> 16
decimalRGB := r "," g "," b
Return decimalRGB
}

A minor problem (because it is irrelevant to your demonstration: that's local color, r, g, b, decimalRGB in the first line... Smile

)

There are several ways to skin a cat, and you demonstrated it brillantly. For some reason, I used here the classical C mindset, instead of a more creative AutoHotkey one... I suppose it seemed more "natural" at the time... And I didn't searched performance then.

Notes on other ways to skin a feline:

The decimalRGB variable isn't necessary, I could have just written Return r "," g "," b (or using the given delimiter).
On the other hand, I could have written decimalRGB = %r%,%g%,%b% and gain some time on your benchmark. This form of string concatenation is slightly faster than by using expressions.
Another way to do the binary stuff is to apply the shift first, then to mask with 0xFF systematically (g := (color >> 8 ) & 0xFF // r := (color >> 16) & 0xFF). I like the above way because it shows graphically where the bits are taken.
The advantage on StringMid might be because the binary expressions must be evaluated by an expression parser, which might be slow.
I must admit I haven't read the whole article, but I can explain the difference between the first and second version of Hex2RGB2: the loop evaluation is time consuming, and probably also is the variable dereferencing (c%A_Index%).
Note an old trick, useful in the times of 1MHz processors, was to create a line of assembly code by screen line, instead of looping on them: this way, the code avoided the loop management (increment, test). It made big but fast code (an old compromise... there wasn't much lines at the time, too).

Very nice and useful article!
_________________

vPhiLho := RegExReplace("Philippe Lhoste", "^(\w{3})\w*\s+\b(\w{3})\w*$", "$1$2")

Display posts from previous:

	AutoHotkey Community Forum Index -> Ask for Help	All times are GMT
Page 1 of 1

You can post new topics in this forum
You can reply to topics in this forum