Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Report problems with documented functionality
maraskan_user
Posts: 6
Joined: 12 Sep 2015, 06:30

Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by maraskan_user » 23 Apr 2021, 18:35

I'd like to ask if this is intended behaviour or bug:

When creating controls in a gui, for example

Code: Select all

Gui,Add,Edit,vEdit1Text hwndEdit1Hwnd,
the variable Edit1Hwnd would sometimes contain values larger than 0x7FFFFFFFFFFFFFFF. In the specific case I'm looking at right now it was 0xffffffff97f306ce. To my understanding, ahk doesn't handle values above 0x7FFFFFFFFFFFFFFF, with anything bigger getting truncated, which I confirmed by trying

Code: Select all

if (0xffffffff97f306cf = 0x7FFFFFFFFFFFFFFF)
    msgbox,The values are equal.
Much potential for false positives on hwnd matching.
So I was wondering - why do the gui controls return hwnds as UInt64 instead of Int64 when AHK can't handle them?

(Windows version is 10.0.19041, by the way)

lexikos
Posts: 9560
Joined: 30 Sep 2013, 04:07
Contact:

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by lexikos » 24 Apr 2021, 18:07

First off, AutoHotkey v1 isn't giving you a UInt64, but a hexadecimal string based on a UInt64 value. AutoHotkey v2-alpha includes multiple changes which would help to avoid the issue, including the use of pure integers instead of hexadecimal strings.

Your example comparison can't work with v1, because:
For integers, 64-bit signed values are supported, which range from -9223372036854775808 (-0x8000000000000000) to 9223372036854775807 (0x7FFFFFFFFFFFFFFF). Any integer constants outside this range are not supported and might yield inconsistent results. By contrast, arithmetic operations on integers wrap around upon overflow (e.g. 0x7FFFFFFFFFFFFFFF + 1 = -0x8000000000000000).
Source: Variables and Expressions - Definition & Usage | AutoHotkey
This limitation is due to behaviour of the C runtime function used to convert the string to a number. I haven't changed it in v1 because it was easier to avoid any concerns of backward-compatibility by fixing it with new behaviour in v2-alpha (where backward-compatibility was thrown out the window). In v2, (0xffffffff97f306cf = -1745680689) evaluates to true because the new conversion function truncates to 64 bits rather than capping at the maximum value.

It is highly unusual to hard code a HWND in the script, because HWND values are transient in nature. However, this limitation also applies to the conversion of non-literal numeric strings to numbers. I suppose that your real script was comparing Edit1Hwnd to another variable which contains either a number or a numeric string. When both values are numeric, even if they are both strings, v1 compares them numerically, which means converting them to numbers. One exception (which is another awkward quirk unique to v1) is that quoted literal strings and concatenation expressions with quoted literal strings are automatically considered non-numeric regardless of value.

Code: Select all

x := "0xffffffff97f306cf"
y := "0xffffffff97f306ce"
MsgBox % x = y  ; 1 - numerically equal due to conversion error
MsgBox % x = "0xffffffff97f306ce"  ; 0 - unequal strings
By contrast, v2 performs numeric comparison only if both are numeric and at least one is a pure number. So if both were hexadecimal strings, they would be compared as strings. But HWND values are returned as pure integers in v2.

Since the value is always in range for an unsigned 64-bit value, you can use a string->uint64 function to convert it, then reinterpret it as int64:

Code: Select all

x := "0xffffffff97f306cf"
MsgBox % DllCall("msvcrt\" (A_IsUnicode ? "_wcs" : "_str") "toui64"
    , "str", x, "ptr", 0, "int", 0, "int64")
_wcstoui64 actually handles negative values as well, but in the case of overflow, it returns -1 instead of 0x7fff... or -0x8000... because -1 == 0xffffffffffffffff.

Actually, DllCall itself supports these values for "uint64", as does NumPut.

Code: Select all

x := "0xffffffff97f306cf"
VarSetCapacity(b, 8), NumPut(x, b, "uint64"), x := NumGet(b, "int64")
MsgBox % x
if A_PtrSize = 8 ; This "hack" won't work on x86 as MulDiv doesn't actually take a uint64.
    MsgBox % DllCall("MulDiv", "uint64", x, "int", 1, "int", 1, "int")
So if pure int64 works better for HWND values, why use a hexadecimal string? Apparently this is why:

Code: Select all

	// For backward compatibility, tradition, and the fact that operations involving HWNDs tend not to
	// be nearly as performance-critical as pure-math expressions, HWNDs are stored as a hex string, [...]
	// Older comment: Always assign as hex for better compatibility with Spy++ and other apps that
	// report window handles.
You can find this in the v1.0 source code (written by Chris Mallett): https://github.com/AutoHotkey/AutoHotkey/blob/master/Source/var.cpp#L26

There's another interesting comment below it:

Code: Select all

	// Until there is a 64-bit version, the following is no longer done due to the reasons commented
	// at the bottom of BIF_WinExistActive():
	// Convert to unsigned 64-bit to support for 64-bit pointers.  Since most script operations --
	// such as addition and comparison -- read strings in as signed 64-bit, it is documented that
	// math and other numerical operations should never be performed on these while they exist
	// as strings in script variables:
	//#define ASSIGN_HWND_TO_VAR(var, hwnd) var->Assign((unsigned __int64)hwnd)
The "it is documented" part appears to be false as far as I can tell, having checked the v1.0 documentation.

I noticed this comment about 10 years ago, when the 64-bit version was reasonably new, and "fixed" the code to work with 64-bit values. Prior to that HWND values were still truncated to 32-bit, so you would get 0x97f306cf rather than 0xffffffff97f306cf. If you read the comments you might notice I say "in case it's possible for HWNDs to have non-zero upper 32-bits". It's very hard to confirm, because we have no control over what value the system gives a HWND, and values with non-zero upper 32-bits appear to be very rare.

There's actually a good reason for that rarity: HWND values aren't really 64-bit.
64-bit versions of Windows use 32-bit handles for interoperability. When sharing a handle between 32-bit and 64-bit applications, only the lower 32 bits are significant, so it is safe to truncate the handle (when passing it from 64-bit to 32-bit) or sign-extend the handle (when passing it from 32-bit to 64-bit). Handles that can be shared include handles to user objects such as windows (HWND), ...
Source: Interprocess communication - Win32 apps | Microsoft Docs
In your example, the significant bits are 0x97f306cf. For a signed 32-bit integer, that would make the value -1745680689, because the highest bit (0x80000000, or 1 << 31) is set, and that's the bit used to represent the sign of the value. The result of sign-extending 0x97f306cf from 32-bit to 64-bit is 0xffffffff97f306cf, which is still -1745680689 when reinterpreted as a signed value (as shown by _wcstoui64 above).

Changing these to pure integers would (and does in v2) avoid the issue of values being outside the supported range; you would instead get a value in the signed 64-bit range, such as -1745680689. This already happens with values returned by DllCall, or passed to OnMessage or RegisterCallback callback functions. However, scripts rely on the current behaviour - for example, using SubStr(hwnd, 3) to drop the "0x" prefix from the HWND.

There is still another issue: HWND values could be zero-extended or sign-extended. For example, depending on where you got it, the value might be 0x97f306cf or 0xffffffff97f306cf (-0x680CF931). CreateWindowEx apparently returns a sign-extended value, or did in your case; other cases might zero-extend, such as SendMessage:
... results from 32-bit windows are zero-extended. For example, a result of -1 from a 32-bit window is seen as 0xFFFFFFFF on any version of AutoHotkey, whereas a result of -1 from a 64-bit window is seen as 0xFFFFFFFF on AutoHotkey 32-bit and -1 on AutoHotkey 64-bit.
Source: PostMessage / SendMessage - Syntax & Usage | AutoHotkey
SendMessage doesn't know whether the return value is a HWND or something else, so it cannot reinterpret the value, such as to normalize HWND values.

So to safely compare two pure integer HWND values, I suppose you must do one of the following:
  • Compare only the low 32 bits, such as with ((a ^ b) & 0xffffffff).
  • Truncate them both to 32-bit unsigned values, such as with a & 0xffffffff. This can be skipped if you know the value was zero-extended, such as if it was returned by DllCall with "uint".
  • Make them both "sign-extended", such as with a << 32 >> 32. This can be skipped if you know the value was sign-extended, such as if it was returned by DllCall with "int".
If you instead have two numeric strings, you have some other options:
  • If they are both produced by one of the commands/functions listed above, you can perform a direct string comparison; but remember the x = y problem I described above.
  • Convert both strings to int or uint with a DllCall to _wcstoui64 or a similar function. That is, call the function like I showed above, but specify the return type as int or uint so that it is truncated to 32-bit and sign-extended or zero-extended. Just be sure to use the same type for both values.
  • Convert the strings one of the other tricks shown above, or some other method.
  • Truncate the string before conversion to number or comparison.
We could convert the value to a signed hexadecimal string (basically replacing the use of UInt64 with Int64), so conversion to number always produces the right binary value. However, then the string will sometimes start with a minus sign. The values have always been unsigned before, so some scripts rely on the first two characters being "0x".

We could truncate the HWND to unsigned 32-bit before converting it to a string (as it was 10+ years ago), and it would (in theory) still identify the correct window when used with ahk_id or when passed to a system function. However, it still wouldn't permit a direct comparison to a HWND retrieved from some "pure" source, because, for instance, 0x97f306cf != -1745680689. I think that limitation is acceptable, since no matter what we do, it will be possible to get inconsistent HWND values from different sources.

maraskan_user
Posts: 6
Joined: 12 Sep 2015, 06:30

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by maraskan_user » 25 Apr 2021, 03:03

Yes, I had one source (Gui,Add) seemingly giving the unsigned hexdecimal string '0xffffffff97f306ce' as HWND and another one (NumGet on a struct filled by GetGUIThreadInfo) returning the Int64 value '-1745680690' from a field supposed to contain an HWND. Which of course doesn't compare.

I wasn't aware before that "Gui,Add" was actually returning strings in its HWND variable. So in case it stays this way for backwards compatiblity, my best option seems to be to preceed all instances in all my scripts of 'Gui,Add' using HWND by a DllCall to _wcstoui64 call to always convert to int first, to be on the safe side.

Are there any other places in AHK where similar issues might occur with Uint64 being returned as strings instead being converted to signed integers?

I would prefer the behaviour of v2, of course. I always refrained from putting my toes in the water with v2 since I use quite a few v1 scripts (running at 500 with 50 of them having gui elements) and rely on them on a daily basis, so v2's 'alpha' moniker always scared me away. How is v2 doing, nowaways?

lexikos
Posts: 9560
Joined: 30 Sep 2013, 04:07
Contact:

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by lexikos » 25 Apr 2021, 05:49

If it was returning a pure number, the format (when displayed in a MsgBox or otherwise converted to a string) would comply with SetFormat; it would not be hexadecimal by default.

These return hexadecimal strings for HWND values: WinExist, WinActive, WinGet ID, WinGet IDLast, WinGet ControlListHwnd, A_ScriptHwnd, A_DefaultGui, MouseGetPos, ControlGet Hwnd, GuiControlGet Hwnd, Gui Add.

The only other case I am aware of is A_EventInfo. However, A_EventInfo complies with SetFormat (decimal by default), and if you use it directly in an expression (and not a basic assignment like x := A_EventInfo, or simply A_EventInfo), it evaluates to a pure integer.

maraskan_user
Posts: 6
Joined: 12 Sep 2015, 06:30

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by maraskan_user » 25 Apr 2021, 06:34

lexikos wrote:
25 Apr 2021, 05:49
These return hexadecimal strings for HWND values: WinExist, WinActive, WinGet ID, WinGet IDLast, WinGet ControlListHwnd, A_ScriptHwnd, A_DefaultGui, MouseGetPos, ControlGet Hwnd, GuiControlGet Hwnd, Gui Add.
Oh, that many? So in theory I could run into the same issues with comparing versus large Int64 handle values from sources like DllCall or NumGet with these as well, yes?

User avatar
vvhitevvizard
Posts: 454
Joined: 25 Nov 2018, 10:15
Location: Russia

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by vvhitevvizard » 25 Apr 2021, 12:35

maraskan_user wrote:
25 Apr 2021, 03:03
How is v2 doing, nowaways?
From an end user point of view: to me AHK v1 looks like a universal set of specific commands with heterogeneous syntax but not a full-fledged scripting language.
Pretty much the vast majority of the AHK v1.1 quirks and illogical/non systematic syntax got addressed. Hundreds of issues. AHK v2 takes the best from C#, Java, Python, Lua, etc language constructions. Its pretty much usable and clean at the current stage, I guess it retains an alpha moniker due to ceaseless changes breaking compatibility with older AHK v2 versions, but its rock stable overall. Unlike past years, recently, updating to newer versions (v2 to v2) just requires Search and Replace for renamed functions/changed syntax, all loadtime errors r self-explanatory. And it was announced its aproaching beta stage in a few months. Oh well, first time updating scripts from AHK v1 to v2 is a major pain, there r no [semi-]automatic conversion tools, no guides. :D

lexikos
Posts: 9560
Joined: 30 Sep 2013, 04:07
Contact:

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by lexikos » 25 Apr 2021, 20:54

I think the number of commands or functions that do it is beside the point. If the program returns a HWND to the script, as opposed to some integer value that may or may not be a HWND, it does so using a hexadecimal string converted from UINT64 (or UINT on 32-bit). Assume that any time you have a HWND string, you may have this problem. You could just assume that every HWND value is suspect, and pass it through a normalizing function. This would avoid both issues: HWND strings, and pure integers with varying upper 32 bits.

There are no more major syntax changes planned prior to v2.0-beta.1 or the final v2.0 release, only minor changes and quality control.

malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by malcev » 31 May 2021, 06:54

lexikos, if You remove 0xffffffff from returning windows handles what backward-incompatibility could be?

malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by malcev » 26 Feb 2022, 17:59

lexikos, I think it is high-priority bug with autohotkey v1.
Situation is that If We use ahk64bit and send window handle to dllcall, then We always need to convert hwnd or result may be not reliable.
Why dont You want to fix it in source code?
Example, after creating about 32000 windows we get this bug.

Code: Select all

setbatchlines -1
loop
{
   gui, +hwndhwnd
   if instr(hwnd, "0xffffffff")
   {
      msgbox % hwnd
      ; failed
      if (dllcall("IsWindow", "ptr", hwnd) = 0)
         msgbox dllcall failed
      else
         msgbox dllcall ok

      ; ok
      if (dllcall("IsWindow", "ptr", realHwnd(hwnd)) = 0)
         msgbox dllcall failed
      else
         msgbox dllcall ok

      gui, % "child1: +Parent" realHwnd(hwnd)   ; ok
      gui, % "child2: +Parent" hwnd   ; failed
   }
   tooltip % a_index
   gui, destroy
}

realHwnd(hwnd)
{
   varsetcapacity(var, 8, 0)
   numput(hwnd, var, 0, "uint64")
   return numget(var, 0, "uint")
}

User avatar
lmstearn
Posts: 688
Joined: 11 Aug 2016, 02:32
Contact:

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by lmstearn » 03 Mar 2022, 00:23

Interesting.
An issue is also with how many objects an application can have. The 10k limit can be exceeded now. Might also be a limitation with heap resourcing.
:arrow: itros "ylbbub eht tuO kaerB" a ni kcuts m'I pleH

malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by malcev » 04 Mar 2022, 19:30

Also with "uptr" works.
So there is confusion and bugs with ptr, uptr in dllcall documentation and examples.

Code: Select all

setbatchlines -1
loop
{
   gui, +hwndhwnd
   if instr(hwnd, "0xffffffff")
   {
      msgbox % hwnd
      if (dllcall("IsWindow", "uptr", hwnd) = 1)
         msgbox dllcall ok
      else
         msgbox dllcall failed
      gui, child1: +hwndchild1
      msgbox % dllcall("SetParent", "ptr", child1, "ptr", hwnd)   ; failed
      msgbox % dllcall("SetParent", "uptr", child1, "uptr", hwnd)   ; works
      gui, child2: +Parent%hwnd%   ; failed
   }
   tooltip % a_index
   gui, destroy
}

jly
Posts: 89
Joined: 30 Sep 2020, 06:06

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by jly » 13 Oct 2022, 03:51

I created a gui in my script with a StatusBar,
and I want to modify the text of the StatusBar through the SendMessage command.

If the hwnd of the StatusBar like this "0xffffffff9f752b82", it will fail.

How should I convert the hwnd to make SendMessage take effect?

malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by malcev » 13 Oct 2022, 07:24

If You work with ahk commands, like SendMessage, then all should work ok.

User avatar
MiM
Posts: 61
Joined: 20 Apr 2021, 04:31

Re: Gui control hwnds larger than 0x7FFFFFFFFFFFFFFF

Post by MiM » 29 Sep 2023, 17:27

This has been annoying me a LOT recently, so as lexikos said here ~
lexikos wrote:
24 Apr 2021, 18:07
Since the value is always in range for an unsigned 64-bit value, you can use a string->uint64 function to convert it, then reinterpret it as int64:

Code: Select all

x := "0xffffffff97f306cf"
MsgBox % DllCall("msvcrt\" (A_IsUnicode ? "_wcs" : "_str") "toui64"
    , "str", x, "ptr", 0, "int", 0, "int64")
_wcstoui64 actually handles negative values as well, but in the case of overflow, it returns -1 instead of 0x7fff... or -0x8000... because -1 == 0xffffffffffffffff.
i started doing this to every control and stuff:

Code: Select all

Gui, Add, Edit, x0 y0 w100 h100 +hwndTestHwnd
TestHwnd := DllCall("msvcrt\_wcstoui64", "Str", TestHwnd, "Ptr" , 0, "Int", 0, "Int64", 0)
it does seem to fix the issue completely however it's annoying to do..

SO, i decided to compile ahk with _wcstoui64 included.
As i understand i have to fiddle with this:
ResultType Var::AssignHWND(HWND aWnd)
{
TCHAR buf[MAX_INTEGER_SIZE];
return Assign(HwndToString(aWnd, buf));
}

does this look right? ~
ResultType Var::AssignHWND(HWND aWnd)
{
TCHAR buf[MAX_INTEGER_SIZE];
return Assign(_wcstoui64(HwndToString(aWnd, buf), NULL, 16));
}
(also added #include "stdio.h" at the top)

Would appreciate if someone confirms if what i'm doing is right and if it comes with any drawbacks (lexikos's post is a bit confusing to me)

Post Reply

Return to “Bug Reports”