script started with shortcut key shows additional window

[T-Rock]

I created an executable from a small test script, which creates an empty gui:
Gui, Show, w100 h100
return

Then I created in Windows 10 (and also in Windows 7) a shortcut to this executable on the desktop.
In the properties of the shortcut you can define a shortcut key (I used F10).
The first time I hit F10 the test gui starts.
But if I hit F10 again with the script already running, an empty standard window of AutoHotkey with a standard toolbar is started beside my gui (seems to be a child window of the main script).
This doesnt't happen with other executables, so I assume this is related to AutoHotkey.

[lexikos]

This is by design.

When you start the program using that shortcut file, the system automatically assigns the hotkey to the first window that the program creates. In our case, that's the script's main window. When you press the hotkey, the system attempts to find and activate a window that has that hotkey. It is empty because you haven't called ListLines, ListHotkeys, ListVars or KeyHistory, or activated any of the corresponding menu items.

I think that when you press the hotkey, the system actually sends WM_GETHOTKEY to every window, and activates whichever one returns the right hotkey code (virtual keycode combined with flags). If no window returns that hotkey code, the system executes the shortcut file.

The default handling of WM_GETHOTKEY is to return whatever value was set by the system or WM_SETHOTKEY, but the default handling can be overridden. For example, if I run this script, it gets activated whenever I press F10 even though it has nothing to do with the shortcut file that I've assigned that hotkey to:

Code: Select all

ListLines  ; Show the script's main window.
OnMessage(0x33, "OnWmGetHotkey")
OnWmGetHotkey() {
    return 0x79  ; F10
}

If you want to disable the window activation part of the hotkey, there are two ways:

Code: Select all

OnMessage(0x33, "OnWmGetHotkey")
OnWmGetHotkey() {
    return 0  ; Disable activation hotkeys for all windows owned by the script.
}

Code: Select all

; Remove any hotkey the system has assigned to our main window.
DetectHiddenWindows On
SendMessage 0x32, 0,,, ahk_id %A_ScriptHwnd%

If the hotkey is not assigned to any windows, pressing it will attempt to run the script again, which may cause it to reload or display a single-instance prompt. If you want the hotkey (whatever hotkey was assigned by the system) to activate your GUI instead, you can do this (fully functional example):

Code: Select all

Gui, Show, w100 h100
Gui, +hwndhGui
DetectHiddenWindows On
WM_GETHOTKEY := 0x33
WM_SETHOTKEY := 0x32
; Get the system-assigned hotkey (if any) of our main window.
SendMessage WM_GETHOTKEY,,,, ahk_id %A_ScriptHwnd%
if ErrorLevel
{
    ; Assign it to our GUI instead.
    SendMessage WM_SETHOTKEY, ErrorLevel,,, ahk_id %hGui%
    ; Remove the hotkey.
    SendMessage WM_SETHOTKEY, 0,,, ahk_id %A_ScriptHwnd%
}
DetectHiddenWindows Off

Without the script running, I see a long delay between pressing the hotkey and the GUI appearing. I believe this is because WM_GETHOTKEY is sent to every top-level window before the shortcut file is executed, including hidden windows. On my system there are hundreds of top-level windows (mostly hidden).

[T-Rock]

Thank you very much lexikos for taking the time for a detailed analysis and providing solutions, this is very helpful.