Understanding SendInput and keyboard hooks

[Descolada]

Introduction
The default Send mode for AutoHotkey is SendInput. It is usually reliable and fast, but it does have downsides that are quite easily encountered.

TL;DR
If you want to get the benefits of SendInput and have it behave consistently, ensure that no other program (including other AutoHotkey scripts) has a keyboard hook installed: other scripts must not use hotstrings; 1:1 remaps; hotkeys in combination with #HotIf; hotkeys with modifiers ~, * or $; InstallKeybdHook(1); active input hook by the InputHook function; use of SetCapsLockState/SetNumLockState/SetScrollLockState functions. In AutoHotkey v2.1 you can check for the presence of another AHK script with a keyboard hook with A_KeybdHookInstalled > 1.

In all other cases I recommend using SendMode "Event" with a suitable key delay (eg SetKeyDelay(-1, 0)), with or without input buffering to stop keys from being interspersed with user input (for an example see the InputBuffer class here).

Low level keyboard hook
This article is going to mention a low level keyboard hook multiple times, so lets first examine what that is. Windows has a function named SetWindowsHookEx which can be used to install global hooks to capture all kinds of system events. A low level keyboard hook is one of those, and works as follows: any program may call SetWindowsHookEx to register a new keyboard hook with Windows, and along with the function call it provides a callback function. Then, once a keystroke happens (either the user pressed a key physically, or a program sends an artificial keystroke), Windows starts going through its list of registered hooks and starts calling their callback functions one by one (starts processing the "chain of hooks"), starting from the most recently installed one. It calls the callback with information about the key (key code, whether it was pressed/released, whether it was artificial or not, and any extra information that might have been provided with SendInput/keybd_event which are discussed further below) and waits for the result. Depending on the result, one of three things might happen:
1) The callback function can block the keystroke. This means no further hooks will be called, and the keystroke won't be sent in any windows.
2) The callback function can let the keystroke through, and pass it on to the next hook in the hook chain. If all the other hooks also let the key through or there are no other hooks, then the key is sent.
2) The callback function can explicitly let the keystroke through. This means no further hooks will be called and the key will be sent, but no further hooks are called.

Windows monitors the callback function to ensure that it doesn't take too long to process the key event, and might remove the hook if it fails to process it fast enough too many times. The timeout value is specified by LowLevelHooksTimeout in the registry key HKEY_CURRENT_USER\Control Panel\Desktop.
Installing and removing hooks happens very fast, almost instantaneously, as they are implemented at a very low level in Windows (kernel-level?).

Due to how Windows has implemented these hooks, programs don't have access to information about hooks: we can't know how many hooks are installed, where in the hook chain we are located, when a hook was installed, or whether our hook has been removed. We also can't remove other hooks than our own.

How hotstrings and hotkeys are implemented
First lets try to understand how AutoHotkey (AHK) implements its hotkeys and hotstrings.
1) Hotstrings are always implemented via a low level keyboard hook. AHK installs it automatically if any hotstrings are used, and what it does is it captures all keyboard input (both artificial and physical) and sends it to AutoHotkey to be "filtered". AHK could block the key or let it through unchanged, and in the case of hotstrings it always lets the keys through and simply records which keys have been pressed. Once a hotstring match is detected, any #HotIf conditions are checked and depending on the result the trigger will be executed (either auto-replace string, or a function).
2) Some hotkeys are registered using RegisterHotkey. What it does is it registers the hotkey in the Windows system, so when a key is pressed, the system looks for a match against all registered hotkeys and notifies the applications that have registered them. RegisterHotkey supports keys that have a virtual key code, also in combination with modification keys Ctrl, Shift, Alt, and Win. All keys registered and activated via RegisterHotkey will be blocked. Also, any hotkeys registered via RegisterHotkey can be triggered by AHK Send functions, as the system processes the key, not AHK:

Code: Select all

Send "a"
a::MsgBox("ok")

triggers the a hotkey, because the system just sees the a key and notifies AHK, and AHK has no way of knowing that its own Send triggered it.

ALL other hotkeys are implemented through a low level keyboard hook:
* Any hotkey that uses #HotIf. This is because in order check whether #HotIf criteria have been met, AHK needs to capture the key, then process #HotIf, and resultingly either let the key through or block it. RegisterHotkey can't easily and reliably be used to implement this.
* Any hotkey that uses modifier symbols ~, * or $
* Any custom combinations such as a & b as custom modifiers aren't supported by RegisterHotkey
* Any hotkey that doesn't have a virtual key code

Send modes
There are three Send modes: Input (the default one in v2), Event, and Play. For us the relevant ones are Input and Event.
1) SendInput uses win32 SendInput function to send keystrokes. All input is sent in one batch and without delays, meaning SetKeyDelay will not have an effect. The main (or perhaps only) benefit of SendInput is that since the keys are sent in batch, then user input can't be interspersed with it even when sending long text. However, that is true if and only if no keyboard hooks are installed (either by AHK or other applications).
2) SendEvent uses win32 . Only one keystroke is sent at a time (either pressed or released), meaning AHK can apply delays in-between key press and release (press delay) or two keys. These can be changed with [url=https://www.autohotkey.com/docs/v2/lib/SetKeyDelay.htm]SetKeyDelay.

SendInput and hooks
Why is all this important? Well, to get the benefits of SendInput, AHK must remove its own keyboard hook before using it, otherwise the benefits are lost. AutoHotkey can detect keyboard hooks installed by other AHK scripts, and if it determines that it is the only one with a hook then it will uninstall the keyboard hook before a SendInput call, send the keys with SendInput, and then reinstall the hook.
AutoHotkey can't detect keyboard hooks installed by other programs, so if any are present then it will still use SendInput but keys will be sent slower (as the other app must process the keys) and sent keys can get interspersed with keys typed by the user while sending. In that case SendInput is effectively equivalent to SendEvent with SetKeyDelay(-1, -1), with the added downside that the keyboard hook is uninstalled for the duration of the Send.

Usually the keyboard hook is installed and uninstalled so quick that it doesn't have a noticable impact. However, if the user types the keys very rapidly it can sometimes happen that keys that should be hotkeys and be blocked can "leak" through. For example, the following hotkey implements a keyboard hook ($ modifier forces the use of a hook), and if "z" is held down then sometimes "z" leaks through inbetween "n" characters.

Code: Select all

$z:: {
    While GetKeyState("z", "P") {
        Send "{n Down}"
        Sleep 200
        Send "{n Up}"
    }
}

In my computer about every 20th character will be "z"!

This problem can be fixed by using SendMode Event instead of Input and setting the KeyDelay to a low value (eg SetKeyDelay(-1, 0)), as SendEvent doesn't uninstall the hook.

Keys sent with SendEvent will by default NOT be detected by hotkeys registered via a keyboard hook, since AHK "tags" the keys before sending with the current SendLevel, and once the key reaches the keyboard hook the SendLevel is read and used to determine whether to trigger hotkeys/hotstrings. By default if the SendLevel is 0 then hotkeys/hotstrings will not be triggered, and the default SendLevel is 0. This means we can trigger hotkeys with SendEvent by changing the SendLevel to a higher value before sending. SendInput on the other hand can NOT trigger hotstrings nor hotkeys implemented via a keyboard hook for the running script, but it can trigger hotkeys implemented by RegisterHotkey.

SendInput reverting
Because of the downsides of SendInput when a keyboard hook is active, if AHK detects that another AHK script is also using a keyboard hook, it will revert SendInput to SendEvent with SetKeyDelay(-1, 0). This might or might not be noticeably slower than SendInput, and sent keys can get interspersed with user input. If the use of SendInput is still desired then we must be careful to ensure that NO other AHK script installs a keyboard hook (eg uses hotstrings, #HotIf etc). In AHK v2.0 there is no way to check whether another script has a hook active, but in v2.1 we can use the A_KeybdHookInstalled built-in variable which will be >1 if another script has a keyboard hook installed.
If you have two AutoHotkey scripts running and both have a low level keyboard hook installed, then neither script can use SendInput without it reverting to Event!

Conclusion
Hopefully this gives some insight into how SendInput works. Because of all the nuances, my personal opinion is that if SendInput isn't much needed or there are problems with keys "leaking", then SendEvent should be used with a suitable key delay (eg SetKeyDelay(-1, 0)).

Please comment below if I forgot to mention any other conditions which will cause a keyboard hook to be installed.

Epilogue
The following is going to inspect the GetKeyState function, which is unrelated to SendInput but is related to hooks. If you don't already know, GetKeyState returns the logical state of the key (what your computer believes the state of the key is), or the physical state of the key (whether the user is physically holding down the key or not). These states can be different, because for example if AHK sends an artificial keystroke with Send "{a down}" then the "a" key will logically be pressed down, but physically it will not.

The first thing we need to understand is Windows doesn't provide a way to get information about the physical state of keys. This means AHK cannot know the actual physical state of a key (except perhaps with third party libraries such as AutoHotInterception which install a driver to access that information). GetKeyStates "physical" state is a misnomer: it actually tries to interpolate information about keystroke physicality by keeping track of any artificial keystrokes captured by a low level keyboard hook.

If there is no keyboard hook present then GetKeyState uses win32 GetKeyState to query the "logical" state of a key as AHK has read from its input queue, and "physical" state is queried with win32 GetAsyncKeyState which returns the current logical state of a key as reported by Windows. These two should usually be the same.

If there IS a keyboard hook present, then AHK monitors all incoming keystrokes (both physical and artificial) and checks whether they have the LLKHF_INJECTED flag set, which is automatically set for all artificial keystrokes sent by win32 SendInput or keybd_event. If it is not set (a "physical" keystroke) then it updates an internal array of key states with the new "physical" state. Then, once GetKeyState with "P" flag is called, AHK checks what the last recorded key state in that array was and returns it. Problems start if the keyboard hook is intermittently removed, which could lead to "physical" keys being pressed but not recorded by AHK as "physical". Eg if the user starts a script with the "a" key already pressed down then the "logical" state is reported as pressed, but "physical" as not pressed because the down-stroke hasn't been recorded by the low level keyboard hook.

AHK keyboard hook removes the LLKHF_INJECTED flag from a key in some very specific circumstances, but generally leaves it untouched, meaning you can't fake "physical" keystrokes with SendInput/SendEvent. However, if you really wanted to, you could trick the hook into removing the flag by writing a custom SendInput function with a specific flag set. This is undocumented behavior and might change in the future (although unlikely to), so keep that in mind.

Code: Select all

InstallKeybdHook(1, 1)
KeyArray := [{sc: GetKeySC("a"), event: "Down"}]
SendInputEx(KeyArray)
MsgBox "Logical state: " GetKeyState("a") "`nPhysical state: " GetKeyState("a", "P")

SendInputEx(KeyArray) {
   static INPUT_KEYBOARD := 1, KEYEVENTF_KEYUP := 2, KEYEVENTF_SCANCODE := 8, InputSize := 16 + A_PtrSize*3
   INPUTS := Buffer(InputSize * KeyArray.Length, 0)
   offset := 0
   for k, v in KeyArray {
    NumPut("int", INPUT_KEYBOARD, "int", 0, "ushort", 0, "ushort", v.sc & 0xFF, "int", (v.event = "Up" ? KEYEVENTF_KEYUP : 0) | KEYEVENTF_SCANCODE | (v.sc >> 8), "int", 0, "int", 0, "int", 0xFFC3D44E, INPUTS, offset)
    offset += InputSize
   }
   DllCall("SendInput", "UInt", KeyArray.Length, "Ptr", INPUTS, "Int", InputSize)
}

Output:
Logical state: 1
Physical state: 1

Code: Select all

InstallKeybdHook(1, 1)
SendInput("{a down}")
MsgBox "Logical state: " GetKeyState("a") "`nPhysical state: " GetKeyState("a", "P")

Output:
Logical state: 1
Physical state: 0

Version history

09.03.24 added additional situations which install the keyboard hook (thanks niCode)
13.03.24 added further explanation about low level keyboard hooks, and an Epilogue about GetKeyState

[niCode]

Thanks for the write-up! I'm always interested in how some of these things work behind-the-scenes and this definitely pulls back the curtain here.

InputHooks use the keyboard hook as well. And apparently, as does SetCaps/Scroll/NumLock AlwaysOn/AlwaysOff according to the documentation.

The only thing I don't understand is

SendInput on the other hand can NOT trigger hotkeys/hotstring for the running script.

I tested this because I was sure I could do it before, and I still clearly can (using the context of the previous sentence where SendLevel is involved). I'm thinking I just misunderstood something. Could you provide an example?

[Descolada]

@niCode thanks, I added those to the list!

I tested this because I was sure I could do it before, and I still clearly can (using the context of the previous sentence where SendLevel is involved). I'm thinking I just misunderstood something. Could you provide an example?

I've now appended the main post:

SendInput on the other hand can NOT trigger hotstrings nor hotkeys implemented via a keyboard hook for the running script, but it can trigger hotkeys implemented by RegisterHotkey.

Code: Select all

#Requires AutoHotkey v2

;SendMode "Event"
SendLevel 1
Send "abc "

::abc::123

This does not trigger the hotstring (Send = SendInput), but after uncommenting SendMode it does (Send = SendEvent).

Code: Select all

#Requires AutoHotkey v2
SendInput "{F1}"
F1::MsgBox("F1 triggered")

This does trigger the hotkey, as it is implemented via RegisterHotkey.

Code: Select all

#Requires AutoHotkey v2
SendInput "{F1}"
*F1::MsgBox("F1 triggered")

This does not trigger the hotkey.

Code: Select all

#Requires AutoHotkey v2
SendLevel 1
SendEvent "{F1}"
*F1::MsgBox("F1 triggered")

This does trigger the hotkey.

[niCode]

Omg, I feel so dumb. The reason I was getting something different was the whole reason for this post. I had another script open which used the keyboard hook so SendInput wasn't being properly utilized and it was still triggering hotkeys with the keyboard hook.

[eugenesv]

Thanks for the clarification, a couple of things I didn't get:

• "if any [external hooks] are present then it will still use SendInput but keys will be sent slower (as the other app must process the keys)"
  
  Why does this not affect SendEvent, do keyboard hooks ignore keys sent via this method?
• Per docs "SendInput automatically reverts to SendEvent" when it detects other keyboard hooks
  
  So then why does AHK need to un/re-install all the hooks instead of behaving just like though the user used SendEvent instead if it's using SendEvent itself? So then there is no pentaly, just not any benefit

[Descolada]

"if any [external hooks] are present then it will still use SendInput but keys will be sent slower (as the other app must process the keys)"

Why does this not affect SendEvent, do keyboard hooks ignore keys sent via this method?

It does affect SendEvent as well: the more keyboard hooks there are, the longer it takes to process any keystroke. This is why it's a good idea to limit the number of keyboard hooks to as few as possible (ideally just one) - especially AHK keyboard hooks, since AHK is a relatively slow language.

So a few words about low level keyboard hooks... Any program can install a keyboard hook, and they specify a callback function in their code that gets called every time a keyboard event happens. Windows keeps track of the hooks and calls the hooks one by one (starting from the most recently installed hook) until a hook tells it to block the keystroke. AHK uses the hook to check whether the key might activate a hotstring or hotkey, and if a match is found then it also calls the associated #HotIf function (if one exists), and finally returns the result of the hook callback to Windows. If multiple AHK scripts have a hook then the one that most recently installed the hook will get called first, and only after the callback finishes may the second scripts' callback be activated, and so on until the chain of hooks is processed.
Imagine if some of your keys are hooked by multiple AHK scripts and every #HotIf call is badly written to take 100ms: the keys would lag badly. This is why #HotIfs should resolve as fast as possible.

Anyway, if SendInput is used and a keyboard hook is installed by a non-AHK program then in addition to the possibility of keystrokes being interspersed with user input, your script might hang for the duration of the SendInput call. For short sends this probably doesn't matter, but sending 4000 characters might take a while. This is why I think the decision was made for it to revert to SendEvent if detectable hooks are present, as SendEvent sends keys one-by-one and I believe AHK checks it's message queue in-between the calls, so the script shouldn't totally freeze for the duration of the call. I haven't verified this theory though, so take it with a grain of salt

Per docs "SendInput automatically reverts to SendEvent" when it detects other keyboard hooks

So then why does AHK need to un/re-install all the hooks instead of behaving just like though the user used SendEvent instead if it's using SendEvent itself? So then there is no pentaly, just not any benefit

I'm not sure I understood the question. If SendInput reverts, then it also doesn't un/reinstall the keyboard hook, it just uses SendEvent with SetKeyDelay(-1, 0).

[eugenesv]

Thanks, the second question stemmed from my misunderstanding of the comparison between Event and Input "SendInput is effectively equivalent to SendEvent ... with the added downside that the keyboard hook is uninstalled" is only applicable when AHK can't detect someone else's hook, so it uninstalls its own to get the benefit, but can't get it. But if it detects a second hook from itself, then you don't get the downside since it simply reverts without the uninstall


P.S.

This reminder of Input reversion helped me find the cause for the unreliable getkeystate viewtopic.php?f=82&t=127078&p=562734#p562734

So glad I've accidentally bumped into your post, thanks!

[eugenesv]

By the way, it seems that hotkey() functions with a call back also install keyboard hook even for regular keys with virtual codes and without any extra ~ * etc.?

[Descolada]

@eugenesv I am unable to reproduce that behavior:

Code: Select all

#Requires AutoHotkey v2

Hotkey("F1", MyCallback)
Hotkey("$F2", MyCallback)
SendInput "{F1}{F2}"
Persistent()

MyCallback(ThisHotkey) => MsgBox(ThisHotkey " pressed")

Shows only "F1 pressed". This is expected if F1 is impemented with RegisterHotkey and F2 with a keyboard hook.

Code: Select all

#Requires AutoHotkey v2

Hotkey("F1", MyCallback)
Hotkey("$F2", MyCallback)
SendLevel 1
SendEvent "{F1}{F2}"
Persistent()

MyCallback(ThisHotkey) => MsgBox(ThisHotkey " pressed")

Shows "$F2 pressed", then "F1 pressed".

[eugenesv]

Ah, I see, it must've been inputlevel that triggered a hook, not the callback

Code: Select all

!f1::Reload()
Hotkey('vk45'   , hkModTap,'I1') ; installs a hook
Hotkey('vk45'   , hkModTap) ; does NOT install a hook
hkModTap(hk_dirty) {
}

Hotkey("F1", MyCallback)
MyCallback(ThisHotkey) => MsgBox(ThisHotkey " pressed " A_KeybdHookInstalled)

[eugenesv]

By the way, is it possible to block SendInput reverting (when two AHK scripts use hooks)? This caused a hard-to-track bug in my scripts recently, so I'm wondering whether it'd be possible to avoid such silent conversions so that an explicit SendInput always sends Input even though it might not get the no-interrupt benefit anymore. Would be less confusing if only Send were ambiguous and managed by AHK, but if an explicit method were given, that method should be used

[Descolada]

@eugenesv there is no way to block the reverting. In v2.1-alpha you can use A_KeybdHookInstalled to check whether an external hook exists, which would mean SendInput will revert.

Would be less confusing if only Send were ambiguous and managed by AHK, but if an explicit method were given, that method should be used

Perhaps it would be less confusing, but it would have other complications. In an alternate universe SendInput would throw an error instead of reverting, and Send would revert from SendInput to SendEvent in case it throws (and also return the used send mode as a return value), but would users like that kind of approach? I'm not sure.

[eugenesv]

In an alternate universe SendInput would throw an error instead of reverting

Why not just send input as usual just like it does when another app has a hook that Ahk cannot detect?

[Descolada]

@eugenesv it is an option, but has the same issue that you won't know whether user keystrokes can become interspersed with it or not. Throwing an error could indicate that, or perhaps an optional output parameter, or SendInput returning a value, or whatever. But having this discussion here is rather pointless: it belongs in the Wish List section (maybe for AHK v3?)

[Jasonosaj]

Almost certainly a remedial question - if ScriptA and ScriptB both use keyboard hooks, would any potential conflicts be resolved by consolidating them both in ScriptC via #Include?

[Descolada]

@Jasonosaj if you can consolidate two hook-using scripts into one without breaking them, then I definitely recommend doing that. The fewer AHK keyboard hooks there are the better (performance-wise), and if you need/want to use SendInput and there are only two scripts with hooks then it's the only way to get the benefits of SendInput.