Creating COM Object Event Handler

Get help with using AutoHotkey and its commands and hotkeys
cpriest
Posts: 20
Joined: 17 Sep 2017, 08:06

Creating COM Object Event Handler

17 Sep 2017, 08:24

I apologize if I have missed something I've read over many hours on COM Objects and AutoHotkey.

I'm not new to Autohotkey, but my head is spinning every direction over how to do what I would like to do.

I need to create an AHK class which implements the IUIAutomationEventHandler Interface:
https://msdn.microsoft.com/en-us/librar ... s.85).aspx

I've read on these pages:
* About ComDispatch0 by @lexikos and I've also seen https://autohotkey.com/boards/viewtopic.php?f=13&t=4558:
So I experimented with adding native support for passing AutoHotkey objects to/from COM APIs. The cost is about 1-2KB of added size to AutoHotkey.exe (32-bit), with the following features...
* ActiveScript for AutoHotkey v1.1 https://autohotkey.com/boards/viewtopic.php?f=6&t=4555
ComObject := ComDispatch0(Object) - not needed in AutoHotkey v1.1.17+
My initial foray into writing an object for the event handler (just to see what was being called/done):

Code: Select all

; NOTE - AHKDebug() is just a wrapper for OutputDebug % Format(...)
#Include <uia>
; global $u := new IUIAutomation
; global $e := new IUIAutomationElement
; global $c := new IUIAutomationCondition
; global $r := new IUIAutomationCacheRequest
; global $t := new IUIAutomationTreeWalker

; UIA_Window_WindowOpenedEventId	20016



UIA := new IUIAutomation

DesktopElem := new IUIAutomationElement(UIA.GetRootElement())

class UIATest {
	__Get(name) {
		AHKDebug("__Get({})", name)
	}
	__Set(name) {
		AHKDebug("__Set({})", name)
	}
	__Call(name) {
		AHKDebug("__Call({})", name)
	}
}

test := new UIATest()

AHKDebug("About to AddAutomationEventHandler")

UIA.AddAutomationEventHandler( evt := UIA_Event("Window_WindowOpened")
	, DesktopElem
	, scope := UIA_Enum("TreeScope_Children")
	, 0
	, &test )

AHKDebug("After AddAutomationEventHandler")
Trying either just test or &test both are giving me back a "Pointer not valid error."

I'm sure this is all because my head is spinning. I'm more than happy to be pointed to similar code or a general "this is what your missing and here's some other code that does it correctly."

Thanks!
qwerty12
Posts: 468
Joined: 04 Mar 2016, 04:33
GitHub: qwerty12

Re: Creating COM Object Event Handler

17 Sep 2017, 13:04

https://msdn.microsoft.com/en-us/library/windows/desktop/ee696044(v=vs.85).aspx wrote:The IUIAutomationEventHandler interface inherits from the IUnknown interface.
No mention of IDispatch there. ;)

You need to implement the interface yourself. Here's my (definitely was buggy) attempt:

Code: Select all

#NoEnv
#Include %A_ScriptDir%\UIA.ahk
#Persistent

IUIAutomationEventHandler_new()
{
	vtbl := IUIAutomationEventHandler_Vtbl()
	evt := DllCall("GlobalAlloc", "UInt", 0x0000, "Ptr", A_PtrSize + 4, "Ptr")
	if (!evt)
		return 0

	NumPut(vtbl, evt+0,, "Ptr")
	NumPut(1, evt+0, A_PtrSize, "UInt") ; thanks, just me

	return evt
}

IUIAutomationEventHandler_Vtbl(ByRef vtblSize := 0)
{
	static vtable
	if (!VarSetCapacity(vtable)) {
		extfuncs := ["QueryInterface", "AddRef", "Release", "HandleAutomationEvent"]

		VarSetCapacity(vtable, extfuncs.Length() * A_PtrSize)

		for i, name in extfuncs
			NumPut(RegisterCallback("IUIAutomationEventHandler_" . name), vtable, (i-1) * A_PtrSize)
	}
	if (IsByRef(vtblSize))
		vtblSize := VarSetCapacity(vtable)
	return &vtable
}

IUIAutomationEventHandler_QueryInterface(this_, riid, ppvObject)
{
	static IID_IUnknown, IID_IUIAutomationEventHandler
	if (!VarSetCapacity(IID_IUnknown))
		VarSetCapacity(IID_IUnknown, 16), VarSetCapacity(IID_IUIAutomationEventHandler, 16)
		,DllCall("ole32\CLSIDFromString", "WStr", "{00000000-0000-0000-C000-000000000046}", "Ptr", &IID_IUnknown)
		,DllCall("ole32\CLSIDFromString", "WStr", "{146c3c17-f12e-4e22-8c27-f894b9b79c69}", "Ptr", &IID_IUIAutomationEventHandler)

	if (DllCall("ole32\IsEqualGUID", "Ptr", riid, "Ptr", &IID_IUIAutomationEventHandler) || DllCall("ole32\IsEqualGUID", "Ptr", riid, "Ptr", &IID_IUnknown)) {
		NumPut(this_, ppvObject+0, "Ptr")
		IUIAutomationEventHandler_AddRef(this_)
		return 0 ; S_OK
	}

	NumPut(0, ppvObject+0, "Ptr")
	return 0x80004002 ; E_NOINTERFACE
}

IUIAutomationEventHandler_AddRef(this_)
{
	NumPut((_refCount := NumGet(this_+0, A_PtrSize, "UInt") + 1), this_+0, A_PtrSize, "UInt")
	return _refCount
}
 
IUIAutomationEventHandler_Release(this_) {
	_refCount := NumGet(this_+0, A_PtrSize, "UInt")
	if (_refCount > 0) {
		_refCount -= 1
		NumPut(_refCount, this_+0, A_PtrSize, "UInt")
		if (_refCount == 0)
			DllCall("GlobalFree", "Ptr", this_, "Ptr")
	}
	return _refCount
}

IUIAutomationEventHandler_HandleAutomationEvent(this_, sender, eventId)
{
	OutputDebug %A_ThisFunc%: %eventId%
	return 0
}

AtExit()
{
	global UIA, pEHTemp, DesktopElem
	OnExit(A_ThisFunc, 0)

	if (IsObject(UIA)) {
		UIA.RemoveAllEventHandlers()
		UIA := ""
	}

	if (pEHTemp) {
		ObjRelease(pEHTemp)
		,pEHTemp := 0
	}

	if (DesktopElem) {
		ObjRelease(DesktopElem)
		,DesktopElem := 0
	}

	return 0
}

UIA := new IUIAutomation
if (!IsObject(UIA))
	ExitApp 1

OnExit("AtExit")

if (!(DesktopElem := UIA.GetRootElement()))
	ExitApp 1

pEHTemp := IUIAutomationEventHandler_new()

UIA.AddAutomationEventHandler( UIA_Event("Window_WindowOpened")
	, DesktopElem
	, UIA_Enum("TreeScope_Subtree")
	, 0
	, pEHTemp )
Last edited by qwerty12 on 19 Sep 2017, 18:10, edited 1 time in total.
cpriest
Posts: 20
Joined: 17 Sep 2017, 08:06

Re: Creating COM Object Event Handler

17 Sep 2017, 19:26

Okay, I see. So I do have to do it the long way. I had read so many posts through the research I'd done and I thought it was handled automatically now.

So the gist of this, effectively is:
* We're returning a globally allocated memory address (vtable) with 4 functions (in this example), each of which has had RegisterCallback() called for them and put into the vtable.
* We're implementing each of the functions as appropriate

I guess one question that comes up, when COM queries the interface for the IUIAutomationEventHandler, it's still going to call the 4th function in the vtable? Seems like that would be black magic to my mind, but COM is a complicated beast I have no serious knowledge of.
qwerty12
Posts: 468
Joined: 04 Mar 2016, 04:33
GitHub: qwerty12

Re: Creating COM Object Event Handler

18 Sep 2017, 03:27

cpriest wrote:I had read so many posts through the research I'd done and I thought it was handled automatically now.
For objects that implement the IDispatch way of doing this, yes. I think there are some examples of listening to IE and Flash events around. (But IDispatch confuses me more than it makes things simpler by being able to refer to COM methods by name, so I avoid that stuff.)
* We're returning a globally allocated memory address (vtable) with 4 functions (in this example), each of which has had RegisterCallback() called for them and put into the vtable.
Have you seen the header file for this interface? It might clear a bit of things up:

Code: Select all

    typedef struct IUIAutomationEventHandlerVtbl
    {
        BEGIN_INTERFACE
        
        HRESULT ( STDMETHODCALLTYPE *QueryInterface )( 
            __RPC__in IUIAutomationEventHandler * This,
            /* [in] */ __RPC__in REFIID riid,
            /* [annotation][iid_is][out] */ 
            _COM_Outptr_  void **ppvObject);
        
        ULONG ( STDMETHODCALLTYPE *AddRef )( 
            __RPC__in IUIAutomationEventHandler * This);
        
        ULONG ( STDMETHODCALLTYPE *Release )( 
            __RPC__in IUIAutomationEventHandler * This);
        
        HRESULT ( STDMETHODCALLTYPE *HandleAutomationEvent )( 
            __RPC__in IUIAutomationEventHandler * This,
            /* [in] */ __RPC__in_opt IUIAutomationElement *sender,
            /* [in] */ EVENTID eventId);
        
        END_INTERFACE
    } IUIAutomationEventHandlerVtbl;

    interface IUIAutomationEventHandler
    {
        CONST_VTBL struct IUIAutomationEventHandlerVtbl *lpVtbl;
    };
IUIAutomationEventHandlerVtbl is a (v)table with space for four pointers. So, yes, IUIAutomationEventHandler_Vtbl() in the script creates one IUIAutomationEventHandlerVtbl struct and populates it with pointers to the AHK functions using RegisterCallback() which is pretty much the inverse of DllCall().
IUIAutomationEventHandler_new() gets the pointer and places it into a IUIAutomationEventHandler struct it creates, filling in the lpVtbl member. The object's refcount needs to be stored so IUIAutomationEventHandler is unofficially extended to do so.

Me making the vtable static is a bit of an oversight, come to think of it. Since IUIAutomationEventHandler_new() creates a new object each time, I guess it only makes sense you'd want the ability to choose which function gets called for HandleAutomationEvent each time. (I used the code I wrote to implement IMMNotificationClient as a base, only that just instantiates one static instance and always returns 1 for Release and AddRef(), so it wasn't something I was concerned with.) If you do want to have each object having its own vtable, keep https://autohotkey.com/docs/commands/Re ... htm#Memory in mind.
I guess one question that comes up, when COM queries the interface for the IUIAutomationEventHandler, it's still going to call the 4th function in the vtable? Seems like that would be black magic to my mind, but COM is a complicated beast I have no serious knowledge of.
Truth be told, I have no serious knowledge of COM either (or OO - I like C - so forgive me if my terminology is off), but I feel comfortable in saying If all goes well, always.

From the other side, here is a good guide on calling COM interfaces in AutoHotkey. Using that for background knowledge, if I call, say, pDesktopWallpaper := ComObjCreate("{C2CF3110-460E-4fc1-B9D0-8A1C0C9CC4BD}", "{B92B56A9-8B55-4E14-9A89-0199BBB6F93B}"), off := 16*A_PtrSize, DllCall(NumGet(NumGet(pDesktopWallpaper+0)+off), "Ptr", pDesktopWallpaper, "Ptr", monitorID, "UInt", 0) then I expect the DllCall to advance the desktop's wallpaper to happen because I requested a certain interface with its unique CLSID and IID pair and the design for that interface says I can expect to find a method in the vtable at the 16th position that will do just that.

In this script, the interface is implemented by ourselves and a pointer to one instance of it passed to AddAutomationEventHandler. As we can see from the header file, IUIAutomationEventHandler has an entry for HandleAutomationEvent in its vtable. The QueryInterface method here (every interface must implement IUnknown, which is where the QI, AddRef and Release methods come from) says "yes, I am a IUIAutomationEventHandler object" when AddAutomationEventHandler calls it with IUIAutomationEventHandler's IID and so it's not unreasonable of it to expect there to be a pointer to a valid HandleAutomationEvent method at the fourth position of the vtable, which it will have been hardcoded to call.
User avatar
jeeswg
Posts: 5939
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Creating COM Object Event Handler

18 Sep 2017, 04:43

Here are some collected links re. UI Automation, in case they're useful:
UIA Automation - Inspect.exe for AHK - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=28866
[AHK_L] Screen Reader -- a tool to get text anywhere - Scripts and Functions - AutoHotkey Community
https://autohotkey.com/board/topic/9461 ... -anywhere/
[AHK_L] ImportTypeLib - extended COM support! - Scripts and Functions - AutoHotkey Community
https://autohotkey.com/board/topic/7820 ... m-support/
Internet Explorer / FireFox - Activate Tab (UIA - 64bit AHK) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=2216
UIA_Interface/UIA_Interface.ahk at master · jethrow/UIA_Interface · GitHub
https://github.com/jethrow/UIA_Interfac ... erface.ahk
TypeLib2AHK - Convert COM Type libraries to AHK code - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=36025

If anyone knows of any useful things that UI Automation can do that Acc (MSAA: Microsoft Active Accessibility) can't, I would be interested. Thanks.

Acc library (MSAA) and AccViewer download links - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=26201
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
cpriest
Posts: 20
Joined: 17 Sep 2017, 08:06

Re: Creating COM Object Event Handler

18 Sep 2017, 07:07

Thanks much for that detailed explanation. That makes sense, I hadn't looked at the definition of the interface as I wasn't sure it was needed. I guess the current state of AHK_L is that it's automated for IDispatch based interfaces, but no others.

I had read over that excellent tutorial, but given all of the posts over the board and timeframes, I just wasn't sure what was current. Thanks also jeeswg for the round up, I'll look these over tonight. I may be wrong but I don't think Acc can tell me what tabbed documents belong to a window which is why I moved on to UIA vs Acc.

I'm working on a Virtual Desktop Manager that will put windows back onto the virtual desktop and into the state they were last moved around.
qwerty12
Posts: 468
Joined: 04 Mar 2016, 04:33
GitHub: qwerty12

Re: Creating COM Object Event Handler

18 Sep 2017, 10:15

jeeswg wrote:If anyone knows of any useful things that UI Automation can do that Acc (MSAA: Microsoft Active Accessibility) can't, I would be interested. Thanks.
It's probably not a concern for you, as you're on Windows 7, but UIA has the advantage that its DoDefaultAction actually works (unlike MSAA's) on Modern/Metro/whatever-they're-calling-them-this-week Windows 10 apps.
cpriest wrote:Thanks much for that detailed explanation. That makes sense, I hadn't looked at the definition of the interface as I wasn't sure it was needed. I guess the current state of AHK_L is that it's automated for IDispatch based interfaces, but no others.
No problem.

IMHO that's never going to change. IDispatch interfaces provide metadata on itself, whereas with interfaces that just derive from IUnknown, you have nothing.
I may be wrong but I don't think Acc can tell me what tabbed documents belong to a window which is why I moved on to UIA vs Acc.
Do you have any recent(-ish) version of the Windows SDK installed? The inspect.exe program from it allows you to quickly see what UIA and MSAA can both find in a given window.
I'm working on a Virtual Desktop Manager that will put windows back onto the virtual desktop and into the state they were last moved around.
Ah, nice. I guess you might also find yourself needing to implement IVirtualDesktopNotification, if you're already using the private virtual desktop interface for Windows 10 (I have a dodgy implementation somewhere that I won't share out of embarrassment, but I can tell you for that I didn't bother to make it create multiple instances if needed - the memory for the IVirtualDesktopNotification struct is allocated in a function using VarSetCapacity, the pointer is stored statically, the AddRef and Release methods just return 1 and subsequent calls to the IVirtualDesktopNotification_new() function just return the same instance)

EDIT:
cpriest wrote: Actually, I've had a script to do stuff with VD for quite some time, based on this project:
https://github.com/Ciantic/VirtualDesktopAccessor

Now I'm extending it to handle my most hated thing about VD's. Re-launch chrome or other multi-window apps and whee, time to start pushing windows to a bunch of other desktops again, ugh.

Thank you again for your help!
Ah, OK, I've heard of that DLL before but I've never touched it because I know pretty much everything can be done in AutoHotkey itself. But again, no problem, and good luck with your script!
Last edited by qwerty12 on 18 Sep 2017, 21:55, edited 1 time in total.
cpriest
Posts: 20
Joined: 17 Sep 2017, 08:06

Re: Creating COM Object Event Handler

18 Sep 2017, 16:26

qwerty12 wrote: Ah, nice. I guess you might also find yourself needing to implement IVirtualDesktopNotification, if you're already using the private virtual desktop interface for Windows 10 (I have a dodgy implementation somewhere that I won't share out of embarrassment, but I can tell you for that I didn't bother to make it create multiple instances if needed - the memory for the IVirtualDesktopNotification struct is allocated in a function using VarSetCapacity, the pointer is stored statically, the AddRef and Release methods just return 1 and subsequent calls to the IVirtualDesktopNotification_new() function just return the same instance)
Actually, I've had a script to do stuff with VD for quite some time, based on this project:
https://github.com/Ciantic/VirtualDesktopAccessor

Now I'm extending it to handle my most hated thing about VD's. Re-launch chrome or other multi-window apps and whee, time to start pushing windows to a bunch of other desktops again, ugh.

Thank you again for your help!
cpriest
Posts: 20
Joined: 17 Sep 2017, 08:06

Re: Creating COM Object Event Handler

19 Sep 2017, 19:21

qwerty12 wrote: You need to implement the interface yourself. Here's my (definitely was buggy) attempt:
@qwerty12: Just wanted to say thank you very much, your code worked flawlessly, even though I read it over and thought I saw a bug, I re-read until I understood what was going on.

Thank you so much, I'll post what I go with for my final implementation for others to reference as well.
qwerty12
Posts: 468
Joined: 04 Mar 2016, 04:33
GitHub: qwerty12

Re: Creating COM Object Event Handler

19 Sep 2017, 20:38

cpriest wrote: @qwerty12: Just wanted to say thank you very much, your code worked flawlessly, even though I read it over and thought I saw a bug, I re-read until I understood what was going on.

Thank you so much, I'll post what I go with for my final implementation for others to reference as well.
No problem, I'm glad it helped. The bug was that the interface is meant to explicitly add a reference when the QueryInterface succeeds. I changed that, and made sure we free our reference at exit. That wasn't needed before because when RemoveAllEventHandlers() was called, UIA would have released its reference to the object and it would have gotten freed anyway since we didn't hold an explicit reference to the object ourselves in the script, but that's not how a COM interface is meant to be designed.

I only realised my mistake when reading Inside COM today on the train, and saw in the reference implementation there what I had neglected to do in the code I posted above, but I was stupid to not see it myself before - after all, when I do ComObjQuery with any other interface, I know I always have to ObjRelease the returned object when I'm done with it...

Return to “Ask For Help”

Who is online

Users browsing this forum: ktbjx, Trej and 184 guests