ACC.AHK - Get IAccessible for Context menu

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 11:36

Hi all,

I'm trying to get the IAccessible interface for the right-click context menu using acc.ahk.

I've noticed that "Context" is a direct child of the Desktop client window, so I'm trying to loop over all children of the Desktop window until accName=="Context" and accRole==0xb

So far I've managed to get the IAccessible interface for the desktop client window:

Code: Select all

getRoot(){
    return acc_ObjectFromWindow(dllcall("GetDesktopWindow"))
}
I can tell this is indeed the desktop window because: msgbox, % getRoot().accChildCount ;==> 351 which is the same as the count that I see in Inspect.exe.

However when I try to use msgbox, % """" Acc_Child(getRoot(),1).accName """" I get "".

Looking deeper into the issue ComObjType(Acc_Child(getRoot(),1)) returns an empty string. I.E. Acc_Child does not return a COM Object...

Any ideas why this doesn't work?

Edit:

When running getRoot().accChild(1), I get an error "The parameter is incorrect." ... Specifically accChild.

Edit2:

I found that someone using AutoIt got a similar error:

https://www.autoitscript.com/forum/topi ... ent=251855
qwerty12
Posts: 468
Joined: 04 Mar 2016, 04:33
Contact:

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 12:40

Hi,

While I don't know what your end goal here is, the first thought that comes to mind is why bother with MSAA here? Use ShellContextMenu if you want the context menu for a file or the desktop (pass "Desktop" as sPath if you want the desktop's context menu).

There are two caveats though:
  • ShellContextMenu hosts everything inside your process, so you get a metric shitton of DLLs loaded - and it shows when the memory usage of your script goes way up
  • You need to add fragman's additions if you want to directly invoke something without even seeing the context menu. However, that code is for 32-bit AHK only. I have a port of it to 64-bit AHK somewhere that I did but I must warn you that I did it to put into practice the IUnknown stuff I learnt after reading maul-esel's tutorial - I still don't know to this day if the code is correct (mainly because I don't have a need for the function any more - with Windows 8, Microsoft finally added a proper interface to move to the next desktop slideshow pic).
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 12:45

It may be that you can only access a context menu via Acc (or by any means) while it's showing. It may be that you can access the submenus of menu bars, while they are not shown, via DllCall (but perhaps not via Acc).

Btw if you mention what you're hoping to do, maybe there's another way. Are you trying to invoke a Desktop context menu item?

Sometimes just sending WM_COMMAND to the right window/control works:
Get Info from Context Menu (x64/x32 compatible) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=31971
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 16:42

qwerty12 wrote:Hi,

While I don't know what your end goal here is, the first thought that comes to mind is why bother with MSAA here? Use ShellContextMenu if you want the context menu for a file or the desktop (pass "Desktop" as sPath if you want the desktop's context menu).
I'm not sure this will work. One of the software applications I use uses a context menu as a 'popup menu' from a button within it's GUI. I need to click the button (which is easy with MSAA) and then click the 'Find' button on the context menu.

Now WM_COMMAND has been suggested, and I would use this but for some reason this software is terribly unstable with WM_COMMAND, which can often cause instantaneous crashes... So I'm trying to stick with the UI as much as I can, because so far that hasn't failed me.

My last resort is using Acc_ObjectFromPoint, which is what other people have used in the past apparently. But it's not clean of course...

@jeeswg It's got nothing to do with accessing the context menu while it's showing, because there are plenty of other windows in the desktop window (literally every window on the desktop), and all windows are inaccessible for some reason... Look at this image:

Image

As you can see, all windows are stored within the desktop client area. This includes the context menu.

Edit:

Hmm actually... I could probably get the context menu's hwnd with UI Automation via CLR... That's not very clean either but it would be general at least...
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 17:04

I have this example for 'Edit, Invert Selection' in Explorer via Acc.
AccViewer crashing, Acc.ahk failing (invoke items, retrieve text, IE/Explorer/VLC) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 39#p127439

Although I later found out that I could do it more directly via WM_COMMAND:
Windows 7 - invert selection, set details view/list view (trigger Explorer/Internet Explorer menu items) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=27564
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 17:17

jeeswg wrote:I have this example for 'Edit, Invert Selection' in Explorer via Acc.
AccViewer crashing, Acc.ahk failing (invoke items, retrieve text, IE/Explorer/VLC) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 39#p127439

Although I later found out that I could do it more directly via WM_COMMAND:
Windows 7 - invert selection, set details view/list view (trigger Explorer/Internet Explorer menu items) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=27564
Sadly in my case it is using windows message's WM_COMMAND which causes the crash and not the other way round. Otherwise I would quite happily use WM_COMMAND for everything.
qwerty12
Posts: 468
Joined: 04 Mar 2016, 04:33
Contact:

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 17:21

sancarn wrote: I'm not sure this will work. One of the software applications I use uses a context menu as a 'popup menu' from a button within it's GUI. I need to click the button (which is easy with MSAA) and then click the 'Find' button on the context menu.

Now WM_COMMAND has been suggested, and I would use this but for some reason this software is terribly unstable with WM_COMMAND, which can often cause instantaneous crashes... So I'm trying to stick with the UI as much as I can, because so far that hasn't failed me.
You might consider an event hook to detect when a popup menu has been displayed. I do this with f.lux so that I can put it into movie mode automatically when I start PotPlayer. I don't know of any synchronous ways of doing this (which I'd like. WM_COMMAND won't work here as f.lux creates its menu on demand using TrackPopupMenu with TPM_RETURNCMD - which I think creates another message loop that I have no idea how to message - and f.lux doesn't listen for menu-related WM_COMMAND messages at any other point). Thanks, jeeswg

I can't give a concrete example (the code I have currently to do this is a mess, and decoupling it would take me too much time), but you can shove a MsgBox in this to see if the context menu you speak of is detected by the event hook (and if so, read the forum post linked to find out what to do next):

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
#Persistent

global EVENT_SYSTEM_MENUPOPUPSTART := 0x0006

OnPopupMenu(hWinEventHook, event, hWnd, idObject, idChild, dwEventThread, dwmsEventTime) {
	if (event == EVENT_SYSTEM_MENUPOPUPSTART) {
		;if (!idChild && idObject)

		; send MN_GETHMENU as described here to get the hMenu from hwnd: https://autohotkey.com/board/topic/16457-controlling-popup-menues/ - if this succeeds, you know you are dealing with a menu
		; get process name from hwnd to discard unwanted processes - note: it is far more efficient 
		; to have the event hook only look at a specific process. you can do that by specifying the target process' PID in the SetWinEventHook call
		 WinGet, pn, ProcessName, ahk_id %hwnd%
		if (pn == "whatever.exe") {
			; now to choose a specific option, you have two choices. Either send the accelerator key pertaining to a specific menu item (much faster):
			ControlSend, ahk_parent, %key%, ahk_id %hwnd%
			; or use Acc:
			;Acc_Get("DoDefaultAction", "1", targetMenuPos+1, "ahk_id " . hwnd)
			
			; However, for the Acc way to work, you need to get the position of the menu item.
			; mugz's third post in the AutoHotkey forum topic linked above will show you how to examine a menu from its handle and get the menu items' names and their positions
			; you also might want to do this anyway to make sure you're dealing with the right menu
		}
	}
}

AtExit()
{
	global hWinEventHook, lpfnWinEventProc
	if (hWinEventHook)
		DllCall("UnhookWinEvent", "Ptr", hWinEventHook), hWinEventHook := 0
	if (lpfnWinEventProc)
		DllCall("GlobalFree", "Ptr", lpfnWinEventProc, "Ptr"), lpfnWinEventProc := 0	
	return 0
}

OnExit("AtExit")
hWinEventHook := DllCall("SetWinEventHook", "UInt", EVENT_SYSTEM_MENUPOPUPSTART, "UInt", EVENT_SYSTEM_MENUPOPUPSTART, "Ptr", 0, "Ptr", (lpfnWinEventProc := RegisterCallback("OnPopupMenu", "")), "UInt", 0, "UInt", 0, "UInt", WINEVENT_OUTOFCONTEXT := 0x0000 | WINEVENT_SKIPOWNPROCESS := 0x0002)

Esc::ExitApp
Last edited by qwerty12 on 16 Jul 2017, 17:26, edited 1 time in total.
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 17:23

In my Acc script for invert selection (no WM_COMMAND), I do this:

Code: Select all

q:: ;click Edit, click Invert Selection (tested on Windows 7)
DetectHiddenWindows, Off
WinGet, hWnd, ID, A
oAcc := Acc_Get("Object", "4.3.3.2", 0, "ahk_id " hWnd)
oAcc.accDoDefaultAction(0) ;click Edit

WinWait, ahk_class #32768
WinGet, hWnd, ID, ahk_class #32768
oAcc := Acc_Get("Object", "1", 0, "ahk_id " hWnd)
oAcc.accDoDefaultAction(13) ;click Invert Selection
return
These lines wait for a context menu to appear, and then click an item:
WinWait, ahk_class #32768
WinGet, hWnd, ID, ahk_class #32768

I would have thought that you could use that to get the context menu hWnd, once it's visible, and then invoke a menu item.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 17:48

jeeswg wrote: These lines wait for a context menu to appear, and then click an item:
WinWait, ahk_class #32768
WinGet, hWnd, ID, ahk_class #32768
I would have thought that you could use that to get the context menu hWnd, once it's visible, and then invoke a menu item.
Hey, I completely forgot that context menus had the class #32768! That does work well, although I worry somewhat that there may be multiple windows with the class #32768.
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 17:50

qwerty12 wrote: I can't give a concrete example (the code I have currently to do this is a mess, and decoupling it would take me too much time), but you can shove a MsgBox in this to see if the context menu you speak of is detected by the event hook (and if so, read the forum post linked to find out what to do next):
Thanks qwerty12, I will give this a go as i think it will have more stable results.

Edit:

Also works like a charm! I think I will settle with this option.
Last edited by sancarn on 16 Jul 2017, 17:55, edited 1 time in total.
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 17:51

The PID and process name/path will match the process, you can add these to the window criteria (in the WinWait and WinGet lines). Also, you can use Acc to confirm the text of the menu items.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 17:57

jeeswg wrote:The PID and process name/path will match the process, you can add these to the window criteria (in the WinWait and WinGet lines). Also, you can use Acc to confirm the text of the menu items.
My worry really is that the program might have 2 menus open with the same window class... If that case does arise there would be no way to get the other window if the found window is incorrect... right?
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: ACC.AHK - Get IAccessible for Context menu

16 Jul 2017, 18:24

You'd just list the windows, and check for the right one:

Code: Select all

q::
DetectHiddenWindows, Off
WinGet, vWinList, List, ahk_class #32768
vOutput := ""
Loop, % vWinList
{
	hWnd := vWinList%A_Index%
	WinGetTitle, vWinTitle, % "ahk_id " hWnd
	;WinGetClass, vWinClass, % "ahk_id " hWnd
	;WinGet, vPID, PID, % "ahk_id " hWnd
	;WinGet, vPName, ProcessName, % "ahk_id " hWnd
	vOutput .= hWnd " " vWinTitle "`r`n"
}
MsgBox, % vOutput
return
If anyone can find a common program that can have 2 context menus open at the same time, I would like to know, to test/to see if it's possible.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

Re: ACC.AHK - Get IAccessible for Context menu

17 Jul 2017, 14:53

jeeswg wrote:If anyone can find a common program that can have 2 context menus open at the same time, I would like to know, to test/to see if it's possible.
I'm quite certain 1 program can't have 2 context menus, but context menus aren't the only windows that I have seen which use the class #32768. So I wouldn't be surprised if you could have more than 1 of these, though I haven't got an explicit example either.
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: ACC.AHK - Get IAccessible for Context menu

17 Jul 2017, 15:14

Well from experience:
context menus: #32768
ComboBox drop-down menus: #32769
dialog windows (and tab controls): #32770

This link doesn't say too much:
About Window Classes (Windows)
https://msdn.microsoft.com/en-us/librar ... s.85).aspx
#32768 The class for a menu.
#32769 The class for the desktop window.
#32770 The class for a dialog box.
#32771 The class for the task switch window.
#32772 The class for icon titles.

If anyone has anything to add. Thanks.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

Re: ACC.AHK - Get IAccessible for Context menu

18 Jul 2017, 07:07

jeeswg wrote:Well from experience:
context menus: #32768
ComboBox drop-down menus: #32769
dialog windows (and tab controls): #32770

This link doesn't say too much:
About Window Classes (Windows)
https://msdn.microsoft.com/en-us/librar ... s.85).aspx
#32768 The class for a menu.
#32769 The class for the desktop window.
#32770 The class for a dialog box.
#32771 The class for the task switch window.
#32772 The class for icon titles.

If anyone has anything to add. Thanks.
Oh... Perhaps I am mistaken then! I didn't realize there were other #.... classes, so that may explain it.
sancarn
Posts: 224
Joined: 01 Mar 2016, 14:52

Re: ACC.AHK - Get IAccessible for Context menu

18 Jul 2017, 07:13

Related to the OP:

Supposedly this also works to get a handle of the context menu:

Code: Select all

FindWindowEx( NULL, NULL, MAKEINTATOM(0x8000), NULL );
Edit:

Apparently the application who the context menu belongs to can execute this to retrieve the HWND of that menu, but not any arbitrary application.
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: ACC.AHK - Get IAccessible for Context menu

14 Nov 2019, 19:59

- @sancarn: I've been doing various tests with Acc.
- Most of the time I haven't been able to get anything useful out of the accChild method.
- Typically these fail: oAcc.accChild(n) and Acc_Child(oAcc, n) (from Acc.ahk), as do attempts to call the vtable directly via DllCall.
- When I test on a child object, I get the error E_INVALIDARG := 0x80070057, and message 'The parameter is incorrect.'
- When I test on a child non-object I get S_FALSE := 1 as expected.

- However, I did find 2 examples in my script libraries where all 3 approaches did work, they were for Firefox and Chrome. I tested the Firefox code and it was still working, it retrieved text from Show All History (Ctrl+Shift+H).

- Virtually every time I've referred to child objects, it's been via the Acc_Children function in the Acc.ahk library, which uses oleacc\AccessibleChildren. That's been something that has seemed to always work.
- So perhaps when accChild fails, it's simply dependent on how something is implemented in the software you're using. So perhaps the message 'The parameter is incorrect.', is misleading/wrong, what we've sent is correct, and should work in theory, but for whatever reason the receiver isn't it handling it.
- When oAcc.accChild(n) has worked, n was simply an integer. I did not need to use oAcc.accChild(ComObject(type, n)).

- Link:
IAccessible::get_accChild (oleacc.h) - Win32 apps | Microsoft Docs
https://docs.microsoft.com/en-us/windows/win32/api/oleacc/nf-oleacc-iaccessible-get_accchild

- Example code to get a pointer from an Acc object:

Code: Select all

WinGet, hWnd, ID, A
oAcc := Acc_Get("Object", "4.4.4", 0, "ahk_id " hWnd) ;Notepad Find dialog 'Match case'

pAcc := ComObjValue(oAcc)
;pAcc := ComObject(oAcc) ;equivalent to line above

- List of methods for IAccessible:

Code: Select all

;INTERFACE - IAccessible
;source: oleacc.h
0 IUnknown::QueryInterface
1 IUnknown::AddRef
2 IUnknown::Release
3 IDispatch::GetTypeInfoCount
4 IDispatch::GetTypeInfo
5 IDispatch::GetIDsOfNames
6 IDispatch::Invoke
7 IAccessible::get_accParent
8 IAccessible::get_accChildCount
9 IAccessible::get_accChild
10 IAccessible::get_accName
11 IAccessible::get_accValue
12 IAccessible::get_accDescription
13 IAccessible::get_accRole
14 IAccessible::get_accState
15 IAccessible::get_accHelp
16 IAccessible::get_accHelpTopic
17 IAccessible::get_accKeyboardShortcut
18 IAccessible::get_accFocus
19 IAccessible::get_accSelection
20 IAccessible::get_accDefaultAction
21 IAccessible::accSelect
22 IAccessible::accLocation
23 IAccessible::accNavigate
24 IAccessible::accHitTest
25 IAccessible::accDoDefaultAction
26 IAccessible::put_accName
27 IAccessible::put_accValue
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: mikeyww, TAC109 and 133 guests