jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

Helpful script writing tricks and HowTo's
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

28 Nov 2017, 03:54

==================================================

Microsoft Active Accessibility (MSAA) (Acc)

- You can use the Acc functions, in the Acc.ahk library, to get information about windows/controls/GUI elements, and to, in some ways, interact with them.
- There is also a key script called AccViewer, a window inspector, which demonstrates much of the info that the Acc functions can retrieve.

GET WINDOW INFO:
- I use AccViewer as a window spy, to get information about windows/controls/GUI elements. E.g. text, positions, focused/selected/hidden states.
- One thing that AccViewer doesn't currently show is the GUI element state (e.g. focused), see AccViewer Basic which does.
- Any information that can be retrieved via AccViewer, can be retrieved using the Acc functions (Acc library).
- The AccViewer Basic script demonstrates how to use the Acc functions.

SET WINDOW INFO/INTERACT WITH WINDOW:
- Acc provides an accDoDefaultAction method, where available, this is usually equivalent to clicking on a GUI element/control.
- The accSelect method can be used to change the focus/selection.
- For *some* but not all GUI elements, you can set the value by specifying the text via accValue.
- Although I haven't been able to get Acc to right-click a GUI element, I have been able to use Acc to get the position of a GUI element, and use non-Acc techniques to perform a right-click.

==================================================

SCRIPTS / FUNCTIONS

Acc library (MSAA) and AccViewer download links - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=26201

[examples of how to get information from a window via Acc]
AccViewer Basic - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=32039

[JEE_AccGetTextAll: a function to get text and ID information for all the GUI elements in a window/control]
GUIs via DllCall: get/set internal/external control text - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=40514
Acc: get text from all window/control elements - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=40615

[JEE_AccGetPath: get Acc path for GUI element]
[e.g. element under cursor, element at a specific point, any element]
How to get the full ACC path for control on cursor? - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=56470&p=254902#p254902

==================================================

REFERENCE LISTS

[list of methods (new/old versions of page)]
IAccessible interface (Windows)
https://msdn.microsoft.com/en-us/library/windows/desktop/dd318466(v=vs.85).aspx
IAccessible interface (Windows)
https://web.archive.org/web/20161224102522/https://msdn.microsoft.com/en-us/library/windows/desktop/dd318466(v=vs.85).aspx

[role constants (describe the type of a GUI element)]
AccViewer Basic - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=32039&p=167451#p167451

[state constants (describe the state of a GUI element)]
AccViewer Basic - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=32039&p=177601#p177601

[selection constants for use with accSelect]
SELFLAG Constants (Windows)
https://msdn.microsoft.com/en-us/library/windows/desktop/dd373634(v=vs.85).aspx

==================================================

LIST OF METHODS

accChild: get an object for one of a GUI element's child elements
accChildCount: number indicating how many child elements a GUI element has
accDefaultAction: text describing what the default action does
accDescription
accDoDefaultAction: perform the element's default action (often equivalent to clicking it)
accFocus: number indicating the focused item index
accHelp
accHelpTopic
accHitTest
accKeyboardShortcut
accLocation: can be used to output ByRef a GUI element's coordinates
accName: text of/relating to a GUI element
accNavigate: (deprecated)
accParent: get an object for a GUI element's parent element
accRole: number indicating a role e.g. title bar
accSelect: change the focused/selected items
accSelection: number indicating the selected item index (or an object indicating the selected item indexes)
accState: number indicating focused/selected state etc
accValue: text of/relating to a GUI element

note the distinction:
accDefaultAction [no 'do']
accDoDefaultAction [contains 'do']

See also:
IAccessible interface (Windows)
https://msdn.microsoft.com/en-us/library/windows/desktop/dd318466(v=vs.85).aspx

==================================================

THE ACC_GET FUNCTION'S 'CMD' PARAMETER

Some notes on the Acc_Get function's 'Cmd' parameter:

pass 'Object' to return an Acc object

or pass the method name minus 'acc' to invoke a method:
e.g. pass 'Name' to invoke accName
e.g. pass 'Value' to invoke accValue
(note: see the list of accXXX methods above)

or pass one of these abbreviations to invoke the corresponding method:
Action (accDefaultAction)
DoAction (accDoDefaultAction)
Keyboard (accKeyboardShortcut)

note: Acc_Get uses some custom handling for these methods: accLocation/accRole/accState

==================================================

FURTHER LINKS AND EXAMPLE SCRIPTS

[introductions]
Microsoft Active Accessibility - Wikipedia
https://en.wikipedia.org/wiki/Microsoft_Active_Accessibility
Microsoft Active Accessibility (Windows)
https://msdn.microsoft.com/en-us/library/windows/desktop/dd373592(v=vs.85).aspx
Introduction to Microsoft Active Accessibility - CodeProject
https://www.codeproject.com/Articles/18290/Introduction-to-Microsoft-Active-Accessibility

[original Acc threads]
Accessibility Standard Library - Scripts and Functions - AutoHotkey Community
https://autohotkey.com/board/topic/22058-accessibility-standard-library/
Acc Library [AHK_L] (updated 09/27/2012) - Scripts and Functions - AutoHotkey Community
https://autohotkey.com/board/topic/77303-acc-library-ahk-l-updated-09272012/
Accessible Info Viewer - Alpha Release (2012-09-20) - Scripts and Functions - AutoHotkey Community
https://autohotkey.com/board/topic/77888-accessible-info-viewer-alpha-release-2012-09-20/

Threads where I use Acc_Get():
- Get the URL of the current (active) browser tab
- Get text from status bar
- Unable to detect Checkbox control using Windows Spy
- GUI COMMANDS: COMPLETE RETHINK
- Hotkey to sort images by Rating/Descending
- Internet Explorer: move tab to new window
- obs studio start and record
- Acc library (MSAA) and AccViewer download links
- Internet Explorer Save Open Cancel Popup
- set text/state of listbox/listview items via Acc?
- AccViewer crashing, Acc.ahk failing (invoke items, retrieve text, IE/Explorer/VLC)
- is it possible to write a script that opens specific chrome or firefox tabs
- Windows Update: detect if important updates available
- Firefox/Chrome, get tab names/focus tab
- Move mouse to taskbar to use AeroPeek
- Acc (MSAA) - Text Without Needing Cursor Position?
- WinGetText and ControlGetText not working on certain windows
- ResetArray() , ExtractURLDomain(), GetChromeURL()
- notepad get/set path (get/set text file path)
- Resizing the preview pane of Windows Explorer
- Acc / AccViewer / iWB2 Learner / Anchor conversion attempts
- IE Com Save Dialog - Auto Save using ACC
- ungettable text (Character Map/Excel/WordPad)
- Chrome update, now winkill not working
- Select text from prompt and select the proper option
- Getting position and size of the scroll bar
- jeeswg's Notepad tutorial
- Close a Window - How
- To Scroll the Bar with Arrow Keys
- Detect if classNN is scrollable
- Selecting a tab in another program
- Grabbing scrollbar control or location
- Explorer column interaction (get/set: which appear, width, ascending/descending order)
- Windows Explorer folder positions and scrolling
- Add "https://" in URL if specific error page is displayed on Google Chrome
- Internet Explorer: activate tab
- ACC.AHK - Get IAccessible for Context menu
- Script idea for an expert: typing in list view
- script to full screen YouTube videos
- Hotkey Control Partially Works
- Explorer window interaction (folder windows/Desktop, file/folder enumeration/selection/navigation/creation)
- Is it possible to get coordinates of selected item in TreeView (TreeViewWndClass1) control ?
- How to click on the control position?
- set value to Acc object children?
- Check if a box is checked in an other program
- Using COM to manipulate Microsoft Word's Navigation task pane
- ControlClick to click on X of window
- ControlSend command to send the problem
- active accessibility get value
- active accessibility click selectable
- Shortage for few auto excel operations ?
- Spotify PostMessage Library
- Assistance with Windows Text via Acc Library
- Clicks on the Notification Area of TaskBar
- Help with IAccessible objects
- how to click the first item in a listview?
- GUIs via DllCall: get/set internal/external control text
- Pin to taskbar in Windows 10
- Acc: get text from all window/control elements
- how to know if windows media player is playing or not?
- Firefox : toggle on/off sidebar - HELP
- Doubleclick On Selected Treeview Item
- Help to find the active tab number in Firefox
- Screen position of items in listview?
- possible to get information from external treeview?
- Open/Save dialogue in IE
- How to open specific dialogue box in MS Excel on pressing a HotKey?
- Sending keystrokes to a background process (hidden)
- IE COM fire click event
- How to get the full ACC path for control on cursor?
- Need Help in using ACC library
- AccessibleObjectFromWindow does not execute method accFocus in Firefox
- Select a line from a menu with the text...?

Threads by other people involving Acc:
- Firefox close tab if there is no more history
Last edited by jeeswg on 29 Apr 2019, 03:21, edited 20 times in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's Acc tutorial (initial content)

28 Nov 2017, 04:19

[RESERVED]
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

14 Apr 2018, 09:37

[RESERVED]
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

14 Apr 2018, 09:38

[RESERVED]
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
neogna2
Posts: 32
Joined: 15 Sep 2016, 15:44

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

27 Apr 2018, 03:26

Do you know how to use Acc to check if an element has keyboard focus? Specifically I want to check if the URL bar in Firefox and Chrome has keyboard focus or not. This MSDN page seems relevant, but I couldn't figure out the syntax for it in AutoHotkey with Acc.ahk

edit: fixed URL

edit2: solved it after a second look at the source of your Acc Viewer Basic. We check if an object ís focused with

Code: Select all

if (oAcc.accFocus = "0")
I was confused by the fact that accFocus returns 0 if the object has focus and nothing if the object does not have focus.

I adapted some of your code into two functions that return 1 if the URL bar has focus

Code: Select all

FirefoxUrlBarFocused(hWnd)
{
  vAccPath := "application.tool_bar3.combo_box1.editable_text"
  ;vAccPath := "4.27.4.2"
  oAcc := Acc_Get("Object", vAccPath, 0, "ahk_id " hWnd)
  Return oAcc.accFocus = "0" ? 1 : 0
}

ChromeUrlBarFocused(hWnd)
{
  vAccPath := "pane.client.client2.client2.client.grouping.editable_text"
  ;vAccPath := "4.1.2.2.3.5.2"
  oAcc := Acc_Get("Object", vAccPath, 0, "ahk_id " hWnd)
  Return oAcc.accFocus = "0" ? 1 : 0
}
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

27 Apr 2018, 06:59

- Take a look at accState:

Code: Select all

vAccStateNum := oAcc.accState(vChildId)
vAccStateNumHex := Format("0x{:X}", vAccStateNum)
- There's an example here.
Help to find the active tab number in Firefox - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 56#p206156
- Also, check the link to state constants above.

- Re. accFocus (and accSelection):
- I hadn't looked too closely at accFocus or accSelection before, they appear to work with certain controls, e.g. listview controls, to report which items are focused/selected.
- accFocus returns the index of the focused item.
- accSelection returns the index of the selected item, or an object if multiple items are selected.
- I had trouble trying to retrieve the indexes from an object, but there was a useful link re. accSelection here (thanks just me and lexikos):
- I'll improve my AccViewer Basic script with my new-found knowledge.
[Solved] IAccessible - Accessible Objects - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 811#p62811

Code: Select all

q:: ;Acc - get the index of the focused item / index(es) of the selected item(s)
;e.g. test this on a listview control
WinGet, hWnd, ID, A
ControlGetFocus, vCtlClassNN, % "ahk_id " hWnd
ControlGet, hCtl, Hwnd,, % vCtlClassNN, % "ahk_id " hWnd
oAcc := Acc_ObjectFromWindow(hCtl)
vFoc := oAcc.accFocus ;gets index of focused item
vSel := oAcc.accSelection ;if one item selected, gets index, if multiple items selected, gets indexes as object

if IsObject(vSel)
{
	oSel := vSel, vSel := ""
	while oSel.Next(vValue, vType)
		vSel .= (A_Index=1?"":",") vValue
	oSel := ""
}

MsgBox, % "foc: " vFoc "`r`n" "sel: " vSel
oAcc := ""
return
Last edited by jeeswg on 28 Apr 2018, 17:22, edited 1 time in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
neogna2
Posts: 32
Joined: 15 Sep 2016, 15:44

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

27 Apr 2018, 07:26

Thanks for the reply
jeeswg wrote: - accFocus returns the index of the focused item.
Ok, that explains why it returns 0 when the URL bar has focus.
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

28 Apr 2018, 17:48

I've improved AccViewer Basic to better support accSelection (multiple items selected) and accState (provide friendly text for all of an item's state flags).
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
neogna2
Posts: 32
Joined: 15 Sep 2016, 15:44

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

01 May 2018, 08:30

jeeswg wrote:I've improved AccViewer Basic to better support accSelection (multiple items selected) and accState (provide friendly text for all of an item's state flags).
Useful! On small thing is that the Name contents placed first in the tooltip can be very long for example when viewing the text pane of Notepad++. In that case the other tooltip variables are pushed out of view below the screen. I worked around that with this change to your script.

Code: Select all

vAccNameShort := SubStr(vAccName, 1, 200)
vOutput = ;continuation section
(
Name: %vAccNameShort%
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

01 May 2018, 08:53

I'd previously added a fix for other variables to AccViewer Basic, I've updated the script to add vAccName to the list:

Code: Select all

;before:
vList := "vWinText,vAccValue"
;after:
vList := "vWinText,vAccName,vAccValue"
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
DRocks
Posts: 443
Joined: 08 May 2018, 10:20

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

28 May 2018, 04:29

hi jeeswg

I have been reading a couple of your topics so thanks so much for the help you provide here.
Are you interested in anything like making short videos about some of the most important discoveries you've explained by text? It is a suggestion that runs my mind as I write because I think it would make your content be even more educationnal. Just a thought like that :)
Nate32
Posts: 1
Joined: 08 Jul 2018, 01:02

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

08 Jul 2018, 09:53

merci beaucoup d'avoir pris la peine de faire tout ça. Thank you
panhartstuff
Posts: 19
Joined: 21 Jan 2018, 07:40

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

21 Dec 2018, 10:28

Hey jeeswg, I'm trying to use your code from OBS start/stop switcher, but it doesn't seem to be working

Code: Select all

WinGet, hWnd, ID, OBS ahk_class Qt5QWindowIcon
ControlGet, hCtl, Hwnd,, Qt5QWindowIcon10, % "ahk_id " hWnd
vAccPath := "client.push_button2"
oAcc := Acc_Get("Object", vAccPath, 0, "ahk_id " hCtl)
vText := oAcc.accName(0)

if InStr(vText, "Start")
	oAcc.accDoDefaultAction(0)
This should press the start button, right? But it's not working. Maybe OBS have changed their class names? I'm using OBS 22.0.2

Also, is it possible to use your Acc scripts to switch scene?
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

27 Dec 2018, 15:04

@panhartstuff: I've had partial success in trying to update the script, in principle it works, however, the clicking is working sometimes but not always. If you'd like to say more about it, please post in that thread. Thanks.
obs studio start and record - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=26124
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
panhartstuff
Posts: 19
Joined: 21 Jan 2018, 07:40

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

01 Jan 2019, 14:19

@jeeswg
Thanks for the help man! Yeah working with AHK + OBS is just a nightmare, OBS is just so bizarrely resistant to AHK fiddling.
cavin12
Posts: 1
Joined: 25 Mar 2019, 04:56
Contact:

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

25 Mar 2019, 05:04

Wow, thanks for sharing this useful tutorial which i was actually looking for, appreciate the efforts
neogna2
Posts: 32
Joined: 15 Sep 2016, 15:44

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

13 Apr 2019, 02:06

How can we use Acc to get the path (for example 4.3.2.4.2.1.1.2.3 ) to an element that is shown only when the mouse hovers over a location of the window but that element is to the side of the mouse and not directly under it? For example statusbar text in some browsers behaves like that, it is temporarily shown hovering above the bottom edge of the window when the user positions the mouse over a hyperlink elsewhere in the window.

AccViewer Basic can't help with this since its Q hotkey only gets us the element under the mouse.

AccViewer can give us the whole treeview structure, but we have to manually walk through it, because we can't use the drag drop crosshair on the target element for the same reason as above.

If there was a way to save all information for all elements in the AccViewer treeview at a point in time to a textfile we could later search the textfile for some string that identifies the element and from that infer what its path is. Is there some method to save all that info? Or some other workaround for this kind of case?
User avatar
jeeswg
Posts: 6466
Joined: 19 Dec 2016, 01:58
Location: UK

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

13 Apr 2019, 03:14

- @neogna2: Above, I mention 2 links (I just added the JEE_AccGetPath link):
- JEE_AccGetTextAll: get the paths for all elements.
- JEE_AccGetPath: get the path for a specific element.
- Note: you can use Acc_ObjectFromPoint to get an Acc object for the element at a specific point, you can pass that object to JEE_AccGetPath.

- (I'd use JEE_AccGetTextAll, because it should always give you the correct path, whereas JEE_AccGetPath is potentially unreliable, as stated in the description for the function. That said, you can test that a path is correct, using Acc_Get.)

- Here's some come for handling regular ToolTips:

Code: Select all

q:: ;get text from ToolTips
ToolTip, hello world

DetectHiddenWindows, On
WinGet, vWinList, List, ahk_class tooltips_class32
vOutput := ""
Loop, % vWinList
{
	hWnd := vWinList%A_Index%
	WinGet, vPName, ProcessName, % "ahk_id " hWnd
	ControlGetText, vText,, % "ahk_id " hWnd
	vOutput .= hWnd " " vPName " " vText "`r`n"
}
Clipboard := vOutput
MsgBox, % vOutput
return
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
neogna2
Posts: 32
Joined: 15 Sep 2016, 15:44

Re: jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA)

13 Apr 2019, 07:11

Thank you, I really appreciate that you're so helpful and post all these great resources in the forum. I had missed that Acc_ObjectFromPoint optionally accepted an X Y screen position as parameters. For example

Code: Select all

x := 200, y := 150
oAcc := Acc_ObjectFromPoint(vChildID, x, y)
That approach works well in the the specific case I was trying to solve. I used it to modify your JEE_AccGetPath to get the child object path I was after. (Side note: it didn't work for browser statusbar text on hyperlink hover, but I didn't really need that, only mentioned it as an example.)

Thanks also for JEE_AccGetTextAll. I first though it didn't work. But turns out it just takes a very long time to finish when testing on a browser window with many tabs open. Much quicker with only one browser tab open and using SetBatchlines, -1 .

A suggestion. Your JEE_AccGetPath uses WinGet, hWnd, ID, A. But to also handle cases where the element under cursor is from a non active window we can replace that line with MouseGetPos,,, hWnd

Return to “Tutorials”

Who is online

Users browsing this forum: No registered users and 6 guests