UIA v2

Post your working scripts, libraries and tools.
Descolada
Posts: 1167
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 23 Sep 2023, 01:54

@songdg, the first link actually does show some of the checkboxes, but at a wrong location:
image.png
image.png (156.45 KiB) Viewed 3262 times
The other links have the checkbox elements in the UIA tree, but they are reported as 0 width and 0 height elements so they aren't highlightable, yet they can still be toggled.
I'm not sure of the exact cause of this problem. It appears it affects multiple browser types (Chrome, Firefox) and the official Accessibility Insights inspection tool has the same issue. Yet Chrome developer consoles (F12) element inspection tool highlights them correctly. Overall this isn't a UIA.ahk problem, but I can't fathom where the cause lies.

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 23 Sep 2023, 11:35

Thanks, much appreciated your help. I still have a few questions, how to get an active tab's title, how to use GetAllLinks() to get a list of links' title and click on a certain link, say on https://www.procon.org/debate-topics/, I want a list of all the topics and click Artificial Intelligence

elbitjusticiero
Posts: 78
Joined: 06 May 2017, 11:07

Re: UIA v2

Post by elbitjusticiero » 27 Sep 2023, 06:29

I can't make a piece of example code from the tutorial work.

The code is:

Code: Select all

gn := wap.FindElement({Type:"Button", or:[Name:"Something", Name:"Else"]})
(wap is a UIA object)

Lifted from https://github.com/Descolada/UIA-v2/wiki/05.-Conditions

It gives me the following error:
Error: Missing "]" before ":"

Descolada
Posts: 1167
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 27 Sep 2023, 06:46

@elbitjusticiero, it's a typo (missing some brackets), I'll soon correct it. It should be gn := wap.FindElement({Type:"Button", or:[{Name:"Something"}, {Name:"Else"}]}) or gn := wap.FindElement({Type:"Button", Name:["Something", "Else"]}) should also work.

elbitjusticiero
Posts: 78
Joined: 06 May 2017, 11:07

Re: UIA v2

Post by elbitjusticiero » 27 Sep 2023, 08:24

Thank you!

elbitjusticiero
Posts: 78
Joined: 06 May 2017, 11:07

Re: UIA v2

Post by elbitjusticiero » 29 Sep 2023, 14:22

Hi, sorry for the double post. I have implemented an ugly workaround for controlling RSS Guard, an awesome RSS reader for Windows. In trying to come up with a way to focus on the article list, I found that there seems to be no method that lets me do that.

SetFocus() gives an error: Error: (0x80040201) An event was unable to invoke any of the subscribers

Click() gives an even more cryptic error: Error: (0x80131509)

Invoke() also fails: Error: Method Invoke not found in UIA.IUIAutomationElement Class.

LegacyIAccessible.DoDefaultAction() is a no-go as well: Error: Property LegacyIAccessible not found in UIA.IUIAutomationElement Class.

The only method that kinda works is ControlClick(). But it clicks somewhere near the middle of the list, so it doesn't preserve the article I was viewing, unless it was in that exact position.

Any ideas?

I am attaching a screenshot:
Attachments
clip_20230929_16.21.54.jpg
clip_20230929_16.21.54.jpg (129.66 KiB) Viewed 3022 times

Descolada
Posts: 1167
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 29 Sep 2023, 14:59

@elbitjusticiero, yeah Qt apps don't play nice with UIA and often the normal UIA actions don't work (SetFocus, LegacyIAccessiblePattern.DoDefaultAction etc). LegacyIAccessible.DoDefaultAction() didn't work because it should be either only Element.DoDefaultAction() (the pattern is automatically detected) or Element.LegacyIAccessiblePattern.DoDefaultAction(), but these throw an error anyway.

I found that ControlClick-ing the header of the tree seems to set the focus while preserving the selected article. It should be the first element of the tree so the path "RRQRQRQNZ" should work (or in my setup it was "RRQRQR/NZ").

elbitjusticiero
Posts: 78
Joined: 06 May 2017, 11:07

Re: UIA v2

Post by elbitjusticiero » 30 Sep 2023, 11:31

Sadly, it doesn't work in my machine. ¯\_(ツ)_/¯

I'll keep trying options. I installed Acc (also by you!) but the same thing seems to happen.

roeleboele
Posts: 33
Joined: 24 Nov 2022, 02:39

Re: UIA v2

Post by roeleboele » 30 Sep 2023, 15:12

Hello
I am also using Edge with this fantastic library.
But i get also thé error comcall ()
I did a workaround with thé dumpall function and fileappend it to an text-file
Then search the info i need and split thé file in different other file if necessary.. With arrays and for loops it kind works

Descolada could you tell me if i use thé library how they library is connecting with thé browser?

Descolada
Posts: 1167
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 01 Oct 2023, 01:05

@elbitjusticiero, both UIA and Acc are based on the same technology so I'm not surprised Acc doesn't work. By "it doesn't work in my machine" do you mean you have determined that rssGuardEl["RRQRQRQNZ"].Highlight() doesn't highlight the Title part of the article list treeview and rssGuardEl["RRQRQRQNZ"].ControlClick() doesn't move keyboard focus to the treeview? In my setup the following highlights and then focuses the treeview when pressing F1 in RSSGuard:

Code: Select all

#Include <UIA>
#HotIf WinActive("ahk_exe rssguard.exe")
F1::
{
    rssGuard := UIA.ElementFromHandle("A")
    articleView := rssGuard.FindElement({AutomationId:"FeedsView"})
    title := articleView[1].Highlight().ControlClick()
}
@roeleboele, the ComCall error simply means that something went wrong on the UIAutomation side. Maybe the element isn't accessible (been destroyed perhaps), or doesn't support the requested pattern/method, or any number of other causes. Without having a testable situation (code + name of website) I can't comment any further.

Both UIA-v2 and UIA_Browser are "connecting" to the browser via the UIAutomation interface which is provided by Microsoft. What are you asking exactly?

elbitjusticiero
Posts: 78
Joined: 06 May 2017, 11:07

Re: UIA v2

Post by elbitjusticiero » 01 Oct 2023, 16:30

@Descolada, I think I misunderstood you. ControlClick()ing the tree does focus the tree. I thought it would focus the article list.

With your code, I can send a tab key so that the article list becomes focused.

Thank you so much! You're the best. ツ

roeleboele
Posts: 33
Joined: 24 Nov 2022, 02:39

Re: UIA v2

Post by roeleboele » 02 Oct 2023, 02:16

Descolada wrote:
01 Oct 2023, 01:05
@elbitjusticiero, both UIA and Acc are based on the same technology so I'm not surprised Acc doesn't work. By "it doesn't work in my machine" do you mean you have determined that rssGuardEl["RRQRQRQNZ"].Highlight() doesn't highlight the Title part of the article list treeview and rssGuardEl["RRQRQRQNZ"].ControlClick() doesn't move keyboard focus to the treeview? In my setup the following highlights and then focuses the treeview when pressing F1 in RSSGuard:

Code: Select all

#Include <UIA>
#HotIf WinActive("ahk_exe rssguard.exe")
F1::
{
    rssGuard := UIA.ElementFromHandle("A")
    articleView := rssGuard.FindElement({AutomationId:"FeedsView"})
    title := articleView[1].Highlight().ControlClick()
}
@roeleboele, the ComCall error simply means that something went wrong on the UIAutomation side. Maybe the element isn't accessible (been destroyed perhaps), or doesn't support the requested pattern/method, or any number of other causes. Without having a testable situation (code + name of website) I can't comment any further.

Both UIA-v2 and UIA_Browser are "connecting" to the browser via the UIAutomation interface which is provided by Microsoft. What are you asking exactly?
@Descolada With my company we use sharepoint. Is it possible that the sites are slower after i use te UIA interface? Some collegues are experiencing that sharepoint is slower. Those problems started when i started to use the uia interface or perhaps it is just an coincidence and is the problerm of the slow sharepoint something else

Descolada
Posts: 1167
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 02 Oct 2023, 04:17

@roeleboele, UIA is known to be resource-intensive if called multiple times in a loop (eg with WaitElement), so it can be the cause. However, any other cause of high CPU usage might cause that as well (eg running an infinite loop without any Sleeps). When Sharepoint is slow then check out the CPU usage in Task Manager to see what the cause is, and if it's the AHK script then try to figure out which step exactly is causing the high CPU usage and try to eliminate that.

WKen
Posts: 185
Joined: 21 Feb 2023, 00:01

Re: UIA v2

Post by WKen » 11 Oct 2023, 07:57

Newbie here, using the example, getting the error.
image.png
image.png (66.94 KiB) Viewed 2033 times

Descolada
Posts: 1167
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 11 Oct 2023, 08:03

@WKen, the error message seems to say that you have the class UIA_Vivaldi declared multiple times, which is not possible with the UIA.ahk and UIA_Browser.ahk libraries. Make sure you have downloaded UIA_Browser.ahk properly (perhaps you've copy-pasted it incorrectly?) and haven't included UIA_Browser.ahk multiple times.

WKen
Posts: 185
Joined: 21 Feb 2023, 00:01

Re: UIA v2

Post by WKen » 11 Oct 2023, 09:05

Yes. I'm having trouble downloading file in my Edge browser, I use the button to copy, but it's not copied completely maybe because the text is too long. Thanks.
image.png
image.png (3.76 KiB) Viewed 1999 times

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 13 Oct 2023, 10:36

https://www.autohotkey.com/boards/viewtopic.php?f=6&t=104999&p=515298#p515298
I try to convert your context menu code, and have some problem:
Error: This value of type "String" has no method named "HasOwnProp".
▶ 2860: timeOut := condition.HasOwnProp("timeOut") ? condition.timeOut : timeOut

Code: Select all

MouseClick "right"
WinWait "ahk_class #32768"
contextmenu := UIA.ElementFromHandle(hwnd := WinExist("ahk_class #32768"))
subMenu := contextmenu.WaitElement({Name:"New", Type:"menuitem"})
expandMenu := subMenu.ExpandCollapsePattern.Expand()
WinWait "ahk_class #32768",,, "ahk_id " hwnd
hwnd := WinExist("ahk_class #32768",, "ahk_id " hwnd)
UIA.ElementFromHandle(hwnd).WaitElement("Folder", "menuitem").click()

Descolada
Posts: 1167
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 13 Oct 2023, 10:59

@songdg, this

Code: Select all

UIA.ElementFromHandle(hwnd).WaitElement("Folder", "menuitem").click()
probably needs to be

Code: Select all

UIA.ElementFromHandle(hwnd).WaitElement({Name:"Folder", Type:"menuitem"}).click()

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 13 Oct 2023, 11:26

Descolada wrote:
13 Oct 2023, 10:59
@songdg, this

Code: Select all

UIA.ElementFromHandle(hwnd).WaitElement("Folder", "menuitem").click()
probably needs to be

Code: Select all

UIA.ElementFromHandle(hwnd).WaitElement({Name:"Folder", Type:"menuitem"}).click()
Thanks, my careless mistake :headwall:

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 18 Oct 2023, 05:24

Code: Select all

MsgBox chromeEl.FindElements({Name:"下载", Type:"Button"}).Length
There are 3 download buttons, why the result is 6?
Image

Post Reply

Return to “Scripts and Functions (v2)”