UIA v2

[eugenesv]

Thanks, the verbosity of every call name (be it Current or Cached) is exactly what I thought to avoid with a single "subelement"!

[gdqb521]

@Descolada, I test "myDataGrid.Scroll(UIA.ScrollAmount.SmallIncrement)", The error is shown in the figure。
but code "myDataGrid.SetScrollPercent(30)" it works。
When the focus is on the datagrid, pageup or up. Down works, but there are other actions in the middle that cause the focus to be lost.
What am I supposed to do?

error

P0016.png (20.75 KiB) Viewed 626 times

Code: Select all

; this can work
myDataGrid.SetScrollPercent(30)
; error
myDataGrid.Scroll(UIA.ScrollAmount.SmallIncrement)

[Descolada]

If myDataGrid.SetScrollPercent(30) works then try myDataGrid.SetScrollPercent(100) - sometimes it causes elements for the whole grid to show up. Otherwise you can scroll some percent (10%?) at a time and gather your elements.

Does myDataGrid.Scroll(UIA.ScrollAmount.LargeIncrement) work?

[wpb]

Hi there. Can anyone report if they've had any luck getting the UIA element of the formula bar in Excel? I tried this but to no avail:

Code: Select all

ExcelElement := UIA.ElementFromHandle("ahk_exe EXCEL.EXE")
FormulaBarElement := ExcelElement.FindElement({AutomationId:"FormulaBar"})

Thanks!

[Descolada]

@wpb that code did get me the formula bar element. Eg highlight it and get the content:

Code: Select all

#Requires AutoHotkey v2
#Include UIA.ahk

ExcelElement := UIA.ElementFromHandle("ahk_exe excel.exe")
FormulaBarElement := ExcelElement.FindElement({AutomationId:"FormulaBar"})
FormulaBarElement.Highlight()
MsgBox FormulaBarElement.DocumentRange.GetText()

Unfortunately you can't set the content of the element with UIA, only get the text and select the text. I recommend automating Excel with its COM interface, which is faster and much more reliable.

[LAPIII]

UIA Browser won't interact with Lastpass and at Spotify the Log In button. I've tried both in chrome and edge, as admin and without running as admin.

[Descolada]

@LAPIII I tried only with LastPass, but it worked as expected in both Chrome and Edge (both running as non-admin):

Code: Select all

#Requires AutoHotkey v2
#Include UIA.ahk
#Include UIA_Browser.ahk

WinActivate "LastPass - Sign In"
WinWaitActive "LastPass - Sign In"
cUIA := UIA_Browser()
cUIA.FindElement({Name:"LOG IN", Type:"Button"}).Click()

[wpb]

@Descolada, thanks for your input (and for the brilliant library!). Your code works perfectly - it was the

Code: Select all

.DocumentRange.GetText()

part that I didn't realize I had to do. How did you determine that a simple

Code: Select all

.Value

wouldn't suffice?

[Descolada]

@wpb inspecting the formula bar with Accessibility Insights showed only one available pattern: Text. For Value to be available the element would need to support LegacyIAccessible, Value, or RangeValue pattern, but none of those are available for that element. Inspecting with UIAViewer did show LegacyIAccessible as being available but that is not actually the case (and I'm not sure how Accessibility Insights figured it out that it in fact isn't available), and trying to access Element.LegacyIAccessiblePattern.Value throws an error accordingly.

[wpb]

Thanks @Descolada - that's interesting. I'm downloading Accessibility Insights now!

[reddyshyam]

Hi @Descolada ,

Good day! May be not but just asking, is there a way to get current downloads list with percentage down etc... on Brave/Chrome browser?

[Descolada]

@reddyshyam, not any easy way, as UIA can only see the download button which has information about how many downloads are active. Otherwise you'd need to click it with UIA and extract the information from the new popup window.
Perhaps it'd be easier and more reliable to use Chrome.ahk or Rufaydium to get that info? I don't know whether those APIs can access download info though.

[reddyshyam]

@reddyshyam, not any easy way, as UIA can only see the download button which has information about how many downloads are active. Otherwise you'd need to click it with UIA and extract the information from the new popup window.
Perhaps it'd be easier and more reliable to use Chrome.ahk or Rufaydium to get that info? I don't know whether those APIs can access download info though.

Thanks @Descolada. You are too kind to reply to every query almost promtly. I assumed it would be the case but still wanted to check with you. I did try Chrome.ahk but didnt find any. At the moment I can get the list of downloads as they appear in the default download location with suffix .crdownload. This mostly addresses my requirement, just thought if I get the percentage downloaded, it would be great. Hence checked with you.

Thanks again for your time and have a good day!

[Spitzi]

@Descolada
This is probalby easy for you: I have an application with a qt-container with image elements in it. When scrolling up/down while holding down the alt-key, these elements increase or decrease in size. I would like to automate that, but it does not work. Here's what I used:

Code: Select all

					
tokenContainerEl := winEl.FindElement({ClassName:"TokenContainerWidget"})
Send("{Alt down}")
tokenContainerEl.ControlClick("WheelDown", 20)
Send("{Alt up}")

any tip as for what might be the reason why it fails? Can this even be done with UIA?

Greets

[Descolada]

@Spitzi, as far as I know you can't zoom elements using UIA. You can use UIA to locate the element and use Click/ControlClick to send WheelUp/WheelDown at that element as you've done in you example.

I don't know any Qt applications where I could test Alt+scrolling, but Anki (Qt6) can be zoomed with Ctrl+scrolling. The following worked:

Code: Select all

#include UIA.ahk
winEl := UIA.ElementFromHandle("ahk_exe anki.exe")
el := winEl.FindElement({ClassName:"MainWebView"})

SetControlDelay(-1)
Loop 10 {
    ControlSend "{Ctrl down}",, "ahk_exe anki.exe"
    el.ControlClick("WheelDown", 1, "NA")
    ControlSend "{Ctrl up}",, "ahk_exe anki.exe"
}

I'm not sure why using ControlClick with ClickCount=10 didn't work, but it scrolled just one time if I did that.

If your version failed, try using Click instead of ControlClick. Perhaps your Qt application doesn't respond to ControlClick...

[fenchai]

I have been having some weird issues with UIA these days, I haven't changed my code in any way.

I am automating google chrome on 2 computers, but on one computer (COMP 1) it works as usual, but on the other (COMP 2) it stopped working so I was trying to debug:

Code: Select all

UIA := new UIA_Browser("ahk_exe " browserExe)

    if !(IsObject(UIA)) {
        msgbox UIA is not loaded
        return
    }

if !(UIA.GetCurrentURL() == "chrome://new-tab-page/") {
            UIA.NewTab()
        }

- for some reason UIA.NewTab() did not work on COMP 2.
- they are both on the same google chrome version
- same UIA version, in this case UIA for AHK v1.
- The surprising thing is... I tried UIAViewer (v2) and launched https://www.autohotkey.com/boards/ but COMP 2 only detected the statusbar and the body, no other elements were detected, where as COMP 1 I launched the same UIAViewer and it was able to detect every single element (by detect I mean it shows the blue rectangle).

what could be happening on COMP 2? My other AHK scripts work as usual, it's only UIA v1 giving me issues, that or chrome... but they are on the same version so why COMP 1 works and COMP 2 not?

[Descolada]

@fenchai browser automation with UIA is highly unreliable, because in addition to webpages user interface changing, also the browsers UIA tree tends to change with browser updates. I've needed to adapt and change UIA v2 UIA_Browser multiple times because of this. I am not updating v1 UIA_Interface any longer, so you can expect UIA_Browser to break as well (and it probably already has). I periodically update v2 UIA_Browser, but it's a losing battle as well...

As to why you are seeing a difference between computer 1 and 2, I can't say. You could try launching Chrome with the --force-renderer-accessibility flag in the one where you were having problems with UIAViewer, in case the Chromium activation method had failed somehow. Also make sure that you are holding the mouse over the window long enough so UIAViewer has time to activate Chromium - sometimes it takes a little bit of time (a few seconds usually).

[fenchai]

@fenchai browser automation with UIA is highly unreliable, because in addition to webpages user interface changing, also the browsers UIA tree tends to change with browser updates. I've needed to adapt and change UIA v2 UIA_Browser multiple times because of this. I am not updating v1 UIA_Interface any longer, so you can expect UIA_Browser to break as well (and it probably already has). I periodically update v2 UIA_Browser, but it's a losing battle as well...

As to why you are seeing a difference between computer 1 and 2, I can't say. You could try launching Chrome with the --force-renderer-accessibility flag in the one where you were having problems with UIAViewer, in case the Chromium activation method had failed somehow. Also make sure that you are holding the mouse over the window long enough so UIAViewer has time to activate Chromium - sometimes it takes a little bit of time (a few seconds usually).

yeah I tried --force-renderer-accessibility and checked with chrome://accessibility/ that accesibility settings where checked and it also did not work. This is weird because on the working computer, no accesibility is needed and it still works... Looks like there is nothing I can do as not even UIAViewer (v2) works in this computer, I will have to use another browser automator as an alternative, can you recommend me one? thanks!

I guess if no better options I'm going to have to rely on python. nvm used rufaydium, recommended for browser automation, relying on python was slower on start load, rufaydium is crazy fast.

[wpb]

@Descolada , as always thank you for the UIA library.

I think there may be an error in Example07_FocusChangedEvent.ahk, since the variable cEl is never used.

[Descolada]

@wpb fixed.

@fenchai Rufaydium is a good option. Another is Chrome.ahk which can be used without installing anything extra, but requires Chrome/Edge to be ran in debugging mode. It should be as fast as Rufaydium.