UIA v2

[wpb]

@Descolada , thanks for the swift response.

I was actually wondering if it was possible to trigger the focus change event only on focus changes to cEl or its descendants, rather than on the whole tree. I thought perhaps that was the purpose of getting cEl in the example?

[k0stell0]

Hi @Descolada

I'm stuck trying to access a menu in Davinci Resolve. Please see the code below. I used this method multiple times before, but this particular menu gives me "No matching window found" error.
It looks like I'm missing something obvious. I would highly appreciate if you could point me in the right direction.
Thanks in advance.

Code: Select all

DR := UIA.ElementFromHandle(WinExist('DaVinci Resolve'))
DR.FindElement({ AutomationId: 'UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Browse.m_pBrowsePanel.frameMediaPageWidgetsContainer.frameMediaPageMediaPool.frameMultipleBinGridLayoutContainer.frameMultipleBinLayoutTopLeft.frameTopLeftContent.m_MediaPoolWidget.frameContainer.frameMediaPoolTitleBars.frameMediaPoolTitleBarTop.btnMediaPoolOptions' }).Click()
contextMenu := UIA.ElementFromHandle(WinExist('ahk_class Qt5152QWindowPopupDropShadowSaveBits'))

2024-04-04_14-58-51.jpg (52.95 KiB) Viewed 724 times

[Descolada]

@k0stell0 in this case AHK is probably running faster than Davinci: AHK calls Click() and then immediately checks WinExist, but Davinci might not had time to create the menu window yet. Try adding a WinWait call before WinExist.

[k0stell0]

Thanks for your prompt reply @Descolada

I tried adding WinWait, but it didn't work. So I decided to try a simple 5 sec Sleep timer, but it was the same.
The context menu opens up and I can see it, unfortunately, I'm still getting "No matching window found" error.
Is there any other workaround I could try? As I'm running out of ideas.

Thanks again.

[Descolada]

@k0stell0 when I tested it in Media tab then the script froze after Click() and resumed after the tooltip menu closed (and then throwing the error). This is a known problem with Davinci, as it seems to have a bug in the UIAutomation interface implementation. The following worked in my setup:

Code: Select all

#Requires AutoHotkey v2
#Include <UIA>

DR := UIA.ElementFromHandle('DaVinci Resolve')
DR.FindElement({ AutomationId: 'UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Browse.m_pBrowsePanel.frameMediaPageWidgetsContainer.frameMediaPageMediaPool.frameMultipleBinGridLayoutContainer.frameMultipleBinLayoutTopLeft.frameTopLeftContent.m_MediaPoolWidget.frameContainer.frameMediaPoolTitleBars.frameMediaPoolTitleBarTop.btnMediaPoolOptions' }).ControlClick()
contextMenu := UIA.ElementFromHandle(WinWait('ahk_class Qt5152QWindowPopupDropShadowSaveBits'))
MsgBox contextMenu.DumpAll()

[k0stell0]

Thanks a lot @Descolada
Your code works for me too. The issue was that I used Click() instead of ControlClick().
I still don't fully understand the difference between them, but at least it's working now.

[MerlinSilk]

So glad that I found this library, and I am still very new at it and have many questions. So, the following might be a bloody newbie problem: I had been looking for a more reliable way to automate Acrobat XI, in particular, popping up the palette with stamps (Comments >> Annotations >> Click Add Stamp icon >> Show Stamps Palette). Investigating with UIA Viewer does not seem to help as I can not get into the group where the Comment button lives.
Further down I do find the 'menu item "Stamps"' and the Macro Creator gives me

Code: Select all

AcrobatEl := UIA.ElementFromHandle("Adobe Acrobat Pro ahk_exe Acrobat.exe")
AcrobatEl.ElementFromPath("YY/YYQOQQB").Click()

But running this does not give me the expected dropdown menu (from which I could then get the desired result with sending "s".
What am I doing wrong

Gezi

[Descolada]

@MerlinSilk do you by chance have a link for Adobe XI download so I could test it out?

Otherwise, some things to check:
1. Does AcrobatEl.ElementFromPath("YY/YYQOQQB").Highlight() highlight the correct element?
2. In UIAViewer, does the element have any "clickable" patterns available (eg Invoke, MenuItem). Try expanding the patterns and double-click Invoke()/Select()/Expand() or something similar
3. Some programs report that they implement patterns, but in actuality they don't. In that case you could try AcrobatEl.ElementFromPath("YY/YYQOQQB").Click("left") or AcrobatEl.ElementFromPath("YY/YYQOQQB").ControlClick()

[jdp]

I'm having trouble with UIA and can't seem to figure out why.

I have an element which i dump() info from into a variable and on the same page click("Left") using automation ID. I keep getting invalid index errors (see screenshot)

image.png (20.34 KiB) Viewed 606 times

Code: Select all

*#1 (1949) : [UIA.IUIAutomationElement.Prototype.ElementFromPath] Throw IndexError(StrReplace(err.Message, "at index", "at path index"), -1, err.Extra)
*#1 (5258) : [<Hotkey>] msedgeEl.ElementFromPath("Y/YYY/YqYYYYYYVRRsRR").Click("left")
> :?:dn
*#1 (5370) : [] Loop Read A_WorkingDir "\Wards with orders.csv"
> Auto-execute

A snippet of the code:

Code: Select all

		datebtn := UIA.ElementFromHandle(WinActivate("CareFlow Medicines Management"))
		datebtn.ElementFromPath("Y/YYY/YqYYYYYVRRs3").Click("left")
		Send FormatTime(A_Now, "dd-MMM-yyyy")

The strange thing is - if I reload the script (i have esc mapped as reload) it works. It doesn't seem to matter if the target window is open at the time of reload, but reloading the entire script makes it work flawlessley. To then get the error to reproduce again, you need to logout of the computer and log back in. Then again, upon calling the hotkey the error throws, click reload and then try the hotkey again and it works!

Any help would be much appreciated

Cheers

[Descolada]

@jdp since I can't access the program you are using, you need to do some debugging yourself. Try logging out of the computer, start the CareFlow program, and then run

Code: Select all

A_Clipboard := UIA.ElementFromHandle(WinActivate("CareFlow Medicines Management")).DumpAll()

paste the output in Notepad and check whether the element you are looking for exists.
Then run the same script again, and check whether the element exists and whether the path is the same. Perhaps there is some kind of problem with UIA activation that it fails on the first time?

By the way, the UIA path you are using looks like something from a Chromium (browser-based) app. The thing with Chromium apps is that they don't have UIA activated by default and UIA.ahk forces the app to activate it. The default timeout or wait value for the activation is 500 milliseconds, but it might sometimes take longer if the webpage is complex (as yours seems to be). You could try datebtn := UIA.ElementFromHandle(WinActivate("CareFlow Medicines Management"),, 2000) to see if a larger timeout value makes a difference.

[emp00]

I am experiencing an issue to the UIA_Browser method "Reload()". My very simple script below is supposed to press the browser's Reload button - script is activated via a hotkey in my main AHK script. I'm using the Vivaldi browser. I explicitly would like to initiate "Reload()" and not an "F5" keypress because this is a workaround for "crashed" browser tabs where "F5" does not work but clicking "Reload" does work. Instead of clicking I would like to do this via a simple hotkey.

Symptom: The script below does not press the "Reload" but instead the "Home" button. Is this my fault, any ideas? Thanks guys.

Code: Select all

#Requires AutoHotkey v2
#SingleInstance Force
#include Libs\UIA-v2-main\Lib\UIA.ahk
#include Libs\UIA-v2-main\Lib\UIA_Browser.ahk
If !WinWaitActive("ahk_exe vivaldi.exe",, 1)
    ExitApp()
SoundBeep(750, 75)
cUIA := UIA_Browser()  ; Initialize UIA_Browser, use Last Found Window (returned by WinWaitActive)
cUIA.Reload() ; UIA-v2 UIA_Browser -- Reload() presses the Reload button https://github.com/Descolada/UIA-v2/wiki/12.-UIA_Browser#reload
Sleep(500)
ExitApp

[gongnl]

I run UIA_Browser_Example03_Login.ahk of UIA2 on my computer(Win7 x64 , vivaldi),the following error appears. What might be the reason for this and how can it be resolved? Thank you very much

image.png (64.56 KiB) Viewed 533 times

[Descolada]

@emp00 I was not aware that Vivaldi had a Home button. Unfortunately that means there is no reliable way to get the navigation buttons in Vivaldi, as they have no discernible characteristics (besides the Name property, but that is locale-specific and will change if the user uses another language). You can manually fix it by changing the -3 to -4 in this.ReloadButton := this.URLEditElement.WalkTree("-3", {Type:"Button"}).

@gongnl the oldest Windows version I test UIA_Browser in is Windows 10, so unfortunately I can't help you there. Windows 7 is deprecated and is missing some newer UIAutomation methods and properties, and I'm not sure whether UIA_Browser uses any of those.
You could try inspecting the tab elements with UIAViewer and see whether the AutomationId starts with "tab-" (as it does in Windows 10+), and modify that line as needed.

[emp00]

@emp00 I was not aware that Vivaldi had a Home button. Unfortunately that means there is no reliable way to get the navigation buttons in Vivaldi, as they have no discernible characteristics (besides the Name property, but that is locale-specific and will change if the user uses another language). You can manually fix it by changing the -3 to -4 in this.ReloadButton := this.URLEditElement.WalkTree("-3", {Type:"Button"}).

Understood - you're using WalkTree, that's the reason! Actually, as you know, Vivaldi is highly customizable and those buttons (and much more) can be shifted to various other positions. Probably I have changed the default button configuration, see screenshot. You can also see the AccessibilityInsights output for Vivaldi's reload button. I understand that the name property might be locale specific - that's why you don't want to use it. Maybe you can find another unique id in the output?

For the time being, I have moved my reload button +1 to the right (next to the URL bar) and for now the script works... I really wonder why such standard buttons don't have distinct AutomationIds? This would solve many of our issues... It's a shame - probably Vivaldi cannot change this because they're relying on the chromium engine code??

[gongnl]

@Descolada When chrome/vivaldi opens an url, how does uia2 get the url Document Loading Completed event and call the event handler function? Thank you very much

[Descolada]

@gongnl try this:

Code: Select all

#Requires AutoHotkey v2
#include UIA.ahk

browser := UIA.ElementFromHandle("ahk_exe vivaldi.exe")
handler := UIA.CreateAutomationEventHandler(AsyncContentLoadedEvent)
UIA.AddAutomationEventHandler(handler, browser, UIA.Event.AsyncContentLoaded)
Persistent

AsyncContentLoadedEvent(documentEl, eventId) {
    url := ""
    try url := documentEl.Value
    if !url
        return
    ToolTip "Webpage loaded: " url
    SetTimer ToolTip, -3000
}

[gongnl]

@Descolada The script is working fine, thank you very much for your help, and two more questions for you,
1.How to find hidden element <INPUT type=hidden name=isWriteOff value=N> with UIA2?
2.How to convert the script "document.parentWindow.execScript("function confirm(){return true}")" to run with uia2?

[Descolada]

@gongnl
1. Try using DumpAll() on the website and see whether the output contains something resembling the hidden element (eg an element with the name "isWriteOff"). UIA is mostly meant for visible UI elements so a hidden one might not be detectable though, in which case you'd probably need to inject Javascript to interact with it.
2. That looks like Javascript? In that case you could use UIA_Browser.ahk JSExecute method, or I have a more optimized one for Chrome (which could be modified for other browsers as well):

Code: Select all

#Requires Autohotkey v2.0
#SingleInstance
#Include UIA.ahk

F1::{
    ChromeExecuteJS('alert("Hello")')
}

F2::{
    ChromeExecuteJS('document.parentWindow.execScript("function confirm(){return true}")')
}

ChromeExecuteJS(js, WinTitle := "Google Chrome ahk_exe chrome.exe") {
    if !(hWnd := WinExist(WinTitle))
        throw TargetError("No matching window found", -1)
    ChromeNavigate("javascript:" js, WinTitle)
}

ChromeNavigate(url?, WinTitle := "Google Chrome ahk_exe chrome.exe") {
    static cache := Map()
    if !(hWnd := WinExist(WinTitle))
        throw TargetError("No matching Chrome window found", -1)
    ActivateChromiumAccessibility(hWnd)

    if !cache.Has(hWnd) {
        try cache[hWnd] := UIA.ElementFromHandle(hWnd).FindElement({Type:"Edit", index:-1})
        catch
            throw TargetError("Failed to get the address bar element", -1)

        for k in [cache*]
            if !WinExist(k)
                cache.Delete(k)
    }

    if !IsSet(url)
        return

    try cache[hWnd].Value := url " "
    catch
        Error("Failed to set the URL value", -1)

    prevDelay := A_KeyDelay, prevDuration := A_KeyDuration
    SetKeyDelay -1, 1
    ControlSend("{LCtrl up}{LAlt up}{LShift up}{RCtrl up}{RAlt up}{RShift up}{Enter}", , hWnd)
    SetKeyDelay prevDelay, prevDuration
}

ActivateChromiumAccessibility(WinTitle := "Google Chrome ahk_exe chrome.exe") {
    static SWP_NOREDRAW := 0x0008, SWP_NOSIZE := 0x0001, SWP_NOMOVE := 0x0002, SWP_NOACTIVATE := 0x0010
    hWnd := IsInteger(WinTitle) ? WinTitle : WinExist(WinTitle)
    if !IsWindowVisible() || !(cHwnd := TryGetChromiumHwnd()) {
        list := WinGetList(), index := 0
        for i, win in list
            if win = hWnd {
                index := i
                break
            }

        active := WinExist("A")
        tp := WinGetTransparent(hWnd)
        prevDelay := A_WinDelay
        SetWinDelay -1
        WinSetTransparent(0, hWnd)
        ; Bring to front
        DllCall("SetWindowPos", "ptr", hWnd, "ptr", 0, "int", 0, "int", 0, "int", 0, "int", 0, "int", SWP_NOMOVE | SWP_NOSIZE | SWP_NOREDRAW)
        WinActivate(hWnd)
        WinActivate(active)
        end := A_TickCount + 100
        while (A_TickCount < end && !(cHwnd := TryGetChromiumHwnd()))
            Sleep -1
        if index > 1
            DllCall("SetWindowPos", "ptr", hWnd, "ptr", list[index-1], "int", 0, "int", 0, "int", 0, "int", 0, "int", SWP_NOMOVE | SWP_NOSIZE | SWP_NOREDRAW | SWP_NOACTIVATE)
        WinSetTransparent(tp, hWnd)
        SetWinDelay prevDelay
    }
    return chWnd

    TryGetChromiumHwnd() {
        try return ControlGetHwnd("Chrome_RenderWidgetHostHWND1", hWnd)
        return 0
    }

    IsWindowVisible() {
        WinGetPosEx(hWnd, &x, &y, &w, &h)
        l := Max(0, x), t := Max(0, y), r := Min(x+w-1, A_ScreenWidth), b := Min(y+h-1, A_ScreenHeight)
        if !(hWnd = UIA.WindowFromPoint(l, t) || hWnd = UIA.WindowFromPoint(r, t) || hWnd = UIA.WindowFromPoint(l, b) || hWnd = UIA.WindowFromPoint(r, b))
            return 0
        return 1
    }

    ; https://www.autohotkey.com/boards/viewtopic.php?t=114183&p=508461
    WinGetPosEx(hWindow, &X := "", &Y := "", &W := "", &H := "") {
        static S_OK := 0x0, DWMWA_EXTENDED_FRAME_BOUNDS := 9
        RECTPlus := Buffer(24,0)
        try {
            DWMRC := DllCall("dwmapi\DwmGetWindowAttribute",
                                "Ptr",  hWindow,                     ;-- hwnd
                                "UInt", DWMWA_EXTENDED_FRAME_BOUNDS, ;-- dwAttribute
                                "Ptr",  RECTPlus,                    ;-- pvAttribute
                                "UInt", 16,                          ;-- cbAttribute
                                "UInt")
        } catch
        return False

        X := NumGet(RECTPlus,  0, "Int"), Y := NumGet(RECTPlus,  4, "Int")
        , R := NumGet(RECTPlus,  8, "Int"), B := NumGet(RECTPlus, 12, "Int")
        , W := R - X, H := B - Y
    }
}

[gongnl]

@Descolada Thank you very much. To add to the second question, there is a button on the web page，when clicked the button ,a confirmation box will pop up. When opening the web page with IE browser, I can use ' wb.document.parentWindow.execScript("function confirm(){return true}") ' to automatically confirm and close the confirmation box. Now use the vivaldi browser to open the web page, how to automatically confirm and close the pop-up confirmation box?

Code: Select all

Gui, Font, s15
Gui, Add, ActiveX, w600 h300 vwb, Shell.Explorer
Gui, Add, Button, w200, Test
Gui, Show

wb.Navigate("about:tabs")
wb.document.write( html() )
Return

ButtonTest:
	wb.document.parentWindow.execScript("function confirm(){return true}")
	wb.document.getElementById("saveButton").click()
return

GuiClose:
ExitApp

html() {
	html =
	(
		<html>
			<script>
				function save() {
					if(confirm("Confirm save？")){
						document.getElementById('result').innerText = "success";
					}
				}
			</script>
			
			<input type=button id="saveButton" onclick="save()" value="Ok" />
			<p>result: <span id=result></span>
		</html>
	)
	return html
}

[shipaddicted]

I'm using UIA to navigate Salesforce, and for the first time EVER with Salesforce, I'm actually having a lot of success! However, I'm stuck on the script below.

I'm trying to automate closing a case, and the chosen ListItem value in the second combobox depends on what the ListItem value is in the first combobox. I get the value of the first one and use it in a Switch statement for the second. However, when I run the script, it just pretends to perform an action from the Switch statement and moves on to finish the script.

Thinking that the menu must somehow be getting 'closed' while the Switch statement is happening, I tried moving the "Case Closed Reason" combobox invoke statement to inside that case -- but the same thing happened.

I have tried adding sleeps around all the commands for the 2nd combobox -- made absolutely no difference.

However -- if I run the script again immediately, the script works exactly as desired. How can I get it to run right the first time??

Code: Select all

!c::	;close Overdue case - paid 
{
	cUIA := ""
	cUIA := UIA_Browser("| Case | Salesforce")
	y := cUIA.GetCurrentDocumentElement() 
	;'open' case to edit 
	c := y.FindElement({Type:"button", Name:"Edit Status"}).Click()

	;get value of first combobox & then change it to 'closed'
	status := y.WaitElement({Type:"ComboBox", Name:"Status - Current Selection",cs:0,mm:2})
	statuss := status.Value 
	status.Invoke() 
	y.WaitElement({Type:"ListItem", Name:"Closed"}).Invoke() 
	
	;'open' 2nd combobox & select option based on value of 1st combobox 
	y.WaitElement({Type:"ComboBox", Name:"Case Closed Reason",cs:0,mm:2}).Invoke() 
	Switch statuss
	{
		case "Stage 1": y.WaitElement({Type:"ListItem", Name:"Paid Stage 1"}).Invoke() 
		case "Stage 2": y.WaitElement({Type:"ListItem", Name:"Paid Stage 2"}).Invoke() 
		case "Stage 3": y.WaitElement({Type:"ListItem", Name:"Paid Stage 3"}).Invoke() 
		case "Stage 4": y.WaitElement({Type:"ListItem", Name:"Paid Stage 4"}).Invoke() 
		case "Stage 5": y.WaitElement({Type:"ListItem", Name:"Paid Stage 5"}).Invoke() 
		case "Stage 6": y.WaitElement({Type:"ListItem", Name:"Paid Stage 6"}).Invoke() 
	}
	MsgBox "done"
	tog := y.FindElement({Type:"Checkbox", Name:"Do Not Send Case Closed Email"})
	If (tog.toggleState = "0")
	{
		tog.Toggle() 
	}
	cUIA := ""
}