Page 26 of 29
Re: UIAutomation with a focus on Chrome
Posted: 11 Jul 2023, 14:34
by Loop
Hello, I always want to click on the last entry. The problem is, AutomationId='one-column-item-22 is sometimes 0 and sometimes 22. So it keeps changing. How can I find out the last AutomationId?
el.FindFirstBy("ControlType=Button AND AutomationId='one-column-item-22'").Click()
Thanks!
Re: UIAutomation with a focus on Chrome
Posted: 11 Jul 2023, 23:25
by Descolada
@Loop, one way would be to use a TreeWalker to get the last element with a condition that matches for substrings (flag=2):
Code: Select all
foundEl := UIA.CreateTreeWalker("ControlType=Button AND AutomationId='one-column-item' FLAGS=2").GetLastChildElement(el)
Another way would be to use that same condition but with FindFirstWithOptions, using LastToFirst traversal option:
Code: Select all
foundEl := el.FindFirstWithOptions(, "ControlType=Button AND AutomationId='one-column-item' FLAGS=2", 2, el)
If you were to use AHK v2 with UIA-v2, then it would be
foundEl := el.FindElement({Type:"Button", AutomationId:"one-column-item", matchmode:"SubString", index:-1})
Re: UIAutomation with a focus on Chrome
Posted: 04 Aug 2023, 14:35
by Gary-Atlan82
@Descolada
UIAutomation SetFocus() is not working for me
In the following example, the
.Highlight() line works fine but the
.SetFocus() line does not. The window get activated but nothing else happens.
UIA cant set the values in this program so I am trying to do it manually myself but I first need to activate the edit field:
Code: Select all
UIA := UIA_Interface()
el := WinExist("Pencil Tool Properties ahk_exe PDFXEdit.exe ahk_class UIX:Window")
WinActivate, ahk_id %el%
WinWaitActive, ahk_id %el%
el := UIA.ElementFromHandle(el)
; el.FindFirstBy("ControlType=Text AND Name='6 pt' AND AutomationId='1@'").Highlight()
el.FindFirstBy("ControlType=Text AND Name='6 pt' AND AutomationId='1@'").SetFocus()
Note: the width property has to have "6" for UIA to find it.
Thanks for any help.
Re: UIAutomation with a focus on Chrome
Posted: 05 Aug 2023, 05:18
by Descolada
@Gary-Atlan82, yeah, for some reason SetFocus doesn't work. I changed the code to target the parent of the Text element (a ListItem type element) and instead Clicked it, which seems to focus it. Then after clicking the UIA tree updates with a new Combobox element that has an "Open" Button element to reveal the pencil size selection menu.
I also changed the way the pencil element is found by using
Name=' pt' along with matchmode=2, so the pencil size element can be found independent of the currently selected value.
Code: Select all
el.FindFirstBy("ControlType=ListItem AND Name=' pt' AND AutomationId='2@'",,2).Click()
; el.FindFirstBy("Type=Button AND Name='Open'").Click() ; opens the combo-box
Re: UIAutomation with a focus on Chrome
Posted: 05 Aug 2023, 07:53
by Gary-Atlan82
Aah! even better. Thanks so much man
Re: UIAutomation with a focus on Chrome
Posted: 11 Aug 2023, 20:03
by Gary-Atlan82
@Descolada
UIAutomation: How to perform Invoke() on a tray icon while the taskbar is hidden?
I am trying to write a script using UIAutomation, that will click on a icon present in the tray bar (right side of the taskbar).
Most of the time, my taskbar is hidden but I found the
Invoke() button in the UIA Viewer window (Found in "UIAutomation Element Patterns" sidebar) works even when the taskbar is hidden!
I am having issue actually invoking this
Inokve() pattern in a AHK script, I have tried:
Code: Select all
UIA := UIA_Interface()
WinID := WinExist("ahk_class Shell_TrayWnd ahk_exe Explorer.EXE")
TaskBar := UIA.ElementFromHandle(WinID)
; TransMisson := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").ControlClick() ;This seems to be a standard AHK mousemove with Click function?
TransMisson := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").LegacyIAccessiblePattern.DoDefaultAction() ;Does Nothing, no mouse movement observed
; TransMisson := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").Click() ;Does Nothing either
What I am expecting to happen is the window being brought to focus, same thing as clicking on the icon.
Also when
.Click() is used without any parameters is it performing one of the AHK mouse clicks or is it doing a UIA method? What method is actually doing the
Invoke() pattern?
Any help would be greatly appreciated!
Re: UIAutomation with a focus on Chrome
Posted: 12 Aug 2023, 02:07
by Descolada
@Gary-Atlan82, Element.Click() will use one of the UIA methods and the first one it tries is Invoke. You can also specifically call Invoke from the InvokePattern by using Element.InvokePattern.Invoke(). Element.ControlClick() will use AHK ControlClick function to click it, and Element.Click("left") will use AHK Click function.
If you were to switch to AHK v2 and UIA-v2 then Element.Click() works the same, but for InvokePattern you could simply use Element.Invoke() because the pattern is auto-detected.
Re: UIAutomation with a focus on Chrome
Posted: 12 Aug 2023, 18:25
by Gary-Atlan82
@Descolada
I was not able to get
Element.InvokePattern.Invoke() to work but this mostly likely due to me hiding the taskbar though, the following works:
Code: Select all
#SingleInstance, force
SetBatchLines, -1
Process, Exist, transmission-qt.exe
If !(ErrorLevel){
Run, "C:\Program Files\Transmission\transmission-qt.exe"
WinWait , ahk_exe transmission-qt.exe,, 1
WinActivate, ahk_exe transmission-qt.exe,, 1
WinWaitActive, ahk_exe transmission-qt.exe,, 1
}
Else If WinExist("ahk_exe transmission-qt.exe"){
WinActivate, ahk_exe transmission-qt.exe,, 1
WinWaitActive, ahk_exe transmission-qt.exe,, 1
}
Else{
If !WinExist("ahk_class Shell_TrayWnd"){
WinShow, ahk_class Shell_TrayWnd
WinShow, ahk_class Shell_SecondaryTrayWnd
TaskbarIsShown := 1
}
UIA := UIA_Interface()
WinID := WinExist("ahk_class Shell_TrayWnd ahk_exe Explorer.EXE")
WinID := WinExist(" ahk_exe Explorer.EXE ahk_class Shell_TrayWnd")
TaskBar := UIA.ElementFromHandle(WinID)
Trans := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").Click()
If (TaskbarIsShown){
WinHide, ahk_class Shell_TrayWnd
WinHide, ahk_class Shell_SecondaryTrayWnd
}
WinActivate, ahk_exe transmission-qt.exe,, 1
WinWaitActive, ahk_exe transmission-qt.exe,, 1
}
ExitApp
If you are thinking "All of this to bring up a window", sadly yes. Its a broken program (maybe broken by the Win11 taskbar) and I have made many thread asking for help about it. Until now I have not been able to solve it.
With UIA it was a piece of work. Thank you for writing this library, I plan to do so much with it! and your clarification with
Invoke() will help allot going forward. Cheers!
Re: UIAutomation with a focus on Chrome
Posted: 13 Aug 2023, 17:16
by Gary-Atlan82
@Descolada
How can I use the RegEX option to solve this problem?
So the program I wrote in the above post yesterday broke today. The issue when there is a file being downloaded, it will show the download speed in the tray, such as
598 kB/s ▾ 0 kB/s ▴. This also apllies to the control name in UIA.
I want to build a script that will send a click to the tray Icon whether its name is
Idle or something like
598 kB/s ▾ 0 kB/s ▴. I have tried the following but its not working:
Code: Select all
UIA := UIA_Interface()
WinID := WinExist(" ahk_exe Explorer.EXE ahk_class Shell_TrayWnd")
MsgBox, % WinId
TaskBar := UIA.ElementFromHandle(WinID)
; TransMission := TaskBar.FindFirstBy("ControlType=Image").Click()
; TransMission := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").Click()
; TransMission := TaskBar.FindFirstBy("ControlType=Button AND Name='▾| Idle' AND AutomationId='NotifyItemIcon'",, RegEx).Click()
; TransMission := TaskBar.FindFirstBy("ControlType=Button AND Name='kb| Idle' AND AutomationId='NotifyItemIcon'",, RegEx).Click()
TransMission := TaskBar.FindFirstBy("'▾| Idle'",, RegEx).Click()
I am using the RegEx option the right way?
Thanks for any help.
Re: UIAutomation with a focus on Chrome
Posted: 13 Aug 2023, 23:41
by Descolada
@Gary-Atlan82, you almost got it, but you are using
RegEx as a variable, not as a string-type option.
TransMission := TaskBar.FindFirstBy("'▾| Idle'",, "RegEx").Click() is correct, or if you do actually want to use a variable then:
Code: Select all
RegEx := "RegEx"
TransMission := TaskBar.FindFirstBy("'▾| Idle'",, RegEx).Click()
Re: UIAutomation with a focus on Chrome
Posted: 14 Aug 2023, 09:02
by Gary-Atlan82
@Descolada Thanks for the help!
I got skewered by such a Noobie mistake. Well I am Noob but AHK has one acomplishing feats so fast, one starts to feel like a level 99 hacker.
I appreciate the varied examples of Var vs String.
Anyways, I want to confirm following works for me:
Code: Select all
UIA := UIA_Interface()
WinID := WinExist(" ahk_exe Explorer.EXE ahk_class Shell_TrayWnd")
TaskBar := UIA.ElementFromHandle(WinID)
Trans := TaskBar.FindFirstBy("ControlType=Button AND Name='/s|Idle' AND AutomationId='NotifyItemIcon'",, "RegEx").Click()
Re: UIAutomation with a focus on Chrome
Posted: 19 Aug 2023, 11:01
by songdg
Thanks again, can UIA_Browser connect to an already existing Chrome tab?
Re: UIAutomation with a focus on Chrome
Posted: 19 Aug 2023, 11:14
by Descolada
@songdg, yes and no. With UIA_Browser you can connect to a *window* with new UIA_Browser(WinTitle), which might be the same thing as the tab you want to connect to. If that Chrome window has the specific tab you wish to connect to, then you can use cUIA.SelectTab to switch to it.
But, I am not actively updating UIA_Browser nor UIA_Inteface for v1, and since browser automation with UIA is tricky and browsers tend to be updated, then it might stop working at some time (if it hasn't already). I recommend switching to AHK v2 and UIA.ahk if at all possible.
Re: UIAutomation with a focus on Chrome
Posted: 19 Aug 2023, 12:38
by songdg
@Descolada, thanks for your reply, I currently use Chrome.ahk to scrape some information from a website, but with little knowledge of javascript, sometime it's very difficult for me to use Chrome.ahk to click a button or fill an edit, yet use UIAutomation is quite easy to do that, so I want to combine them to do my job. Unfortunately Chrome.ahk no longer update, I wonder if UIA.ahk@V2 will be easy to perform simple scrape jobs if possible.
Re: UIAutomation with a focus on Chrome
Posted: 25 Aug 2023, 10:30
by Gary-Atlan82
@Descolada
Search for a element and return the Hwnd of its Window?
I want to heavily utilise AutoHotkey with Adobe Illustrator, unfortunately, aside from the main window and document windows, Illustrator "Panels" windows do not have window titles at all, every "Panel" window has only the following information:
Code: Select all
ahk_class OWL.Dock
ahk_exe Illustrator.exe
ahk_pid 27784
I cant use any of the cool Functions/Libraries I have, If I cant tell what's what.
What I am trying to do
At first launch of the program or any change of "WorkSpace", run AHK script, that will look for windows that do not have titles and set them, via
WinsetTitle, so the Layers window when first created goes from:
Code: Select all
ahk_class OWL.Dock
ahk_exe Illustrator.exe
ahk_pid 27784
to
Code: Select all
Layers
ahk_class OWL.Dock
ahk_exe Illustrator.exe
ahk_pid 27784
As for the problem of identifying which window is which, I think UIA can help with this, because it can actually get allot of information from Illustrator:
I want UIA to go through the Illustrator application tree and find all the
tab item "Layers" or
tab item "Navigator" and return a series of
Hwnd for the windows that elements belongs to.
If two or more
tab item "..." belong to one window, such as in
THIS example, that is fine as I want to do something like this anyways:
Code: Select all
Layers | Transparency | Document Info | Info
ahk_class OWL.Dock
ahk_exe Illustrator.exe
ahk_pid 27784
I only know abit about UIA, certainly not enough to recurse a series of windows. Any help or pointers would be really appreciated. Thanks.
Re: UIAutomation with a focus on Chrome
Posted: 25 Aug 2023, 10:52
by Descolada
@Gary-Atlan82, if you are dealing with separate windows (not controls), then looping over them shouldn't be too difficult, something like this:
Code: Select all
#include <UIA_Interface>
UIA := UIA_Interface()
WinGet, wList, List, ahk_class OWL.Dock ahk_exe Illustrator.exe
Loop %wList%
{
wId := wList%A_index%
WinGetTitle, wTitle, ahk_id %wId%
if (wTitle)
continue
winEl := UIA.ElementFromHandle("ahk_id" wId)
if (foundEl := winEl.FindFirst("Name='Layers' OR Name='Navigator' AND Type=TabItem")) { ; multiple and/or statements are interpreted left-to-right, so this is (Name='Layers' OR Name='Navigator') AND Type=TabItem
WinActivate, ahk_id %wId%
foundEl.Highlight()
MsgBox, % "Window with winId " wId " contains TabItem Layers or Navigator"
;WinSetTitle, ahk_id %wId%,, Layers
}
}
If you need access to a control that is embedded in a window, then you'd need to loop the windows and for every window loop over WinGet ControlList for all controls, and run an UIA search on those.
Re: UIAutomation with a focus on Chrome
Posted: 25 Aug 2023, 12:27
by Gary-Atlan82
@Descolada
Amazing, you make it look so easy, that OR operator(?) looks super handy, I will cherish your example and apply it to Photoshop next. Cheers as always!
Re: UIAutomation with a focus on Chrome
Posted: 25 Aug 2023, 17:34
by Gary-Atlan82
@Descolada
Retreive all the "name" values of a control type in one query?
I got your solution working!
I am trying to emulate a feature I use all the time in FindText,
Ok.[1].id but I cant seem to figure it out in UIA.
Given a illustrator window like the one in the picture, that has three tabs (Layers, Info and Artboards), is it possible to make a single query to UIA like:
Code: Select all
UIA := UIA_Interface()
AI := UIA.ElementFromHandle(0x4b1b18) ;Lets say its the panels Window ID
Var := AI.FindAllByName("ControlType=TabItem")
Somewhere in
Var, there should be
Artboards ,
Layers and
Info.
Var can be an array or even one long string.
I have tried to see what is actually inside the
AI/
Var object in VsCode debugger, its just one deeply nested rabbit hole.
If I can do this it would immensely simplify many tasks I have in mind for UIA
Re: UIAutomation with a focus on Chrome
Posted: 26 Aug 2023, 00:55
by Descolada
@Gary-Atlan82, you can do that in almost the same way as in the last example, but instead of FindFirst use FindAll:
Code: Select all
winEl := UIA.ElementFromHandle("ahk_id" wId)
foundEls := winEl.FindAll("Name='Layers' OR Name='Info' OR Name='Artboards' AND Type=TabItem")
if (foundEls.MaxIndex()) {
WinActivate, ahk_id %wId%
foundEls[1].Highlight()
MsgBox, % "Window with winId " wId " contains element " foundEls[1].Dump()
;WinSetTitle, ahk_id %wId%,, Layers
}
Re: UIAutomation with a focus on Chrome
Posted: 26 Aug 2023, 08:21
by Gary-Atlan82
@Descolada
Amazing, thank you. Cannot wait to test this as soon as I get home.