UIAutomation with a focus on Chrome
Re: UIAutomation with a focus on Chrome
Hello, I always want to click on the last entry. The problem is, AutomationId='one-column-item-22 is sometimes 0 and sometimes 22. So it keeps changing. How can I find out the last AutomationId?
el.FindFirstBy("ControlType=Button AND AutomationId='one-column-item-22'").Click()
Thanks!
el.FindFirstBy("ControlType=Button AND AutomationId='one-column-item-22'").Click()
Thanks!
Re: UIAutomation with a focus on Chrome
@Loop, one way would be to use a TreeWalker to get the last element with a condition that matches for substrings (flag=2):
Another way would be to use that same condition but with FindFirstWithOptions, using LastToFirst traversal option:
If you were to use AHK v2 with UIA-v2, then it would be foundEl := el.FindElement({Type:"Button", AutomationId:"one-column-item", matchmode:"SubString", index:-1})
Code: Select all
foundEl := UIA.CreateTreeWalker("ControlType=Button AND AutomationId='one-column-item' FLAGS=2").GetLastChildElement(el)
Code: Select all
foundEl := el.FindFirstWithOptions(, "ControlType=Button AND AutomationId='one-column-item' FLAGS=2", 2, el)
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada
UIAutomation SetFocus() is not working for me
In the following example, the .Highlight() line works fine but the .SetFocus() line does not. The window get activated but nothing else happens.
UIA cant set the values in this program so I am trying to do it manually myself but I first need to activate the edit field:
Note: the width property has to have "6" for UIA to find it.
Thanks for any help.
UIAutomation SetFocus() is not working for me
In the following example, the .Highlight() line works fine but the .SetFocus() line does not. The window get activated but nothing else happens.
UIA cant set the values in this program so I am trying to do it manually myself but I first need to activate the edit field:
Code: Select all
UIA := UIA_Interface()
el := WinExist("Pencil Tool Properties ahk_exe PDFXEdit.exe ahk_class UIX:Window")
WinActivate, ahk_id %el%
WinWaitActive, ahk_id %el%
el := UIA.ElementFromHandle(el)
; el.FindFirstBy("ControlType=Text AND Name='6 pt' AND AutomationId='1@'").Highlight()
el.FindFirstBy("ControlType=Text AND Name='6 pt' AND AutomationId='1@'").SetFocus()
Thanks for any help.
Re: UIAutomation with a focus on Chrome
@Gary-Atlan82, yeah, for some reason SetFocus doesn't work. I changed the code to target the parent of the Text element (a ListItem type element) and instead Clicked it, which seems to focus it. Then after clicking the UIA tree updates with a new Combobox element that has an "Open" Button element to reveal the pencil size selection menu.
I also changed the way the pencil element is found by using Name=' pt' along with matchmode=2, so the pencil size element can be found independent of the currently selected value.
I also changed the way the pencil element is found by using Name=' pt' along with matchmode=2, so the pencil size element can be found independent of the currently selected value.
Code: Select all
el.FindFirstBy("ControlType=ListItem AND Name=' pt' AND AutomationId='2@'",,2).Click()
; el.FindFirstBy("Type=Button AND Name='Open'").Click() ; opens the combo-box
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
Aah! even better. Thanks so much man
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada
UIAutomation: How to perform Invoke() on a tray icon while the taskbar is hidden?
I am trying to write a script using UIAutomation, that will click on a icon present in the tray bar (right side of the taskbar).
Most of the time, my taskbar is hidden but I found the Invoke() button in the UIA Viewer window (Found in "UIAutomation Element Patterns" sidebar) works even when the taskbar is hidden!
I am having issue actually invoking this Inokve() pattern in a AHK script, I have tried:
What I am expecting to happen is the window being brought to focus, same thing as clicking on the icon.
Also when .Click() is used without any parameters is it performing one of the AHK mouse clicks or is it doing a UIA method? What method is actually doing the Invoke() pattern?
Any help would be greatly appreciated!
UIAutomation: How to perform Invoke() on a tray icon while the taskbar is hidden?
I am trying to write a script using UIAutomation, that will click on a icon present in the tray bar (right side of the taskbar).
Most of the time, my taskbar is hidden but I found the Invoke() button in the UIA Viewer window (Found in "UIAutomation Element Patterns" sidebar) works even when the taskbar is hidden!
I am having issue actually invoking this Inokve() pattern in a AHK script, I have tried:
Code: Select all
UIA := UIA_Interface()
WinID := WinExist("ahk_class Shell_TrayWnd ahk_exe Explorer.EXE")
TaskBar := UIA.ElementFromHandle(WinID)
; TransMisson := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").ControlClick() ;This seems to be a standard AHK mousemove with Click function?
TransMisson := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").LegacyIAccessiblePattern.DoDefaultAction() ;Does Nothing, no mouse movement observed
; TransMisson := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").Click() ;Does Nothing either
Also when .Click() is used without any parameters is it performing one of the AHK mouse clicks or is it doing a UIA method? What method is actually doing the Invoke() pattern?
Any help would be greatly appreciated!
Re: UIAutomation with a focus on Chrome
@Gary-Atlan82, Element.Click() will use one of the UIA methods and the first one it tries is Invoke. You can also specifically call Invoke from the InvokePattern by using Element.InvokePattern.Invoke(). Element.ControlClick() will use AHK ControlClick function to click it, and Element.Click("left") will use AHK Click function.
If you were to switch to AHK v2 and UIA-v2 then Element.Click() works the same, but for InvokePattern you could simply use Element.Invoke() because the pattern is auto-detected.
If you were to switch to AHK v2 and UIA-v2 then Element.Click() works the same, but for InvokePattern you could simply use Element.Invoke() because the pattern is auto-detected.
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada
I was not able to get Element.InvokePattern.Invoke() to work but this mostly likely due to me hiding the taskbar though, the following works:
If you are thinking "All of this to bring up a window", sadly yes. Its a broken program (maybe broken by the Win11 taskbar) and I have made many thread asking for help about it. Until now I have not been able to solve it.
With UIA it was a piece of work. Thank you for writing this library, I plan to do so much with it! and your clarification with Invoke() will help allot going forward. Cheers!
I was not able to get Element.InvokePattern.Invoke() to work but this mostly likely due to me hiding the taskbar though, the following works:
Code: Select all
#SingleInstance, force
SetBatchLines, -1
Process, Exist, transmission-qt.exe
If !(ErrorLevel){
Run, "C:\Program Files\Transmission\transmission-qt.exe"
WinWait , ahk_exe transmission-qt.exe,, 1
WinActivate, ahk_exe transmission-qt.exe,, 1
WinWaitActive, ahk_exe transmission-qt.exe,, 1
}
Else If WinExist("ahk_exe transmission-qt.exe"){
WinActivate, ahk_exe transmission-qt.exe,, 1
WinWaitActive, ahk_exe transmission-qt.exe,, 1
}
Else{
If !WinExist("ahk_class Shell_TrayWnd"){
WinShow, ahk_class Shell_TrayWnd
WinShow, ahk_class Shell_SecondaryTrayWnd
TaskbarIsShown := 1
}
UIA := UIA_Interface()
WinID := WinExist("ahk_class Shell_TrayWnd ahk_exe Explorer.EXE")
WinID := WinExist(" ahk_exe Explorer.EXE ahk_class Shell_TrayWnd")
TaskBar := UIA.ElementFromHandle(WinID)
Trans := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").Click()
If (TaskbarIsShown){
WinHide, ahk_class Shell_TrayWnd
WinHide, ahk_class Shell_SecondaryTrayWnd
}
WinActivate, ahk_exe transmission-qt.exe,, 1
WinWaitActive, ahk_exe transmission-qt.exe,, 1
}
ExitApp
With UIA it was a piece of work. Thank you for writing this library, I plan to do so much with it! and your clarification with Invoke() will help allot going forward. Cheers!
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada
How can I use the RegEX option to solve this problem?
So the program I wrote in the above post yesterday broke today. The issue when there is a file being downloaded, it will show the download speed in the tray, such as 598 kB/s ▾ 0 kB/s ▴. This also apllies to the control name in UIA.
I want to build a script that will send a click to the tray Icon whether its name is Idle or something like 598 kB/s ▾ 0 kB/s ▴. I have tried the following but its not working:
I am using the RegEx option the right way?
Thanks for any help.
How can I use the RegEX option to solve this problem?
So the program I wrote in the above post yesterday broke today. The issue when there is a file being downloaded, it will show the download speed in the tray, such as 598 kB/s ▾ 0 kB/s ▴. This also apllies to the control name in UIA.
I want to build a script that will send a click to the tray Icon whether its name is Idle or something like 598 kB/s ▾ 0 kB/s ▴. I have tried the following but its not working:
Code: Select all
UIA := UIA_Interface()
WinID := WinExist(" ahk_exe Explorer.EXE ahk_class Shell_TrayWnd")
MsgBox, % WinId
TaskBar := UIA.ElementFromHandle(WinID)
; TransMission := TaskBar.FindFirstBy("ControlType=Image").Click()
; TransMission := TaskBar.FindFirstBy("ControlType=Button AND Name=' Idle' AND AutomationId='NotifyItemIcon'").Click()
; TransMission := TaskBar.FindFirstBy("ControlType=Button AND Name='▾| Idle' AND AutomationId='NotifyItemIcon'",, RegEx).Click()
; TransMission := TaskBar.FindFirstBy("ControlType=Button AND Name='kb| Idle' AND AutomationId='NotifyItemIcon'",, RegEx).Click()
TransMission := TaskBar.FindFirstBy("'▾| Idle'",, RegEx).Click()
Thanks for any help.
Re: UIAutomation with a focus on Chrome
@Gary-Atlan82, you almost got it, but you are using RegEx as a variable, not as a string-type option. TransMission := TaskBar.FindFirstBy("'▾| Idle'",, "RegEx").Click() is correct, or if you do actually want to use a variable then:
Code: Select all
RegEx := "RegEx"
TransMission := TaskBar.FindFirstBy("'▾| Idle'",, RegEx).Click()
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada Thanks for the help!
I got skewered by such a Noobie mistake. Well I am Noob but AHK has one acomplishing feats so fast, one starts to feel like a level 99 hacker.
I appreciate the varied examples of Var vs String.
Anyways, I want to confirm following works for me:
I got skewered by such a Noobie mistake. Well I am Noob but AHK has one acomplishing feats so fast, one starts to feel like a level 99 hacker.
I appreciate the varied examples of Var vs String.
Anyways, I want to confirm following works for me:
Code: Select all
UIA := UIA_Interface()
WinID := WinExist(" ahk_exe Explorer.EXE ahk_class Shell_TrayWnd")
TaskBar := UIA.ElementFromHandle(WinID)
Trans := TaskBar.FindFirstBy("ControlType=Button AND Name='/s|Idle' AND AutomationId='NotifyItemIcon'",, "RegEx").Click()
Re: UIAutomation with a focus on Chrome
Thanks again, can UIA_Browser connect to an already existing Chrome tab?
Re: UIAutomation with a focus on Chrome
@songdg, yes and no. With UIA_Browser you can connect to a *window* with new UIA_Browser(WinTitle), which might be the same thing as the tab you want to connect to. If that Chrome window has the specific tab you wish to connect to, then you can use cUIA.SelectTab to switch to it.
But, I am not actively updating UIA_Browser nor UIA_Inteface for v1, and since browser automation with UIA is tricky and browsers tend to be updated, then it might stop working at some time (if it hasn't already). I recommend switching to AHK v2 and UIA.ahk if at all possible.
But, I am not actively updating UIA_Browser nor UIA_Inteface for v1, and since browser automation with UIA is tricky and browsers tend to be updated, then it might stop working at some time (if it hasn't already). I recommend switching to AHK v2 and UIA.ahk if at all possible.
Re: UIAutomation with a focus on Chrome
@Descolada, thanks for your reply, I currently use Chrome.ahk to scrape some information from a website, but with little knowledge of javascript, sometime it's very difficult for me to use Chrome.ahk to click a button or fill an edit, yet use UIAutomation is quite easy to do that, so I want to combine them to do my job. Unfortunately Chrome.ahk no longer update, I wonder if UIA.ahk@V2 will be easy to perform simple scrape jobs if possible.
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada
Search for a element and return the Hwnd of its Window?
I want to heavily utilise AutoHotkey with Adobe Illustrator, unfortunately, aside from the main window and document windows, Illustrator "Panels" windows do not have window titles at all, every "Panel" window has only the following information:
I cant use any of the cool Functions/Libraries I have, If I cant tell what's what.
What I am trying to do
At first launch of the program or any change of "WorkSpace", run AHK script, that will look for windows that do not have titles and set them, via WinsetTitle, so the Layers window when first created goes from:
to
As for the problem of identifying which window is which, I think UIA can help with this, because it can actually get allot of information from Illustrator:
I want UIA to go through the Illustrator application tree and find all the tab item "Layers" or tab item "Navigator" and return a series of Hwnd for the windows that elements belongs to.
If two or more tab item "..." belong to one window, such as in THIS example, that is fine as I want to do something like this anyways:
I only know abit about UIA, certainly not enough to recurse a series of windows. Any help or pointers would be really appreciated. Thanks.
Search for a element and return the Hwnd of its Window?
I want to heavily utilise AutoHotkey with Adobe Illustrator, unfortunately, aside from the main window and document windows, Illustrator "Panels" windows do not have window titles at all, every "Panel" window has only the following information:
Code: Select all
ahk_class OWL.Dock
ahk_exe Illustrator.exe
ahk_pid 27784
What I am trying to do
At first launch of the program or any change of "WorkSpace", run AHK script, that will look for windows that do not have titles and set them, via WinsetTitle, so the Layers window when first created goes from:
Code: Select all
ahk_class OWL.Dock
ahk_exe Illustrator.exe
ahk_pid 27784
Code: Select all
Layers
ahk_class OWL.Dock
ahk_exe Illustrator.exe
ahk_pid 27784
I want UIA to go through the Illustrator application tree and find all the tab item "Layers" or tab item "Navigator" and return a series of Hwnd for the windows that elements belongs to.
If two or more tab item "..." belong to one window, such as in THIS example, that is fine as I want to do something like this anyways:
Code: Select all
Layers | Transparency | Document Info | Info
ahk_class OWL.Dock
ahk_exe Illustrator.exe
ahk_pid 27784
Re: UIAutomation with a focus on Chrome
@Gary-Atlan82, if you are dealing with separate windows (not controls), then looping over them shouldn't be too difficult, something like this:
If you need access to a control that is embedded in a window, then you'd need to loop the windows and for every window loop over WinGet ControlList for all controls, and run an UIA search on those.
Code: Select all
#include <UIA_Interface>
UIA := UIA_Interface()
WinGet, wList, List, ahk_class OWL.Dock ahk_exe Illustrator.exe
Loop %wList%
{
wId := wList%A_index%
WinGetTitle, wTitle, ahk_id %wId%
if (wTitle)
continue
winEl := UIA.ElementFromHandle("ahk_id" wId)
if (foundEl := winEl.FindFirst("Name='Layers' OR Name='Navigator' AND Type=TabItem")) { ; multiple and/or statements are interpreted left-to-right, so this is (Name='Layers' OR Name='Navigator') AND Type=TabItem
WinActivate, ahk_id %wId%
foundEl.Highlight()
MsgBox, % "Window with winId " wId " contains TabItem Layers or Navigator"
;WinSetTitle, ahk_id %wId%,, Layers
}
}
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada
Amazing, you make it look so easy, that OR operator(?) looks super handy, I will cherish your example and apply it to Photoshop next. Cheers as always!
Amazing, you make it look so easy, that OR operator(?) looks super handy, I will cherish your example and apply it to Photoshop next. Cheers as always!
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada
Retreive all the "name" values of a control type in one query?
I got your solution working!
I am trying to emulate a feature I use all the time in FindText, Ok.[1].id but I cant seem to figure it out in UIA.
Given a illustrator window like the one in the picture, that has three tabs (Layers, Info and Artboards), is it possible to make a single query to UIA like:
Somewhere in Var, there should be Artboards , Layers and Info. Var can be an array or even one long string.
I have tried to see what is actually inside the AI/Var object in VsCode debugger, its just one deeply nested rabbit hole.
If I can do this it would immensely simplify many tasks I have in mind for UIA
Retreive all the "name" values of a control type in one query?
I got your solution working!
I am trying to emulate a feature I use all the time in FindText, Ok.[1].id but I cant seem to figure it out in UIA.
Given a illustrator window like the one in the picture, that has three tabs (Layers, Info and Artboards), is it possible to make a single query to UIA like:
Code: Select all
UIA := UIA_Interface()
AI := UIA.ElementFromHandle(0x4b1b18) ;Lets say its the panels Window ID
Var := AI.FindAllByName("ControlType=TabItem")
I have tried to see what is actually inside the AI/Var object in VsCode debugger, its just one deeply nested rabbit hole.
If I can do this it would immensely simplify many tasks I have in mind for UIA
Re: UIAutomation with a focus on Chrome
@Gary-Atlan82, you can do that in almost the same way as in the last example, but instead of FindFirst use FindAll:
Code: Select all
winEl := UIA.ElementFromHandle("ahk_id" wId)
foundEls := winEl.FindAll("Name='Layers' OR Name='Info' OR Name='Artboards' AND Type=TabItem")
if (foundEls.MaxIndex()) {
WinActivate, ahk_id %wId%
foundEls[1].Highlight()
MsgBox, % "Window with winId " wId " contains element " foundEls[1].Dump()
;WinSetTitle, ahk_id %wId%,, Layers
}
-
- Posts: 74
- Joined: 07 Mar 2023, 05:20
Re: UIAutomation with a focus on Chrome
@Descolada
Amazing, thank you. Cannot wait to test this as soon as I get home.
Amazing, thank you. Cannot wait to test this as soon as I get home.
Return to “Scripts and Functions (v1)”
Who is online
Users browsing this forum: submeg and 120 guests