I made some personal modifications to UIAViewer.ahk, in this version I added:
- "Test Macro" button that runs the recorded macro
- Macro creator is now the default window
- press esc twice to exit the app: prevents exiting the gui if you press esc twice while recording
- use only window class and exe for recognition, in case there are different titles for windows, as with some text editors
- the print button now does not call the Windows screenshot
Change line 9 with your path to "UIA_Interface.ahk"
UIAutomation with a focus on Chrome
Re: UIAutomation with a focus on Chrome
- Attachments
-
- UIAViewer.ahk
- (223.79 KiB) Downloaded 307 times
Re: UIAutomation with a focus on Chrome
Hi @Descolada
Im trying to find a ListItem using the FindByPath() function, but not work.
Here is my code:
element := cUIA.FindFirstBy("Name='ds4' AND ControlType=Text")
hola := element.FindByPath("-1", "ControlType=ListItem")
Element pointer is finded correctly but the list not. The debugger error level say: "Step 1 with ""-1"" was out of bounds (GetPreviousSiblingElement failed)"
Im trying to find a ListItem using the FindByPath() function, but not work.
Here is my code:
element := cUIA.FindFirstBy("Name='ds4' AND ControlType=Text")
hola := element.FindByPath("-1", "ControlType=ListItem")
Element pointer is finded correctly but the list not. The debugger error level say: "Step 1 with ""-1"" was out of bounds (GetPreviousSiblingElement failed)"
Re: UIAutomation with a focus on Chrome
@mora145, I think it's because the ListItem is not a sibling element, but is a parent. Try instead hola := element.FindByPath("p", "ControlType=ListItem")
-
- Posts: 10
- Joined: 23 Dec 2023, 01:57
Re: UIAutomation with a focus on Chrome
Hi @Descolada
I appreciate for the powerful UIA library and make all automation flow come true,
However, I am still struggle in pick up (select) a specific list item by it.
May please guide me how to get all available list item and select on of them by name?
I try following, but could only ExpandCollapse this drop down menu and get error message as attached.
ERROR: Method Call not support by the UIA_interface
My AHK version is V1.1.37, Should I upgrade to v2?
Appreciate for your time in advance.
[Mod edit: Removed redundant codebox tags inside of code tags.]
I appreciate for the powerful UIA library and make all automation flow come true,
However, I am still struggle in pick up (select) a specific list item by it.
May please guide me how to get all available list item and select on of them by name?
I try following, but could only ExpandCollapse this drop down menu and get error message as attached.
ERROR: Method Call not support by the UIA_interface
My AHK version is V1.1.37, Should I upgrade to v2?
Appreciate for your time in advance.
Code: Select all
expandCollapsePattern := VcdEl.FindAllByType("ComboBox")[3].GetCurrentPatternAs("ExpandCollapse")
expandCollapsePattern.Expand()
sleep 500
UIA.WaitElementExistByNameAndType("320x240" "ListItem").Click()
Re: UIAutomation with a focus on Chrome
@chamber_wang, WaitElementExist is not an UIA method, but an UIA element method: you probably need to use VcdEl.WaitElementExistByNameAndType("320x240", "ListItem").Click() instead. (By the way, make sure there is a comma between "320x240" and "ListItem").
You don't necessarily need to upgrade to AHK v2, but I recommend it for new scripts or if your script isn't too large either since overall I think v2 is much easier to use. Additionally, I don't actively develop UIA_Interface for AHK v1, so any updates will come to UIA-v2.
For example, the code snippet in UIA-v2 would look something like this:
You don't necessarily need to upgrade to AHK v2, but I recommend it for new scripts or if your script isn't too large either since overall I think v2 is much easier to use. Additionally, I don't actively develop UIA_Interface for AHK v1, so any updates will come to UIA-v2.
For example, the code snippet in UIA-v2 would look something like this:
Code: Select all
VcdEl.FindElement({Type:"Combobox", i:3}).Expand()
Sleep 500
VcdEl.WaitElement({Name:"320x240", Type:"ListItem"}).Click()
-
- Posts: 10
- Joined: 23 Dec 2023, 01:57
Re: UIAutomation with a focus on Chrome
Hi @Descolada
Thanks for reply and point my error.
However, after doing so, the script exit without error but still no item be selected.
Any better idea? Please help.
Thanks for reply and point my error.
However, after doing so, the script exit without error but still no item be selected.
Any better idea? Please help.
-
- Posts: 10
- Joined: 23 Dec 2023, 01:57
Re: UIAutomation with a focus on Chrome
My bad not post code here,
Code: Select all
expandCollapsePattern := VcdEl.FindAllByType("ComboBox")[3].GetCurrentPatternAs("ExpandCollapse")
expandCollapsePattern.Expand()
Sleep 100
VcdEl.WaitElementExistByNameAndType("640x480", "ListItem").Click()
Re: UIAutomation with a focus on Chrome
@chamber_wang, I took a look at OBS and for some reason that list item doesn't respond to Click() nor SelectionItemPattern.Select(). However, ControlClick() appears to work:
A faster and more reliable way is to set the combobox's value directly, because it also has an edit element as a child:
Code: Select all
#Requires AutoHotkey v1
#include UIA_Interface.ahk
SetTitleMatchMode 2
WinActivate Properties for
WinWaitActive Properties for
UIA := UIA_Interface()
prop := UIA.ElementFromHandle("A")
cb := prop.FindFirstByNameAndType("Resolution", "Text").FindByPath("+1")
cb.ExpandCollapsePattern.Expand()
sel := cb.WaitElementExistByName("640x360").ControlClick()
Code: Select all
#Requires AutoHotkey v1
#include UIA_Interface.ahk
SetTitleMatchMode 2
WinActivate Properties for
WinWaitActive Properties for
UIA := UIA_Interface()
prop := UIA.ElementFromHandle("A")
cb := prop.FindFirstByNameAndType("Resolution", "Text").FindByPath("+1", "Type=Edit")
cb.Value := "640x360"
-
- Posts: 10
- Joined: 23 Dec 2023, 01:57
Re: UIAutomation with a focus on Chrome
Hi @Descolada
Appreciate for your reply and guide me this solution. I have tried both code work perfectly
Now I am more confident to upgrade AHK to v2 as you mentioned will be more advantages by previous thread.
Appreciate for your reply and guide me this solution. I have tried both code work perfectly
Now I am more confident to upgrade AHK to v2 as you mentioned will be more advantages by previous thread.
-
- Posts: 10
- Joined: 23 Dec 2023, 01:57
Re: UIAutomation with a focus on Chrome
Hi @Descolada ,
I suffer a list item does not interact with invoke pattern and workaround it by click button as Figure
I understand this may cause by OBS does not implement Invoke method.
However, this has been an entry barrier for beginners (as me) to get things done.
May please let me know any better insight to trial more efficiently?
I am confused the variety ways to access elements and associated methods.
Path, Control, Elements ... @.@
I suffer a list item does not interact with invoke pattern and workaround it by click button as Figure
I understand this may cause by OBS does not implement Invoke method.
However, this has been an entry barrier for beginners (as me) to get things done.
May please let me know any better insight to trial more efficiently?
I am confused the variety ways to access elements and associated methods.
Path, Control, Elements ... @.@
Code: Select all
UIA := UIA_Interface() ; Initialize UIA interface
WinWaitActive, ahk_exe obs64.exe
obsEl := UIA.ElementFromHandle("ahk_exe obs64.exe") ; Get the element for the Notepad window
WinEl := obsEl.FindFirstBy("Name=Sources")
VideoSrc := WinEl.FindFirstBy("Name=Video Capture Device") ; Find Video Src list item
; Select
SelPattern := VideoSrc.GetCurrentPatternAs("SelectionItemPattern")
sel := SelPattern.Select()
; Wait properties button exist and click, invoke pattern seems not work
Prop := obsEl.FindFirstByNameAndType("Properties", "Button")
PropinvokePattern := Prop.InvokePattern
PropinvokePattern.Invoke()
Re: UIAutomation with a focus on Chrome
@chamber_wang, I'm not sure that I understood your question (if there was one?). The code you posted works fine on my computer: it selects Video Capture Device and then clicks Properties. Perhaps you have some kind of timing issue, meaning that after sel := SelPattern.Select() the Properties button may not be active yet before Prop := obsEl.FindFirstByNameAndType("Properties", "Button") and thus Invoke fails? Try adding a small Sleep after selecting.
But yes, UIA is complicated. The Wiki might be of some help, and for UIA-v2 I've also created a small video tutorial series.
But yes, UIA is complicated. The Wiki might be of some help, and for UIA-v2 I've also created a small video tutorial series.
-
- Posts: 10
- Joined: 23 Dec 2023, 01:57
Re: UIAutomation with a focus on Chrome
Hi @Descolada ,
Thanks for reply. Let me explain more clear my question.
Please refer to following picture.
First, I would like to invoke the yellow blocked element but it does not work by Invoke pattern or Controlclick. (By double click the yellow block directly)
Therefore, I workaround it to click Properties button which marked by cobalt blue arrow.
Thanks for reply. Let me explain more clear my question.
Please refer to following picture.
First, I would like to invoke the yellow blocked element but it does not work by Invoke pattern or Controlclick. (By double click the yellow block directly)
Therefore, I workaround it to click Properties button which marked by cobalt blue arrow.
Re: UIAutomation with a focus on Chrome
@chamber_wang, UIA doesn't have a native built-in method to double-click an element (unfortunately...). But we can use ControlClick to double-click it:
Code: Select all
VideoSrc := WinEl.FindFirstBy("Name=Video Capture Device") ; Find Video Src list item
VideoSrc.ControlClick(,,,2)
-
- Posts: 10
- Joined: 23 Dec 2023, 01:57
Re: UIAutomation with a focus on Chrome
Hi @Descolada ,
Thanks for reply, It works.
Now I understand the ControlClick() parameters by this real practice.
And also get better insight by refer to wiki page you mentioned.
https://github.com/Descolada/UIAutomation/wiki/04.-Elements#controlclick
Happy New Year!
Thanks for reply, It works.
Now I understand the ControlClick() parameters by this real practice.
And also get better insight by refer to wiki page you mentioned.
https://github.com/Descolada/UIAutomation/wiki/04.-Elements#controlclick
Happy New Year!
Re: UIAutomation with a focus on Chrome
Thanks, macro testing straight from the viewer is a welcome addition.d_romeo wrote: ↑08 Dec 2023, 12:47I made some personal modifications to UIAViewer.ahk, in this version I added:
- "Test Macro" button that runs the recorded macro
- Macro creator is now the default window
- press esc twice to exit the app: prevents exiting the gui if you press esc twice while recording
- use only window class and exe for recognition, in case there are different titles for windows, as with some text editors
- the print button now does not call the Windows screenshot
Change line 9 with your path to "UIA_Interface.ahk"
-
- Posts: 110
- Joined: 11 Mar 2023, 14:16
Re: UIAutomation with a focus on Chrome
@Descolada
What are the purposes for the UIA_Browser,UIA_Constants and UIA_Interface .ahk files?
I have been using this library for some time now and I always get tripped up with the script provided in the /lib/ directory on Github. Which one am I supposed to include in my script and which one I am meant to use for purely discovering UI elements, their names etc during development.
This confusion especially becomes pronounced when I come back to programming for UIA after a long hiatus. I skimmed the Wiki, I was not able to get an answer.
Any help would be greatly appreciated!
What are the purposes for the UIA_Browser,UIA_Constants and UIA_Interface .ahk files?
I have been using this library for some time now and I always get tripped up with the script provided in the /lib/ directory on Github. Which one am I supposed to include in my script and which one I am meant to use for purely discovering UI elements, their names etc during development.
This confusion especially becomes pronounced when I come back to programming for UIA after a long hiatus. I skimmed the Wiki, I was not able to get an answer.
Any help would be greatly appreciated!
Re: UIAutomation with a focus on Chrome
@Ralf_Reddings200244 the purposes of the files are described on the GitHub main page:
Also, I highly recommend switching to AHK v2 and UIA-v2 library instead, because I am not actively supporting UIA_Interface.ahk any more, and UIA-v2 is overall much more pleasant to use.
Meaning the only one you need to include is UIA_Interface.ahk, either with #include <UIA_Interface> if it's in the Lib directory, or #include UIA_Interface.ahk if in the same folder as the script.1. UIA_Interface.ahk, which is based on jethrow's project, and contains wrapper functions for the UIAutomation framework. In addition it has some helper functions for easier use of UIA.
2. UIA_Browser.ahk, which contains helper functions for Chrome (and Edge mostly too) automation (fetching URLs, switching tabs etc).
3. OPTIONAL: UIA_Constants.ahk, which contains most necessary constants to use UIA. It is mostly based on an Autoit3 project by LarsJ. Note that this file creates a lot of global variables, so make sure your script doesn't change any of these during runtime. The preferred option is to use constants/enumerations from the UIA_Interface.ahk UIA_Enum class (details are available in the AHK forum thread).
Also, I highly recommend switching to AHK v2 and UIA-v2 library instead, because I am not actively supporting UIA_Interface.ahk any more, and UIA-v2 is overall much more pleasant to use.
-
- Posts: 110
- Joined: 11 Mar 2023, 14:16
Re: UIAutomation with a focus on Chrome
Ahh how did I miss that... forgive me. That answers what I wanted to know. Thank you.
Could I ask one other follow up question please?
I am trying to querry a UIA object saved to a variable, to get one or more window IDs for the trees child windows. I am not sure if this possible, I have tried to do it and was not successful. I am hoping to get some clarification.
Lets say I have a application that has a few child windows, amongst other stuff and I want to save the entire applications and all of its tree structure to a variable:
My reasoning for wanting to do this could be to avoid making successive queries to UIA, maybe the application is very expensive to query with UIA and so can only afford to make a single query, or I just want to work with the information at the time of saving the tree to the variable and that may have changed since then.
Is it possible to query this AIL object independent of the current/live state of the Adobe Illustrator tree structure? For example after running the above code, if I closed the Illustrator window, can I still work with AIL using UIA methods?
This may seem very obvious, so pardon me for asking if it is. I tried to do this a few times on my own and did not succeed:
I am thinking FindAllByName("ControlType=Pane") should match a number of child windows that have this windows Pane control type. so I am expecting IDs to have a series of window IDs that correspond to the number of illustrator child windows but in my MsgBox I get nothing.
I have been following the project and its progression into AHKV2, I am really excited to give it go along with AHK V2, unfortunetly there are a few libraries that still have not transitioned/ported to V2, one that is critical to me and so until some time I am stuck on AHK V1. I am so glad FindText and UIA have made smooth transitions though, soon enough I will make a decision to move forward.Also, I highly recommend switching to AHK v2 and UIA-v2 library instead, because I am not actively supporting UIA_Interface.ahk any more, and UIA-v2 is overall much more pleasant to use.
Could I ask one other follow up question please?
I am trying to querry a UIA object saved to a variable, to get one or more window IDs for the trees child windows. I am not sure if this possible, I have tried to do it and was not successful. I am hoping to get some clarification.
Lets say I have a application that has a few child windows, amongst other stuff and I want to save the entire applications and all of its tree structure to a variable:
Code: Select all
WinId := WinExist("ahk_exe Illustrator.exe")
UIA := UIA_Interface()
AIL := UIA.ElementFromHandle(WinId)
Is it possible to query this AIL object independent of the current/live state of the Adobe Illustrator tree structure? For example after running the above code, if I closed the Illustrator window, can I still work with AIL using UIA methods?
This may seem very obvious, so pardon me for asking if it is. I tried to do this a few times on my own and did not succeed:
Code: Select all
WinId := WinExist("ahk_exe Illustrator.exe")
UIA := UIA_Interface()
AIL := UIA.ElementFromHandle(WinId)
IDs := AIL.FindAllByName("ControlType=Pane").GetParentHwnd()
MsgBox,% IDs[1] ;Nothing is shown
Re: UIAutomation with a focus on Chrome
If you don't mind me asking: what is the library that isn't ported over?Ralf_Reddings200244 wrote: ↑27 Jan 2024, 13:42I have been following the project and its progression into AHKV2, I am really excited to give it go along with AHK V2, unfortunetly there are a few libraries that still have not transitioned/ported to V2, one that is critical to me and so until some time I am stuck on AHK V1. I am so glad FindText and UIA have made smooth transitions though, soon enough I will make a decision to move forward.
If an element that an UIA object represents is destroyed (for example it usually happens if a child window is closed) then the UIA object is invalid and won't respond to queries any more, including querying for window ID.I am trying to querry a UIA object saved to a variable, to get one or more window IDs for the trees child windows. I am not sure if this possible, I have tried to do it and was not successful. I am hoping to get some clarification.
What do you mean by "saving the entire application and all of its tree structure"? You can query for multiple elements at a time with FindAllBy, and/or you can cache properties via caching. I can't say whether you can prefetch the AIL object beforehand and interact with it even if it's closed, because it depends on how Illustrator handles its UIA implementations. Usually the UIA object would be invalid, but in some applications I've seen that the object remains valid for some time. You'd need to experiment with it and find out.Lets say I have a application that has a few child windows, amongst other stuff and I want to save the entire applications and all of its tree structure to a variable:
First of all you are trying to use GetParentHwnd on an array which will not work. Second, GetParentHwnd returns the hWnd for the main Illustrator window, not the controls/subwindows (so it should be the same as the WinId variable in your code). If the Pane elements are implemented as win32 controls (meaning inspection with WindowSpy shows control names) then you can get the control hWnd by normalizing to the NativeWindowHandle property, something like UIA.CreateTreeWalker(UIA.CreateNotCondition(UIA.CreatePropertyCondition(UIA_Enum.UIA_PropertyId("NativeWindowHandle"), 0))).NormalizeElement(Element).GetCurrentPropertyValue(UIA_Enum.UIA_PropertyId("NativeWindowHandle"))This may seem very obvious, so pardon me for asking if it is. I tried to do this a few times on my own and did not succeed:Code: Select all
WinId := WinExist("ahk_exe Illustrator.exe") UIA := UIA_Interface() AIL := UIA.ElementFromHandle(WinId) IDs := AIL.FindAllByName("ControlType=Pane").GetParentHwnd() MsgBox,% IDs[1] ;Nothing is shown
How useful that is depends on what you are planning to do with the panes/subwindows...
Return to “Scripts and Functions (v1)”
Who is online
Users browsing this forum: No registered users and 181 guests