UIA v2

Post your working scripts, libraries and tools.
songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 16 Nov 2023, 03:33

https://cn.bing.com/search?q=define+metaphorical&FORM=BESBTB&mkt=zh-CN&ensearch=1
How to set an Edit's value through another element, in that case through the Edit's upper text "Dictionary".

Code: Select all

word := "help"
dic_text := cUIA.WaitElement({Name:"Dictionary", Type:"Text"}).Highlight()
dic_text := UIA.TreeWalkerTrue.GetParentElement(dic_text)
dic_edit := dic_text.WalkTree("+3").Highlight()
dic_edit.Value := word
That code doesn't work, also it seems there is a problem of identify such Edit.
Image

Descolada
Posts: 1168
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 16 Nov 2023, 11:15

@songdg
I'm not exactly sure what you mean by "through another element" and your provided code doesn't work in my setup (WalkTree throws an error). I assume that you are trying to get the Edit element from the "Dictionary" Text element? In that case something like

Code: Select all

word := "help"
dic_text := cUIA.WaitElement({Name:"Dictionary", Type:"Text"}).Highlight()
dic_edit := cUIA.FindElement({Type:"Edit", startingElement:dic_text})
dic_edit.Value := word
might be more reliable: it just finds the next Edit-type element in the tree. If you know that the Edit element is a sibling or a child then you can also use WalkTree with a filter condition, such as if it's a sibling then dic_edit := dic_text.WalkTree("+1", {Type:"Edit"})

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 17 Nov 2023, 04:29

@Descolada
Thanks a lot, it's a clever work around, and in fact I can use cUIA.WaitElement({Name:"Enter a word", Type:"Edit"}).Highlight() to locate the edit directly.

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 27 Nov 2023, 10:43

What's the chrome version of scrolling, the code from the example dosen't work on chrome.

Code: Select all

cUIA := UIA_Browser()
docEl := cUIA.GetCurrentDocumentElement()
ToolTip "Current scroll percent: " docEl.VerticalScrollPercent
; Lets scroll down in steps of 10%
for percent in [10, 20, 30, 40, 50, 60, 70, 80, 90, 100] {
	docEl.VerticalScrollPercent := percent
	Sleep 500
	ToolTip "Current scroll percent: " docEl.VerticalScrollPercent
}

Descolada
Posts: 1168
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 27 Nov 2023, 10:46

@songdg, Chrome doesn't support the ScrollPattern for its Document element, so it's not possible. However, it's possible to use Element.ScrollIntoView() to scroll certain elements into view, or you can inject Javascript to scroll.

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 27 Nov 2023, 11:16

@Descolada
I want to scroll to the Full Editor & Preview button on this page, but there's an error.
Error: Method ScrollIntoView not found in UIA.IUIAutomationElement Class.

Full_Editor := cUIA.FindElement({Name:"Full Editor & Preview", Type:"Button"}).Highlight()
Full_Editor.ScrollIntoView()

Descolada
Posts: 1168
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 27 Nov 2023, 11:25

Worked fine without any errors here...

Code: Select all

#Requires Autohotkey v2
#Include UIA.ahk
#Include UIA_Browser.ahk

WinActivate "ahk_exe chrome.exe"
WinWaitActive "ahk_exe chrome.exe"
chrome := UIA.ElementFromHandle("ahk_exe chrome.exe")
Full_Editor := chrome.FindElement({Name:"Full Editor & Preview", Type:"Button"}).Highlight()
/* Alternatively:
cUIA := UIA_Browser("ahk_exe chrome.exe")
Full_Editor := cUIA.FindElement({Name:"Full Editor & Preview", Type:"Button"}).Highlight()
*/
Full_Editor.ScrollIntoView()

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 27 Nov 2023, 11:49

@Descolada
The problem persists.

Call stack:
D:\UIA-v2-main\Lib\UIA.ahk (1962) : [UIA.IUIAutomationElement.Prototype.__Call] Throw MethodError("Method " Name " not found in " this.__Class " Class.",-1,Name)
D:\UIA-v2-main\scroll2.ahk (14) : [] Full_Editor.ScrollIntoView()
> Auto-execute
Image

MiguelCt
Posts: 2
Joined: 28 Nov 2023, 05:08

Re: UIA v2

Post by MiguelCt » 28 Nov 2023, 08:15

Hi!
How do I find the path of an element that I have found with findElement?
I want to find out the path of this Edit element with the name: Razón social" (example: [6,1,1,X])

Code: Select all

 
#SingleInstance Force
#include UIA.ahk
#include UIA_Browser.ahk
+z::
{
firefoxEl := UIA.ElementFromHandle("Mozilla Firefox ahk_exe firefox.exe")
elementoInicial := firefoxEl.FindElement({Type:"Edit", Name:"Razón social:"}).Highlight().Dump()
msgbox elementoInicial
}
Thanks!

Descolada
Posts: 1168
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 28 Nov 2023, 08:42

@MiguelCt, firefoxEl.GetNumericPath(elementoInicial) should return the numeric path, and firefoxEl.GetUIAPath(elementoInicial) the UIA path. Note that these methods will enumerate the starting elements subtree (in this case firefoxEl), so it will most likely be very slow and should be used sparingly. Also paths are unreliable in Firefox because they will depend on the tab order (if your starting point is the Firefox window, not a tab), and websites tend to have varying content which often breaks paths.

MiguelCt
Posts: 2
Joined: 28 Nov 2023, 05:08

Re: UIA v2

Post by MiguelCt » 28 Nov 2023, 10:09

Descolada wrote:
28 Nov 2023, 08:42
@MiguelCt, firefoxEl.GetNumericPath(elementoInicial) should return the numeric path, and firefoxEl.GetUIAPath(elementoInicial) the UIA path. Note that these methods will enumerate the starting elements subtree (in this case firefoxEl), so it will most likely be very slow and should be used sparingly. Also paths are unreliable in Firefox because they will depend on the tab order (if your starting point is the Firefox window, not a tab), and websites tend to have varying content which often breaks paths.
Thanks! Then I'll try with chrome, but my aproach (the only one I can think of (besides getting the whole html with your UIA_Browser ) is to find out the path of that element , and the others are suposed to be always in the same position relative to this one.
Thanks for UIA and for your help.
Greetings from Spain

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 29 Nov 2023, 11:02

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/select
How to get a list of the items of a combobox control both the Standard and Custom for later compare if there are items been added or deleted, and select an item that match or in that item's text, say "pea" then select the Peas item.
Image

Descolada
Posts: 1168
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 29 Nov 2023, 14:12

@songdg, without injecting Javascript there isn't a way other than expanding the combobox, getting all the items, and then selecting the desired one. Eg for the "Custom controls" combobox in Chrome:

Code: Select all

#Requires AutoHotkey v2
#include UIA.ahk

WinActivate "ahk_exe chrome.exe"
WinWaitActive "ahk_exe chrome.exe"
chrome := UIA.ElementFromHandle("ahk_exe chrome.exe")
example := chrome.FindElement({Name:"Custom <select> example", Type:"Document"})
secondCB := example.FindElement({Name:"↓", Type:"Group", MatchMode:"Substring"})
secondCB.Invoke()
while (children := secondCB.FindElements({Type:"Group"}, UIA.TreeScope.Children)).Length < 2
    Sleep 40
items := ""
for child in children {
    items .= child.Name "|"
    if InStr(child.Name, "peas")
        pea := child
}
if IsSet(pea)
    pea.Invoke()
MsgBox items
Personally I wouldn't use UIA to do this task, but I would either inject Javascript or use Chrome.ahk/Rufaydium etc to automate the browser directly.

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 30 Nov 2023, 10:07

@Descolada
https://business.linkedin.com/sales-solutions/contact-us
Thanks, maybe this page is more similar to the problem I currently face, I try to apply your example to select one item, but the code stuck at some point.

Code: Select all

WinActivate "ahk_exe chrome.exe"
WinWaitActive "ahk_exe chrome.exe"
chrome := UIA.ElementFromHandle("ahk_exe chrome.exe")
role_text := chrome.FindElement({Name:"Your role", Type:"Text"}).Highlight()
role_group := role_text .WalkTree("+1", {Type:"Group"}).Highlight()
role_group.ControlClick()
;role_group.Click()    dosen't work
;role_group.Invoke()   dosen't work
while (children := role_group.FindElements({Type:"Text"}, UIA.TreeScope.Children)).Length < 2
    Sleep 40

items := ""
for child in children {
    items .= child.Name "%"
    if InStr(child.Name, "Job Seeker")
        Job_Seeker := child
}
if IsSet(Job_Seeker)
    Job_Seeker.Invoke()
MsgBox items

Descolada
Posts: 1168
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 30 Nov 2023, 14:34

@songdg, apparently that LinkedIn pages combobox creates a new Chrome window to display the options, and that window isn't inspectable with Acc nor UIA. This unfortunately means it can't be automated with UIA.

Again, just use Javascript to get the values... Type this into the address bar and it shows all the roles:
javascript:var roles = document.querySelector("#role"); var result = ''; for (let item of roles) { result += item.value + "\n";}; alert(result);

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 01 Dec 2023, 00:09

@Descolada
What about click on a particular item? Since this is a new Chrome window, I wonder does that can be implemented like the context menu.
And what are the right syntax of using JSExecute, cUIA.JSExecute("roles = document.querySelector('#role'); var result = ''; for (let item of roles) { result += item.value + '\n';}; alert(result);"), that doesn't work.

songdg
Posts: 605
Joined: 04 Oct 2017, 20:04

Re: UIA v2

Post by songdg » 03 Dec 2023, 11:29

Descolada wrote:
30 Nov 2023, 14:34
@songdg, apparently that LinkedIn pages combobox creates a new Chrome window to display the options, and that window isn't inspectable with Acc nor UIA. This unfortunately means it can't be automated with UIA.

Again, just use Javascript to get the values... Type this into the address bar and it shows all the roles:
javascript:var roles = document.querySelector("#role"); var result = ''; for (let item of roles) { result += item.value + "\n";}; alert(result);
I find out just use textContent method also can get all the roles.
javascript:var roles = document.querySelector("#role"); alert(roles.textContent);

neogna2
Posts: 598
Joined: 15 Sep 2016, 15:44

Re: UIA v2

Post by neogna2 » 08 Dec 2023, 03:54

Small suggestion: in the viewer make the ~F1 hotkey able to toggle capture on and off (not just start capture). Because hotkey ~Esc passing through Esc in some windows causes an active element to close, whereas a passed through F1 more seldom does.

Seems the only code change needed is to comment out this line
https://github.com/Descolada/UIA-v2/blob/main/Lib/UIA.ahk#L7303

too hasty, will test more and suggest working change

Descolada
Posts: 1168
Joined: 23 Dec 2021, 02:30

Re: UIA v2

Post by Descolada » 09 Dec 2023, 02:09

@neogna2, I actually don't like the F1 hotkey either, because in some apps it opens a help dialog. How about the Print Screen button for both starting and stopping capturing?

neogna2
Posts: 598
Joined: 15 Sep 2016, 15:44

Re: UIA v2

Post by neogna2 » 09 Dec 2023, 04:50

Good point. Print Screen is seldomly used by apps so good in that way, though I've seen some new keyboards without a Print Screen key. Maybe default to PrintScreen and if not too much work make it user configurable in a static variable in the Viewer class?
https://github.com/Descolada/UIA-v2/blob/main/Lib/UIA.ahk#L7084
e.g. static CaptureKey := "PrintScreen"

Post Reply

Return to “Scripts and Functions (v2)”