UIA v2

[Descolada]

@songdg, I don't speak chinese so I don't understand most of the UIAViewer content. If the close button is a separate UIA element then you can try El.Click(), El.ControlClick() or El.Click("left"). If it isn't, then you can get the tab location and calculate where to click/controlclick from that.
Though if at all possible I'd recommend Chrome.ahk or Rufaydium libraries for browser automation instead, since they are much more reliable.

[songdg]

@Descolada
The close button without name and AutomationID, does that mean it not a separate UIA element? I can get the subtab name through DumpAll() method, how to get the tab location?

[Descolada]

@songdg, you can get the position by using Element.Location that contains screen x,y,w,h, or you can use Element.GetPos() which returns x,y,w,h in whatever the current CoordMode Mouse is set to. Then using ControlClick or Click should be fairly straight-forward.

[songdg]

@songdg, you can get the position by using Element.Location that contains screen x,y,w,h, or you can use Element.GetPos() which returns x,y,w,h in whatever the current CoordMode Mouse is set to. Then using ControlClick or Click should be fairly straight-forward.

Thanks, much appreciated

[songdg]

Element[path], can the path be a condition path of an element of chrome browser?

[Descolada]

@songdg, yes that syntax accepts all the types of paths: condition path (Element[{Type:"Button"}, {Type:"Link"}), UIA path (Element["YYY"]), numeric path (Element[1.2.3]).

[songdg]

@songdg, yes that syntax accepts all the types of paths: condition path (Element[{Type:"Button"}, {Type:"Link"}), UIA path (Element["YYY"]), numeric path (Element[1.2.3]).

Thanks, it's very convenient, I can use ElementFromPath method to set value or click a button

[songdg]

How to detect the OK button on a chrome base popup window(Pane type), which on a fixed location and can not be move, use UIA I can only focus on the titlebar, neither the OK \ CANCEL button nor the information.

[Descolada]

@songdg, I'm not sure why that alert dialog isn't inspectable with UIAViewer, but it still exists in the UIA tree as the last child element of the Chrome window element:

image.png (225.67 KiB) Viewed 2516 times

You can get the alert dialog element like this and extract the OK button from there:

Code: Select all

WinTitle := "ahk_exe chrome.exe"
WinActivate WinTitle
WinWaitActive WinTitle
chrome := UIA.ElementFromHandle("ahk_exe chrome.exe")
MsgBox chrome[-1].DumpAll() ; Gets the last child element

[songdg]

@Descolada
Thanks, I'll try it later after the weekend break.

[arcylix]

@Descolada

Is there a faster way to achieve these same results?

Code: Select all

getRowElements(base) {
   WebMedEl := UIA.ElementFromHandle("A")
   rowUser := WebMedEl.FindElement({AutomationId: base}).Name
   rowPage := WebMedEl.FindElement({AutomationId: base+1}).Name
   rowID := WebMedEl.FindElement({AutomationId: base+3}).Name

   rowPageArray := StrSplit(RegExReplace(Trim(rowPage), "\s+", " "), A_Space)
   rowIDArray := StrSplit(RegExReplace(Trim(rowID), "\s+", " "), A_Space)

   rowPages := rowPageArray[1]
   rowDate := rowPageArray[2]

   rowTime := rowIDArray[1]
   rowFaxID := rowIDArray[3]

   return rowUser ", " rowPages ", " rowDate ", " rowTime ", " rowFaxID
}

rowOne := getRowElements(16)
rowTwo := getRowElements(21)
rowThr := getRowElements(26)
rowFou := getRowElements(31)
rowFiv := getRowElements(36)
rowSix := getRowElements(41)

The snippet of code above runs for the element dump below. However, it does seem relatively slow, taking about 5 seconds or more to go through the 6 rows I have to go through before it scrolls down to add more to an array. The interface only displays 6 rows at a time, so after it collects the information, it will scroll down for the next 6 rows. Given that there's currently 500 rows, it takes a good long while before I can use my computer again. If there's not, no worries. I'm just making sure I'm not missing something obvious. For the record, the automationID will always be: 16, 17, 19 (and then add 5 to each one until 41, 42, 44).

Code: Select all

3: Type: 50033 (Pane) LocalizedType: "pane" AutomationId: "3" ClassName: "White Area Class"
3,1: Type: 50033 (Pane) Name: "Fax Imaging                                                               " LocalizedType: "pane" AutomationId: "246640624" ClassName: "Title Window"
3,2: Type: 50033 (Pane) LocalizedType: "pane" AutomationId: "1" ClassName: "Child"
3,2,1: Type: 50033 (Pane) LocalizedType: "pane" ClassName: "Scrl Class"
3,2,2: Type: 50033 (Pane) LocalizedType: "pane" ClassName: "Scrl Class"
3,2,3: Type: 50033 (Pane) Name: "Inbound Fax Images" LocalizedType: "pane" ClassName: "Label Class"
3,2,4: Type: 50033 (Pane) Name: "Images for Patient: 'No Patient Selected'" LocalizedType: "pane" AutomationId: "1" ClassName: "Label Class"
3,2,5: Type: 50033 (Pane) Name: "Code" LocalizedType: "pane" AutomationId: "2" ClassName: "Label Class"
3,2,6: Type: 50033 (Pane) Name: "Description" LocalizedType: "pane" AutomationId: "3" ClassName: "Label Class"
3,2,7: Type: 50033 (Pane) Name: "User" LocalizedType: "pane" AutomationId: "4" ClassName: "Label Class"
3,2,8: Type: 50033 (Pane) Name: "Date" LocalizedType: "pane" AutomationId: "5" ClassName: "Label Class"
3,2,9: Type: 50033 (Pane) Name: "Print/Fax Ver#" LocalizedType: "pane" AutomationId: "6" ClassName: "Label Class"
3,2,10: Type: 50033 (Pane) Name: "No Patient Selected" LocalizedType: "pane" AutomationId: "7" ClassName: "Label Class"
3,2,11: Type: 50033 (Pane) Name: "Preview" LocalizedType: "pane" AutomationId: "8" ClassName: "Label Class"
3,2,12: Type: 50033 (Pane) Name: "User" LocalizedType: "pane" AutomationId: "9" ClassName: "Label Class"
3,2,13: Type: 50033 (Pane) Name: "Pages" LocalizedType: "pane" AutomationId: "10" ClassName: "Label Class"
3,2,14: Type: 50033 (Pane) Name: "Date" LocalizedType: "pane" AutomationId: "11" ClassName: "Label Class"
3,2,15: Type: 50033 (Pane) Name: "Time" LocalizedType: "pane" AutomationId: "12" ClassName: "Label Class"
3,2,16: Type: 50033 (Pane) Name: "Description" LocalizedType: "pane" AutomationId: "13" ClassName: "Label Class"
3,2,17: Type: 50033 (Pane) Name: "Id" LocalizedType: "pane" AutomationId: "14" ClassName: "Label Class"
3,2,18: Type: 50033 (Pane) Name: "View Fax Image" LocalizedType: "pane" AutomationId: "15" ClassName: "CHwndCppBase Window Class"
3,2,19: Type: 50033 (Pane) LocalizedType: "pane" AutomationId: "18" ClassName: "Label Class"
3,2,20: Type: 50033 (Pane) Name: "View Fax Image" LocalizedType: "pane" AutomationId: "20" ClassName: "CHwndCppBase Window Class"
3,2,21: Type: 50033 (Pane) LocalizedType: "pane" AutomationId: "23" ClassName: "Label Class"
3,2,22: Type: 50033 (Pane) Name: "View Fax Image" LocalizedType: "pane" AutomationId: "25" ClassName: "CHwndCppBase Window Class"
3,2,23: Type: 50033 (Pane) LocalizedType: "pane" AutomationId: "28" ClassName: "Label Class"
3,2,24: Type: 50033 (Pane) Name: "View Fax Image" LocalizedType: "pane" AutomationId: "30" ClassName: "CHwndCppBase Window Class"
3,2,25: Type: 50033 (Pane) LocalizedType: "pane" AutomationId: "33" ClassName: "Label Class"
3,2,26: Type: 50033 (Pane) Name: "View Fax Image" LocalizedType: "pane" AutomationId: "35" ClassName: "CHwndCppBase Window Class"
3,2,27: Type: 50033 (Pane) Name: "View Fax Image" LocalizedType: "pane" AutomationId: "45" ClassName: "CHwndCppBase Window Class"
3,2,28: Type: 50033 (Pane) LocalizedType: "pane" AutomationId: "48" ClassName: "Label Class"
3,2,29: Type: 50033 (Pane) LocalizedType: "pane" AutomationId: "38" ClassName: "Label Class"
3,2,30: Type: 50033 (Pane) Name: "Locale2" LocalizedType: "pane" AutomationId: "36" ClassName: "Label Class"
3,2,31: Type: 50033 (Pane) Name: "Locale3" LocalizedType: "pane" AutomationId: "16" ClassName: "Label Class"
3,2,32: Type: 50033 (Pane) Name: "9                                                          09/06/23" LocalizedType: "pane" AutomationId: "17" ClassName: "Label Class"
3,2,33: Type: 50033 (Pane) Name: " 8:35a                                                         Fax                                                                   4848824" LocalizedType: "pane" AutomationId: "19" ClassName: "Label Class"
3,2,34: Type: 50033 (Pane) Name: "Locale2" LocalizedType: "pane" AutomationId: "21" ClassName: "Label Class"
3,2,35: Type: 50033 (Pane) Name: "6                                                          09/06/23" LocalizedType: "pane" AutomationId: "22" ClassName: "Label Class"
3,2,36: Type: 50033 (Pane) Name: " 5:48a                                                         Fax                                                                   4848659" LocalizedType: "pane" AutomationId: "24" ClassName: "Label Class"
3,2,37: Type: 50033 (Pane) Name: "Locale3" LocalizedType: "pane" AutomationId: "26" ClassName: "Label Class"
3,2,38: Type: 50033 (Pane) Name: "7                                                          09/06/23" LocalizedType: "pane" AutomationId: "27" ClassName: "Label Class"
3,2,39: Type: 50033 (Pane) Name: "12:02a                                                        Fax                                                                   4848649" LocalizedType: "pane" AutomationId: "29" ClassName: "Label Class"
3,2,40: Type: 50033 (Pane) Name: "Locale1" LocalizedType: "pane" AutomationId: "31" ClassName: "Label Class"
3,2,41: Type: 50033 (Pane) Name: "6                                                          09/06/23" LocalizedType: "pane" AutomationId: "32" ClassName: "Label Class"
3,2,42: Type: 50033 (Pane) Name: "12:02a                                                        Fax                                                                   4848648" LocalizedType: "pane" AutomationId: "34" ClassName: "Label Class"
3,2,43: Type: 50033 (Pane) Name: "2                                                          09/05/23" LocalizedType: "pane" AutomationId: "37" ClassName: "Label Class"
3,2,44: Type: 50033 (Pane) Name: " 9:09p                                                         Fax                                                                  4848630" LocalizedType: "pane" AutomationId: "39" ClassName: "Label Class"
3,2,45: Type: 50033 (Pane) Name: "Locale1" LocalizedType: "pane" AutomationId: "46" ClassName: "Label Class"
3,2,46: Type: 50033 (Pane) Name: "8                                                          09/05/23" LocalizedType: "pane" AutomationId: "47" ClassName: "Label Class"
3,2,47: Type: 50033 (Pane) Name: " 7:01p                                                         Fax                                                                  4848623" LocalizedType: "pane" AutomationId: "49" ClassName: "Label Class"

[Descolada]

@songdg, I have updated UIA.ahk with an improved UIAViewer that should now display that button as well. Also it is able to show some windows/controls that it wasn't able to before, such as Discord or Notepad++ treeview control.

@arcylix, this can be improved in two main ways: using FindElements once instead of FindElement multiple times to avoid traversing the tree unnecessarily, and caching.

First lets see how FindElements could be used here:

Code: Select all

getRowElements(base) {
   WebMedEl := UIA.ElementFromHandle("A")
   els := WebMedEl.FindElement([{AutomationId: base}, {AutomationId: base+1}, {AutomationId: base+3}])
   for el in els {
      elId := el.AutomationId ; avoid getting it multiple times, every call is slow
      if elId = base
         rowUser := el.Name
      else if elId = (base+1)
         rowPage := el.Name
      else if elId = (base+3)
         rowId := el.Name
   }

   rowPageArray := StrSplit(RegExReplace(Trim(rowPage), "\s+", " "), A_Space)
   rowIDArray := StrSplit(RegExReplace(Trim(rowID), "\s+", " "), A_Space)

   rowPages := rowPageArray[1]
   rowDate := rowPageArray[2]

   rowTime := rowIDArray[1]
   rowFaxID := rowIDArray[3]

   return rowUser ", " rowPages ", " rowDate ", " rowTime ", " rowFaxID
}

rowOne := getRowElements(16)
rowTwo := getRowElements(21)
rowThr := getRowElements(26)
rowFou := getRowElements(31)
rowFiv := getRowElements(36)
rowSix := getRowElements(41)

This is still calling getRowElements multiple times which depending on the application might or might not be possible to partially eliminate as well. You could simply pass an array of the 6 visible rows and get all of them with one FindElements call:

Code: Select all

getRowElements([16, 21, 26, 31, 36, 41])
getRowElements(bases) {
   condition := []
   for base in base {
      condition.Push({AutomationId: base})
      condition.Push({AutomationId: base+1})
      condition.Push({AutomationId: base+3})
   }
   WebMedEl := UIA.ElementFromHandle("A")
   els := WebMedEl.FindElement(condition)
   ; do stuff with rows and columns
}

Next, caching will definitely improve the search speed. It seems you are only using AutomationId and Name, so we could cache those and not even get the real "live" element (this means you can't interact with it or get any non-cached elements, remove UIA.AutomationElementMode.None to also get the live element).

Code: Select all

getRowElements(base) {
   static cacheRequest := UIA.CreateCacheRequest(["AutomationId", "Name"],,5, UIA.AutomationElementMode.None)
   WebMedEl := UIA.ElementFromHandle("A", cacheRequest)
   els := WebMedEl.FindCachedElement([{AutomationId: base}, {AutomationId: base+1}, {AutomationId: base+3}])
   for el in els {
      elId := el.CachedAutomationId ; this is actually fast, but elId is shorter to write...
      if elId = base
         rowUser := el.CachedName
      else if elId = (base+1)
         rowPage := el.CachedName
      else if elId = (base+3)
         rowId := el.CachedName
   }

   rowPageArray := StrSplit(RegExReplace(Trim(rowPage), "\s+", " "), A_Space)
   rowIDArray := StrSplit(RegExReplace(Trim(rowID), "\s+", " "), A_Space)

   rowPages := rowPageArray[1]
   rowDate := rowPageArray[2]

   rowTime := rowIDArray[1]
   rowFaxID := rowIDArray[3]

   return rowUser ", " rowPages ", " rowDate ", " rowTime ", " rowFaxID
}

rowOne := getRowElements(16)
rowTwo := getRowElements(21)
rowThr := getRowElements(26)
rowFou := getRowElements(31)
rowFiv := getRowElements(36)
rowSix := getRowElements(41)

Note that this is untested so it might throw unforeseen errors

[SteveMylo]

Hi @Descolada In Version 1, I used to do a search with caseSensitive and Matchmode method e.g. cb := dvEl.FindFirstBy("ItemStatus=Asc",,2,False)

But this method doesn't work in V2 e.g. chromeEl.ElementFromPathExist({T:30}, {T:26, i:2}, {T:26}, {T:26}, {T:0,N:"Cinema mode keyboard shortcut t"},,2,False)
So I tried doing research as to the new methods and I found this cs:0, mm:2 but it doesn't work when I add it to the end.
chromeEl.ElementFromPathExist({T:30}, {T:26, i:2}, {T:26}, {T:26}, {T:0,N:"Cinema mode keyboard shortcut t"}, cs:0, mm:2)
Any Thoughts as to where I put them and how I write it?

[SteveMylo]

I think I just figured it out, it has to be inside the curly brackets yes? {T:0,N:"Cinema mode keyboard shortcut t", cs:0, mm:2}) .. ?

[Descolada]

I think I just figured it out, it has to be inside the curly brackets yes? {T:0,N:"Cinema mode keyboard shortcut t", cs:0, mm:2}) .. ?

Yes, this is correct. Every search condition can be assigned it's own CaseSense and MatchMode properties for greater flexibility, so one part of the condition path could be made to match Exact while another part Substring, etc. The same applies to creating and/or conditions.

[songdg]

@Descolada
Thanks bro, now it works perfectly !

[arcylix]

@Descolada

I wound up using the second example (and corrected some mistakes from the example), and it seems considerably faster than it had been before.

I would use the Caching option, except I am refreshing the group of elements through a loop to get the next set to insert them into an array. Still have some kinks to work out, but I greatly appreciate the feedback!

[songdg]

It seems there's a problem of identify checkbox, the checkbox inside a Group item.

[Descolada]

@songdg, I see that the element has a child element as well, so have you checked whether that one is toggle-able? Other than that I can't help you without having a way to test it myself: I haven't had such problems before so this might be related to that specific page.

[songdg]

@Descolada
There are checkboxes on the pages below UIAViewer can not identify.
https://www.w3schools.com/howto/howto_css_custom_checkbox.asp
https://designsystem.digital.gov/components/checkbox/
https://baseweb.design/components/checkbox/