@malcev, interesting, now it doesn't activate any window at all, regardless of the AutoSetFocus property. I'm guessing a Windows update in the last year has changed something, because it used to auto-activate every window it was called on. Chromium has gotten a lot of UIAutomation improvements as well in the meanwhile. I'll play around with it a bit more when I get time, and see if they have fixed the reliability issue as well.
Although since LegacyIAccessible is a legacy pattern, I'm not sure how wise it would be to continue using it...
UIAutomation with a focus on Chrome
Re: UIAutomation with a focus on Chrome
"Continuation section too long." What's the problem when I include the UIA_Interface library?
Re: UIAutomation with a focus on Chrome
@songdg, not exactly sure what the problem is.
1) Try downloading it again from GitHub by pressing the green Code button -> Download ZIP
2) Make sure you are running a relatively recent AHK v1 version (preferably 1.1.36.02)
1) Try downloading it again from GitHub by pressing the green Code button -> Download ZIP
2) Make sure you are running a relatively recent AHK v1 version (preferably 1.1.36.02)
Re: UIAutomation with a focus on Chrome
Thanks for answer my question, now I can use UIAutomation. I found your liberary can select sub_menu item of context menu from the forumviewtopic.php?f=76&t=108384&p=484315&hilit=context+menu+submenu#p484315, but when I try to create a new folder on the desktop it doesn't work.
Code: Select all
Click Right
Sleep, 2000
WinWait, ahk_class #32768
WinGet, hwnd, ID, ahk_class #32768
contextmenu := UIA.ElementFromHandle(hwnd)
item := "New"
subitem := "Folder"
subMenu := contextmenu.FindFirstByNameAndType(item, "menuitem")
expandMenu := subMenu.GetCurrentPatternAs("ExpandCollapse")
expandMenu.Expand()
subMenu.FindFirstByNameAndType(subitem, "menuitem").click()
Winclose
Re: UIAutomation with a focus on Chrome
@songdg, rather unfortunately every new context menu is its own window, so you need to wait for it and get the handle individually.
F1 is your code with a slight modification to get the correct window element for the second context menu. F2 is similar, but I added some modifications of my own: WaitElementExist usually helps a bit with timing issues, and accessing patterns can be done without GetCurrentPatternAs.
F1 is your code with a slight modification to get the correct window element for the second context menu. F2 is similar, but I added some modifications of my own: WaitElementExist usually helps a bit with timing issues, and accessing patterns can be done without GetCurrentPatternAs.
Code: Select all
#Requires AutoHotkey v1.1
#SingleInstance Force
#Persistent
#include <UIA_Interface>
UIA := UIA_Interface()
return
F1::
Click Right
WinWait, ahk_class #32768
WinGet, hwnd, ID, ahk_class #32768
contextmenu := UIA.ElementFromHandle(hwnd)
item := "New"
subitem := "Folder"
subMenu := contextmenu.FindFirstByNameAndType(item, "menuitem")
expandMenu := subMenu.GetCurrentPatternAs("ExpandCollapse")
expandMenu.Expand()
WinWait, ahk_class #32768,,, ahk_id %hwnd%
WinGet, hwnd, ID, ahk_class #32768,, ahk_id %hwnd%
UIA.ElementFromHandle(hwnd).FindFirstByNameAndType(subitem, "menuitem").click()
return
F2::
Click Right
WinWait, ahk_class #32768
contextmenu := UIA.ElementFromHandle(hwnd := WinExist("ahk_class #32768"))
item := "New"
subitem := "Folder"
subMenu := contextmenu.WaitElementExistByNameAndType(item, "menuitem")
expandMenu := subMenu.ExpandCollapsePattern.Expand()
WinWait, ahk_class #32768,,, ahk_id %hwnd%
hwnd := WinExist("ahk_class #32768",, "ahk_id " hwnd)
UIA.ElementFromHandle(hwnd).WaitElementExistByNameAndType(subitem, "menuitem").click()
return
Re: UIAutomation with a focus on Chrome
Thanks, much appreciated I should further explore what else this amazing tool can do.Descolada wrote: ↑30 Mar 2023, 06:52@songdg, rather unfortunately every new context menu is its own window, so you need to wait for it and get the handle individually.
F1 is your code with a slight modification to get the correct window element for the second context menu. F2 is similar, but I added some modifications of my own: WaitElementExist usually helps a bit with timing issues, and accessing patterns can be done without GetCurrentPatternAs.Code: Select all
#Requires AutoHotkey v1.1 #SingleInstance Force #Persistent #include <UIA_Interface> UIA := UIA_Interface() return F1:: Click Right WinWait, ahk_class #32768 WinGet, hwnd, ID, ahk_class #32768 contextmenu := UIA.ElementFromHandle(hwnd) item := "New" subitem := "Folder" subMenu := contextmenu.FindFirstByNameAndType(item, "menuitem") expandMenu := subMenu.GetCurrentPatternAs("ExpandCollapse") expandMenu.Expand() WinWait, ahk_class #32768,,, ahk_id %hwnd% WinGet, hwnd, ID, ahk_class #32768,, ahk_id %hwnd% UIA.ElementFromHandle(hwnd).FindFirstByNameAndType(subitem, "menuitem").click() return F2:: Click Right WinWait, ahk_class #32768 contextmenu := UIA.ElementFromHandle(hwnd := WinExist("ahk_class #32768")) item := "New" subitem := "Folder" subMenu := contextmenu.WaitElementExistByNameAndType(item, "menuitem") expandMenu := subMenu.ExpandCollapsePattern.Expand() WinWait, ahk_class #32768,,, ahk_id %hwnd% hwnd := WinExist("ahk_class #32768",, "ahk_id " hwnd) UIA.ElementFromHandle(hwnd).WaitElementExistByNameAndType(subitem, "menuitem").click() return
Re: UIAutomation with a focus on Chrome
Is it possible with this library to call context menu functions? For example to add shortcut keys to the context menu of explorer or better still for some software, such as 7zip, and avoid doing "RButton>compress and send by email..." every time Maybe without necessarily opening the context menu if possible.
Re: UIAutomation with a focus on Chrome
@d_romeo, with this library you can only navigate the context menu, not bypass it completely. For that you would need task-specific commands, such as for compressing files and then call a command to send it...
Re: UIAutomation with a focus on Chrome
Ok, well, it's not a problem if you see for a while. What I'm looking for is a more refined method of calling specific commands from the menu, not using:
Can this library automate similar behavior?
Code: Select all
Sleep, 300
Send, {AppsKey}
Sleep, 20
WinSet, Transparent, 0, ahk_class #32768
Sleep, 20
Send, {Down 6}
Sleep, 20
Send, {Enter}
Sleep, 20
Send, {Right 1}
Sleep, 20
Send, {Enter}
Re: UIAutomation with a focus on Chrome
@d_romeo, see a few posts up for my reply to user songdg, where we discussed exactly what you are asking (automating a context menu, in his case the one which appears after right-clicking the Desktop).
Re: UIAutomation with a focus on Chrome
in my program, when I click on the element of my interest the only UIA parameter that changes is this (seen with accessibility insights)
LegacyIAccessiblePattern.State 1048576
to
LegacyIAccessiblePattern.State 1048580
How could I ask to UIA to change this property?
Thank you
LegacyIAccessiblePattern.State 1048576
to
LegacyIAccessiblePattern.State 1048580
How could I ask to UIA to change this property?
Thank you
Re: UIAutomation with a focus on Chrome
@jekko1976, LegacyIAccessiblePattern.State is a read-only property which you can't change programatically. If you need to change it then you need a click the element of interest (perhaps with LegacyIAccessiblePattern.DoDefaultAction()?), or do some other action that will change the state (toggling etc).
Re: UIAutomation with a focus on Chrome
@Descolada
Unfortunately the element is an EDIT element, but never change for any value that you insert.
DoDefaultAction does nothing.
I will do like I always did. Searching for the bounding area and clicking in the middle of it.
I Leave herewith the report of Accessibility Insights if you have some different idea.
Thank you
Properties:
AcceleratorKey Property does not exist
AccessKey Property does not exist
AutomationId Property does not exist
BoundingRectangle [l=50,t=373,r=324,b=395]
ControlType Edit(50004)
Culture 0
FillType 0
HasKeyboardFocus True
HeadingLevel HeadingLevel_None (80050)
HelpText Property does not exist
IsContentElement True
IsControlElement True
IsDataValidForForm False
IsDialog False
IsEnabled True
IsKeyboardFocusable True
IsOffscreen False
IsPassword False
IsPeripheral False
IsRequiredForForm False
LegacyIAccessiblePattern.ChildId 1
LegacyIAccessiblePattern.Name Row 6, Column 1
LegacyIAccessiblePattern.Role ROLE_SYSTEM_TEXT(42)
LegacyIAccessiblePattern.State 1048580
LegacyIAccessiblePattern.Value Ritenute e Contributo
LiveSetting 0
LocalizedControlType Modifica
Name Row 6, Column 1
NativeWindowHandle 0
OptimizeForVisualContent False
Orientation None(0)
ProcessId 18228
ProviderDescription [pid:18228,providerId:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:UIAutomationCore.dll)]
ValuePattern.IsReadOnly False
ValuePattern.Value Ritenute e Contributo
VisualEffects 0
Available Patterns:
ValuePattern
- IsReadOnly: False
- Value: Ritenute e Contributo
LegacyIAccessiblePattern
- ChildId: 1
- DefaultAction:
- Description:
- Help:
- KeyboardShorcut:
- Name: Row 6, Column 1
- Role: ROLE_SYSTEM_TEXT(42)
- State: 1048580
- Value: Ritenute e Contributo
Unfortunately the element is an EDIT element, but never change for any value that you insert.
DoDefaultAction does nothing.
I will do like I always did. Searching for the bounding area and clicking in the middle of it.
I Leave herewith the report of Accessibility Insights if you have some different idea.
Thank you
Properties:
AcceleratorKey Property does not exist
AccessKey Property does not exist
AutomationId Property does not exist
BoundingRectangle [l=50,t=373,r=324,b=395]
ControlType Edit(50004)
Culture 0
FillType 0
HasKeyboardFocus True
HeadingLevel HeadingLevel_None (80050)
HelpText Property does not exist
IsContentElement True
IsControlElement True
IsDataValidForForm False
IsDialog False
IsEnabled True
IsKeyboardFocusable True
IsOffscreen False
IsPassword False
IsPeripheral False
IsRequiredForForm False
LegacyIAccessiblePattern.ChildId 1
LegacyIAccessiblePattern.Name Row 6, Column 1
LegacyIAccessiblePattern.Role ROLE_SYSTEM_TEXT(42)
LegacyIAccessiblePattern.State 1048580
LegacyIAccessiblePattern.Value Ritenute e Contributo
LiveSetting 0
LocalizedControlType Modifica
Name Row 6, Column 1
NativeWindowHandle 0
OptimizeForVisualContent False
Orientation None(0)
ProcessId 18228
ProviderDescription [pid:18228,providerId:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:UIAutomationCore.dll)]
ValuePattern.IsReadOnly False
ValuePattern.Value Ritenute e Contributo
VisualEffects 0
Available Patterns:
ValuePattern
- IsReadOnly: False
- Value: Ritenute e Contributo
LegacyIAccessiblePattern
- ChildId: 1
- DefaultAction:
- Description:
- Help:
- KeyboardShorcut:
- Name: Row 6, Column 1
- Role: ROLE_SYSTEM_TEXT(42)
- State: 1048580
- Value: Ritenute e Contributo
Re: UIAutomation with a focus on Chrome
@jekko1976, yeah, an edit element isn't "clickable" in the traditional sense. You can try Element.SetFocus(), but otherwise I don't have better ideas than Click("left") or ControlClick() to focus the element.
Re: UIAutomation with a focus on Chrome
I'm trying to automate Google Lens on chrome using UIA_Browser.ahk, more specifically "image translation":
Error: This class does not contain method "WaitElementExistByName"
Can you help my why and how to succeed in automating this last click?
Here's my complete AHK code - many thanks for looking into this!
Here's the UIAViewer output showing the Google Lens browser window with the Translate-button (marked yellow). I concluded the button name must be "Switch to Translate mode" and thus I'm trying to use the above WaitElementExistByName.Click() method. But somehow it does not work.
- Step 1: Starting SnippingTool.exe and let user select a screenshot
- Step 2: Starting chrome.exe, open the Google Lens page and paste the screenshot with CTRL-V
- Step 3: Wait for the upload to finish and then click the "Translate" button at the bottom of the Google Lens window
Error: This class does not contain method "WaitElementExistByName"
Can you help my why and how to succeed in automating this last click?
Here's my complete AHK code - many thanks for looking into this!
Code: Select all
#include <UIA>
#include <UIA_Browser>
RunWait "SnippingTool.exe /clip"
; This runs the good old windows snipping tool starting in clipping mode; RunWait will wait until the program finishes before continuing.
Run "GoogleChromePortable.exe https:\lens.google.com\search?p="
; Startup Chrome with new Google Lens tab and wait until it's ready
WinWaitActive "ahk_exe chrome.exe"
; Initialize UIA_Browser, use Last Found Window (returned by WinWaitActive)
cUIA := UIA_Browser()
cUIA.WaitPageLoad("Google Lens", 1000)
; Wait until the Google Lens pages has fully loaded
ControlClick "x100 y100", "ahk_exe chrome.exe",,,, "Pos"
; Click left mouse button at coordinates relative to active Chrome window, this is important -> before sending CTRL-V !
Send "^v"
Sleep 500
; Paste the snipped image from clipboard to the Google Lens window by sending CTRL-V and wait 500ms as precaution if WaitPageLoad does not work reliably
cUIA.WaitPageLoad("Google Lens", 1000)
; Wait again until Google Lens upload has finished and the new pages has fully loaded, not sure if this really does anything
cUIA.WaitElementExistByName("Switch to Translate mode").Click()
; THIS SHOULD wait for the Translate button to appear and then click it.... But it throws an error :-(
MsgBox "Now were here and should be done!"
ExitApp
Re: UIAutomation with a focus on Chrome
@emp00, it appears that you are using AHK v2 code and UIA-v2? UIA-v2 and UIA_Interface (which is a AHK v1 library) have some differences, in this case that all UIA_Interface WaitElementExist methods have been replaced by WaitElement. So in UIA-v2 it would be cUIA.WaitElement({Name:"Switch to Translate mode"}).Click(). The condition syntax is explained in the UIA-v2 wiki.
Also I recommend posting v2 code in the v2 section (perhaps the UIA v2 main thread?), otherwise readers might get confused
Also I recommend posting v2 code in the v2 section (perhaps the UIA v2 main thread?), otherwise readers might get confused
-
- Posts: 90
- Joined: 22 Jul 2016, 16:28
Re: UIAutomation with a focus on Chrome
Hi @Descolada !
I have an Idea:
Why not make the functions after and before? that will search the first object after or before a specified one
Example:
cUIA.WaitElementExistByName("Submit").after("Full Editor & Preview").click()
cUIA.WaitElementExistByName("Submit").before("Post Reply").click()
Its noob friendly and helps to better identify something when there are no names, duplicate names, and etc.
I have an Idea:
Why not make the functions after and before? that will search the first object after or before a specified one
Example:
cUIA.WaitElementExistByName("Submit").after("Full Editor & Preview").click()
cUIA.WaitElementExistByName("Submit").before("Post Reply").click()
Its noob friendly and helps to better identify something when there are no names, duplicate names, and etc.
Re: UIAutomation with a focus on Chrome
Hi, @Descolada,
The Windows Editor is now a UWP app and offers tabs. My problem is that the Window Spy tool doesn't display the window title of the current tab. However, I can access the window titles using your UIA class. But I need to get the title of the current tab when multiple tabs are open. Do you have any ideas on how to get only the title of the active tab?
The Windows Editor is now a UWP app and offers tabs. My problem is that the Window Spy tool doesn't display the window title of the current tab. However, I can access the window titles using your UIA class. But I need to get the title of the current tab when multiple tabs are open. Do you have any ideas on how to get only the title of the active tab?
Code: Select all
#NoEnv
#Singleinstance Force
#Include <UIA_Interface>
UIA := UIA_Interface()
el := WinExist("Editor ahk_exe Notepad.exe ahk_class Notepad")
el := UIA.ElementFromHandle(el)
Tabs := el.FindAllBy("ControlType=TabItem AND ClassName=ListViewItem")
For _,Tab in Tabs
MsgBox,% Tab.Name
Re: UIAutomation with a focus on Chrome
@Loop, I can't test this at the moment (running Windows 10), but usually tabs have "SelectionItemIsSelected" property. This means you can either do selectedTab := el.FindFirstBy("ControlType=TabItem AND ClassName=ListViewItem AND SelectionItemIsSelected=1"), or loop over the tabs and test if Tab.SelectionItemIsSelected == 1.
Re: UIAutomation with a focus on Chrome
Thank you, that worked!
Tab.SelectionItemIsSelected == 1
Tab.SelectionItemIsSelected == 1
Return to “Scripts and Functions (v1)”
Who is online
Users browsing this forum: submeg and 121 guests