UIAutomation with a focus on Chrome
Re: UIAutomation with a focus on Chrome
You are very welcome my friend.
Re: UIAutomation with a focus on Chrome
Hello,
i want to move mouse on an element without clicking, how can I do that?
Thx
i want to move mouse on an element without clicking, how can I do that?
Thx
Re: UIAutomation with a focus on Chrome
@Loop, currently there is no built-in method to do that. Easiest way would be to use
Though be careful if your coordmode is not set to Screen that the window is active...
Code: Select all
pos := Element.GetCurrentPos()
MouseMouse, % pos.x + pos.w//2, % pos.y+pos.h//2
Re: UIAutomation with a focus on Chrome
many thanks @Descolada
I have the following problem, I move the mouse over a button after 1sec, then a Pic appears to the right of the button, on this picture I must make a left click.
My script works:
but 280 is the problem, on different computers the distance is different, is there a reliable way.
So click left on an image that is located to the right of a button.
I have the following problem, I move the mouse over a button after 1sec, then a Pic appears to the right of the button, on this picture I must make a left click.
My script works:
Code: Select all
AudioWalker := UIA.CreateTreeWalker(UIA.CreateCondition("ControlType=Button AND Name='Voicemail'"))
NextAudio := AudioWalker.GetNextSiblingElement(FirstAudio)
CurrPos := NextAudio.GetCurrentPos()
MouseMove, % CurrPos.x + CurrPos.w//2 , % CurrPos.y + CurrPos.h//2
Sleep, 1500
Click, % CurrPos.x + 280 " " CurrPos.y " left"
So click left on an image that is located to the right of a button.
Re: UIAutomation with a focus on Chrome
@Loop, if the image is an element identifiable by UIA then you could use that to position the cursor (you could probably use a TreeWalker to get it). Otherwise if it always appears right next to the button then perhaps Click, % CurrPos.x + CurrPos.w + some value " " CurrPos.y " left" would work more reliably.
Re: UIAutomation with a focus on Chrome
No, unfortunately not! The only distinguishing factor is ControlType=imageif the image is an element identifiable by UIA
Thanks, for the suggestion
Re: UIAutomation with a focus on Chrome
Is it possible with UIA to "connect" to an already open Chrome session/page?
In other words, I want to start Chrome manually, then navigate some pages, then make my script "get" this page and finish the job navigating/clicking/ecc...
Thank you
In other words, I want to start Chrome manually, then navigate some pages, then make my script "get" this page and finish the job navigating/clicking/ecc...
Thank you
Re: UIAutomation with a focus on Chrome
@jekko1976, yes you can "connect" to a Chrome window at any point. If you are using UIA_Browser then just use cUIA := new UIA_Browser("browser title or ahk_id/ahk_exe").
Re: UIAutomation with a focus on Chrome
@Descolada
Yes!
I have written a test code with this exact page and it seems to work!
Thank you so much!
Yes!
I have written a test code with this exact page and it seems to work!
Thank you so much!
Code: Select all
#include Lib\UIA_Interface.ahk
#include Lib\UIA_Browser.ahk
#SingleInstance Force
ch := new UIA_Browser("UIAutomation with a focus on Chrome - Page 16 - AutoHotkey Community - Google Chrome")
ch.WaitElementExistByName("Login").Click()
UIA for Chromium windows (Skype, Discord etc)
I think I've had a breakthrough for the annoying problem of getting UIA access to Chromium windows such as Skype and Discord! The secret appears to be to get the Chrome_RenderWidgetHostHWND1 control and then use that hwnd to get the element! I've added a new method UIA_Interface.ElementFromChromium(winTitle="A", activateChromiumAccessibility=True), and made all other activateChromiumAccessibility flags (eg for ElementFromHandle) ByRef so they will also return that element on the first run.
Example:
Example2:
Example:
Code: Select all
#include <UIA_Interface>
UIA := UIA_Interface()
skypeEl := UIA.ElementFromChromium("ahk_exe skype.exe")
MsgBox, % skypeEl.DumpAll()
Code: Select all
#include <UIA_Interface>
UIA := UIA_Interface()
winEl := UIA.ElementFromHandle("ahk_exe skype.exe", skypeEl)
MsgBox, % winEl.DumpAll() ; will not show the content
MsgBox, % skypeEl.DumpAll() ; WILL show the content, but only on the first run (since UIA will be activated only once per run)
; The following will NOT work, since it would be ran for a second time. Use ElementFromChromium for multiple accesses
; winEl := UIA.ElementFromHandle("ahk_exe skype.exe", skypeEl)
; MsgBox, % skypeEl.DumpAll()
-
- Posts: 90
- Joined: 22 Jul 2016, 16:28
Re: UIA for Chromium windows (Skype, Discord etc)
Wow congratulations!! You are the best ahk developer right now!Descolada wrote: ↑09 Sep 2022, 12:36I think I've had a breakthrough for the annoying problem of getting UIA access to Chromium windows such as Skype and Discord! The secret appears to be to get the Chrome_RenderWidgetHostHWND1 control and then use that hwnd to get the element! I've added a new method UIA_Interface.ElementFromChromium(winTitle="A", activateChromiumAccessibility=True), and made all other activateChromiumAccessibility flags (eg for ElementFromHandle) ByRef so they will also return that element on the first run.
Example:Example2:Code: Select all
#include <UIA_Interface> UIA := UIA_Interface() skypeEl := UIA.ElementFromChromium("ahk_exe skype.exe") MsgBox, % skypeEl.DumpAll()
Code: Select all
#include <UIA_Interface> UIA := UIA_Interface() winEl := UIA.ElementFromHandle("ahk_exe skype.exe", skypeEl) MsgBox, % winEl.DumpAll() ; will not show the content MsgBox, % skypeEl.DumpAll() ; WILL show the content, but only on the first run (since UIA will be activated only once per run) ; The following will NOT work, since it would be ran for a second time. Use ElementFromChromium for multiple accesses ; winEl := UIA.ElementFromHandle("ahk_exe skype.exe", skypeEl) ; MsgBox, % skypeEl.DumpAll()
Re: UIAutomation with a focus on Chrome
@Descolada - stumbled over this, (probably outdated but worth reading): https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/
Hell, and that started in 2013 with now > 40 pages https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/
...where fellow @malcev has provided an AHK script (@ page44) without having been tortured by AU3 fundamentalist afterward
To whom it may concern: a bunch of "viewers" kindly recommended by malcev: https://github.com/blackrosezy/gui-inspect-tool
Hell, and that started in 2013 with now > 40 pages https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/
...where fellow @malcev has provided an AHK script (@ page44) without having been tortured by AU3 fundamentalist afterward
To whom it may concern: a bunch of "viewers" kindly recommended by malcev: https://github.com/blackrosezy/gui-inspect-tool
Re: UIAutomation with a focus on Chrome
Any way to create a uia element for ahk_class Framework::CFrame? Seems like UIA_Interface isn't a fan of modern Microsoft Office apps?
Re: UIAutomation with a focus on Chrome
@Skrell, it should be possible to use ANY window identifier you use with other AHK functions to also get UIA elements. That means ElementFromHandle("ahk_class Framework::CFrame") is valid, and even using control hwnds is valid (this can be gotten with ControlGet Hwnd and then used with "ahk_id ...").
I haven't played much with Office apps - is there any specific problem you are having trouble with?
I haven't played much with Office apps - is there any specific problem you are having trouble with?
Re: UIAutomation with a focus on Chrome
My apologies, it was user error. Other topic, is there a way to limit how far down each node the tree builds when using UIA.ElementFromHandle? So for instance, if I print out the results using npEl.DumpAll(), there are a lot of sub-sub-sub nodes displayed. ANy way to limit how deep of these nodes gets built?Descolada wrote: ↑15 Sep 2022, 23:00@Skrell, it should be possible to use ANY window identifier you use with other AHK functions to also get UIA elements. That means ElementFromHandle("ahk_class Framework::CFrame") is valid, and even using control hwnds is valid (this can be gotten with ControlGet Hwnd and then used with "ahk_id ...").
I haven't played much with Office apps - is there any specific problem you are having trouble with?
Code: Select all
1 Type: 50033 (Pane) LocalizedControlType: "Pane" ClassName: "NetUIPanViewer"
1.1 Type: 50026 (Group) LocalizedControlType: "Group" ClassName: "NetUITWMenuItemGroup"
1.1.1 Type: 50011 (MenuItem) Name: "Copy" LocalizedControlType: "Menu Item" ClassName: "NetUITWBtnMenuItem"
1.1.2 Type: 50011 (MenuItem) Name: "Quick Print" LocalizedControlType: "Menu Item" ClassName: "NetUITWBtnMenuItem"
1.2 Type: 50026 (Group) LocalizedControlType: "Group" ClassName: "NetUITWMenuItemGroup"
1.2.1 Type: 50011 (MenuItem) Name: "Reply" LocalizedControlType: "Menu Item" ClassName: "NetUITWBtnMenuItem"
1.2.2 Type: 50011 (MenuItem) Name: "Reply All" LocalizedControlType: "Menu Item" ClassName: "NetUITWBtnMenuItem"
1.2.3 Type: 50011 (MenuItem) Name: "Forward" LocalizedControlType: "Menu Item" ClassName: "NetUITWBtnMenuItem"
Re: UIAutomation with a focus on Chrome
@Skrell, you can limit DumpAll with the maxDepth argument. el.DumpAll(0) will only "build" once layer, el.DumpAll(1) two layers etc. Though I just noticed this is weird notation (1 should show 1 layer...), so I just fixed this and el.DumpAll(1) will show one layer and el.DumpAll(0) the node itself (the updated UIA_Interface.ahk is in Github).
Re: UIAutomation with a focus on Chrome
Sorry, I wasn't clear, I want to speed up UIA.ElementFromHandle by having it build an element with only part of the tree by limiting its depth... is that a possibility?Descolada wrote: ↑16 Sep 2022, 10:36@Skrell, you can limit DumpAll with the maxDepth argument. el.DumpAll(0) will only "build" once layer, el.DumpAll(1) two layers etc. Though I just noticed this is weird notation (1 should show 1 layer...), so I just fixed this and el.DumpAll(1) will show one layer and el.DumpAll(0) the node itself (the updated UIA_Interface.ahk is in Github).
Re: UIAutomation with a focus on Chrome
@Skrell, ElementFromHandle doesn't "build" anything, it only gets an element from an already existing UIA tree. Building the tree is handled by Microsoft UIAutomation library itself and the application. The only way to speed up ElementFromHandle is to set the "activateChromiumAccessibility" flag to False, which otherwise might cause slowdowns if the app is a Chromium app (Skype, VSCode etc) and doesn't require activating.
Re: UIAutomation with a focus on Chrome
Got it ty!Descolada wrote: ↑16 Sep 2022, 12:44@Skrell, ElementFromHandle doesn't "build" anything, it only gets an element from an already existing UIA tree. Building the tree is handled by Microsoft UIAutomation library itself and the application. The only way to speed up ElementFromHandle is to set the "activateChromiumAccessibility" flag to False, which otherwise might cause slowdowns if the app is a Chromium app (Skype, VSCode etc) and doesn't require activating.
Re: UIAutomation with a focus on Chrome
i'd need to fire a select option on a site, on a html like:
and eventually fire an change event.
in javascript i would use Setattribute('selected','selected')
Is there a way to do this action with UIAutomation?
Thank you
Code: Select all
<select name="pagamenti_table_length" aria-controls="pagamenti_table" class="custom-select custom-select-sm form-control form-control-sm">
<option value="10">10</option>
<option value="25">25</option>
<option value="50">50</option>
<option value="100">100</option>
</select>
in javascript i would use Setattribute('selected','selected')
Is there a way to do this action with UIAutomation?
Thank you