@mora145, leosouza85's answer should work. Perhaps you aren't setting your variable LanguagueTarget correctly?
Re: UIAutomation with a focus on Chrome
Posted: 31 Jul 2022, 14:04
by mora145
I just tested, and yes, it works After testing for about 24 hours. It was the capital letter at the beginning that didn't match. Thanks @Descolada and @leosouza85
Re: UIAutomation with a focus on Chrome
Posted: 02 Aug 2022, 08:59
by crocodile
If there is an image under the mouse, but it has no image menu to save. Can I use UIAutomation to save it locally? Thanks.
Re: UIAutomation with a focus on Chrome
Posted: 02 Aug 2022, 10:06
by Descolada
@crocodile, AFAIK you cannot save UIAutomation image elements into files using UIAutomation. But you can get the exact position and size of the element with UIA, then use another library to take a screenshot of that area and save it. For example the Gdip library function Gdip_BitmapFromScreen does that.
Re: UIAutomation with a focus on Chrome
Posted: 02 Aug 2022, 11:14
by crocodile
thank you very much.
Re: UIAutomation with a focus on Chrome
Posted: 03 Aug 2022, 07:44
by kasandra
Hi,
I want to use this UIA library to automate some functions in MSPaint. I noted that it doesn't run from certain depth of this app, e.g. element by name "Obrót w prawo o 90°" (eng. "90° right rotation"). Do you have any solution for that case?
@kasandra, the menu item "Obrót w prawo o 90°" will not show up in DumpAll until it is actually visible: you need to put DumpAll after the line firststepE1 := pI0.FindFirstByNameAndType("Obróć", "SplitButton").Click(), or inspect it with an inspect tool like UIAViewer.
Your provided code worked perfectly in my computer, though I did have to use English names for buttons:
firststepE1 := pI0.FindFirstByNameAndType("Rotate", "SplitButton").Click()
secondstepE1 := pI0.WaitElementExistByName("Rotate right 90°")
Note that you need to save the code in UTF-8-BOM encoding, otherwise the ° sign won't be recognized. Or alternatively avoid using ° in your code: secondstepE1 := pI0.WaitElementExistByName("Obrót w prawo o 90",,2)
Re: UIAutomation with a focus on Chrome
Posted: 03 Aug 2022, 09:09
by kasandra
@Descolada Thanks for quick reply, it works correctly now
Re: UIAutomation with a focus on Chrome
Posted: 05 Aug 2022, 10:59
by BoBo
Here another challenge with probably a billion people desperately waiting in line to get it solved
There are several command line tools that can cast local media files on a Google Chromecast enabled device, but I haven’t seen any that will establish the connection to simply display the PC's desktop instead (ie to show a livestream within a fullscreen browser session).
Here's an attempt to do this manually, but I'd guess it'll be a hell smarter using UIA…
@BoBo, I don't have experience with Chromecasting, but gave it a shot anyway. Couldn't debug it thoroughly (only the "Cast tab" option was available), also I have no idea why it has to be done in a new Chrome window.
@Descolada - unfortunately it's throwing an error.
I've updated Chrome to its latest release: Version 104.0.5112.81...
executed the script with an active Chrome instance - and no instance.
Ran the script with RunNewChrome := false ;and true...
to no avail *sigh*
The current thread will exit.
---------------------------
OK
---------------------------
@malcev would love to see that in action, BUT I wouldn't be able to sort this out on my own, so ending up torturing a fellow supporter with a bunch of stupid questions. Not that attractive perspective, right?
@BoBo, I was using a relatively new syntax of UIA_Interface/UIA_Browser: chrome := UIA.ElementFromHandle("ahk_exe chrome.exe") instead of chrome := UIA.ElementFromHandle(WinExist("ahk_exe chrome.exe")). Try changing that line, or updating UIA_Interface (and UIA_Browser).
Re: UIAutomation with a focus on Chrome
Posted: 06 Aug 2022, 10:06
by BoBo
@Descolada - got it. Pushed it into a function.
Question: is there a chance to extract the current "cast mode" setting/label from this pane for further validation?
... because it's possible using that same function to toggle the connection to a device, but only if that label hasn't changed meanwhile (ie by manual intervention).
#SingleInstance, Force
SetBatchLines, -1
#include <UIA_Interface>
Global en, de
en := ["Cast…","Sources","Cast tab","Cast screen","Cast file"]
de := ["Streamen…","Quellen","Tab streamen","Bildschirm streamen","Datei streamen"]
!c::cast("kitchen") ;press ALT+c to connect/cast to an active chromecast device
cast(myDevice:="living room",mode:="en",RunNewChrome:=false) { ;mode:="de" ;myDevice (case sensitive, matching partial names)
UIA := UIA_Interface()
if RunNewChrome
Run, chrome.exe --fullscreen --start-maximized ;--lang=en-US,en
WinWaitActive, ahk_exe chrome.exe
chrome := UIA.ElementFromHandle("ahk_exe chrome.exe")
chrome.FindFirstBynameAndType("Chrome", "MenuItem").Click()
chrome.WaitElementExistByNameAndType(%mode%[1], "MenuItem",,2).Click() ;selecting Chrome's streaming option
castTab := chrome.WaitElementExistByNameAndType(%mode%[3], "Pane",,1) ;waiting till selection pane occurs
castTab.FindFirstByNameAndType(%mode%[2], "Button").Click() ;activating 'Sources' drop down list
castTab.FindFirstByNameAndType(%mode%[3], "MenuItem",,2).Click() ;selecting casting mode 3=tab/4=screen/5=file
castTab.FindFirstByNameAndType(myDevice, "Button",,2).Click() ;selecting device
if RunNewChrome
WinClose, ahk_exe chrome.exe
return
}
@malcev - I'll have a look at it. Thx a lot (and no, I wouldn't interfere if you would play with it meanwhile)
Re: UIAutomation with a focus on Chrome
Posted: 07 Aug 2022, 01:41
by Descolada
@BoBo, I think you can check whether the device is available or not by the second line of its name: if InStr(StrSplit(castTab.FindFirstByNameAndType(DeviceName, "Button",,2).CurrentName, "`n")[2], "Available") { ... }.
myDevice:="living room"
;
;
;
MsgBox % castTab.FindFirstByNameAndType(myDevice, "Button",,2) ;empty
MsgBox % StrSplit(castTab.FindFirstByNameAndType(myDevice, "Button",,2), "`n")[2] ;spliting emptyness
MsgBox % InStr(StrSplit(castTab.FindFirstByNameAndType("myDevice", "Button",,2), "`n")[2], "Available") ;0 not changing bc of the above
The challenge, I need to identify that cast mode to be able to toggle/switch off a currently running session in case I wanna change the cast mode from 'screen' to 'tab' or vice versa if done for the same device.
If I change the cast mode and the device at the same time it'll simply terminate the running session.
Interesting side fact: you can cast a Chrome "tab" but not a "screen" to a screenless device (like the Google Home Mini)
Descolada:
Couldn't debug it thoroughly (only the "Cast tab" option was available),...
Isn't that depending on the source application that has the focus at the time you initialize the cast??
...also I have no idea why it has to be done in a new Chrome window.
Weird, as I can use an already YT/AHK/whatever loaded Chrome tab and it connects fine.
PS. do I remember it correctly that all of these UIA events can't be executed 'hidden'-style?
Re: UIAutomation with a focus on Chrome
Posted: 07 Aug 2022, 11:52
by Descolada
@BoBo, I'm so sorry, in a hurry it seems I omitted the Name property of the element I changed my last post to reflect that: InStr(StrSplit(castTab.FindFirstByNameAndType(DeviceName, "Button",,2).CurrentName, "`n")[2], "Available") is correct.
Regarding the current cast mode:
1) Your option of castTab.GetCurrentPropertyValue("Name") works, gets the pane name. This is equivalent to castTab.CurrentName, and in newer UIA_Interface versions also castTab.Name
2) Second option would be to get the TitleBar name: castTab.FindFirstByType("TitleBar").CurrentName
3) Third option is to get the toggle state of the menu item button:
Isn't that depending on the source application that has the focus at the time you initialize the cast??
Yeah, I have no idea. As I said, I haven't used Chromecast and I only managed to get it working in YouTube...
Weird, as I can use an already YT/AHK/whatever loaded Chrome tab and it connects fine.
Yes, it works for me too. I was referring to the StackOverflow post you originally posted (https://stackoverflow.com/a/67202498) which did that for whatever reason.
do I remember it correctly that all of these UIA events can't be executed 'hidden'-style?
Best to do it when Chrome is active, but you can try it out in unfocused mode. For that remove the WinWaitActive line and set AutoSetFocus to False:
Might not be reliable though, so caution is advised
Re: UIAutomation with a focus on Chrome
Posted: 07 Aug 2022, 12:32
by BoBo
@Descolada - Thx a lot for your explanations, much appreciated. Stay tuned
Re: UIAutomation with a focus on Chrome
Posted: 09 Aug 2022, 15:26
by TomDonovan
Does anyone have a sample of creating an event to monitor for a change in a UIA Property
In Outlook desktop, there are tabs across the top. Home tab is the one I am interested in. I am interested in the IsSelected Property
A real simplification would be
OutlookMainID := WinExist("ahk_exe outlook.exe")
elementMain := uia.ElementFromHandle(OutlookMainID)
homeTab := elementMain.FindFirstByNameAndType("Home","TabItem")
SelectionPattern := HomeTab.GetCurrentPatternAs("SelectionItemPattern")
if SelectionPattern.IsSelected
make my gui visible
else
hide my gui
I can do this thru a timer, but think it would be more efficient to have an event handler notify me with IsSelected changes. From Accessibility Insights, I know the event is raised - Property ID 30079, SelectionItemPattern.IsSelected is what shows in Accessibility Insights.
But I haven't figure out how to connect to the even.
Any and all help (or kicks in the butt if this is more obvious than I am seeing) appreciated.