Code: Select all
modeEl := davinci.FindFirstBy("Name='Mode' AND Type=Text")
modeEl.FindByPath("-2").Highlight() ; moves two elements to the left (or I guess the upward direction in the UIAViewer image)Interacting with DaVinci using UIAutomation
Re: Interacting with DaVinci using UIAutomation
@M0T0GP, you can't use BoundingRectangle to get that element, because the location of the element will change every time you move or scale the window. Since the element doesn't have any identifying features, then you should first find an identifiable element and then travel to the element you actually want. In this case you could use the "Mode" text element:
Note that this is untested code: you might have to play around with FindByPath values a bit if it doesn't work on the first try.
Re: Interacting with DaVinci using UIAutomation
I replace highlight to click('left') or ControlClick but it didn't click anything. thanks.
Re: Interacting with DaVinci using UIAutomation
it is working with controlclick() and click(left). amazing @ Descolada, you saved my tons of time. thanks....I will learn your tool, I found your tool from Joe Automator and Isieas baez, I also check you explain on that channel and Axlefublr. 
Re: Interacting with DaVinci using UIAutomation
@Descolada Hi !
Can UIA look behind open windows?
I have a script which suspends my scripts running for DaVinci Resolve when I Left click in certain areas, then it enables them when I click on other areas, like the Timeline for example.
There is a new Feature in DaVinci Resolve that allows me to transcribe my Audio, but it opens up in a new window (not a sperate window in the Task Bar, but inside in Davinci.
When this new 'Auto Caption window' is Open, UIA only sees the Open window 1st and nothing behind it (e.g. the timeline)
As far as I know, UIA only sees windows On top and nothing behind it, which stops my 'Suspend ON/OFF script' from running when the 'Auto Caption' Window is Open.
Is there a way around this?
Maybe Cache the entire DaVinci Resolve Library? Would that solve UIA not finding things behind a window?
With my script, It does a search every time I click LButton, it kind of has to use that method.
Every time I click on the timeline or anywhere, UIA doesn't know, because the Top window is the new' Auto Caption' window.
Anyways, if you want to see my script here it is below. Sorry if you are a little confused, happy for you to ask questions.
Can UIA look behind open windows?
I have a script which suspends my scripts running for DaVinci Resolve when I Left click in certain areas, then it enables them when I click on other areas, like the Timeline for example.
There is a new Feature in DaVinci Resolve that allows me to transcribe my Audio, but it opens up in a new window (not a sperate window in the Task Bar, but inside in Davinci.
When this new 'Auto Caption window' is Open, UIA only sees the Open window 1st and nothing behind it (e.g. the timeline)
As far as I know, UIA only sees windows On top and nothing behind it, which stops my 'Suspend ON/OFF script' from running when the 'Auto Caption' Window is Open.
Is there a way around this?
Maybe Cache the entire DaVinci Resolve Library? Would that solve UIA not finding things behind a window?
With my script, It does a search every time I click LButton, it kind of has to use that method.
Every time I click on the timeline or anywhere, UIA doesn't know, because the Top window is the new' Auto Caption' window.
Anyways, if you want to see my script here it is below. Sorry if you are a little confused, happy for you to ask questions.
Spoiler
Re: Interacting with DaVinci using UIAutomation
@SteveMylo, since you are running FindFirstBy search on DaVincis main window, then when that "Auto Capture" window is open and TimeLine FindFirstBy search fails, it means that the elements behind that window no longer exist. Caching the UIA tree would only create a snapshot of the time the caching was performed, so the BoundingRectangles from the cache might not contain the real locations if the elements are moved in the meanwhile.
One thought that occurs is that you could try running AHK v2 TreeInspector, which can also inspect controls inside windows. If perchance a control still contains the tree (sometimes it happens that elements are inaccessible from the main window, but ARE accessible from a control) then you could perhaps get it from there.
Another way would be to see whether ElementFromPoint still works. Then you could simply test whether the AutomationId for that element is TimeLine or Viewers: that solution would probably also be faster than the current one.
Other than that I don't have good ideas on how to solve this...
One thought that occurs is that you could try running AHK v2 TreeInspector, which can also inspect controls inside windows. If perchance a control still contains the tree (sometimes it happens that elements are inaccessible from the main window, but ARE accessible from a control) then you could perhaps get it from there.
Another way would be to see whether ElementFromPoint still works. Then you could simply test whether the AutomationId for that element is TimeLine or Viewers: that solution would probably also be faster than the current one.
Other than that I don't have good ideas on how to solve this...
Re: Interacting with DaVinci using UIAutomation
No worries. Thank you. I don’t think i’ll upgrade to V2 for a while. But if I do i can try that.
But I might try element from point. That works in Version 1 doesn’t it?
Many thanks
But I might try element from point. That works in Version 1 doesn’t it?
Many thanks
Re: Interacting with DaVinci using UIAutomation
@SteveMylo, you don't have to upgrade to AHK v2 completely, only use it to use the tool this once. As for ElementFromPoint, it is available in UIA_Interface.ahk/AHK v1 as well.
Re: Interacting with DaVinci using UIAutomation
Great thank you
Re: Interacting with DaVinci using UIAutomation
@Descolada Hi, I just donated 3 coffees to you for all your help 
And I encourage others to think about it 
I tried out the V2 UIAViewer in AutoHotkey V2 and it's fantastic, I love how easy the new PATHS system works.
I'm so impressed that I will now convert a few scripts to V2.
But... previously you wrote me 3 functions in V1 to help with my scripts viewtopic.php?p=497380#p497380
They were great but I'm not sure how to convert those 3 small functions for V2.
I'll paste a mock script with the functions in it, hopefully you can get me out of trouble
And I encourage others to think about it I tried out the V2 UIAViewer in AutoHotkey V2 and it's fantastic, I love how easy the new PATHS system works.
I'm so impressed that I will now convert a few scripts to V2.
But... previously you wrote me 3 functions in V1 to help with my scripts viewtopic.php?p=497380#p497380
They were great but I'm not sure how to convert those 3 small functions for V2.
I'll paste a mock script with the functions in it, hopefully you can get me out of trouble
Spoiler
Many thanks- Attachments
-
- image.png (70.17 KiB) Viewed 985 times
Re: Interacting with DaVinci using UIAutomation
@SteveMylo, thanks, I appreciate it! Also I'm glad to hear you are considering v2, since it's so much more sane
The functions translate fairly easily, albeit needed some extra validation:
You can add caching to maybe get some speed improvements:
The functions translate fairly easily, albeit needed some extra validation:
Code: Select all
#Requires AutoHotkey v2.0
#include <UIA>
x::
{
if !(dvEl := GetDavinciElement())
return
MediaPool := dvEl.ElementExist({AutomationId:"UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Conform.m_pConformPanel.frameEditPageWidgetsContainer.frameEditPageMediaPool.frameMultipleBinGridLayoutContainer.frameMultipleBinLayoutTopLeft.frameTopLeftContent.mediaPoolWidget.frameContainer.frameMediaPoolViews.mediaPoolViewSplitter.frameMediaPoolClipViewContainer"}) ; Media Pool
Media2 := dvEl.ElementExist({AutomationId:"UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Conform.m_pConformPanel.frameEditPageWidgetsContainer.frameEditTimelinesHolder.frameConformTimelines.frameEditTimelineAndAudioView.frameEditTimelinesContainer.splitterEditTimelinesContainer.frameTimelineContainer.frameTimelineViewContainer.timelineViewsSplitter"}) ; timeline
if IsMouseInElement(MediaPool, Media2) {
soundbeep 900, 20
}
}
;\\\\\\\\\\\\\\\\\\\\\\\ FUNCTION \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
GetDavinciElement() {
static davinciId := 0, davinciElement := 0
if (!(dvId := WinExist("ahk_exe Resolve.exe")))
return
if ((davinciId == dvId) && davinciElement)
return davinciElement
else {
davinciElement := "", davinciId := dvId
while (!davinciElement && WinExist("ahk_exe Resolve.exe"))
try davinciElement := UIA.ElementFromHandle(dvId)
return davinciElement
}
}
IsMouseInElement(els*) { ; Returns 1 in the mouse is inside any of the provided UIA elements
for el in els {
if !IsObject(el)
continue
Coords := el.BoundingRectangle
if IsMouseInRect(Coords.l, Coords.t, Coords.r, coords.b)
return 1
}
return 0
}
IsMouseInRect(X1, Y1, X2, Y2, relative:="screen") {
savedCoordMode := A_CoordModeMouse
CoordMode "Mouse", relative
MouseGetPos &mouseX, &mouseY
CoordMode "Mouse", savedCoordMode
return (mouseX > X1 && mouseX < X2) && (mouseY > Y1 && mouseY < Y2)
}Code: Select all
#Requires AutoHotkey v2.0
#include <UIA>
#HotIf WinActive("ahk_exe Resolve.exe")
x::
{
static cacheRequest := UIA.CreateCacheRequest(["AutomationId", "BoundingRectangle"],,, "None")
if !(dvEl := GetDavinciElement())
return
; Find all elements with one search, and if need to determine which elements were found use el.CachedAutomationId
els := dvEl.FindElements([{AutomationId:"UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Conform.m_pConformPanel.frameEditPageWidgetsContainer.frameEditPageMediaPool.frameMultipleBinGridLayoutContainer.frameMultipleBinLayoutTopLeft.frameTopLeftContent.mediaPoolWidget.frameContainer.frameMediaPoolViews.mediaPoolViewSplitter.frameMediaPoolClipViewContainer"} ; Media Pool
, {AutomationId:"UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Conform.m_pConformPanel.frameEditPageWidgetsContainer.frameEditTimelinesHolder.frameConformTimelines.frameEditTimelineAndAudioView.frameEditTimelinesContainer.splitterEditTimelinesContainer.frameTimelineContainer.frameTimelineViewContainer.timelineViewsSplitter"}],,,,cacheRequest) ; timeline
;for el in els {
; br := el.CachedBoundingRectangle
; MsgBox el.CachedAutomationId " is located at x" br.l " y" br.t
;}
if IsMouseInElement(els*) {
soundbeep 900, 20
}
}
;\\\\\\\\\\\\\\\\\\\\\\\ FUNCTION \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
GetDavinciElement() {
static davinciId := 0, davinciElement := 0
if (!(dvId := WinExist("ahk_exe Resolve.exe")))
return
if ((davinciId == dvId) && davinciElement)
return davinciElement
davinciElement := 0, davinciId := dvId
while (!davinciElement && WinExist("ahk_exe Resolve.exe"))
try davinciElement := UIA.ElementFromHandle(dvId)
return davinciElement
}
IsMouseInElement(els*) { ; Returns 1 in the mouse is inside any of the provided UIA elements
for el in els {
if !IsObject(el)
continue
Coords := el.CachedBoundingRectangle
if IsMouseInRect(Coords.l, Coords.t, Coords.r, coords.b)
return 1
}
return 0
}
IsMouseInRect(X1, Y1, X2, Y2, relative:="screen") {
savedCoordMode := A_CoordModeMouse
CoordMode "Mouse", relative
MouseGetPos &mouseX, &mouseY
CoordMode "Mouse", savedCoordMode
return (mouseX > X1 && mouseX < X2) && (mouseY > Y1 && mouseY < Y2)
}Re: Interacting with DaVinci using UIAutomation
@Descolada Thankyou Appreciate it.
Although the script definitely works, it seems AUTOHOTKEY V2 throws up an error window if the window isn't present, eg another Davinci window or the Color Page etc. Even if I'm still in Davinci
See pic attached
In version 1 , we previously avoided this with the line of code that you made for me, which worked to stop an error message:
if !(dvEl := GetDavinciElement())
return
It seems to not do anything in V2.
Anyways, I'll paste my V2 code below, I'm wondering if we can prevent V2 from showing an error message when an Element is not active?:
Although the script definitely works, it seems AUTOHOTKEY V2 throws up an error window if the window isn't present, eg another Davinci window or the Color Page etc. Even if I'm still in Davinci
See pic attached
In version 1 , we previously avoided this with the line of code that you made for me, which worked to stop an error message:
if !(dvEl := GetDavinciElement())
return
It seems to not do anything in V2.
Anyways, I'll paste my V2 code below, I'm wondering if we can prevent V2 from showing an error message when an Element is not active?:
Spoiler
- Attachments
-
- image.png (12.47 KiB) Viewed 843 times
Re: Interacting with DaVinci using UIAutomation
@Descolada I seemed to have fixed this by using WaitElementFromPath with a timeout of '55' , but is this a hacky way of doing it?
Re: Interacting with DaVinci using UIAutomation
UIA-v2 FindElement and ElementFromPath throw errors if the element is not found, because they assume you've done all the checking necessary to ensure the elements are there. This is for easier debugging, because you'll instantly get feedback about what went wrong. In this case you can instead use the corresponding "Exist" methods: ElementExist or ElementFromPathExist, which return either 0 or the element. FindElements doesn't throw an error, instead an empty array is returned in that case.
This change is similar to AHK v2 native functions in general: for example WinActivate(WinTitle) will also throw an error if no matching window is found.
Re: Interacting with DaVinci using UIAutomation
Thanks, all seems good.
I'm now a little curious about caching.
Regarding the code you mentioned static cacheRequest := UIA.CreateCacheRequest(["AutomationId", "BoundingRectangle"],,, "None")
I noticed it doesn't work with Paths, is that right? Just AutomationId: etc
Because I now like working with 'Condition Paths' with the 'UIA viewer' Autohotkey V2. It is more accurate for sure. e.g. ({T:11,CN:"UiMenuItemAction", i:15})
I'm now a little curious about caching.
Regarding the code you mentioned static cacheRequest := UIA.CreateCacheRequest(["AutomationId", "BoundingRectangle"],,, "None")
I noticed it doesn't work with Paths, is that right? Just AutomationId: etc
Because I now like working with 'Condition Paths' with the 'UIA viewer' Autohotkey V2. It is more accurate for sure. e.g. ({T:11,CN:"UiMenuItemAction", i:15})
Re: Interacting with DaVinci using UIAutomation
@SteveMylo, the answer is yes and no. First some explanations for why the cached version code should be faster:
1) Reason number 1 is not related to caching at all, but to using FindElements instead of multiple FindElement (or FindFirstBy in the case of UIA_Interface). FindElements traverses the whole tree just once and returns all the found elements, whereas calling FindElement multiple times might need to traverse more of the tree in sum. For example, if the target element is right in the middle of the tree, than a single FindElement will be faster than a single FindElements. Calling FindElement twice will be as fast as FindElements; calling it three times is already slower than FindElement once. So while it depends on where the elements are in the tree, I usually try to use FindElements instead of FindElement if I need to call it more than once, or I use FindElement and specify the startingElement argument. In this case, since I don't know the order of the elements I opted for FindElements.
2) Caching the properties will make reading them faster and also prevents problems such as the element being invalidated in the time between finding it and accessing its properties.
3) Using AutomationElementMode.None ("None" in the CreateCacheRequest call) will not return the live element at all, which also gets big speed improvements.
ElementFromPath is essentially calling FindElement with TreeScope.Children for each condition in the condition path. Caching all these elements only to not use them afterwards doesn't make sense, but if you wish to cache the found element (and its properties) then you can use Element.BuildUpdatedCache() to do that.
Another way would be to cache the whole tree and then walk the path using cached properties (the corresponding function is CachedElementFromPath). This may or may not give a speed improvement depending on how many times you plan to use ElementFromPath. Since ElementFromPath uses Type, AutomationId, ClassName and Name, then you need to cache these along with any other property you want to use. Something like this:
IsMouseInElement would need to use CachedBoundingRectangle as well, like in my example.
If you want to also access the live element (along with live properties, use patterns etc) then omit the "None" from the cacheRequest. That will make the caching significantly slower though.
1) Reason number 1 is not related to caching at all, but to using FindElements instead of multiple FindElement (or FindFirstBy in the case of UIA_Interface). FindElements traverses the whole tree just once and returns all the found elements, whereas calling FindElement multiple times might need to traverse more of the tree in sum. For example, if the target element is right in the middle of the tree, than a single FindElement will be faster than a single FindElements. Calling FindElement twice will be as fast as FindElements; calling it three times is already slower than FindElement once. So while it depends on where the elements are in the tree, I usually try to use FindElements instead of FindElement if I need to call it more than once, or I use FindElement and specify the startingElement argument. In this case, since I don't know the order of the elements I opted for FindElements.
2) Caching the properties will make reading them faster and also prevents problems such as the element being invalidated in the time between finding it and accessing its properties.
3) Using AutomationElementMode.None ("None" in the CreateCacheRequest call) will not return the live element at all, which also gets big speed improvements.
ElementFromPath is essentially calling FindElement with TreeScope.Children for each condition in the condition path. Caching all these elements only to not use them afterwards doesn't make sense, but if you wish to cache the found element (and its properties) then you can use Element.BuildUpdatedCache() to do that.
Another way would be to cache the whole tree and then walk the path using cached properties (the corresponding function is CachedElementFromPath). This may or may not give a speed improvement depending on how many times you plan to use ElementFromPath. Since ElementFromPath uses Type, AutomationId, ClassName and Name, then you need to cache these along with any other property you want to use. Something like this:
Code: Select all
static cacheRequest := UIA.CreateCacheRequest(["Type", "AutomationId", "ClassName", "Name", "BoundingRectangle"],,5, "None") ; 5 == Descendants + Element
if !(dvEl := GetDavinciElement())
return
cachedEl := dvEl.BuildUpdatedCache(cacheRequest)
found := cachedEl.CachedElementFromPath(({T:11,CN:"UiMenuItemAction", i:15})If you want to also access the live element (along with live properties, use patterns etc) then omit the "None" from the cacheRequest. That will make the caching significantly slower though.
Who is online
Users browsing this forum: No registered users and 121 guests