Interacting with DaVinci using UIAutomation

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: Interacting with DaVinci using UIAutomation

Post by SteveMylo » 19 Dec 2022, 21:25

G'day @Descolada , thanks for everything, it really helps me.
Is there a way to shorten the below example code?
The basic script involves, When I push x:: a SoundBeep occurs in either of two UIA areas.

I'm Trying to avoid writing two different CurrentBoundingRectangle areas and Coords cause The code will get BIG if I start adding a lot more UIA areas.

....maybe activating it 1st the see if the mouse is focused under the activated UIA area :?: :problem: 😕
P.s. I'm happy to pay for you The PAID version of DaVinci Resolve STUDIO if you don't have it already, if you need it 😊

SEE CODE BELOW:
Spoiler

Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: Interacting with DaVinci using UIAutomation

Post by Descolada » 20 Dec 2022, 00:24

@SteveMylo, if you see yourself using the same code all over again, it is usually a good idea to put it in a function:

Code: Select all

x::
  if !(dvEl := GetDavinciElement())
    return
  MediaPool := dvEl.FindFirstBy("AutomationId=UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Conform.m_pConformPanel.frameEditPageWidgetsContainer.frameEditPageMediaPool.frameMultipleBinGridLayoutContainer.frameMultipleBinLayoutTopLeft.frameTopLeftContent.mediaPoolWidget.frameContainer.frameMediaPoolViews.mediaPoolViewSplitter.frameMediaPoolClipViewContainer") ; Media Pool
  Media2 := dvEl.FindFirstBy("AutomationId=UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Conform.m_pConformPanel.frameEditPageWidgetsContainer.frameEditTimelinesHolder.frameConformTimelines.frameEditTimelineAndAudioView.frameEditTimelinesContainer.splitterEditTimelinesContainer.frameTimelineContainer.frameTimelineViewContainer.timelineViewsSplitter") ; timeline

  if IsMouseInElement(MediaPool, Media2)
  {
    soundbeep, 900, 20
    return
  }
  return

 ;\\\\\\\\\\\\\\\\\\\\\\\ FUNCTION \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
GetDavinciElement() {
    static davinciId, davinciElement

    if (!(dvId := WinExist("ahk_exe Resolve.exe")))
    	return
    if ((davinciId == dvId) && davinciElement)
        return davinciElement
    else {
        davinciElement := "", davinciId := dvId
        while (!davinciElement && WinExist("ahk_exe Resolve.exe")) {
            try davinciElement := UIA_Interface().ElementFromHandle(dvId)
        }
        return davinciElement
    }
}
IsMouseInElement(els*) { ; Returns 1 in the mouse is inside any of the provided UIA elements
   for _, el in els {
       Coords := el.CurrentBoundingRectangle
       if IsMouseInRect(Coords.l, Coords.t, Coords.r, coords.b)
           return 1
    }
   return 0
}
IsMouseInRect(X1, Y1, X2, Y2, relative:="screen") {
   savedCoordMode := A_CoordModeMouse
   CoordMode, Mouse, %relative%
   MouseGetPos, mouseX, mouseY
   CoordMode, Mouse, %savedCoordMode%
   return (mouseX > X1 && mouseX < X2) && (mouseY > Y1 && mouseY < Y2)
}
It would probably be best though to first check which panel is active (Media, Cut, Edit etc), and then only check the corresponding elements. Otherwise you will be doing a lot of wasted FindFirst-st.
Also, in your hotkey, if you don't care whether the mouse is in Media Pool or Timeline, then you could just do one FindFirstBy (which will be faster):
ElOfInterest := dvEl.FindFirstBy("AutomationId=UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Conform.m_pConformPanel.frameEditPageWidgetsContainer.frameEditPageMediaPool.frameMultipleBinGridLayoutContainer.frameMultipleBinLayoutTopLeft.frameTopLeftContent.mediaPoolWidget.frameContainer.frameMediaPoolViews.mediaPoolViewSplitter.frameMediaPoolClipViewContainer OR AutomationId=UiMainWindow.bigBossWidget.widgetStack_Panel.WStackPage_Conform.m_pConformPanel.frameEditPageWidgetsContainer.frameEditTimelinesHolder.frameConformTimelines.frameEditTimelineAndAudioView.frameEditTimelinesContainer.splitterEditTimelinesContainer.frameTimelineContainer.frameTimelineViewContainer.timelineViewsSplitter")

I'm gonna keep you offer in mind for now since I don't use DaVinci that regularly, but thanks :)

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: Interacting with DaVinci using UIAutomation

Post by SteveMylo » 20 Dec 2022, 03:06

@Descolada those functions worked a treat thank so much :-)

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: Interacting with DaVinci using UIAutomation

Post by SteveMylo » 28 Jan 2023, 22:27

@Descolada Hey there,
Do you know if Python coding can use the same elements found the UIA tools?
Does a UIA library exist for Python?

Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: Interacting with DaVinci using UIAutomation

Post by Descolada » 29 Jan 2023, 00:34

@SteveMylo, I don't think you can "share" the same element objects between AHK and other languages (e.g find the element with AHK, pass the pointer to Python, click it with Python). Perhaps if you passed both the UIA interface object pointer and the element pointer, and created the objects in the other language from those pointers? Though I'm not sure why you'd want to do that...

I haven't used Python for UIAutomation, but a quick Google search does return some wrapper libraries. In fact, most languages have some kind of wrappers available for it. Whether they implement any helper functions (like easier Find methods), I cannot say.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: Interacting with DaVinci using UIAutomation

Post by SteveMylo » 29 Jan 2023, 00:56

@Descolada Yeah the reason I ask is because I'm in partnership to sell a large amounts of scripts, over 10,000 plus..... but we use AUTOHOTKEY obviously.

And I was told that AUTOHOTKEY isn't the best because it's easy for others to look into your code and steal it. Even if you convert scripts into an .exe file with MPRESS, or even Obfuscator the code.
Our codes are very unique and ground breaking in our industry and need to protect the Intellectual property of the idea. It's the idea that no one has thought of regarding script.
I don't mind if people eventually steal our programs and share it 'free illegally' because that can never be stopped. Look at ADOBE!

But it's the Intellectual property of the ide inside the code that we want to protect.
I have heard Python probably better cause it has better encryption tools?

So I'm wondering if we should pay someone to completely pay someone to rewrite all our code into python for safer business perspective?
Any thoughts on that at all?

Many Thanks

Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: Interacting with DaVinci using UIAutomation

Post by Descolada » 29 Jan 2023, 03:40

@SteveMylo, while I can help you with UIA-related questions, I am unable to give business advice :)

If you are worried about intellectual property, then be assured that every language you use can be cracked by someone who has enough time and knowledge, be it AHK, Python, C++, or anything else.
I think there are a few encryption methods for AHK as well, for example feiyue made such a script (search the forums for it).

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: Interacting with DaVinci using UIAutomation

Post by SteveMylo » 29 Jan 2023, 03:44

@Descolada Ah yes I have seen feiyue's script about encryption .
Yes no problem at all. Thanks Descolada

malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: Interacting with DaVinci using UIAutomation

Post by malcev » 02 Feb 2023, 05:51

I think that the best way for You is rewrite Your code on c++.
From compiled c++ code You cannot get sources.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: Interacting with DaVinci using UIAutomation

Post by SteveMylo » 27 Feb 2023, 19:35

@malcev Thanks, but does C++ have access to UlAutomation the same as Autohotkey?

malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: Interacting with DaVinci using UIAutomation

Post by malcev » 28 Feb 2023, 03:49

It depends on how do You call UlAutomation from autohotkey.
If You use wrapper, then You need understand how it works, what uia methods it calls.
And of course read and try to run examples from manual.
For example:
https://learn.microsoft.com/en-us/windows/win32/winauto/uiauto-howto-find-ui-elements

Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: Interacting with DaVinci using UIAutomation

Post by Descolada » 28 Feb 2023, 04:21

Every general purpose programming language, including C++, has access to UIAutomation. But you should be aware that you will be dealing with it on a lower level, meaning you would have to implement some wrappers yourself, deal with memory management etc.
This actually goes for a lot of AHK functions - for example WinExist is an AHK wrapper, which means that as far as I'm aware you can't just get a window handle from an exe name like with "ahk_exe", or match by title, you would have to implement those yourself.

Personally I would avoid using C++ if I needed to use win32 and UIAutomation heavily. Python might be a good option, since it has obfuscation libraries available, and it can be compiled into byte-code.

malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: Interacting with DaVinci using UIAutomation

Post by malcev » 28 Feb 2023, 04:39

Good discussion:
https://stackoverflow.com/questions/261638/how-do-i-protect-python-code-from-being-read-by-users
So, If I choose to hire somebody to write protected code on python or c++, I would choose c++.

M0T0GP
Posts: 6
Joined: 16 Oct 2016, 06:56

Re: Interacting with DaVinci using UIAutomation

Post by M0T0GP » 31 Mar 2023, 02:39

Image
Image
I wanted to click that button, I tried but no automation id, BoundingRectangle is not reliable idea cause, if i used different screen, it change, and scripts broke. Sorry for weak English.

Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: Interacting with DaVinci using UIAutomation

Post by Descolada » 31 Mar 2023, 03:27

@M0T0GP, from the UIAViewer screenshot it looks like you might be able to use FullDescription instead. Something like davinci.FindFirst("FullDescription='Track Forward One Frame'").Highlight(). Have you tried that?

M0T0GP
Posts: 6
Joined: 16 Oct 2016, 06:56

Re: Interacting with DaVinci using UIAutomation

Post by M0T0GP » 06 Apr 2023, 09:28

Code: Select all

#SingleInstance, force
#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
SetTitleMatchMode, 2

#include <UIA_Interface>

UIA := UIA_Interface()
WinActivate, DaVinci Resolve
WinWaitActive, DaVinci Resolve
davinci := UIA.ElementFromHandle(WinExist("DaVinci Resolve"))
MsgBox, Clicking Track Forward One Frame panel
davinci.FindFirst("FullDescription='Track Forward One Frame'").Highlight()
I wanted to click that button, but it was not working.I changed this code also, it was not working. please fix it.

Code: Select all

davinci.FindFirst("FullDescription='Track Forward One Frame'").Click()

Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: Interacting with DaVinci using UIAutomation

Post by Descolada » 07 Apr 2023, 02:38

@M0T0GP, davinci.FindFirstBy("FullDescription='Track Forward One Frame'").ControlClick() works.

1) Thanks for helping me find a bug with FindFirst: FindFirstBy and FirstFirst should have acted equally when used with an expression, but they weren't. This has now been fixed.
2) It seems that specific element doesn't respond to normal UIA calls (that is, Click()), so either ControlClick() or Click("left") have to be used.

M0T0GP
Posts: 6
Joined: 16 Oct 2016, 06:56

Re: Interacting with DaVinci using UIAutomation

Post by M0T0GP » 08 Apr 2023, 10:31

Thanks man it worked,
how to trigger BoundingRectangle(only this one I can see from UIA viewer option), FindFirst and FindFirstBy is not working.

Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: Interacting with DaVinci using UIAutomation

Post by Descolada » 08 Apr 2023, 11:04

What do you mean by "trigger" BoundingRectangle? It's a property all elements have defining their location on the screen.

Code: Select all

br := davinci.BoundingRectangle ; bounding rectangle for the Davinci window element. It contains {l, t, r, b}, corresponding to left, top, right and bottom screen coordinates for the element bounds
MsgBox % "X (left): " br.l " Y (top): " br.t

M0T0GP
Posts: 6
Joined: 16 Oct 2016, 06:56

Re: Interacting with DaVinci using UIAutomation

Post by M0T0GP » 08 Apr 2023, 13:04

Image
I wanted to clicked that better button, you see no option, click that button except bounding rectangle. I hope you understand it. sorry for weak english.

Post Reply

Return to “Ask for Help (v1)”