How do I click and edit text fields in apps using Acc Libraries?

[SteveMylo]

Well, was just for the records. So is this, the (AFAICS) updated "AccViewer" that is able to identify acc-paths more reliably, kindly provided by @rommmcek : viewtopic.php?p=382050#p382050 + [Download]

That one you provided is definitely better. But yes, UIA viewer does the same. Works great, But I just tested it in Adobe Premiere Pro video editing tool and it doesn't work there though for changing video attributes like ZOOM etc.
Menu's on the other hand always worked, that's a given.
Editing programs are notorious for hiding their inner workings. I'm not a pro though.

But the point of all this, is about 'moving' his mouse to the path. That was the issue.

[BoBo]

But the point of all this, is about 'moving' his mouse to the path. That was the issue.

... to click an element, putting it on focus, right?

What I haven't seen (correct me if I'm wrong) is that the 2nd command of these two (taken from @jeeswg's ACC tutorial) has been used/tested (so far)?
Instead, an AHK standard Click has been used. Why?
- accDefaultAction: get text describing what the default action does (e.g. 'Execute', often equivalent to clicking it)
- accDoDefaultAction: perform the element's default action (often equivalent to clicking it)

[SteveMylo]

to click an element, putting it on focus, right?

Almost all areas of these editing programs have the same focus and Controls names, so it's really hard. We just aren't clever enough.
Well, Myself and the guy who originally posted aren't that good at coding.
BUT..... After searching through the forums, I finally found how to move the mouse where we need it from the ACC Viewer path.
It's not perfect and it doesn't work when the (MediaPool Window) is open as it always goes there 1st.
You see when certain widows GUI windows are open in the editing program, it takes on the same path number as something else that you're trying to get to. makes it very hard to get an exact pass. Probably need some convoluted checks 1st to see what window is open and then apply a certain script to continue.
But I'll paste the code for the mouse to click on the 'zoom' Parameter in 'DaVinci Resolve'
I tried this code to access menu items but I can't seem to make it go to the sub menu's. Not even in Notepad cause I'm not clever enough.

Code: Select all

#include <Acc>
#include <Get_AccPath>

WinGet, hWnd, ID, A
oAcc := Acc_Get("Object", "4.2.2.1.1.2.2.1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1", 0, "ahk_id " hWnd)
oRect := Acc_Location(oAcc)
CoordMode, Mouse, Screen
vCurX := oRect.x + oRect.w/2
vCurY := oRect.y + oRect.h/2
MouseClick,, % vCurX, % vCurY, 2
oAcc := oRect := ""

return

[BoBo]

[To whom it may concern]
The following scripts are "stolen" from here: viewtopic.php?style=19&p=289306#p289306 (created by @jeeswg)

All scripts are acting based on the window that has currently the focus AKA "the active window".

a) the first script (press q to execute it) will deliver the paths for all elements of that window and copy it to the clipboard for further processing/review).
JFTR, that needs time. Be patient!
b) you should find the path for your "element of interest" (ie. '4.4.4' for Notepad's find dialog)
c) use that path and add/replace it to the following (w::;e::,r::) script snippets.

Code: Select all

;[JEE_AccGetTextAll function]
;Acc: get text from all window/control elements - AutoHotkey Community
;https://autohotkey.com/boards/viewtopic.php?f=6&t=40615

q:: ;notepad find dialog - get Acc info
   WinGet, hWnd, ID, A
   Clipboard := JEE_AccGetTextAll(hWnd, "`r`n")
   MsgBox, % "done"
   return

w::                                                      				; notepad find dialog - toggle 'Match case' checkbox via accDoDefaultAction (and get element info)
   WinGet, hWnd, ID, A
   oAcc := Acc_Get("Object", "4.4.4", 0, "ahk_id " hWnd)	; change that path to meet your requirements
   oAcc.accDoDefaultAction(0)
   oRect := Acc_Location(oAcc)
   vEltPos := Format("x{} y{} w{} h{}", oRect.x, oRect.y, oRect.w, oRect.h)
   MsgBox, % oAcc.accName(0) "`r`n" oAcc.accValue(0) "`r`n" vEltPos
   oAcc := oRect := ""
   return

e::                                                      				; notepad find dialog - toggle 'Match case' checkbox via accDoDefaultAction
   WinGet, hWnd, ID, A
   Acc_Get("DoDefaultAction", "4.4.4", 0, "ahk_id " hWnd)	; change that path to meet your requirements
   return

r::                                                      				; notepad find dialog - toggle 'Match case' checkbox via MouseClick
   WinGet, hWnd, ID, A
   oAcc := Acc_Get("Object", "4.4.4", 0, "ahk_id " hWnd)	; change that path to meet your requirements
   oRect := Acc_Location(oAcc)
   CoordMode, Mouse, Screen
   vCurX := oRect.x + oRect.w/2
   vCurY := oRect.y + oRect.h/2
   MouseClick,, % vCurX, % vCurY
   oAcc := oRect := ""
   return

[SteveMylo]

That's the one! thanks, I should have referenced it but I figured you guys have already linked to their threads already which is how I found it. Cheers.
I think inspect.ahk could really help with these editing programs but It looks incredibly complex

[BoBo]

That's the one! thanks, I should have referenced it but I figured you guys have already linked to their threads already which is how I found it. Cheers.
I think inspect.ahk could really help with these editing programs but It looks incredibly complex

So the 'e-script' isn't clicking/activating your '4.2.2.1.1.2.2.1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1'-path ("Zoom-InputField"??) element?
Have you checked if that element is labeled as "editable" in ACCViewer as well as in the parsed "all elements" output?
If OK with you, you can drop your parsed output here, so others can see what its format looks like. Thx

[SteveMylo]

So the 'e-script' isn't clicking/activating your '4.2.2.1.1.2.2.1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1'-path element?

ha well you are very advanced and it's hard to follow. But.. The script I just shared before works perfectly. But when a certain part of the GUI is open in the editing program, the (Media Pool), the path to 'ZOOM' changes a little.
The 7th number is different.
Zoom Path when Medial Pool: 4.2.2.1.1.2.3.1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1 (open)
Zoom Path when Medial Pool: 4.2.2.1.1.2.2.1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1 (NOT open)

So if I use the correct path 4.2.2.1.1.2.2.1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1 it works , but if the Media Pool is Open, it'll click on the 'MediaPool'. So it's very frustrating.

[BoBo]

I'm pretty sure @Descolada,@rommmcek & friends can shed some light on how to distinguish your two conditions above in a smart(er) way (fingers crossed)

So...

Code: Select all

; <detect if mediapool is active>
pathNode := (mediapool = active) ? 3 : 2
MsgBox % path := "4.2.2.1.1.2." pathNode ".1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1"

; or

; <make mediapool in-/active>
; <so you can use either '4.2.2.1.1.2.2.1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1' OR '4.2.2.1.1.2.3.1.3.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1'> as a static value.

[Descolada]

@BoBo, @SteveMylo, I tried UIAutomation on DaVinci Resolve 17 and it worked fairly well. Clicking the "Media Pool" button is especially easy:

Code: Select all

#SingleInstance, force
#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
SetTitleMatchMode, 2

#include <UIA_Interface>

UIA := UIA_Interface()
WinActivate, DaVinci Resolve
WinWaitActive, DaVinci Resolve
davinci := UIA.ElementFromHandle(WinExist("DaVinci Resolve"))
MsgBox, Clicking Edit panel
davinci.FindFirstBy("AutomationId=UiMainWindow.bigBossWidget.m_pWidgetsForm.m_pMiddleFrame.uiColorControlPanel.frameControlPanelBackground.frameControlPanelMiddle.buttonFusionPanel").Click()
MsgBox, Clicking Cut panel
davinci.FindFirstBy("AutomationId=UiMainWindow.bigBossWidget.m_pWidgetsForm.m_pMiddleFrame.uiColorControlPanel.frameControlPanelBackground.frameControlPanelMiddle.buttonFirstCutPanel").Click()
MsgBox, Clicking Media Pool
davinci.FindFirstByName("Media Pool").Click()

Some elements like the timeline element cannot be accessed by UIA though... And since a lot of elements can only be identified by AutomationId, I also changed UIAViewer to show that.

When I get time, I'll try Adobe as well

[SteveMylo]

Clicking the "Media Pool" button is especially easy:

Thanks, although i'll have to learn the UIAutomation way of coding, clicking the media pool wasn't the problem.
if you see my post just before, ,.....when I try to click on the ZOOM parameter using ACC Viewer path coding, it'll always click on the Media Pool instead, but only if the Media Pool is active.
If it's not active, the code works and clicks on ZOOM.

I would honestly love it if you could use that UIAutomation code you just used to show how you would CLICK on the ZOOM parameter when the Inspector tab is open for example.
If you show me that, I could figure out the rest.
Thanks so much ALready been a great help.

[Descolada]

@SteveMylo, oops, seems I glazed over that bit Getting to that Zoom x-coordinate is indeed trickier. Here I cheated a bit and just looked for the first Text element with content X, and got the next Text element from that. Since the UIA tree is quite messed up in that program (eg the Text "Zoom" is actually after the coordinates), if you need to access other values, you would need to inspect the UIA tree and select elements accordingly. Also these Text elements can't be Invoked (clicked with UIA)...

Code: Select all

#SingleInstance, force
#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
SetTitleMatchMode, 2

#include <UIA_Interface>

UIA := UIA_Interface()
WinActivate, DaVinci Resolve
WinWaitActive, DaVinci Resolve
davinci := UIA.ElementFromHandle(WinExist("DaVinci Resolve"))
XText := davinci.FindFirstByNameAndType("X", "Text")
tw := UIA.CreateTreeWalker(UIA.CreateCondition("ControlType", "Text"))
XValue := tw.GetNextSiblingElement(XText)
XValueCoords := XValue.CurrentBoundingRectangle
CoordMode, Mouse, Screen
Click, % (XValueCoords.l+((XValueCoords.r-XValueCoords.l) // 2)) " " (XValueCoords.t + ((XValueCoords.b-XValueCoords.t)//2)) " " 2
Sleep, 1000
Send, 1.5 ; Alternative option: UIA.CreateTreeWalker(UIA.CreateCondition("ControlType", "Edit")).GetNextSiblingElement(XValue).SetValue(1.5)

[SteveMylo]

@Descolada Ah nice.
Yeah, it's a bit of a workaround. But great work. Already learned a bit.
I managed to get the rotation angle parameters working just by guessing XText := davinci.FindFirstByNameAndType("Position", "Text") ; this takes you to Rotation Angle
I can't figure out how to find out if the Inspector Tab is checked or unchecked yet. Cause your script only works if it can see the 'X' near the 'ZOOM' etc.
Wouldn't mind knowing if it's unchecked then I can use davinci.FindFirstByName("Inspector").Click() to (Check) open the Inspector TAB. so the script can find the 'X' near the 'ZOOM'
I know with Acc Viewer you can see the 'Action' and it tells you whether it's 'Checked' or 'unchecked' , but no idea how to script that it.
Anyways , cheers

Screenshot 2022-06-23 194718.jpg (35.47 KiB) Viewed 1877 times

[Descolada]

@SteveMylo, simplest way would be to do if XTest { .... }. But you can also toggle the Inspector button:

Code: Select all

#SingleInstance, force
#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
SetTitleMatchMode, 2

#include <UIA_Interface>

UIA := UIA_Interface()
WinActivate, DaVinci Resolve
WinWaitActive, DaVinci Resolve
davinci := UIA.ElementFromHandle(WinExist("DaVinci Resolve"))
InspectorCB := davinci.FindFirstByNameAndType("Inspector", "CheckBox")
if InspectorCB {
	togglePattern := InspectorCB.GetCurrentPatternAs("Toggle")
	if !togglePattern.CurrentToggleState {
		togglePattern.Toggle()
		Sleep, 500
	}
	XText := davinci.FindFirstByNameAndType("X", "Text")
	if !XText {
		MsgBox, % "Zoom X-coordinate was not found, exiting..."
		ExitApp
	}
	tw := UIA.CreateTreeWalker(UIA.CreateCondition("ControlType", "Text"))
	XValue := tw.GetNextSiblingElement(XText)
	XValueCoords := XValue.CurrentBoundingRectangle

	CoordMode, Mouse, Screen
	Click, % (XValueCoords.l+((XValueCoords.r-XValueCoords.l) // 2)) " " (XValueCoords.t + ((XValueCoords.b-XValueCoords.t)//2)) " " 2
	Sleep, 1000
	Send, 1.5 ; Alternative option: UIA.CreateTreeWalker(UIA.CreateCondition("ControlType", "Edit")).GetNextSiblingElement(XValue).SetValue(1.5)
} Else
	MsgBox, "Inspector" CheckBox was not found!

ExitApp

[SteveMylo]

@SteveMylo, simplest way would be to do if XTest { .... }. But you can also toggle the Inspector button:

Genius. Thanks. Learned a lot. It works and I would love to learn more. Must go through all the paperwork now Thanks Champ.

[SteveMylo]

@Descolada Everything is working great, great tool!
Two Questions if you have time.
1. When you created the TreeWalker, Can you go to the 2nd or 3rd sibling? not just the next or previous? I'll paste the Next Sibling code below for your convenience.

2. In the inspector.exe tool when you click on the Rectangle icon 'watch focus' button only and not the 'watch cursor', it gives you access to a few more spots (in Davinci for example) that the watch Cursor can't.
But I can't use the data from it, for example, the AutomationID.
If you see the pictures below. I used it to separate the Video and Audion panels, normally they are one big Panel. But any data given by the 'watch focus' method doesn't do anything even though it shows you a unique AutomationID.
Is it possible? It would be a great help for me in automating.

Video Layer only.jpg (22.11 KiB) Viewed 1712 times

Code: Select all

#include <UIA_Interface>
UIA := UIA_Interface()
WinActivate, DaVinci Resolve
WinWaitActive, DaVinci Resolve
davinci := UIA.ElementFromHandle(WinExist("DaVinci Resolve"))
XText := davinci.FindFirstByNameAndType("X", "Text")
tw := UIA.CreateTreeWalker(UIA.CreateCondition("ControlType", "Text"))
XValue := tw.GetNextSiblingElement(XText)
XValueCoords := XValue.CurrentBoundingRectangle
CoordMode, Mouse, Screen
Click, % (XValueCoords.l+((XValueCoords.r-XValueCoords.l) // 2)) " " (XValueCoords.t + ((XValueCoords.b-XValueCoords.t)//2)) " " 2

[BoBo]

@SteveMylo @Descolada - Gents, as your latest contributions in this thread are purely UIA-related (instead of ACC like its subject line suggests) I'd recommend to create a separate thread and move your conversation starting here: viewtopic.php?p=469345#p469345 to that threat. Your thoughts please?!

@gregster JFYI

Edit: Descolada has created a follow-up thread regarding the UIA-related postings from above. At your service: viewtopic.php?f=76&t=105890

[GimpyGosling]

Geez Louis! I didn't check this board in a while.

I guess I should explain what I ended up doing @SteveMylo ; basically I made a bunch of presets and resets for work spaces in davinci resolve that basically
set everything up and keeps everything ready for any hotkey functions I made.

I wound up making functions for clicking coordinates. Said functions and coordinates are saved to a separate ahk file containing all pertinent coordinates saved to variables,
and I made a bunch of different functions that perform multiple repetitive tasks like double clicks, single clicks, ctrl + click, click text fields and replacing values.

As to how I fit a bunch of functions onto 20 macro keys, I utilized a system of additional autohotkey files that bring up pop up windows where you choose from a menu of items,
and depending on what you click, a different function is performed, including importing images from the power bins, loading video effects, and custom fusion effects.
The best part about this approach is I can add more functions down the road.

It's not the most elegant solution but it works, and gosh darn it, ain't that the point of autohotkey?

[SteveMylo]

@GimpyGosling Haha. Yeah we've been at it. Your functions sound great. Well done. Fun to tinker isn't it

[GimpyGosling]

@GimpyGosling Haha. Yeah we've been at it. Your functions sound great. Well done. Fun to tinker isn't it

Definitely a good jumping off point towards other scripting languages! Getting stuff to work inspired me to tackle javascript next!