Acc v2

[neogna2]

v2 AccViewer throws error on right click on left side listview headers, at L1164
ToolTip("Copied: " (A_Clipboard := GuiCtrlObj.GetText(Info,2)))
because Info in this case is a large number not matching any existing row.

Smallest fix: add above L1164
if Info <= GuiCtrlObj.GetCount()

Or this to let right click on the header copy all row values and right click in listview copy selected row values (if 2+ rows selected) or otherwise copy the clicked row's value.

Code: Select all

LVData := Info > GuiCtrlObj.GetCount()
    ? ListViewGetContent("Col2", GuiCtrlObj)
    : GuiCtrlObj.GetCount("Selected") > 1
    ? ListViewGetContent("Selected Col2", GuiCtrlObj)
    : GuiCtrlObj.GetText(Info,2)
ToolTip("Copied:`n" (A_Clipboard := LVData))

[Descolada]

@neogna2, your AccGetPath is probably the fastest option and is very similar to what Acc v1 AccViewer uses to generate the paths. The downside is that in some windows the bottom-up approach of traversing the tree will give you the wrong path. I implemented your AccGetPath as Element.GetPath, which also checks that the path is valid and if not then uses FindFirst (the top-down approach) to get the correct path. If no path is found then an empty string is returned. Usage: Acc.ObjectFromWindow("A").GetPath(Acc.ObjectFromPoint())
By the way, in Chrome your function will return a wrong path every time if used with ObjectFromWindow. To avoid top-down searches with the GetPath method (which are much much slower than bottom-up), it's better to use Acc.ObjectFromChromium instead, which will return only the client area (Chrome renderer element, so for example the window close button won't be included in the Acc tree), but bottom-up searches will return a correct path.

I fixed the ListView right-clicking error with your proposed fix with a small change: it also includes the property name in the output.

Todays update has a lot of changes and improvements in addition to the afore-mentioned ones. The biggest is that default hWnds are now Last Found Window not active window, so Acc.ObjectFromWindow() calls might need to be changed to Acc.ObjectFromWindow("A").

[neogna2]

v2 Acc - better every day!

in some windows the bottom-up approach of traversing the tree will give you the wrong path.

Aha, had forgotten about that issue. But thinking on it now, isn't the extra check you've added also vulnerable to the same issue but with lower probability? Check my reasoning: the problem is when somewhere in the path there's two or more elements with the same parent and identical properties so the IsEqual test returns a false positive. For example if the true path we want to find is (unbeknownst) 4,1,2,4 but there's a 4,1,1 that's property identical to 4,1,2 so the bottom-up approach outputs 4,1,1,4. Your extra step validates the output: if 4,1,1,4 fails an IsEqual check against the input object then FindFirst is used to get the path instead. But wait! There *could* exist a false positive 4,1,1,4 too! With identical properties to 4,1,2,4. Then the FindFirst step is never reached. Imagine 4,1,1,4 failed the IsEqual check though. FindFirst starts looking. There could also exist a 4,1,1,2 that is a false positive to the input object. FindFirst would then return it, it seems, since it searches the whole 4,1,1 branch before the 4,1,2 branch. Or am I misunderstanding? If I'm right and we want to reduce the probability further we could also validate against the input object's whole child tree (if it has any children). Trade-off between speed and lower false positive chance I suppose. (edited)

[Descolada]

@neogna2, you are correct that IsEqual can fail. Since Microsoft hasn't provided unique identifiers for Acc elements nor a way to compare two Acc objects for equality, and accessing properties has a non-trivial time cost, then a compromise between speed and reliability has to be made. With IsEqual if an element has a non-zero position and size then only the Name and Role properties are additionally compared, but this leaves out a bunch of other properties which might differ (State, DefaultAction etc). If the position is zero (x, y, w and h all 0) then all the properties are compared, but still I've stumbled upon "identical" different elements using this method.

Your reasoning about possible false positives is totally correct. I think if we wanted to reduce the probability of a conflict as much as possible, then we should reconstruct the whole tree both in upwards and downwards direction from the found element, and then from the starting point element as well, and check whether the trees match. Naturally this would be very time consuming, so I didn't implement that. But usually elements that give a false-positive IsEqual result are irrelevant anyway (hidden elements with 0 sizes and no actual ways of interaction), so in real life it seems to work.

Also, IsEqual isn't actually the reason for why the bottom-up approach fails. For some reason (Microsofts bad implementation? server-side implementation problems?) sometimes the element returned by accParent will NOT have the child element we called accParent from (at least as a direct child, it might be somewhere among the deeper descendants). Especially for big windows with deep nested structures this is a huge problem and the path returned will frequently be incorrect. It is still quite a common problem with Acc v1 AccViewer, mostly with browsers and other Chromium-based windows. This is why I think validating the result from a bottom-up approach is super important, and so far I haven't seen actually relevant "false positives" with the FindFirst top-down approach.

[viv]

The path of Firefox seems to change frequently
It doesn't change much, it just increases or decreases by 1-2 at the some node

like the address bar
it is sometimes 4,20,9,1
sometimes 4,21,9,1
...

FindFirst sometimes takes too long
don't know where it stuck
Is there any better way to solve this situation?

By the way
Does FindFirst have an option similar to setting the starting point to find the node?

For example, in the above case
It may look up a lot faster starting from 4.19

[Descolada]

@viv, you can start FindFirst search from any valid Acc element, thus this is also okay: oFirefox[4,19].FindFirst({Name:"...."}). With the recent update (yesterday) you can also limit the depth of the search, so it wouldn't evaluate very deep elements and perhaps speed up the search: oFirefox.FindFirst({Name:"..."},,4) ; max depth of 4. Though if it's always on of [4,20,9,1] or [4,21,9,1], perhaps you could just evaluate those two if they are the address bar?
For even faster searches you would need to use UIAutomation though.

[neogna2]

@Descolada Ok thanks. Yeah I also haven't encountered any false positive cases with the new GetPath so far, will report back if I do.

@viv on top of what Descolada wrote remember also that you can chain several FindFirst calls with limited scope and with some name/value/role condition where each call uniquely picks out the next step in the path.

[viv]

@neogna2

thank you very much
I thought of another way,
I found that its path changes not randomly, it may be reopened or something

So I saved the last successful search path
then try this path first next time
If unsuccessful, search again and save the path.

About 16-32ms when direct path search
FindFirst I tried limiting the depth or starting point, didn't improve much it took 90-110ms

In fact, this difference is acceptable to me.

The reason I didn't switch to another method last time is that it got stuck somehow.
See if recent updates to this LIb improve the situation

I wish you a happy life

[dJeePe]

Hello

Sorry to bother you but i''ve download the github source with samples and some of them are not working !? Do i miss something ?
Fyi, i use V2-beta.12

image.png (27.14 KiB) Viewed 2531 times

[Descolada]

@dJeePe, should be fixed now - I haven't yet upgraded to Win 11

[dJeePe]

Thank you @Descolada

[Zedcars]

Do you know why I get the following error?

Windows 10
AHK 2.0.rc3
I’m using Example 1 on the GitHub page

774508F5-82E9-46C0-9232-38E9F9A8517A.jpeg (1.5 MiB) Viewed 2460 times

[Zedcars]

Do you know why I get the following error?

Windows 10
AHK 2.0.rc3
I’m using Example 1 on the GitHub page

774508F5-82E9-46C0-9232-38E9F9A8517A.jpeg

It’s ok, I figured it out. Windows was still opening the ahk files using an older build from last year. I just renamed the older build (which I want to keep) and placed the new build in the same folder. It seems to be working ok now.

Sorry to waste your time, and thanks for a great tool!

[Descolada]

@Zedcars glad you got it working I was thinking that you're probably accidentally running an older version, but since I still have v2-rc2 then I couldn't be sure that rc3 didn't break something. But now I can safely postpone upgrading

[crocodile]

Thank you for providing the tools.
I would like to know if there is a built-in method to get content in bulk based on rectangular coordinates? For example in firefox to get all links within the rectangle [200,200,500,500].

[Descolada]

@crocodile, there isn't such a built-in method, but it's rather easy to create one with a recursive function:

Code: Select all

els := GetElementsInRect(Acc.ObjectFromWindow("A"), 200, 200, 500, 500)
out := ""
for el in els
    out .= el.Dump() "`n"
MsgBox out


GetElementsInRect(oAcc, l, t, r, b, containFully:=False, childrenAllowed:=False) {
    RecurseElements(oAcc, &(out := []), l, t, r, b)
    return out
    RecurseElements(oAcc, &elements, l, t, r, b) {
        for el in oAcc {
            len := el.Length, loc := el.Location, l2 := loc.x, t2 := loc.y, r2 := loc.x+loc.w, b2 := loc.y+loc.h
            if !IntersectRect(l, t, r, b, l2, t2, r2, b2)
                continue
            if (childrenAllowed || !len) && (!containFully || (containFully && l2 >= l && t2 >= t && r2 <= r && b2 <= b))
                elements.Push(el)
            if len
                RecurseElements(el, &elements, l, t, r, b)
        }
    }
}
IntersectRect(l1, t1, r1, b1, l2, t2, r2, b2) {
	rect1 := Buffer(16), rect2 := Buffer(16), rectOut := Buffer(16)
	NumPut("int", l1, "int", t1, "int", r1, "int", b1, rect1)
	NumPut("int", l2, "int", t2, "int", r2, "int", b2, rect2)
	if DllCall("user32\IntersectRect", "Ptr", rectOut, "Ptr", rect1, "Ptr", rect2)
		return {l:NumGet(rectOut, 0, "Int"), t:NumGet(rectOut, 4, "Int"), r:NumGet(rectOut, 8, "Int"), b:NumGet(rectOut, 12, "Int")}
}

This method assumes that elements can only contain smaller elements, and that all child elements are located within the bounds of the parent element. I'm not 100% sure this is correct though.
If this turns out to work as expected, I'll think about adding it in the main library as well.

[crocodile]

It works fine in Firefox and is very fast.
The only problem is that it doesn't just get the link of the current tab, it also gets links to the same coordinates of all the tabs. Whereas in Chrome, it only gets the link of the current tab.
Thanks for your help.

-------

I'm using the H version, using static struct, and I added a Rect property.

Code: Select all

Rect {
	get {
		static rc := struct("l,t,r,b"), li := [ComValue(0x4003, rc[], 1), ComValue(0x4003, rc[] + 4, 1), ComValue(0x4003, rc[] + 8, 1), ComValue(0x4003, rc[] + 12, 1), 0]
		return (this.oAcc.accLocation(li*), rc.r += rc.l, rc.b += rc.t, rc.clone())
	}
}

This makes it easy to call IntersectRect() as well.

Code: Select all

	IntersectRect2(r1, r2) {
		static t := struct("l,t,r,b")
		if DllCall("user32\IntersectRect", "Ptr", t[], "Ptr", r1[], "Ptr", r2[])
			return t.clone()
	}

[Descolada]

@crocodile, the problem with Chrome and Firefox acting differently is not new, the same problem exists with UIA as well. This happens because Firefox always lets you access Acc elements even if they are not visible (on another tab), and doesn't report their size as 0. I don't know a good solution to that, but perhaps you could loop through the elements and check if Acc.ObjectFromPoint from the element position returns the same element?

The Rect structure seems interesting. Unfortunately I'm not familiar with AHK_H, so I can't comment on it But the part this.oAcc.accLocation(li*) you should recheck - as the last argument you are always passing 0, but I think this.childId should be correct (as done in Acc.IAccessible class Location property).

[neogna2]

I don't know a good solution to that, but perhaps you could loop through the elements and check if Acc.ObjectFromPoint from the element position returns the same element?

Or a path checking process:
1. get the Acc path of the first/"base" element of the active Tab in Firefox
2. get all elements within a rectangle
3. for each element use GetPath() and discard if not starting with the "base" of the active Tab's path.

[Skrell]

Sorry for being daft, but the enumeration of this library is confusing the hell out of me. Does this library compile in AHK1.1 or ONLY in AHK2.x+ ?