AutoHotkey Community

It is currently May 27th, 2012, 8:20 am

All times are UTC [ DST ]




Post new topic Reply to topic  [ 14 posts ] 
Author Message
PostPosted: January 26th, 2012, 7:45 am 
Offline
User avatar

Joined: May 24th, 2009, 5:35 am
Posts: 2099
Location: Iowa, USA
Thread is now Inactive. Please refer to Accessible Info Viewer.

Would anyone like to help with or take over an Acc Viewer project I started? The premise was to create a window info tool that would also gather Accessible Information. I wanted to be able to create a navigable tree-view that would allow the user to gather info about relative Acc objects. Also, I wanted to be able to parse from the selected object back up the to window object & populate the structure back down to the selected object. I believe this would open the ability to easily get data from those controls that cannot be accessed via the control commands. As it stands right now, my AHK involvement has drastically dropped off, so I probably won't finish this project. Let me know if you want to help.

NOTE - this project requires a relatively good understanding of the IAccessible Interface - particularly for fixing the bugs.

_________________
Image
Recommended: AutoHotkey_L
Basic Webpage Controls


Last edited by jethrow on March 4th, 2012, 9:40 pm, edited 4 times in total.

Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 26th, 2012, 1:43 pm 
Offline

Joined: October 11th, 2010, 6:15 pm
Posts: 1211
Location: Right behind you
I would if I knew jack about Acc. :oops:

I've noticed you've not been around much lately. What are you up to?

_________________
COM Tutorial for Webpages
COM Tutorial for Excel


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 26th, 2012, 3:53 pm 
Offline
User avatar

Joined: March 19th, 2008, 12:43 am
Posts: 5482
Location: the tunnel(?=light)
Why don't you release it as a collaborative project like tank did with the IE Web Recorder?

_________________
Image
Try Quick Search for Autohotkey or see the tutorial for newbies.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 26th, 2012, 7:08 pm 
Offline
User avatar

Joined: May 24th, 2009, 5:35 am
Posts: 2099
Location: Iowa, USA
tank's IE Web Recorder was already complete - his project was to further develop & enhance the recorder. In my situation, the viewer is still needs bugs worked out. I will probably still post the source for anyone who may want to help, but I need to go back & add a lot of comments so it makes sense.

Additionally, there may need to be some collaborative though process & brainstorming that needs to occur outside of posting in the forum.

_________________
Image
Recommended: AutoHotkey_L
Basic Webpage Controls


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 26th, 2012, 7:28 pm 
Offline
User avatar

Joined: December 21st, 2007, 3:14 pm
Posts: 3826
Location: Louisville KY USA
I have been waiting for so long for you to post this
I didnt want to start over from scratch
I have a wretch of a schedule and workload but honestly the results of this solve so many problems. I will help when and if i can.

_________________
No matter what your oppinion Please join this discussion
Formal request to Polyethene
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 26th, 2012, 7:30 pm 
Offline
User avatar

Joined: March 19th, 2008, 12:43 am
Posts: 5482
Location: the tunnel(?=light)
I'm not an expert on Accessibility but I've worked with it a few times, I'm game to help as time permits.

_________________
Image
Try Quick Search for Autohotkey or see the tutorial for newbies.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 26th, 2012, 7:31 pm 
Offline
User avatar

Joined: December 21st, 2007, 3:14 pm
Posts: 3826
Location: Louisville KY USA
As to the bugs. no it was far from complete and remains so. but i just never needed it to do more so i lost interest.

At least share the source code with me and Sinkfaze?

but i think an open development thread is the best plan. Would you 2 have responded as you did if i hadnt posted such

_________________
No matter what your oppinion Please join this discussion
Formal request to Polyethene
Image


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 26th, 2012, 7:52 pm 
Offline
User avatar

Joined: February 28th, 2011, 7:28 pm
Posts: 625
Location: Germany
I would be interested in the source ;-) and willing to contribute from time to time. However, I already have a lot of projects I'm working more or less on, so I might help or I might not. I see there are already some experts interested in it, it seems it's in good hands. :)

Quote:
Additionally, there may need to be some collaborative though process & brainstorming that needs to occur outside of posting in the forum.
github :!: github :!:    :lol::lol:

_________________
RECOMMENDED: AutoHotkey_L
Image
github - ImportTypeLib
Win7 HP SP1 32bit | AHK_L U 32bit


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 30th, 2012, 5:32 am 
Offline
User avatar

Joined: May 24th, 2009, 5:35 am
Posts: 2099
Location: Iowa, USA
I posted a link to the source in the original post. Once I find the time/ambition to properly comment the source code, I'll start a thread in Scripts & Functions (unless someone else can properly take over this project).

The primary issue that I have not yet found a way to resolve/work around/bypass is what I asked in this post. I added a modified copy of the Acc_EnumChildren() function to the bottom of the source code. If utilized, this will show a MsgBox when the Acc_Children() call fails.

_________________
Image
Recommended: AutoHotkey_L
Basic Webpage Controls


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 30th, 2012, 1:31 pm 
Offline

Joined: October 17th, 2006, 4:15 pm
Posts: 7503
Location: Australia
jethrow wrote:
The primary issue that I have not yet found a way to resolve/work around/bypass is what I asked in this post.

When you retrieve the parent object, query for the IAccessible interface.
Code:
Parent := ComObj(9, ComObjQuery(AccObj.accParent
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)

Note that IAccessible::get_accParent() returns an IDispatch interface pointer. Because IAccessible derives from IDispatch, an object which implements IAccessible doesn't necessarily need a separate implementation of IDispatch: it can simply return an IAccessible pointer when an IDispatch pointer is requested. There's absolutely no guarantee that this will happen, so you need to query for the appropriate interface.


Report this post
Top
 Profile  
Reply with quote  
PostPosted: January 30th, 2012, 8:19 pm 
Offline

Joined: August 3rd, 2007, 8:01 am
Posts: 555
Location: Houston, TX
Looks like you might have to deal with a similar issue getting children as well as parents: http://msdn.microsoft.com/en-us/library ... p/dd317975
Here are modified versions of ACC_EnumChildren and GetAccPath to use IAccessible instead of IDispatch.
Code:
Acc_EnumChildren(Acc) {      
    Children := []      
    Loop, % Acc_Children(Acc, Acc.accChildCount, varChildren){
           i := (A_Index-1)*(A_PtrSize*2+8)+8
       child := NumGet(varChildren,i)
           aChild := NumGet(varChildren,i-8)=3?child:ComObj(9,child,1)
      accChild := ComObj(9, ComObjQuery(aChild
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)   
      Children.Insert(accChild ? accChild : aChild)
}
   return, {   Items:      Children
         ,   _NewEnum:   Func("Acc_Enum_NewEnum")
         ,   Next:      Func("Acc_Enum_Next")   
         ,   base:      {__Get:Func("Acc_Enum_Get")}   }
}


GetAccPath(AccObj) {
/* returns:
   object
      AccObj:   Acc Object for Window/Control
      Path:   * Delimited List of child positions for each Acc Level down to the object
*/
   
   ; return object if it already is the window object
        parent := ComObj(9, ComObjQuery(AccObj.accParent
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)
   parent := ComObj(9, ComObjQuery(parent.accParent
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)      
   if Acc_WindowFromObject(parent) != Acc_WindowFromObject(AccObj)
      return {AccObj:AccObj}
   
   Levels := []
   ; get the Window Acc Object
   AccWinObj := Acc_ObjectFromWindow(Acc_WindowFromObject(AccObj))
   AccWinLocation := GetAccLocation(AccWinObj)
   ; Parse up until you reach the Window Acc Object
   Loop {
      AccObjLocation := GetAccLocation(AccObj)
      if (AccObjLocation = AccWinLocation)
         break
      Levels.Insert(1,AccObjLocation)
   ;   AccObj := AccObj.accParent
                AccObj := ComObj(9, ComObjQuery(AccObj.accParent
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)
   }
   AccObj := AccWinObj
   ; Parse back down to identify the Child Path
   For LevNum, Location in Levels
      for child,type in Acc_EnumChildren(AccObj)
         if (type=9 and GetAccLocation(child)=Location) {
            ChildPath .= A_Index "*"
            AccObj := child
            break
         }
   return {AccObj:AccWinObj, Path:SubStr(ChildPath,1,-1)}
}

I had opened a related issue on github. Some relevant links:
IUIAutomation: http://msdn.microsoft.com/en-us/library ... p/ee671406
bewildr using IUIAutomation in ironruby: http://www.natontesting.com/2010/07/20/ ... -ironruby/
https://github.com/GraemeF/Fluid
http://www.autoitscript.com/forum/topic ... utomation/


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 31st, 2012, 2:57 am 
Offline
User avatar

Joined: May 24th, 2009, 5:35 am
Posts: 2099
Location: Iowa, USA
Thanks for the solution Lexikos :D , & thanks for the example modifications tinku99. You guys make this seem so easy.

I suppose this means I get to reembark on this project :) :( :? :P


If anyone knows how to get the all ChildId's (to be used with get_accChild) of an object, that would be a great help. Acc_Children() can be used to get Child Element Id's, but not Child Object Id's, afaik.

_________________
Image
Recommended: AutoHotkey_L
Basic Webpage Controls


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: January 31st, 2012, 4:34 am 
Offline

Joined: October 17th, 2006, 4:15 pm
Posts: 7503
Location: Australia
tinku99 wrote:
Code:
      accChild := ComObj(9, ComObjQuery(aChild
                        , "{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)   
      Children.Insert(accChild ? accChild : aChild)
I think you'll find that accChild will always be used, since ComObj() returns an object even if the pointer is NULL.

jethrow wrote:
If anyone knows how to get the all ChildId's (to be used with get_accChild) of an object, that would be a great help.
Given that you don't have an ID of a child object in the first place, what use is get_accChild?

After reading the documentation for several IAccessible methods, I get the impression that only children which aren't objects have IDs. Even the documentation for get_accChild seems to confirm this (while possibly being contradictory):
IAccessible::get_accChild wrote:
Servers expose elements as either elements (child IDs) or full objects (IAccessible interface pointers).

Objects do not provide information about child objects; only child elements:
Quote:
If a child is an element, get_accChild returns S_FALSE, and the parent will provide information for that child. If the child is a full object, get_accChild will return the IAccessible interface pointer and the parent will not provide information about that child.

So aside from get_accChild, there doesn't appear to be any reason for a child object to have an ID. If a child object has no ID, what use is get_accChild? If you rely only on MSDN, probably no use. However, MSDN only defines the interface; it is open to interpretation by the developer of each accessibility server. For instance, this bug report indicates Firefox uses child IDs from 1 to the number of children, and some of the examples in that thread rely on it.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: March 3rd, 2012, 8:22 am 
Offline
User avatar

Joined: May 24th, 2009, 5:35 am
Posts: 2099
Location: Iowa, USA
Lexikos wrote:
So aside from get_accChild, there doesn't appear to be any reason for a child object to have an ID. If a child object has no ID, what use is get_accChild? If you rely only on MSDN, probably no use. However, MSDN only defines the interface; it is open to interpretation by the developer of each accessibility server.

Thank you for looking into this. Based on this, I feel that the accChild method should not be encouraged as a standard way to access children.

I wrote:
Also, I wanted to be able to parse from the selected object back up the to window object & populate the structure back down to the selected object.

I think I finally accomplished this, with sinkfaze's help. Here is an example:
Code:
ComObjError(false)
F1:: ; access the object under then mouse, then re-access it from the window object
   PtObj := Acc_ObjectFromPoint(child)
   child_path := GetAccPath(PtObj, hwnd)
   WinObj := Acc_ObjectFromWindow(hwnd)
   
   ChildObj := Acc_GetChild(WinObj, child_path)
   
   MsgBox %   "--Window + Path--`nName:`t" ChildObj.accName(child) "`nValue:`t" ChildObj.accValue(child) "`nLocation:`t" Acc_Location(ChildObj, child).pos
         .   "`n`n`n`n"
         .   "--From Point--`nName:`t" PtObj.accName(child) "`nValue:`t" PtObj.accValue(child) "`nLocation:`t" Acc_Location(PtObj, child).pos
   return


Acc_GetChild(Acc, child_path) {
   Loop Parse, child_path, csv
      Acc := A_LoopField="P"? Acc_Parent(Acc):Acc_Children(Acc)[A_LoopField]
   return Acc
}
GetAccPath(Acc, byref hwnd="") {
   hwnd := Acc_WindowFromObject(Acc)
   WinObj := Acc_ObjectFromWindow(hwnd)
   while Acc_WindowFromObject(Parent:=Acc_Parent(WinObj)) = hwnd
      t1.="P,", WinObj:=Parent
   while Acc_WindowFromObject(Parent:=Acc_Parent(Acc)) = hwnd
      t2:=GetEnumIndex(Acc) "," t2, Acc:=Parent
   return SubStr(t1 t2,1,-1)
}
GetEnumIndex(Acc, ChildId=0) {
   if Not ChildId {
      ChildPos := Acc_Location(Acc).pos
      For Each, child in Acc_Children(Acc_Parent(Acc))
         if IsObject(child) and Acc_Location(child).pos=ChildPos
            return A_Index
   } else {
      ChildPos := Acc_Location(Acc,ChildId).pos
      For Each, child in Acc_Children(Acc)
         if Not IsObject(child) and Acc_Location(Acc,child).pos=ChildPos
            return A_Index
   }
}


The data should be the same, even though one was accessed via Acc_ObjectFromPoint & one via Acc_ObjectFromWindow + a child path. An exception may be if the object under the mouse doesn't expose much IAccessible information - such as a Google Chrome Webpage. Besides these types of exceptions, I would like to know if anyone identifies any IAccessible object that the above script provides differentiating data for.

_________________
Image
Recommended: AutoHotkey_L
Basic Webpage Controls


Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC [ DST ]


Who is online

Users browsing this forum: No registered users and 7 guests


You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group