Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

IE HTML Element Spy


  • Please log in to reply
42 replies to this topic
Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007
It retrieves the info of a DOM element under the cursor when the cursor is hovering over Internet Explorer_Server.
The hotkey is currently set to ^E.

REQUIRE COM Standard Library.

^E::MsgBox, % IE_HtmlElement()

IE_HtmlElement()
{
	CoordMode, Mouse
	MouseGetPos, xpos, ypos,, hWnd, 3
	Loop
	{
		If Not	hWnd
		Exit
		WinGetClass, sClass, ahk_id %hWnd%
		If Not	sClass == "Internet Explorer_Server"
			hWnd :=	DllCall("GetAncestor", "Uint", hWnd, "Uint", 1)
		Else	Break
	}
	If	pwin := IE_GetWindow(hWnd)
	{
	If	pelt := COM_Invoke(pwin, "document.elementFromPoint", xpos-xorg:=COM_Invoke(pwin,"screenLeft"), ypos-yorg:=COM_Invoke(pwin,"screenTop"))
	{
		While	(type:=COM_Invoke(pelt,"tagName"))="IFRAME" || type="FRAME"
		{
		selt .= "[" type "]" (A_Index==1 ? "":"." A_Index-1) "`n" COM_Invoke(pelt,"src") "`n", COM_Release(pwin)
		pwin := COM_QueryService(pbrt:=COM_Invoke(pelt,"contentWindow"), "{332C4427-26CB-11D0-B483-00C04FD90119}"), COM_Release(pbrt)
		pbrt := COM_Invoke(pwin, "document.elementFromPoint", xpos-xorg+=COM_Invoke(pelt,"getBoundingClientRect.left"), ypos-yorg+=COM_Invoke(pelt,"getBoundingClientRect.top")), COM_Release(pelt), pelt:=pbrt
		}
		pbrt :=	COM_Invoke(pelt, "getBoundingClientRect")
		l  :=	COM_Invoke(pbrt, "left")
		t  :=	COM_Invoke(pbrt, "top")
		r  :=	COM_Invoke(pbrt, "right")
		b  :=	COM_Invoke(pbrt, "bottom")
		COM_Release(pbrt)
		selt .=	"[HTML]`n" COM_Invoke(pelt, "outerHTML") "`n[TEXT]`n" COM_Invoke(pelt, "outerText") "`n[RECTANGLE]`n(" l+xorg "," t+yorg "," r+xorg "," b+yorg ")"
		COM_Release(pelt)
	}
	COM_Release(pwin)
	Return	selt
	}
}

IE_GetWindow(hWnd)
{
	Static	h:=0
	If	h =0
		h:=COM_AccInit()
	If	pacc:=COM_AccessibleObjectFromWindow(hWnd)
	Return	COM_QueryService(pacc, "{332C4427-26CB-11D0-B483-00C04FD90119}") + COM_Release(pacc)*0
}

IE_GetDocument(hWnd)
{
	Static	h:=0,msg,iid
	If	h =0
		h:=COM_AccInit(), msg:=DllCall("user32\RegisterWindowMessage", "str", "WM_HTML_GETOBJECT"), COM_GUID4String(iid, "{332C4425-26CB-11D0-B483-00C04FD90119}")
	If	DllCall("user32\SendMessageTimeout", "Uint", hWnd, "Uint", msg, "Uint", 0, "Uint", 0, "Uint", 2, "Uint", 1000, "UintP", lr) && DllCall("oleacc\ObjectFromLresult", "Uint", lr, "Uint", &iid, "Uint", 0, "UintP", pdoc)=0
	Return	pdoc
}


tank
  • Moderators
  • 4240 posts
  • Last active: Aug 22 2014 10:52 PM
  • Joined: 21 Dec 2007
Thanks for this for this totally different approach it will probably be easier for most users to digest than any element parsers i have posted


I tried this out just to see if it resolved an issue i have yet to solve well
delving into elements within frames
example


I even tried something similar to this once based on some functionality Lexikos demonstrated in scrollmomentum
GetDeepestScrollElement which worked exceptionally well but i couldnt seem to have any luck mapping a path out so users could easily get the path i.e. return something like
document.frames[0].contentwindow.document.all.item[blah].innerhtml
in particular the frame index(when no name or id is avail) was the point of dificulty

since your on the IE subject i am curious if you have any thoughts on how to return a frame index

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007

I tried this out just to see if it resolved an issue i have yet to solve well delving into elements within frames

Good question. As a matter of fact, I was a little hesitated about whether should try to recurse into frames or not, however, decided not, to keep it simple.

I believe the key function with (i)frames is contentWindow (or contentDocument if supported) which represents the Window (or Document) object of (i)frames. So, can apply the same method to it as done in the function posted.

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007
OK, I updated the function to recurse into (i)frames.

tank
  • Moderators
  • 4240 posts
  • Last active: Aug 22 2014 10:52 PM
  • Joined: 21 Dec 2007
Nice try :idea:
Perhaps i explained the issue incorrectly

first i start by what i think i see in your function
[frame] seems to mean the first frame in the recursion :D
.....
<body> <!-- [frame] -->
<iframe src="blah.html"></iframe>
 .....
blah.html
.....
<body> <!-- [frame].1 -->
<iframe src="blah2.html"></iframe>
where i was asking for a way to get the actual frame index
.....
<body> <!-- [frame] -->
<iframe src="blah.html"></iframe>
<!-- [frame].1 -->
<iframe src="bleh.html"></iframe> .....
blah.html
.....
<body> <!-- [frame] -->
<iframe src="blah2.html"></iframe>
<!-- [frame].1 -->
<iframe src="bleh2.html"></iframe> .....

thus giving the user the information necesary to easily write the following code to get content from bleh2.html
;first frame of the browsers top window
;second frame of the child frames window
COM_Invoke(pwb,document.frames.item[0].contentwindow.frames.item[1].contentwindow.document.body.innerhtml")
I want to be clear the only tactic i have been able to come up with is taking the bodies of each of the frames and comparing with the returned frame body to determin which frame in the collection of frames i have

this is of course heavy on overhead and perhaps (i hope) not the best way

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007
Why don't you just use sourceIndex?
idx0 := COM_Invoke(frame0, "sourceIndex")
idx1 := COM_Invoke(frame1, "sourceIndex")
...
COM_Invoke(pwb,"document.all.item[" idx0 "].contentwindow.document.all.item[" idx1 "].contentwindow.document.body.innerhtml")


tank
  • Moderators
  • 4240 posts
  • Last active: Aug 22 2014 10:52 PM
  • Joined: 21 Dec 2007

Why don't you just use sourceIndex?

because some layouts are dynamic and would shift source index often the frame index is more reliable as a point of reference :wink:

either way thanks for this function. :D I suggest if there is not a way to reference the frame index, then the delving into frames might be represented as you have suggested instead of the way it is now, but thats just my simple short sited opinion :wink:

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007

I suggest if there is not a way to reference the frame index, then the delving into frames might be represented as you have suggested instead of the way it is now, but thats just my simple short sited opinion :wink:

Yes, I'm not aware of a way to return the frame index. From the standpoint of AHK, firefox may be easier to use when dealing with frames as a frame creates/uses a separate (its own) window in FF, which is not the case with IE. So, I may add also to display the screen coordinates of the bounding rectangle of a frame at next update.

tank
  • Moderators
  • 4240 posts
  • Last active: Aug 22 2014 10:52 PM
  • Joined: 21 Dec 2007
Thanks :D

tank
  • Moderators
  • 4240 posts
  • Last active: Aug 22 2014 10:52 PM
  • Joined: 21 Dec 2007
Since Sean has seen fit to entertain my notions (thanks :D )

I feel inclined to share my wrapping of in a gui and some alterations
as usual this link is 90% Sean's genious and 50% my frakking it up but.... hopefully it helps :lol: My screw up :D

ctrl / to pause or resume

Posted Image
nuthing at all fancy

I hope you dont mind me posting my adaption in your thread but since its based on your above posted code i thought it might be appropriate if not let me know and I'll get it moved off your thread

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007
Of course I don't mind. BTW, I was more interested in providing GUI to highlight the bounding rectangle of the element, however, I haven't decided which one to use yet.

DeWild1
  • Members
  • 369 posts
  • Last active: Feb 28 2014 08:15 PM
  • Joined: 30 Apr 2006

Tank + Sean work of fricken ART :D

ctrl / to pause or resume


This is very pretty, functional beyond my understanding but my head is full of rocks.... :roll:

Could you make a small example of how to put the different parts into a script.
Most may see google log in as a useful example or craigslist or for fun, make an example script with the differnet things you could automate on like Mafia Wars on myspace.com. 8)

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007

Could you make a small example of how to put the different parts into a script.

I'm not sure of what you try to achieve. You can retrieve the whole document too, instead of sneaking into each element via this script.

tank
  • Moderators
  • 4240 posts
  • Last active: Aug 22 2014 10:52 PM
  • Joined: 21 Dec 2007
I'll explain as this is something i have discussed before. an html equiv of the script recorder records which elements clicked and automatically generates a rudimentary script.
I have neither the time nor interest to take it to this level any time soon

Sean
  • Members
  • 2462 posts
  • Last active: Feb 07 2012 04:00 AM
  • Joined: 12 Feb 2007

I'll explain as this is something i have discussed before. an html equiv of the script recorder records which elements clicked and automatically generates a rudimentary script.

Oh, I see. The basic idea of the solution is simple: set-up events via COM_ConnectObject. But, the details will be tedious, so, I'll also leave the details to the users.