[DISCONTINUED]IE Web Recorder Developers Needed

Please see Jethrows new thread
Why ?

Rather than continue to hijack other threadswith this subject or continue already badly maintained threads that weren’t really set for this subject I have decided to start this thread. Its purpose is to create useful tools for web based automation developers. a tool kit that will consist of the following tools


If you find this tool usefull and want me to add features i have to be able to afford not to have to work over time. you can help here
Secure paypal link
I will Split a portion of any donations with Jethrow and Sinkfaze.

[*:1yk45m8t]iWebBrowser2 Learner Build ID:2.5 - This tool is under major development effort due to recent interest by serious members of the forum. This is intended to eventually turn into a web simple script writer much like the packaged script recorder for other applications
[*:1yk45m8t]iWeb Functions - Library that relieves the user of intimate understanding of DOM, COM, or JavaScript to control a web siteUpdate from Jethrow 2/19/2011

I updated my compact version of iWebBrowser2 (iWB2) Learner. Here are the main updates:[*:1yk45m8t]Updated to work with Native COM. Now uses Sean's ComUtils[*:1yk45m8t]Added a Menu Item: Show Parent Structure[*:1yk45m8t]Show Parent Structure shows the parent Nodes in a second GUI. Double Clicking an item will highlight the element for a moment. You can copy an element's id/name via Right-Click[*:1yk45m8t]The application now Calculates the Tag Index (in the "document.all.tags(tagName)" collection) after searching the webpage with the Cross Hair. It is calculated when the LButton is released. The index is shown in the WinTitle, next to the tag name.I'm open to any feedback. Here's the iWB2 Learner Download.

I rewrote my Compact iWebBrowser2 Learner script for AHKL. Here's the link if you want to check it out. The primary changes:[*:1yk45m8t]Removed ~LButton Hotkey. Now tests if the A_GuiEvent = "Normal"[*:1yk45m8t]Grouped similar variables into an arrays[*:1yk45m8t]Made the Outline an object[*:1yk45m8t]Used FileInstall to embed Cross-Hair images in .exe

I've already done a re-write of the original iWeb functions (adding a few of my own) for AHK_L.

The tool will be designed so that users who know little more than the following can create web automation

At minimum the user should know how to create
[*:1yk45m8t]scripts
[*:1yk45m8t]hotkeys
[*:1yk45m8t]write functions
[*:1yk45m8t]Understand the concept and use of the standard library
What should you post in this thread?
Code changes
[*:1yk45m8t]Proposed changes with comments in code
[*:1yk45m8t]Reason why the code change suggested have globally useful implications[*:1yk45m8t]Proposed code changes should be easy for intermediate AHK scripter to use
[*:1yk45m8t]Service the end goal of creating a useful simple web based script writer
[*:1yk45m8t]Structured, clean, plain language documentation on the use of iWeb functions. Please hold documenting the recorder till it is in a more final design
What should you not post in this thread?

[*:1yk45m8t]Questions about how to use the recorder?
[*:1yk45m8t]Bugs without proposed fix
[*:1yk45m8t]Questions about syntax
I will post an example of a real world deployment of a web based application that fires command line driven scripts

An example usage of the iWeb functions to demonstrate real world automation needs

Real world example usages
[*:1yk45m8t]Open a new browser
[*:1yk45m8t]Reuse an already open browser
[*:1yk45m8t]Use browser pointer to navigate
[*:1yk45m8t]Wait for navigation to complete
[*:1yk45m8t]Use a browser pointer to fill out and submit a form
[*:1yk45m8t]Use a browser pointer to search the page for specific text in a link and click it
[*:1yk45m8t]Search dynamic content and return an element with an offset
[*:1yk45m8t]Read/write value of global variables already in existence in the page
[*:1yk45m8t]Modify the browser properties
The recorder should be designed to support Scripter’s who intend to create applications with the above guidelines

A screen shot of the tool in use
This tool is based on Sean’s IE HTML Element Spy

Sinkfaze is responsible for some of the features being added


Special thanks to
Sean for COM
Jethrow for the Excellent tutorial he wrote for new members
Sinkfaze for suggestions and some changes to the tool to date

Jethrow, Sinkfaze, and Sean, for the most excellent work in the forum helping users with code problems
Honorable mention of ahklerner who though I havent seen him active in some time actually wrote the function (IE_InjectJS) that got me interested in this subject

Example web deployment of ahk code and example of iWeb function usage
Notice also there is a clickable link to start the application from

it is necesary to open this file either as a local file or with the domain set as a trusted site

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Starting Script...</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<script language="jscript" type="text/jscript">
function regFix(){
   var shell = new ActiveXObject("WScript.shell");
   try{
     shell.RegWrite("HKCR\\.ahk\\", "AutoHotkeyScript", "REG_SZ");
   	shell.RegWrite("HKCR\\AutoHotkeyScript\\Shell\\Open\\Command\\","\"C:\\Program Files\\AutoHotkey\\AutoHotkey.exe\" \"%1\" %*")
   }//end try
   catch(e){
   }//end  catch

    
  }//end of function

</script>
</head>
<body onLoad="regFix();Downloader();window.close();void(0);">
<p>This script clean up after itself by deleting the downloaded files once its done</p>
<p>This window will Close once your application starts</p>
<p>If it doesnt start Click <a href="javascript:void(0);" onClick="regFix();Downloader();window.close();void(0);">Here</a> </p>
</body>
<script language="VBScript">
Function Downloader()
	on error resume next
	Set oShell = CreateObject( "WScript.Shell" )
	Set objFSO = CreateObject("Scripting.FileSystemObject")
	 
	ahk=objFSO.FileExists("C:\Program Files\AutoHotkey\AutoHotkey.exe")
	If not ahk Then
		Set netSys = CreateObject("WScript.Network")  
		MsgBox "Autohotkey is not installed on  " & chr(13)  & netSys.ComputerName ,1,"Setup Issue" 
		Set netSys = nothing
		Set objFSO = nothing
		Set oShell = nothing
		exit function
	End If
	strDirectory = "C:\temp\Scripts\"
	If objFSO.FolderExists(strDirectory) Then
		objFSO.DeleteFolder(strDirectory)
	End If
	Set objFolder = objFSO.CreateFolder(strDirectory)
	Set objFile = objFSO.OpenTextFile(strDirectory & "Scripts.ahk", 8, True) 
	objFile.WriteLine("URLDownloadToFile,<serverpath to file>,"&strDirectory&"Downloaded.ahk"&chr(10)&"if errorlevel"&chr(10)&"{"&chr(10)&"msgbox script failed to start"&chr(10)&""&chr(10)&"}"&chr(10)&"run,"&strDirectory&"Downloaded.ahk"&chr(10)&"detecthiddenwindows,on"&chr(10)&"winwait,"&strDirectory&"Download.ahk,,5")
	objFile.Close
	oShell.Run """C:\temp\Scripts\Scripts.ahk""", 6, true 

	If objFSO.FolderExists(strDirectory) Then
		objFSO.DeleteFolder(strDirectory)
	End If
	Set oShell = nothing
	Set objFSO = nothing
End Function
</script>
</html>

Downloaded.ahk

MsgBox, 0, Search results, This file was on the server`nand now its been downloaded and executed with a command line parameter of %1%
COM_CoInitialize()
pwb:=iWeb_Model()
iWeb_nav(pwb,"http://www.google.com")
iWeb_clickText(pwb,"images")
iWeb_complete(pwb)
iWeb_setDomObj(pwb,"q", "dragons")
iWeb_clickValue(pwb,"search images")
iWeb_complete(pwb)
maxLimit:=iWeb_getDomObj(pwb,"maxLimit")
StringReplace,maxLimit,maxLimit,&#[color=blue]44;[/color],`,,all
MsgBox, 0, Search results,% "wow " maxLimit " results"
iWeb_clickDomObj(pwb,"276")
iWeb_complete(pwb)
COM_Release(pwb)
COM_CoUninitialize()

Very Nice tank! I amended iWebBrowser2 Learner to outline webpage elements as you hover over them. Also, I modified it such that if you are holding the RButton, iWebBrowser2 Learner doesn't update. (I put a commented line around the parts I modified)

<!-- m -->https://ahknet.autoh... ... corder.ahk<!-- m --> - Updated 12/2

Great you implemented Seans idea I love it also more than i thought i would
The screen shot unfortunately does not show it but its there

also note the changelog in the zip file
by the way i have made minor updates ti iWeb and other updates to the learner since you seem to have gotten it last might be best to re download

Very Nice tank! I amended iWebBrowser2 Learner to outline webpage elements as you hover over them. Also, I modified it such that if you are holding the RButton, iWebBrowser2 Learner doesn't update. (I put a commented line around the parts I modified)

<!-- m -->https://ahknet.autoh... ... corder.ahk<!-- m -->

That is really very nice, thank you
When you add NA option it's perfect

Gui, 2:Show, % "[color=red]NA[/color] X" x1-2 " Y" y1-2 " W" x2-x1+4 " H" 2
	Gui, 3:Show, % "[color=red]NA[/color] X" x2 " Y" y1 " W" 2 " H" y2-y1
	Gui, 4:Show, % "[color=red]NA[/color] X" x1-2 " Y" y2 " W" x2-x1+4 " H" 2
	Gui, 5:Show, % "[color=red]NA[/color] X" x1-2 " Y" y1 " W" 2 " H" y2-y1

Good call HotKeyIt I have added your suggestion to the change log and the code and updated the files in the download links at the top of the post

jethrow/tank,

Do you suppose there's any way to prevent the red border from bleeding over onto the recorder (or "flickering" once over the recorder while in "Always on Top" mode)? In addition, the red border seems to assume a "maximized window" mode, so if you're operating on a non-maximized window the border runs all over the screen. I don't see anything obvious that can remedy this but I'm generally the most dense person on any given project. :wink:

I'm still trying to come up with a way to effectively output pre-made function calls based on the element under the mouse pointer, but I just don't see any otherwise efficient way to do it without Accessibility. What functionality I can create with Accessibility is quite good, but it's yet another library to include to run the file when I won't even use most of ACC's functionality.

And FYI jethrow, based on your suggestion I double-checked with Sean and not all ACC functionality was absorbed into the standard library and unfortunately what is available is not enough for what I would need on my development idea.

My workaround is to add only the necessary ACC calls to the script directly and append the prefix "iWeb" to all of the ACC calls so 1) it's not necessary to download the entire ACC library and 2) the script will not present a conflict for those who currently have ACC in their library. Would this be an acceptable fix for you tank if my revisions prove worthy?

Here's an expansion on my earlier idea, Ctrl+e is again the operating hotkey.

I suppose it would require doing something without the gui like wrapping the selected elements outerhtml with a dive with a red boarder and removing it on each cycle but i dont know it may offset some other peice of html because of bad page writing

another option might be to use the window.screen.availHeight/width properties to set limits to the current window/frame. However i dont have time ATM to make the necesary code changes. As to the flickering,
changing

Gui, % A_Index+1 ": -Caption +ToolWindow +AlwaysOnTop"

to

Gui, % A_Index+1 ": -Caption +ToolWindow"

Seems to do the trick

I updated my link with a version that will only outline the element within the dimensions of the Internet Explorer_Server1 control.

I have mercged these changes and am due to this adding a build ID: to the title of the tool starting now Build ID:1.0

I'm still trying to come up with a way to effectively output pre-made function calls based on the element under the mouse pointer, but I just don't see any otherwise efficient way to do it without Accessibility. What functionality I can create with Accessibility is quite good, but it's yet another library to include to run the file when I won't even use most of ACC's functionality.

I am kind of dumb so i really dont understand what result ACC gives you towards this that is not already available with COM? could you elaborate what effect your aiming for and why?

When I test your code it only works on the limited elements ACC can access( links and stuff) when it may be necesary to write automation for all elements

If your woried about clicking links I already told you clicking the elements nested inside of an a tag procuces the same results.

is iWeb_clicktext not adequet and if not why?

As to the flickering,
changing

Gui, % A_Index+1 ": -Caption +ToolWindow +AlwaysOnTop"

to

Gui, % A_Index+1 ": -Caption +ToolWindow"

Seems to do the trick

To some extent. If you check "Always on top" while in a maximized IE window and stay in that window it works perfectly. In a non-maximized window the lines still roll over the GUI and every time you navigate to a new window it will not force the borders back under the GUI until you uncheck and recheck "Always on top".

Can you tell I have this thing open quite a bit? :lol:

I am kind of dumb so i really dont understand what result ACC gives you towards this that is not already available with COM? could you elaborate what effect your aiming for and why?

Sorry, I didn't mean it as an either/or proposition except to the extent that the ACC functionality native to the standard library is insufficient for what I'm exploring.

Neither COM nor ACC will solely provide a viable solution to accurately identify page elements, but working together they can identify (and thus classify) a great deal of them. The more pages elements we can clearly identify, the more specific results we can output to the end user and the less work we have to do in support on the backend.

When I test your code it only works on the limited elements ACC can access( links and stuff) when it may be necesary to write automation for all elements.

It's nowhere near done but hopefully you can see where it's headed for whatever limited functionality it has. It creates a sample function call (or even multiple samples using different information) which can (eventually) be output to some kind of recorder or even a script.

i still dont understand whats innacurate for the element info?


is there an example?

whats inaccurate about the current example function calls i cant find any differences?
except for when acc moves up the dom tree from a nested unsupported by acc tag. which is entirely irrelivant or isit can you give any example of how for instance clikcing the nested element isnt effective?

i still dont understand whats innacurate for the element info? is there an example?

Take INPUT elements, for example. These can be edit fields, check boxes or submit buttons and they all may need to be handled differently. AFAIK the recorder identifies an element as an INPUT and nothing more, so while an iWeb_clickDomObj will work by default for a checkbox or a button it will not for an edit field. Conversely, iWeb_setDomObj is appropriate for an edit field but not necessarily for a check box and probably not for a button. In addition we may run into the button that has no name or id but can be clicked on by value, but it would kind of defeat the purpose of having iWeb_clickValue if the recorder never gave the end user a reason to know it exists, much less use it.

By retrieveing the additional information about the page element from ACC I can more effectively sort those elements and ensure a better rate of successful code output.

Keep in mind that I'm not proposing that you change anything about how the recorder itself or how your iWeb function field works, this is functionality I'm working on looking forward to an automated script output. Making sure the sample hotkey outputs an appropriate function for the page element under the mouse is important, and ACC as a complement to COM appears to be very valuable in that regard. It's not perfect by any means, but at least better.

I've uploaded an amended version of the web recorder, hopefully with all of the gaps in the hotkey functionality closed off. Again, I just want to test and make sure the hotkey is outputting function calls appropriate to the element being hovered over. Also my coding is generally awful so feel free to correct me in whatever way seems best. :oops:

Take INPUT elements, for example. These can be edit fields, check boxes or submit buttons and they all may need to be handled differently. AFAIK the recorder identifies an element as an INPUT and nothing more, so while an iWeb_clickDomObj will work by default for a checkbox or a button it will not for an edit field. Conversely, iWeb_setDomObj is appropriate for an edit field but not necessarily for a check box and probably not for a button. In addition we may run into the button that has no name or id but can be clicked on by value, but it would kind of defeat the purpose of having iWeb_clickValue if the recorder never gave the end user a reason to know it exists, much less use it.

all true but the information is available
we could for instance determin the input type by asking for the type attribute from the element

the dom viewer does show you if check box exists if by nothing else the source index

we can ask if its checked by its checked property
No need to involve another library of functions for this since its all accessable within COM

example

com_invoke(pwb,"document.all[inputID].type")

now we know if its a check box or a button or what

I think what your looking to do is decide what actions are apporpriate

if thats the case this is the path i would go COM_ConnectObject did the user click something type something etc
set the page being examined to editable with
designMode then if a user edits something we know they need iweb_setdomobj etc

com_invoke(pwb,"document.all[inputID].type")

So there's the trick you COM wizards hide amongst yourselves.

I think what your looking to do is decide what actions are apporpriate

if thats the case this is the path i would go COM_ConnectObject did the user click something type something etc

Absolutely correct, I want to anticipate to present the appropriate function call. Since I wasn't aware of that call I was working to hell and back to replicate it via ACC. The COM_Invoke "type" call I can probably work my way through, but the COM_ConnectObject method might require a little explaining.

set the page being examined to editable with
designMode then if a user edits something we know they need iweb_setdomobj etc

:idea: Wow you're way ahead of me, I was just looking to create the call and let them type whatever they want into the field manually after the fact (whether in a recorder or script).