AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

[Project Development]IE Web Recorder Developers Needed
Goto page 1, 2, 3 ... 9, 10, 11  Next
 
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Scripts & Functions
View previous topic :: View next topic  
Author Message
tank



Joined: 21 Dec 2007
Posts: 2294
Location: Louisville KY USA
PostPosted: Wed Nov 18, 2009 4:41 am    Post subject: [Project Development]IE Web Recorder Developers Needed Reply with quote
Why ?

Rather than continue to hijack other threads with this subject or continue already badly maintained threads that weren’t really set for this subject I have decided to start this thread. Its purpose is to create useful tools for web based automation developers. a tool kit that will consist of the following tools

  • iWebBrowser2 Learner Build ID:2.5 - This tool is under major development effort due to recent interest by serious members of the forum. This is intended to eventually turn into a web simple script writer much like the packaged script recorder for other applications
  • iWeb Functions - Very stable well tested library the relieves the user of intimate understanding DOM, COM, or JavaScript to control a web site


The tool will be designed so that users who know little more than the following can create web automation


What should you post in this thread?
    Code changes
  • Proposed changes with comments in code
  • Reason why the code change suggested have globally useful implications
  • Proposed code changes should be easy for intermediate AHK scripter to use
  • Service the end goal of creating a useful simple web based script writer
  • Structured, clean, plain language documentation on the use of iWeb functions. Please hold documenting the recorder till it is in a more final design


What should you not post in this thread?

  • Questions about how to use the recorder?
  • Bugs without proposed fix
  • Questions about syntax


I will post an example of a real world deployment of a web based application that fires command line driven scripts

An example usage of the iWeb functions to demonstrate real world automation needs

    Real world example usages
  • Open a new browser
  • Reuse an already open browser
  • Use browser pointer to navigate
  • Wait for navigation to complete
  • Use a browser pointer to fill out and submit a form
  • Use a browser pointer to search the page for specific text in a link and click it
  • Search dynamic content and return an element with an offset
  • Read/write value of global variables already in existence in the page
  • Modify the browser properties


The recorder should be designed to support Scripter’s who intend to create applications with the above guidelines

A screen shot of the tool in use
This tool is based on Sean’s IE HTML Element Spy

Sinkfaze is responsible for some of the features being added


Special thanks to
Sean for COM
Jethrow for the Excellent tutorial he wrote for new members
Sinkfaze for suggestions and some changes to the tool to date

Jethrow, Sinkfaze, and Sean, for the most excellent work in the forum helping users with code problems
Honorable mention of ahklerner who though I havent seen him active in some time actually wrote the function (IE_InjectJS) that got me interested in this subject
_________________
Basic Webpage Controls with JavaScript / COM - Tutorial by Jethrow


Last edited by tank on Sun Nov 29, 2009 3:23 pm; edited 6 times in total
Back to top
View user's profile Send private message
tank



Joined: 21 Dec 2007
Posts: 2294
Location: Louisville KY USA
PostPosted: Wed Nov 18, 2009 4:49 am    Post subject: Example application Reply with quote
Example web deployment of ahk code and example of iWeb function usage
Notice also there is a clickable link to start the application from

it is necesary to open this file either as a local file or with the domain set as a trusted site
Code:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Starting Script...</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<script language="jscript" type="text/jscript">
function regFix(){
   var shell = new ActiveXObject("WScript.shell");
   try{
     shell.RegWrite("HKCR\\.ahk\\", "AutoHotkeyScript", "REG_SZ");
      shell.RegWrite("HKCR\\AutoHotkeyScript\\Shell\\Open\\Command\\","\"C:\\Program Files\\AutoHotkey\\AutoHotkey.exe\" \"%1\" %*")
   }//end try
   catch(e){
   }//end  catch

   
  }//end of function

</script>
</head>
<body onLoad="regFix();Downloader();window.close();void(0);">
<p>This script clean up after itself by deleting the downloaded files once its done</p>
<p>This window will Close once your application starts</p>
<p>If it doesnt start Click <a href="javascript:void(0);" onClick="regFix();Downloader();window.close();void(0);">Here</a> </p>
</body>
<script language="VBScript">
Function Downloader()
   on error resume next
   Set oShell = CreateObject( "WScript.Shell" )
   Set objFSO = CreateObject("Scripting.FileSystemObject")
   
   ahk=objFSO.FileExists("C:\Program Files\AutoHotkey\AutoHotkey.exe")
   If not ahk Then
      Set netSys = CreateObject("WScript.Network") 
      MsgBox "Autohotkey is not installed on  " & chr(13)  & netSys.ComputerName ,1,"Setup Issue"
      Set netSys = nothing
      Set objFSO = nothing
      Set oShell = nothing
      exit function
   End If
   strDirectory = "C:\temp\Scripts\"
   If objFSO.FolderExists(strDirectory) Then
      objFSO.DeleteFolder(strDirectory)
   End If
   Set objFolder = objFSO.CreateFolder(strDirectory)
   Set objFile = objFSO.OpenTextFile(strDirectory & "Scripts.ahk", 8, True)
   objFile.WriteLine("URLDownloadToFile,<serverpath to file>,"&strDirectory&"Downloaded.ahk"&chr(10)&"if errorlevel"&chr(10)&"{"&chr(10)&"msgbox script failed to start"&chr(10)&""&chr(10)&"}"&chr(10)&"run,"&strDirectory&"Downloaded.ahk"&chr(10)&"detecthiddenwindows,on"&chr(10)&"winwait,"&strDirectory&"Download.ahk,,5")
   objFile.Close
   oShell.Run """C:\temp\Scripts\Scripts.ahk""", 6, true

   If objFSO.FolderExists(strDirectory) Then
      objFSO.DeleteFolder(strDirectory)
   End If
   Set oShell = nothing
   Set objFSO = nothing
End Function
</script>
</html>

Downloaded.ahk
Code:
MsgBox, 0, Search results, This file was on the server`nand now its been downloaded and executed with a command line parameter of %1%
COM_CoInitialize()
pwb:=iWeb_Model()
iWeb_nav(pwb,"http://www.google.com")
iWeb_clickText(pwb,"images")
iWeb_complete(pwb)
iWeb_setDomObj(pwb,"q", "dragons")
iWeb_clickValue(pwb,"search images")
iWeb_complete(pwb)
maxLimit:=iWeb_getDomObj(pwb,"maxLimit")
StringReplace,maxLimit,maxLimit,&#44;,`,,all
MsgBox, 0, Search results,% "wow " maxLimit " results"
iWeb_clickDomObj(pwb,"276")
iWeb_complete(pwb)
COM_Release(pwb)
COM_CoUninitialize()

_________________
Basic Webpage Controls with JavaScript / COM - Tutorial by Jethrow


Last edited by tank on Wed Nov 18, 2009 8:30 pm; edited 1 time in total
Back to top
View user's profile Send private message
jethrow



Joined: 24 May 2009
Posts: 722
Location: Iowa, USA
PostPosted: Wed Nov 18, 2009 8:44 am    Post subject: Reply with quote
Very Nice tank! Very Happy I amended iWebBrowser2 Learner to outline webpage elements as you hover over them. Also, I modified it such that if you are holding the RButton, iWebBrowser2 Learner doesn't update. (I put a commented line around the parts I modified)

http://www.autohotkey.net/~jethrow/Revised_ahk_web_recorder.ahk - Updated 12/2
_________________
AHKL, COM_L,Webpage Controls,Donate to AHK

Last edited by jethrow on Thu Dec 03, 2009 4:27 am; edited 10 times in total
Back to top
View user's profile Send private message
tank



Joined: 21 Dec 2007
Posts: 2294
Location: Louisville KY USA
PostPosted: Wed Nov 18, 2009 3:50 pm    Post subject: Reply with quote
Great you implemented Seans idea I love it also more than i thought i would
The screen shot unfortunately does not show it but its there

also note the changelog in the zip file
by the way i have made minor updates ti iWeb and other updates to the learner since you seem to have gotten it last might be best to re download
_________________
Basic Webpage Controls with JavaScript / COM - Tutorial by Jethrow
Back to top
View user's profile Send private message
HotKeyIt



Joined: 18 Jun 2008
Posts: 1994
Location: GERMANY
PostPosted: Wed Nov 18, 2009 5:40 pm    Post subject: Reply with quote
jethrow wrote:
Very Nice tank! Very Happy I amended iWebBrowser2 Learner to outline webpage elements as you hover over them. Also, I modified it such that if you are holding the RButton, iWebBrowser2 Learner doesn't update. (I put a commented line around the parts I modified)

http://www.autohotkey.net/~jethrow/Revised_ahk_web_recorder.ahk

That is really very nice, thank you Wink
When you add NA option it's perfect Very Happy
Code:
   Gui, 2:Show, % "NA X" x1-2 " Y" y1-2 " W" x2-x1+4 " H" 2
   Gui, 3:Show, % "NA X" x2 " Y" y1 " W" 2 " H" y2-y1
   Gui, 4:Show, % "NA X" x1-2 " Y" y2 " W" x2-x1+4 " H" 2
   Gui, 5:Show, % "NA X" x1-2 " Y" y1 " W" 2 " H" y2-y1

_________________
AutoHotFile - ToolTip(n,text,title,options) Wink
Back to top
View user's profile Send private message
tank



Joined: 21 Dec 2007
Posts: 2294
Location: Louisville KY USA
PostPosted: Wed Nov 18, 2009 5:48 pm    Post subject: Reply with quote
Good call HotKeyIt I have added your suggestion to the change log and the code and updated the files in the download links at the top of the post
_________________
Basic Webpage Controls with JavaScript / COM - Tutorial by Jethrow
Back to top
View user's profile Send private message
sinkfaze



Joined: 18 Mar 2008
Posts: 2424
PostPosted: Wed Nov 18, 2009 7:09 pm    Post subject: Reply with quote
jethrow/tank,

Do you suppose there's any way to prevent the red border from bleeding over onto the recorder (or "flickering" once over the recorder while in "Always on Top" mode)? In addition, the red border seems to assume a "maximized window" mode, so if you're operating on a non-maximized window the border runs all over the screen. I don't see anything obvious that can remedy this but I'm generally the most dense person on any given project. Wink

I'm still trying to come up with a way to effectively output pre-made function calls based on the element under the mouse pointer, but I just don't see any otherwise efficient way to do it without Accessibility. What functionality I can create with Accessibility is quite good, but it's yet another library to include to run the file when I won't even use most of ACC's functionality.

And FYI jethrow, based on your suggestion I double-checked with Sean and not all ACC functionality was absorbed into the standard library and unfortunately what is available is not enough for what I would need on my development idea.

My workaround is to add only the necessary ACC calls to the script directly and append the prefix "iWeb" to all of the ACC calls so 1) it's not necessary to download the entire ACC library and 2) the script will not present a conflict for those who currently have ACC in their library. Would this be an acceptable fix for you tank if my revisions prove worthy?

Here's an expansion on my earlier idea, Ctrl+e is again the operating hotkey.
_________________
Try Quick Search for Autohotkey or see the tutorial for newbies.
Back to top
View user's profile Send private message
tank



Joined: 21 Dec 2007
Posts: 2294
Location: Louisville KY USA
PostPosted: Wed Nov 18, 2009 7:44 pm    Post subject: Reply with quote
I suppose it would require doing something without the gui like wrapping the selected elements outerhtml with a dive with a red boarder and removing it on each cycle but i dont know it may offset some other peice of html because of bad page writing

another option might be to use the window.screen.availHeight/width properties to set limits to the current window/frame. However i dont have time ATM to make the necesary code changes. As to the flickering,
changing
Code:
Gui, % A_Index+1 ": -Caption +ToolWindow +AlwaysOnTop"
to
Code:
Gui, % A_Index+1 ": -Caption +ToolWindow"
Seems to do the trick

jethrow wrote:
I updated my link with a version that will only outline the element within the dimensions of the Internet Explorer_Server1 control.


I have mercged these changes and am due to this adding a build ID: to the title of the tool starting now Build ID:1.0
_________________
Basic Webpage Controls with JavaScript / COM - Tutorial by Jethrow
Back to top
View user's profile Send private message
tank



Joined: 21 Dec 2007
Posts: 2294
Location: Louisville KY USA
PostPosted: Wed Nov 18, 2009 8:12 pm    Post subject: Reply with quote
Sinkfaze wrote:
I'm still trying to come up with a way to effectively output pre-made function calls based on the element under the mouse pointer, but I just don't see any otherwise efficient way to do it without Accessibility. What functionality I can create with Accessibility is quite good, but it's yet another library to include to run the file when I won't even use most of ACC's functionality.
I am kind of dumb so i really dont understand what result ACC gives you towards this that is not already available with COM? could you elaborate what effect your aiming for and why?

When I test your code it only works on the limited elements ACC can access( links and stuff) when it may be necesary to write automation for all elements

If your woried about clicking links I already told you clicking the elements nested inside of an a tag procuces the same results.

is iWeb_clicktext not adequet and if not why?
_________________
Basic Webpage Controls with JavaScript / COM - Tutorial by Jethrow
Back to top
View user's profile Send private message
sinkfaze



Joined: 18 Mar 2008
Posts: 2424
PostPosted: Wed Nov 18, 2009 9:25 pm    Post subject: Reply with quote
tank wrote:
As to the flickering,
changing
Code:
Gui, % A_Index+1 ": -Caption +ToolWindow +AlwaysOnTop"
to
Code:
Gui, % A_Index+1 ": -Caption +ToolWindow"
Seems to do the trick


To some extent. If you check "Always on top" while in a maximized IE window and stay in that window it works perfectly. In a non-maximized window the lines still roll over the GUI and every time you navigate to a new window it will not force the borders back under the GUI until you uncheck and recheck "Always on top".

Can you tell I have this thing open quite a bit? Laughing
_________________
Try Quick Search for Autohotkey or see the tutorial for newbies.
Back to top
View user's profile Send private message
sinkfaze



Joined: 18 Mar 2008
Posts: 2424
PostPosted: Wed Nov 18, 2009 9:52 pm    Post subject: Reply with quote
tank wrote:
I am kind of dumb so i really dont understand what result ACC gives you towards this that is not already available with COM? could you elaborate what effect your aiming for and why?


Sorry, I didn't mean it as an either/or proposition except to the extent that the ACC functionality native to the standard library is insufficient for what I'm exploring.

Neither COM nor ACC will solely provide a viable solution to accurately identify page elements, but working together they can identify (and thus classify) a great deal of them. The more pages elements we can clearly identify, the more specific results we can output to the end user and the less work we have to do in support on the backend.

tank wrote:
When I test your code it only works on the limited elements ACC can access( links and stuff) when it may be necesary to write automation for all elements.


It's nowhere near done but hopefully you can see where it's headed for whatever limited functionality it has. It creates a sample function call (or even multiple samples using different information) which can (eventually) be output to some kind of recorder or even a script.
_________________
Try Quick Search for Autohotkey or see the tutorial for newbies.
Back to top
View user's profile Send private message
tank



Joined: 21 Dec 2007
Posts: 2294
Location: Louisville KY USA
PostPosted: Wed Nov 18, 2009 9:54 pm    Post subject: Reply with quote
i still dont understand whats innacurate for the element info?


is there an example?

whats inaccurate about the current example function calls i cant find any differences?
except for when acc moves up the dom tree from a nested unsupported by acc tag. which is entirely irrelivant or isit can you give any example of how for instance clikcing the nested element isnt effective?
_________________
Basic Webpage Controls with JavaScript / COM - Tutorial by Jethrow
Back to top
View user's profile Send private message
sinkfaze



Joined: 18 Mar 2008
Posts: 2424
PostPosted: Wed Nov 18, 2009 10:55 pm    Post subject: Reply with quote
tank wrote:
i still dont understand whats innacurate for the element info? is there an example?


Take INPUT elements, for example. These can be edit fields, check boxes or submit buttons and they all may need to be handled differently. AFAIK the recorder identifies an element as an INPUT and nothing more, so while an iWeb_clickDomObj will work by default for a checkbox or a button it will not for an edit field. Conversely, iWeb_setDomObj is appropriate for an edit field but not necessarily for a check box and probably not for a button. In addition we may run into the button that has no name or id but can be clicked on by value, but it would kind of defeat the purpose of having iWeb_clickValue if the recorder never gave the end user a reason to know it exists, much less use it.

By retrieveing the additional information about the page element from ACC I can more effectively sort those elements and ensure a better rate of successful code output.

Keep in mind that I'm not proposing that you change anything about how the recorder itself or how your iWeb function field works, this is functionality I'm working on looking forward to an automated script output. Making sure the sample hotkey outputs an appropriate function for the page element under the mouse is important, and ACC as a complement to COM appears to be very valuable in that regard. It's not perfect by any means, but at least better.

I've uploaded an amended version of the web recorder, hopefully with all of the gaps in the hotkey functionality closed off. Again, I just want to test and make sure the hotkey is outputting function calls appropriate to the element being hovered over. Also my coding is generally awful so feel free to correct me in whatever way seems best. Embarassed
_________________
Try Quick Search for Autohotkey or see the tutorial for newbies.
Back to top
View user's profile Send private message
tank



Joined: 21 Dec 2007
Posts: 2294
Location: Louisville KY USA
PostPosted: Wed Nov 18, 2009 11:03 pm    Post subject: Reply with quote
Quote:
Take INPUT elements, for example. These can be edit fields, check boxes or submit buttons and they all may need to be handled differently. AFAIK the recorder identifies an element as an INPUT and nothing more, so while an iWeb_clickDomObj will work by default for a checkbox or a button it will not for an edit field. Conversely, iWeb_setDomObj is appropriate for an edit field but not necessarily for a check box and probably not for a button. In addition we may run into the button that has no name or id but can be clicked on by value, but it would kind of defeat the purpose of having iWeb_clickValue if the recorder never gave the end user a reason to know it exists, much less use it.
all true but the information is available
we could for instance determin the input type by asking for the type attribute from the element

the dom viewer does show you if check box exists if by nothing else the source index

we can ask if its checked by its checked property
No need to involve another library of functions for this since its all accessable within COM

example
Code:
com_invoke(pwb,"document.all[inputID].type")
now we know if its a check box or a button or what

I think what your looking to do is decide what actions are apporpriate

if thats the case this is the path i would go COM_ConnectObject did the user click something type something etc
set the page being examined to editable with
designMode then if a user edits something we know they need iweb_setdomobj etc
_________________
Basic Webpage Controls with JavaScript / COM - Tutorial by Jethrow
Back to top
View user's profile Send private message
sinkfaze



Joined: 18 Mar 2008
Posts: 2424
PostPosted: Thu Nov 19, 2009 12:26 am    Post subject: Reply with quote
tank wrote:
Code:
com_invoke(pwb,"document.all[inputID].type")


So there's the trick you COM wizards hide amongst yourselves. Surprised

tank wrote:
I think what your looking to do is decide what actions are apporpriate

if thats the case this is the path i would go COM_ConnectObject did the user click something type something etc


Absolutely correct, I want to anticipate to present the appropriate function call. Since I wasn't aware of that call I was working to hell and back to replicate it via ACC. The COM_Invoke "type" call I can probably work my way through, but the COM_ConnectObject method might require a little explaining.

tank wrote:
set the page being examined to editable with
designMode then if a user edits something we know they need iweb_setdomobj etc


Idea Wow you're way ahead of me, I was just looking to create the call and let them type whatever they want into the field manually after the fact (whether in a recorder or script).
_________________
Try Quick Search for Autohotkey or see the tutorial for newbies.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Scripts & Functions All times are GMT
Goto page 1, 2, 3 ... 9, 10, 11  Next
Page 1 of 11

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group