This post is just to explain the principles I have used : I'll write an "user manual" (a big job) only if a sufficient number of people ask for it.
Background
Previous Attempts - iMacros
I have before AutoHotkey tried "iMacros for Firefox" but the problem is that it is not a complete system : they give some instructions to access the content of the screen, wait for it or fill fields but to do loop, if then else, or string functions you need to go back to the javascript (run by the built-in Firefox machine) that called their instructions : so many lines of code in js.
"iMacros for Firefox" is the free-ware low-end of a line of commercial products. As they say "We hope you'll enjoy iMacros so much that you'll consider upgrading to the business editions ..." There is no much hope of big improvement : they have to earn their living with the paid for products.
For us, the interest of "iMacros" is their documentation and their demos that may give ideas. Also the URL against which the demos are run are simple and with direct access : this is easier to post than the same problem in the 5th screen of an application with userid and password !
Compared with AutoHotkey, the interface with FireFox is far better because it is a FireFox extension but the supporting language is far worst !
Problems with Firefox & AutoHotkey
I have soon discovered that AutoHotkey cannot read the content of a Firefox screen except the title line : AutoHotkey intercept the Windows interface but FireFox, which runs on multiple OS has its own (common for the various OS) interface to screen. So many of AutoHotkey screen functions do not work.
The keyboard interface, at least the "Send, keys" command works, the mouse interface also.
The Solution
This seems very limited and not usable but there is a solution : The FireFox short-cuts.
AutoHotkey script can type them to FireFox with "Send, keys" e.g. "Send, {CTRLDOWN}l{CTRLUP}" selects the Location Bar (e.g. the URL of the page shown by FireFox) and with "Send, {CTRLDOWN}c{CTRLUP}" you have access to it in the clipboard common to AutoHotkey and FireFox.
With the sequence ctrl+u (show the source screen), ctrl+a (select all), ctrl+c (copy to clipboard), AutoHotkey command (it works !) to close source screen : you have the htlm source of the page in the clipboard.
With ctrl+f (search in text) + typing the string_to_search in the status bar that open with the cursor in it + {ESC} the string is selected and you can with "enter" activate the link under it or with "tab" go to the next input area.
The list of standard short-cuts is at :
<!-- m -->http://support.mozil...board shortcuts<!-- m -->
The Firefox Keyconfig extenshion - More Functions
You need also other short-cuts, they can be coded with the Keyconfig extension see :
<!-- m -->http://forums.mozill...pic.php?t=72994<!-- m -->
<!-- m -->http://kb.mozillazin...onfig_extension<!-- m -->The idea is that, if you need a function that does not work or exist in AutoHotkey or to extract some information from FireFox, you use a short-cuts that does it for you, you call it with "Send, keys" command.
In most of the cases the input to and output from the short-cuts are the clipboard.
To write a short-cut, you need to know very well the coding of FireFox to put together some already existing functions of it : "Dorando" the author of Keyconfig is a very helpful expert, personally I am just able to ask, in the forum, for a short-cut that does what I wish to have !
Development of the Keyconfig shortcuts for AutoHottkey
To ease the understanding of my reasons, I'll do an historical presentation.
1. The initial Shortcuts
first batch of short-cuts :
My experiences have shown that :
- clicking at X and Y coordinates was not reliable because too often the application move slightly the components of the screen to make it better looking. If you have several PC with different sizes of screen, you may also have problems.
- navigating by tabs from a corner of the screen was not reliable because too often the application added or subtracted links or areas to click.
So I used the context with the "search in text" short-cut to move by tab from an nearby explanatory text. This text should be unique on the screen to ease the coding of the script. E.g. for login screen often also used for registration, you find "user" twice (in both parts), password and password forgotten (or lost or equivalent) in the login area, password and repeat password in registration area : I searched for "password forgotten" then tabed back to user input area, entered it, then tab to normal password area.
I obtained 3 short-cuts :
1-1. When executed, puts "NotBusy" in the clipboard if the page has finished to load else "Busy". I have built around it an AutoHotkey function that loops with an 0.5 seconds "Sleep" and return when loading has finished or exit if time out.
1-2. Checks if ctrl+f has found the string and puts "found" in the clipboard else "not_found". I have built around it an AutoHotkey function that send ctrl+f, then the string_to_search, then the 2nd short-cut and exit if "not_found", then {ESC} and return leaving the string selected and you can with "enter" activate the link under it or with "tab" go to the next input area.
1-3. Puts the source in the clipboard. It is equivalent (but easier to synchronize with AutoHotkey) to the sequence ctrl+u, ctrl+a, ctrl+c, AutoHotkey command to close source screen. I have built around it an AutoHotkey function that return the source. Then I extract the information I need with a sub-pattern of RegExMatch. Or do an other string search function to verify if I am on the right screen.
With this you can write and run some scripts without knowledge of html language : you need just your eyes to see the text to select.
Limitations
The limitations are :
- Not all application support navigation by tab.
- "false texts" : they are not in character mode in the source of the page, they are drawn on an image. In this case they are not found with ctrl+f.
- national characters when you are abroad. E.g. in a French application the button to disconnect is often called "Déconnexion" (the French word for the action of disconnecting where the 2nd character is "e with acute accent"). This word was found by ctrl+f without problem on a French PC with French Windows and French FireFox but was not found on a English PC with English Windows and English FireFox : I don't know the reason, nevertheless "e with acute accent" was shown on the screen...
2. The Second Shortcuts
Second batch of short-cuts :
2.1.
'Is this possible to define a new function "find in source and click" that has some similarities to ctrl+f but works on the source'?
The answer of the author of Keyconfig was 'it does exceed my abilities in multiple ways'. I said : 'Something similar is done by the "tag" command of iMacros for Firefox' The author : 'what iMacros does seems to translate the input to "XPath"' FireFox function and gave me a shortcut .
The input parameter is put in the clipboard before short-cut : he said:
You can test some on the iMacros demo page
<!-- m -->http://www.iopus.com...6/pos/index.htm<!-- m -->
by copying one of the following to your clipboard:
nigelle;2;//a[contains(text(),"Click to open this page")]
nigelle;1;//a[@href="page1.htm"]
nigelle;1;//input[@value="your E-mail here"] The format is
identifier;occurrence;XPath while for XPath
//tag[@attribute="value"] or
//tag[contains(text(),"text content")] should cover many cases.'
The clipboard after short-cut contains "Found" (success) or "NotFound" (failure to find) or the text of the internal error in FireFox (generally bad format of the input)...
With this you can activate a link or put the cursor in an input area : this is very powerful, it can do the same as ctrl+f and far more.
For documentation on "XPath" see :
<!-- m -->https://developer.mozilla.org/En/XPath<!-- m -->
<!-- m -->http://www.w3.org/TR/xpath<!-- m -->
<!-- m -->http://www.w3schools.com/xpath/<!-- m -->
<!-- m -->http://www.examplede... ... html?l=rel<!-- m -->With this you can write and run many scripts but with knowledge of html language and format of Xpath.
2.2. Maximizes the window (the AutoHotkey function does not works except in the "run" command). Use this when you don't need to see the background.
2.3. Restores the window to the non-maximized state, I have not used it because often it gives a size of 1x1 inch on my 20" monitor !
3. The Third Shortcuts
Third batch of short-cuts :
3.1. Puts the source of the DOM in clipboard
With an application that modified the content of the window with Javascript, I discovered that there are 2 sources defined in FireFox;
a) source1 the classical source shown by Ctrl+u on a screen with the title : "Source of : <!-- m -->http://url_with_parameters<!-- m --> - Mozilla Firefox". Source1 is not modified by Javascript and is given by 1.3.
source2 shown by selecting an area on a screen, right click and choosing "source code of the selection" : the title of the screen is :
"DOM source of the selection - Mozilla Firefox". Normally it shows from the <div> before the selection to the </div> after it. It works also for the full page after ctrl+a .
The present short-cut puts the source of the DOM (of the whole window) in clipboard, I have built an AutoHotkey function that return it.
3.2. Set the window to 3 quarters of the size of the screen. Use this when you need to see the background.
4. The Final Shortcuts
fourth batch of short-cuts
4.1. URL in clipboard
It is equivalent to the sequence ctrl+l, ctrl+c but without the problem of synchronizing the 2 short-cuts.
Known limitations
- The Xpath short-cut does not start a Flash application. As flash simulation is low in my priority list, I have not tested the methods that Imacros uses in this case : clicking at X and Y coordinates when the image of the Flash application does not move or ImageSearch function and click on the found coordinates. I assume they work.
- ControlSend command does not works properly, only Send does the job : you are forced to keep the focus on the FireFox screen and you cannot run another thing during your script, this is annoying when its duration is 10 minutes or more !
I have tried to change, in one of my script that worked fine and in all my functions, "send, keys" to "ControlSend,, keys, %Wtit%" where %Wtit% is the wintittle field corresponding to the application.
a) First I run it keeping the focus on the FireFox screen to see what occurred, the short-cuts did their job but 'Send, "my_userid" ' (after an Xpath call to put the cursor in the userid input area) entered nothing.
I changed this to
clipboard := "my_userid" ControlSend,, {CTRLDOWN}v{CTRLUP}, %Wtit%and it worked.
Then I tried to run this corrected script, switching to an other window then going back to see what occurred : the script was blocked : the Xpath has put the cursor in the userid area, nothing was entered in this field and the next Xpath to the password has time outed (without putting any result in the clipboard) after more than 40 seconds when normally it takes from 0.3 to 1s to execute with "send, keys" ...
I am an user and not a developer, I cannot give any reason or cure to this behaviour. FireFox ? Its javascript or XUL machine ? AutoHotkey ?
With the fore mentioned limitations, I run happily a few dozens of scripts.
As I am lazy, and does not wish to write more than 3 times the same group of lines, I have many more functions than described here...
So I ask again the question : Do you need that I write an "user manual" ?
This is a big job, I'll not start it without strong request.
AutoHotkey Scripts to Automate Firefox
This post is just to explain the principles I have used : I'll write an "user manual" (a big job) only if a sufficient number of people ask for it.
Background
Previous Attempts - iMacros
I have before AutoHotkey tried "iMacros for Firefox" but the problem is that it is not a complete system : they give some instructions to access the content of the screen, wait for it or fill fields but to do loop, if then else, or string functions you need to go back to the javascript (run by the built-in Firefox machine) that called their instructions : so many lines of code in js.
"iMacros for Firefox" is the free-ware low-end of a line of commercial products. As they say "We hope you'll enjoy iMacros so much that you'll consider upgrading to the business editions ..." There is no much hope of big improvement : they have to earn their living with the paid for products.
For us, the interest of "iMacros" is their documentation and their demos that may give ideas. Also the URL against which the demos are run are simple and with direct access : this is easier to post than the same problem in the 5th screen of an application with userid and password !
Compared with AutoHotkey, the interface with FireFox is far better because it is a FireFox extension but the supporting language is far worst !
Problems with Firefox & AutoHotkey
I have soon discovered that AutoHotkey cannot read the content of a Firefox screen except the title line : AutoHotkey intercept the Windows interface but FireFox, which runs on multiple OS has its own (common for the various OS) interface to screen. So many of AutoHotkey screen functions do not work.
The keyboard interface, at least the "Send, keys" command works, the mouse interface also.
The Solution
This seems very limited and not usable but there is a solution : The FireFox short-cuts.
AutoHotkey script can type them to FireFox with "Send, keys" e.g. "Send, {CTRLDOWN}l{CTRLUP}" selects the Location Bar (e.g. the URL of the page shown by FireFox) and with "Send, {CTRLDOWN}c{CTRLUP}" you have access to it in the clipboard common to AutoHotkey and FireFox.
With the sequence ctrl+u (show the source screen), ctrl+a (select all), ctrl+c (copy to clipboard), AutoHotkey command (it works !) to close source screen : you have the htlm source of the page in the clipboard.
With ctrl+f (search in text) + typing the string_to_search in the status bar that open with the cursor in it + {ESC} the string is selected and you can with "enter" activate the link under it or with "tab" go to the next input area.
The list of standard short-cuts is at :
<!-- m -->http://support.mozil...board shortcuts<!-- m -->
The Firefox Keyconfig extenshion - More Functions
You need also other short-cuts, they can be coded with the Keyconfig extension see :
<!-- m -->http://forums.mozill...pic.php?t=72994<!-- m -->
<!-- m -->http://kb.mozillazin...onfig_extension<!-- m -->The idea is that, if you need a function that does not work or exist in AutoHotkey or to extract some information from FireFox, you use a short-cuts that does it for you, you call it with "Send, keys" command.
In most of the cases the input to and output from the short-cuts are the clipboard.
To write a short-cut, you need to know very well the coding of FireFox to put together some already existing functions of it : "Dorando" the author of Keyconfig is a very helpful expert, personally I am just able to ask, in the forum, for a short-cut that does what I wish to have !
Development of the Keyconfig shortcuts for AutoHottkey
To ease the understanding of my reasons, I'll do an historical presentation.
1. The initial Shortcuts
first batch of short-cuts :
My experiences have shown that :
- clicking at X and Y coordinates was not reliable because too often the application move slightly the components of the screen to make it better looking. If you have several PC with different sizes of screen, you may also have problems.
- navigating by tabs from a corner of the screen was not reliable because too often the application added or subtracted links or areas to click.
So I used the context with the "search in text" short-cut to move by tab from an nearby explanatory text. This text should be unique on the screen to ease the coding of the script. E.g. for login screen often also used for registration, you find "user" twice (in both parts), password and password forgotten (or lost or equivalent) in the login area, password and repeat password in registration area : I searched for "password forgotten" then tabed back to user input area, entered it, then tab to normal password area.
I obtained 3 short-cuts :
1-1. When executed, puts "NotBusy" in the clipboard if the page has finished to load else "Busy". I have built around it an AutoHotkey function that loops with an 0.5 seconds "Sleep" and return when loading has finished or exit if time out.
1-2. Checks if ctrl+f has found the string and puts "found" in the clipboard else "not_found". I have built around it an AutoHotkey function that send ctrl+f, then the string_to_search, then the 2nd short-cut and exit if "not_found", then {ESC} and return leaving the string selected and you can with "enter" activate the link under it or with "tab" go to the next input area.
1-3. Puts the source in the clipboard. It is equivalent (but easier to synchronize with AutoHotkey) to the sequence ctrl+u, ctrl+a, ctrl+c, AutoHotkey command to close source screen. I have built around it an AutoHotkey function that return the source. Then I extract the information I need with a sub-pattern of RegExMatch. Or do an other string search function to verify if I am on the right screen.
With this you can write and run some scripts without knowledge of html language : you need just your eyes to see the text to select.
Limitations
The limitations are :
- Not all application support navigation by tab.
- "false texts" : they are not in character mode in the source of the page, they are drawn on an image. In this case they are not found with ctrl+f.
- national characters when you are abroad. E.g. in a French application the button to disconnect is often called "Déconnexion" (the French word for the action of disconnecting where the 2nd character is "e with acute accent"). This word was found by ctrl+f without problem on a French PC with French Windows and French FireFox but was not found on a English PC with English Windows and English FireFox : I don't know the reason, nevertheless "e with acute accent" was shown on the screen...
2. The Second Shortcuts
Second batch of short-cuts :
2.1.
'Is this possible to define a new function "find in source and click" that has some similarities to ctrl+f but works on the source'?
The answer of the author of Keyconfig was 'it does exceed my abilities in multiple ways'. I said : 'Something similar is done by the "tag" command of iMacros for Firefox' The author : 'what iMacros does seems to translate the input to "XPath"' FireFox function and gave me a shortcut .
The input parameter is put in the clipboard before short-cut : he said:
You can test some on the iMacros demo page
<!-- m -->http://www.iopus.com...6/pos/index.htm<!-- m -->
by copying one of the following to your clipboard:
nigelle;2;//a[contains(text(),"Click to open this page")]
nigelle;1;//a[@href="page1.htm"]
nigelle;1;//input[@value="your E-mail here"] The format is
identifier;occurrence;XPath while for XPath
//tag[@attribute="value"] or
//tag[contains(text(),"text content")] should cover many cases.'
The clipboard after short-cut contains "Found" (success) or "NotFound" (failure to find) or the text of the internal error in FireFox (generally bad format of the input)...
With this you can activate a link or put the cursor in an input area : this is very powerful, it can do the same as ctrl+f and far more.
For documentation on "XPath" see :
<!-- m -->https://developer.mozilla.org/En/XPath<!-- m -->
<!-- m -->http://www.w3.org/TR/xpath<!-- m -->
<!-- m -->http://www.w3schools.com/xpath/<!-- m -->
<!-- m -->http://www.examplede... ... html?l=rel<!-- m -->With this you can write and run many scripts but with knowledge of html language and format of Xpath.
2.2. Maximizes the window (the AutoHotkey function does not works except in the "run" command). Use this when you don't need to see the background.
2.3. Restores the window to the non-maximized state, I have not used it because often it gives a size of 1x1 inch on my 20" monitor !
3. The Third Shortcuts
Third batch of short-cuts :
3.1. Puts the source of the DOM in clipboard
With an application that modified the content of the window with Javascript, I discovered that there are 2 sources defined in FireFox;
a) source1 the classical source shown by Ctrl+u on a screen with the title : "Source of : <!-- m -->http://url_with_parameters<!-- m --> - Mozilla Firefox". Source1 is not modified by Javascript and is given by 1.3.
source2 shown by selecting an area on a screen, right click and choosing "source code of the selection" : the title of the screen is :
"DOM source of the selection - Mozilla Firefox". Normally it shows from the <div> before the selection to the </div> after it. It works also for the full page after ctrl+a .
The present short-cut puts the source of the DOM (of the whole window) in clipboard, I have built an AutoHotkey function that return it.
3.2. Set the window to 3 quarters of the size of the screen. Use this when you need to see the background.
4. The Final Shortcuts
fourth batch of short-cuts
4.1. URL in clipboard
It is equivalent to the sequence ctrl+l, ctrl+c but without the problem of synchronizing the 2 short-cuts.
Known limitations
- The Xpath short-cut does not start a Flash application. As flash simulation is low in my priority list, I have not tested the methods that Imacros uses in this case : clicking at X and Y coordinates when the image of the Flash application does not move or ImageSearch function and click on the found coordinates. I assume they work.
- ControlSend command does not works properly, only Send does the job : you are forced to keep the focus on the FireFox screen and you cannot run another thing during your script, this is annoying when its duration is 10 minutes or more !
I have tried to change, in one of my script that worked fine and in all my functions, "send, keys" to "ControlSend,, keys, %Wtit%" where %Wtit% is the wintittle field corresponding to the application.
a) First I run it keeping the focus on the FireFox screen to see what occurred, the short-cuts did their job but 'Send, "my_userid" ' (after an Xpath call to put the cursor in the userid input area) entered nothing.
I changed this to
clipboard := "my_userid" ControlSend,, {CTRLDOWN}v{CTRLUP}, %Wtit%and it worked.
Then I tried to run this corrected script, switching to an other window then going back to see what occurred : the script was blocked : the Xpath has put the cursor in the userid area, nothing was entered in this field and the next Xpath to the password has time outed (without putting any result in the clipboard) after more than 40 seconds when normally it takes from 0.3 to 1s to execute with "send, keys" ...
I am an user and not a developer, I cannot give any reason or cure to this behaviour. FireFox ? Its javascript or XUL machine ? AutoHotkey ?
With the fore mentioned limitations, I run happily a few dozens of scripts.
As I am lazy, and does not wish to write more than 3 times the same group of lines, I have many more functions than described here...
So I ask again the question : Do you need that I write an "user manual" ?
This is a big job, I'll not start it without strong request.
P. S.
The format of the original post has been improved by Markup done by Michael@oz .
There is now a Wiki, see :
<!-- m -->http://www.autohotke... ... te_FireFox<!-- m -->
structure given by hugov .