AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Upgrade suggestions to help control difficult windows

 
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Wish List
View previous topic :: View next topic  
Author Message
Jerry



Joined: 24 Jun 2004
Posts: 39

PostPosted: Mon May 28, 2007 7:07 pm    Post subject: Upgrade suggestions to help control difficult windows Reply with quote

I've been controlling a program which is a remote program controlled through Citrix (therefore there are no controls) and have come up with a couple of suggestions to assist in controlling the window. P.S. I have tried to get the sendmessage and controlclick commands to work for this window but have not succeeded. (I have used those methods successfully on other windows)

1) Upgrade to MouseGetPos.
If x and y co-ordinates are given then the command gives back the window/control under that
point as opposed to under the mouse. This would make it so that I don't have to move the mouse to
find out if a point I am trying to "click" on is visible on the screen.

2) click,mouseclick,etc.
Upgraded to only click on a window that is specified.
So if a tooltip/balloontip/"window that is top most", etc. happens to be displayed over the
co-ordinates to be clicked on. The command could either
a) wait for the tooltip/balloon/"window that is top most" gets out of the way/ vanishes and then issue the command
b) (my favourite) poke a "hole" through the window (like using the command "WinSet Region") and then
issue the mouseclick and finally undo the "hole"
(The "hole" idea could be extended to keep creating "holes"
until it hits the desktop and then an error could be returned to the script indicating the window was not found.

Also upgrade the click command to check if the left/right/middle mouse buttons are being currently pressed.
If they are, wait until the user releases the button to issue the command.
This is needed as if I have a command to wait until a window is active and then issue the mouseclick.
The script will issue the mouseclick command before I release the mouse and the window will be moved (this is a citrix window). This happens only if I activate the window by clicking on the title line.
I know I could use the "block" mouse input command for the script, but I really want the script to wait until I have finished with the other window before continuing.

3) A new command to get the color depth of the desktop.
I think this is needed because I get the color of points to figure out where to
click the mouse and the color changes depending on the users desktop. (it can also be changed if
I use remote desktop to control my work PC and I pick a different color depth to reduce
network traffic.

4) Upgrades to PixelSearch.
a) a way to scan the row first and then the column.
Currently for points x,y. Y is scanned first then x is incremented/decremented until the end of the area.
This makes it so that the command can not find certain points.
EG. I have on the screen the following icons A and B positioned as
A B A
...A
The bottom A can not be found by the PixelSearch command regardless if the
points (supplied to the command) are reversed. So as to search in the opposite direction.


b) Allow the command to be able to scan until it can NOT find the color indicated. The return co-ordinates would be where the color stops.
With this upgrade, a line/edge of a control could be followed to where it ends or the window ends.
For example, inside a window there is a table and if the table's header changes color if it happens to be
highlighted the command could find the table by scanning from a point where the color does not change (normal background) until it
finds the edge of the table. The enhanced command could also be used to discover where the table header is and control what happens when a user
clicks on it (since you would now discover the edges of the control)

c) This is a far fetched idea but I thought I would throw it out there to see what people think of it.
Upgrade to pixelsearch to be able to have multiple points identified to be consider a "find"
For example, I could ask the command to indicate a find if the first color is red and
the pixel 1 above and 2 to the right is black and ....
I came up with this idea because I had to find an image of an image which is only unique depending on the character that follows the image.
The imageseach command (at least how I'm using it) just can't find images that are
in the citrix window. (outside the citrix window it works great)
I also thought this could be a kind of rudimentary type of OCR since I could set up the condition to find a letter(s) at a point on the window.

(again remember this is for windows that don't have controls like a Citrix window.)
Back to top
View user's profile Send private message
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10467

PostPosted: Wed May 30, 2007 11:29 pm    Post subject: Re: Upgrade suggestions to help control difficult windows Reply with quote

Jerry wrote:
1) Upgrade to MouseGetPos.
If x and y co-ordinates are given then the command gives back the window/control under that
point as opposed to under the mouse. This would make it so that I don't have to move the mouse to
find out if a point I am trying to "click" on is visible on the screen.
I don't think MouseGetPos can be easily changed in that way (without breaking existing scripts), so maybe there's some other solution. Since it might be a long time before such a feature becomes built-in, perhaps it would be good to ask for ideas in Ask for Help in case someone else knows how to do it.

Quote:
2) click,mouseclick,etc.
Upgraded to only click on a window that is specified.
So if a tooltip/balloontip/"window that is top most", etc. happens to be displayed over the
co-ordinates to be clicked on. The command could either
a) wait for the tooltip/balloon/"window that is top most" gets out of the way/ vanishes and then issue the command
b) (my favourite) poke a "hole" through the window (like using the command "WinSet Region") and then
issue the mouseclick and finally undo the "hole"
(The "hole" idea could be extended to keep creating "holes"
until it hits the desktop and then an error could be returned to the script indicating the window was not found.
I can see that this would be useful, but given the daunting size of the to-do list, this is unlikely to get done sooner than a year from now unless someone else codes it.

Quote:
Also upgrade the click command to check if the left/right/middle mouse buttons are being currently pressed.
If they are, wait until the user releases the button to issue the command.
This is needed as if I have a command to wait until a window is active and then issue the mouseclick.
The script will issue the mouseclick command before I release the mouse and the window will be moved (this is a citrix window). This happens only if I activate the window by clicking on the title line.
I know I could use the "block" mouse input command for the script, but I really want the script to wait until I have finished with the other window before continuing.
The benefit/cost doesn't seem enough to justify it as a built-in feature, so you might be on your own (or ask for help).

Quote:
3) A new command to get the color depth of the desktop.
I think this is needed because I get the color of points to figure out where to
click the mouse and the color changes depending on the users desktop. (it can also be changed if
I use remote desktop to control my work PC and I pick a different color depth to reduce
network traffic.
This may someday become a part of the standard library, which will hopefully begin development soon.

Quote:
4) Upgrades to PixelSearch.
Those all seem like good ideas, but benefit/cost doesn't seem high enough to put them in front of dozens of other things on the to-do list. Therefore, it's unlikely to get done (at least by me) anytime in the next year or two. Based on this, I usually don't put such things on the to-do list because it already contains over 1000 items, most of which will realistically never get done.

Thanks for the ideas and for giving so many details.
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Wish List All times are GMT
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group