FindClick() - ImageSearch, Clicking, & More [Newest Version]

[milkygirl90]

May I just check the following modifier means I'm searching at the top middle of screen with a search area of 1200x1500px wide? I combined it with o30 but it doesn't click on the correct area. Is there any way to make the visible area show up on screen so I can easily adjust the values?

Code: Select all

a1000,0,1200,1500

[boiler]

Not sure what you mean by top middle of the screen, but you have it starting at 1000 pixels from the left of the screen at the very top (assuming your monitor starts at 0,0 which is likely) and is 1200 wide and 1500 high.

This script will represent the area on the screen visually:

Code: Select all

area := "a1000,0,1200,1500"
Gui, -Caption -Border +HwndID
Gui, Color, Red,
RegExMatch(area, "(\d+),(\d+),(\d+),(\d+)", a)
Gui, Show, x%a1% y%a2% w%a3% h%a4%
WinSet, Transparent, 150, ahk_id %ID%
return

Esc::ExitApp

The above script will only work when 4 numbers are provided, not in the other cases FindClick allows (blank numbers or widths or heights preceded by +/-), although it could be modified to do so with some effort.

[milkygirl90]

...or a more compact version:

Code: Select all

F11::FindClick((Toggle := !Toggle) ? "C:\MyImage1.png" : "C:\MyImage2.png", "a,,,+500 o12")

Regarding this toggle, just wondering if the modifiers at the end apply to only the latter image instead of BOTH images? I tried adding modifiers to the front image but it is not allowed:

[boiler]

Each parameter is its own expression. You can't have one ternary expression span both parameters (the file name parameter and the options parameter). If you want both to be based on the toggle value, then you need a separate ternary expression for each:

Code: Select all

Toggle := !Toggle
FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-" . (Toggle ? "29-56.png" : "30-36.png"), Toggle ? "o30" : "o60")

[milkygirl90]

Each parameter is its own expression. You can't have one ternary expression span both parameters (the file name parameter and the options parameter). If you want both to be based on the toggle value, then you need a separate ternary expression for each:

Code: Select all

Toggle := !Toggle
FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-" . (Toggle ? "29-56.png" : "30-36.png"), Toggle ? "o30" : "o60")

I see. If both files are in different directories, what is the right way to write it then?

[boiler]

If both files are in different directories, what is the right way to write it then?

It would be like this:

Code: Select all

Toggle := !Toggle
FindClick(Toggle ? "E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-29-56.png" : "E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-30-36.png", Toggle ? "o30" : "o60")

But the advantages of more compact code and readability from using ternary expressions are getting lost with it getting so long, so you might as well just go with a traditional if/else with separate statements:

Code: Select all

if (Toggle := !Toggle)
	FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-29-56.png", "o30")
else
	FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-30-36.png", "o60")

Or you could assign the file paths to variables and use the variable names in the function call rather than the literal strings:

Code: Select all

; at top of script:
File1 := "E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-29-56.png"
File2 := "E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-30-36.png"

; ...

Toggle := !Toggle
FindClick(Toggle ? File1 : File2, Toggle ? "o30" : "o60")

[milkygirl90]

If both files are in different directories, what is the right way to write it then?

It would be like this:

Code: Select all

Toggle := !Toggle
FindClick(Toggle ? "E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-29-56.png" : "E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-30-36.png", Toggle ? "o30" : "o60")

But the advantages of more compact code and readability from using ternary expressions are getting lost with it getting so long, so you might as well just go with a traditional if/else with separate statements:

Code: Select all

if (Toggle := !Toggle)
	FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-29-56.png", "o30")
else
	FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-30-36.png", "o60")

Or you could assign the file paths to variables and use the variable names in the function call rather than the literal strings:

Code: Select all

; at top of script:
File1 := "E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-29-56.png"
File2 := "E:\Dropbox\Screenshots\AutoHotkey_2021-03-05_17-30-36.png"

; ...

Toggle := !Toggle
FindClick(Toggle ? File1 : File2, Toggle ? "o30" : "o60")

I see. Thanks for the explanation. If the only variable is the drive letter, and I have hundreds of FindClick code nested inside #IfWinActive of various applications, is there a way to globally define the drive letter as being C or D or E? I know I should have parked my screenshots folder with the script, but it's now too late I guess

[boiler]

If the only variable is the drive letter, and I have hundreds of FindClick code nested inside #IfWinActive of various applications, is there a way to globally define the drive letter as being C or D or E? I know I should have parked my screenshots folder with the script, but it's now too late I guess

It’s not too late. There are various ways to modify the script to replace the hard-coded drive letter with the proper syntax to implement a variable. You could use your editor to do a global find/replace to modify the appropriate lines of code as necessary. Depending on the details, you probably need to use an editor with RegEx find/replace capability (such as Notepad++), or you could write a script to modify the text of the script.

It’s really out of scope for this thread, so if you want some more direction on how to do that, I suggest you start a new “Ask For Help” thread with some details on the way your code looks now and how you want to implement a variable for the drive letter. Then I or someone else could provide some further guidance.

[milkygirl90]

I have posted here.. Thank you!
https://www.autohotkey.com/boards/viewtopic.php?f=76&t=87954

[milkygirl90]

by the way, will I also be able to use Findclick as a conditional toggle? i.e.

Code: Select all

 If ImageFoundUsingFindClick
Do A
Else
Do B

[boiler]

Yes, if ImageFoundUsingFindClick is a variable to which you have assigned the result of a call to FindClick(). Or you could put the FindClick() call itself there:

Code: Select all

if FindClick(...)
	Do A
else
	Do B

[milkygirl90]

I tried exactly that and it clicked on the image instead of performing A or B (I just want to see if the correct image is found, not click on it). No msgbox was shown:

Code: Select all

f12::
IF FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-12_22-03-37.png", "o60")
Msgbox, tested
else 
return

[boiler]

If you don’t want FindClick() to click the image, you have to tell it not to by using the n (number of clicks) option set for 0.

I’m surprised the MsgBox didn’t display if it actually found the image, which it would have if it clicked on it. What does the MsgBox show using the script below? (both when the image appears and when it doesn’t)

Code: Select all

f12::
MsgBox, % FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-12_22-03-37.png", "n0 o60")
return

[milkygirl90]

I tried again below, and sometimes it displays my coordinates, sometimes it doesn't.

Code: Select all

#If WinActive("Gmail")
!IF FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-15_07-57-03.png", "n0 o60") ;Gmail Email View. If I don't see my Gmail avatar, then the hotkeys below will be valid 
{
Del::#
z::k
x::msgbox, test
y::MsgBox, % FindClick("E:\Dropbox\Screenshots\AutoHotkey_2021-03-15_07-57-03.png", "n0 o60")
}
return

[boiler]

This should be taken to an “Ask For Help” thread because it’s no longer really about how to use FindClick(), it’s about correctly writing/structuring AHK scripts. There are a few non-FindClick related things wrong with this code:

The only condition on the hotkeys is whether Gmail is the active window because that correctly uses an #If directive.

A regular If statement doesn’t put conditions on hotkeys. Also, putting a “!” meaning “not” doesn’t go before the “If”. It goes after it, in the expression: If !FindClick(.... But again, a regular If doesn’t belong there.

I suggest trying to address the things I mentioned and if you still aren’t getting it to work, start a new thread about it.

[milkygirl90]

A regular If statement doesn’t put conditions on hotkeys.

But again, a regular If doesn’t belong there.

I suggest trying to address the things I mentioned and if you still aren’t getting it to work, start a new thread about it.

Hmm.. I don't really understand the 2 statements above and have started a separate thread.

[milkygirl90]

May I ask what are the instances that FindClick() doesn't work reliably on images? I have had great success 99% of the time picking the right snips for detection, but this image below is the only one that has issues:



An example of my snip:


After configuring, it usually works within the session itself. However if I exit and re-enter the session by visiting the URL again, FindClick doesn't reliably find the image. What's the reason why? I have tried snipping different parts of the icon and also allowed for variations in shade (from o12 to o60). What else can I do? it doesn't look like the icon changed shape at all so I don't understand why it fails to work only on this page..

https://app.mural.co/

Lastly, I also noticed sometimes that FindClick will identify the right coordinates (because the mouse cursor moves to the right spot) but click is not activated. Again this only happens on the site above. Any idea why? Sample code below:

Code: Select all

Home::
FindClick("E:\Dropbox\Screenshots\chrome_2021-02-26_09-38-21.png", "o60") ; Click Align
Sleep 50 
FindClick("E:\Dropbox\Screenshots\chrome_2021-02-26_09-39-06.png", "o60") ;Align Middle
Sleep 100 
FindClick("E:\Dropbox\Screenshots\chrome_2021-02-26_09-38-21.png", "o60") ;Align 
Sleep 100 
FindClick("E:\Dropbox\Screenshots\chrome_2021-02-26_09-46-40.png", "o60") ;Align Top
return

[boiler]

The reason it only finds it within the current session is almost certainly because it draws it differently from session to session. Although it may look the same, if you zoom in on it and compare the side edges, you will probably find differences in the darkness of the edge pixels. This is a result of antialiasing, which is more often seen in text (to make edges appear smoother). I can see the edges are not a sharp distinction from one color to the other, so it’s pretty clear that’s the issue.

You can handle this issue by coloring over the edge pixels in your reference image with a certain color and use the option for defining a particular color to be skipped in looking for a match. In ImageSearch, it’s the *Trans option. See the FindClick documentation for how to implement it with its options.

I’m not certain what is causing your second issue. Those kind of things can often be handled using a loop and checking for some expected reaction to your click and clicking again if it didn’t happen.

[milkygirl90]

I checked the docs but didn't find any modifier like *Trans mentioned above

Code: Select all

o – ImageSearch Options
What to give comma-delimited string of imagesearch options
Description The optional parameters for ImageSearch, as shown in the AutoHotkey documentation. Use a comma to separate options instead of a space. You may omit the asterisk (*). For example: oTransBlack,20 makes black transparent and allows 20 shades of variation.
a – Search Area Modifications What to give x[,y,w,h] or mn
Description Region within which to search for the image. Indicate either of the following: 1) A comma-delimited list in the format x,y,w,h. List items may be blank or absent to leave that value unchanged. Coordinates are relative to the window if r is used. For w and h, use + and - to indicate a change to the normal endpoint (depending on a, either the edge of the screen or active window). For instance, a,,-300 will search the entire screen (or window if r is used) except for the last 300 pixels of the right side – the width of the search area has been reduced by 300. 2) The letter m and then a number to search a square region centered at the cursor position and with a width and height twice that number.
r – Relative to Window Coords
What to give window title criteria OR a window hwnd
Description The search area becomes the area of the given window instead of the entire screen area. This is useful when scanning for an element which only appears on a particular window – reducing the search area improves performance. Use of the a option in conjunction with r will apply these offsets to the window coordinates.
x – X Offset
What to give any integer Description This many pixels will be incremented to the click coordinates before clicking. A positive x value will click left of the image, a negative value will click to the right. This change will also show up in the string that is returned by the function and the coordinates stored in the ByRef variables.
y – Y Offset
What to give any integer Description This many pixels will be incremented to the click coordinates before clicking. A positive y value will click below the image, and a negative value will click above it. This change will also show up in the string that is returned by the function and the coordinates stored in the ByRef variables.
n – Number of clicks (No click)
What to give any integer Description The number of times to click on each image. Indicate 0 (the user default) to do no clicking whatsoever and just return the coordinates of the found image(s). In this sense, specifying just n is like saying “no click”. Without clicks, the m, Sleep, and k options are irrelevant, however, the x and y values will still be added to the output coordinates.
e – Find Every Image
What to give positive fraction between 0 and 1
Description If this option is nonblank then FindClick will find (and click) EVERY instance found. This means that after any successful ImageSearch execution the script will queue up a new call of ImageSearch which represents the screen area left unsearched by the first call. (Note that this therefore makes FindClick slower for any use in which you don’t expect to find more than one image.) The value of e signifies the overlap between search areas, in terms of the fraction of the image width and height. Overlap should normally be around 0.9 except if the image is uniform throughout, e.g. a 10x10 square image containing mostly white pixels. The t, d, and Count options all require the “find every image” behavior as part of their execution and so using any of these other options implies e, however, you may still specify a custom value for e to change the overlap for these other behaviors.
w – Wait until image is found
What to give number of milliseconds [, number of milliseconds]
Description If the image is not found, the function will wait this many milliseconds for it to appear before returning. You may add a comma and then a second number which indicates the delay in milliseconds between subsequent ImageSearches. For example 2000,50 means the function will look for the image every 50ms until either the image is found or 2000ms elapse, for a maximum of 40 ImageSearch operations. If omitted, this delay will be the smaller of either the wait time divided by 10 or 100ms.
dx – Diagnostic Mode
What to give comma delimited list of diagnostic mode options
Description Will produce an AutoHotkey GUI before the function completes showing a step-by-step history of how the function executed. The value of the string passed to the dx option may be used to send certain flags to the debugger GUI before it displays. For more info on this see the section on the FindClick debugger.
k – Keystroke(s)
What to give key to send (in the same format as the AutoHotkey Send command) Description Indicate the keys to press (if any) when each image is found. k can include multiple keypresses and even non-mouse keys, for instance, ^{Space}. If m (SendMode) is ControlClick then the format is different (see the ControlClick documentation) and a left click will be assumed if an incorrect k is given.
Stay – Do not restore mouse What to give true or false (1 or 0)
Description Normally after any clicks executed by FindClick the mouse will be immediately returned to its initial location. If Stay is true then the mouse will NOT return to its original position after FindClick finishes.
Count – Return found count
What to give true or false (1 or 0)
Description If Count is true then the function will return the number of items found instead of their
coordinates. e is assumed if absent.
d – Direction Of Search
What to give left, right, bottom, or the first letter of any of these words
Description This setting attempts to emulate the mode of PixelSearch where you can search from right to left
or top to bottom – for instance, specifying Bottom will find the image closest to the bottom of
the search area. However, to accomplish this the script needs to first find every image and then
choose the coordinate pair that is furthest in the requested direction, making it a timeconsuming
process.
m – SendMode
What to give event, play, input, default, controlclick, or the first letter of any of these words
Description The send mode (or first letter thereof) to use for clicks, i.e. Input, Play, or Event. Specify
ControlClick (or c) to use a controlclick instead of a simulated keystroke. If m is blank or the
letter d (for default), the current SendMode will be used.
Func – Function Callout
What to give the name of a function in the script
Description Each time an image is found, instead of clicking on the image or executing keystrokes the given
function will be called. The function must accept at least two parameters: the x-coordinate of the
image will be passed in the first parameter, and the y-coordinate will be passed in the second
parameter. Note that any thread settings such as CoordMode that are changed within the
function will carry over to FindClick when the function is done.
Sleep – Sleep between clicks
What to give any number of milliseconds
Description Number of milliseconds to sleep between each click if n > 1 or e and multiple images found.
t – Image Tracking
What to give percent OR number of pixels
Description This option is used to improve performance if an image will be found nearby where its last
location. The script will first search nearby the last found position and if it is not found there then
the rest of the area will be searched. Indicate a percentage to search within that percentage of
the search area in each direction. For instance, the string 20% will first search a box that begins
20% of the distance between the starting x position and the last found x position, and likewise
for the other three directions. Indicating a number will simply search a box that many pixels from
the last found area before searching the rest of the s area. Indicate 0 to search EXACTLY the last
found location first. If t is negative (including -0) then it will ONLY search in the region
requested, and will not go on to search the rest of the screen if the image is not found there.
Silent – No Dialogs
What to give true or false (1 or 0)
Description Instead of displaying an error dialog when an error prevents the function from executing
properly (for instance, if the image file is not found), it will silently return a blank string and set
the ErrorLevel to the error message. The errors concerned should not appear during normal
execution and so Silent is generally not recommended.
Center – Start at center of image
What to give true or false (1 or 0)
Description If Center is true then clicks will occur at the center of the image, i.e. its found position plus half
its width and half its height. The x and y options treat this as 0, 0 – for instance, by default an x
of 2 will click 2 pixels to the right of the image center. Indicate false to instead start at the top
left corner of the image.
Delim – Delimiter for multiple images
What to give any character or characters
Description If e is used and multiple images are found then each image’s info is separated by this character(s)
in the returned string. The format of the image info is determined by the value of f.
f – Format of output string
What to give template string that includes CharX and/or CharY and/or CharN characters
Description The given string represents a template into which image x-coordinate, y-coordinate, and instance
number will be substituted. Each time an image is found, the CharX, CharY, and CharN
characters will be replaced with these values. The final string is returned by the function. See the
examples section of the documentation for an example.
CharX – Character spaceholder for x-coordinate
What to give any character
Description Any instance of this character, if present, in the string given for f will be replaced with the
image’s x-coordinate.
CharY – Character spaceholder for y-coordinate
What to give any character
Description Any instance of this character, if present, in the string given for f will be replaced with the
image’s y-coordinate.
CharN – Character spaceholder for image instance number
What to give any character
Description Any instance of this character, if present, in the string given for f will be replaced with the order
in which the image was found among all images found. This character is only relevant when e is
being used and is therefore not normally included in f.

[boiler]

It’s the “o” option, which says it’s so you can identify the optional parameters of ImageSearch (of which *Trans is one). The details of the “o” option later in the document even uses “TransBlack” as an example, which means any pixels colored black (pure black with value 0x000000) in your reference image will be skipped. That is to say that those pixels can be any color and won’t prevent the image from being found, which is how you will get it to ignore the pixels at the edges of that image.