ImageSearch Tips & Tricks

Put simple Tips and Tricks that are not entire Tutorials in this forum
Miguel7
Posts: 186
Joined: 08 Sep 2014, 07:06

ImageSearch Tips & Tricks

Post by Miguel7 » 11 Sep 2014, 20:19

Overview

In all my work with AHK, one of the most mind-blowingly awesome, and at the same time intensely frustrating, functions I've ever used is ImageSearch. If you're trying to automate a program that doesn't use standard GUI controls, it can be a game-changer; most games fall into this category, as they usually draw the game onto a control (such C#'s System.Drawing, as Java's AWT, or HTML5's Canvas object) but it's not just limited to games. It's a big part of one of my projects at work, and at home I used it to set up hotkeys for Netflix.

But at times, it can be soooooo annoyingly hard to make it work! Don't get me wrong - AHK's developers are amazing just for creating it; I am not commenting on them in any way. They rock just for having developed AHK, my all-time favorite programming language. But ImageSearch does tend to be a colossal PITA (Pain In The And I'll let you fill in the blank) to work with. This is because in most cases, every little nano-detail has to be just so; otherwise it'll fail without much of a clue as to why.

But you can get ImageSearch working, and in this tutorial I hope to help you with some tips and tricks I've picked up from experience. And of course, if you have others you'd like to share, I'd be interested to hear them. So let's get started.

Tip #1: Play with your numbers

This one alone fixes the overwhelming majority of my ImageSearch problems. The documentation on ImageSearch lays it out like this:

Code: Select all

ImageSearch, OutputVarX, OutputVarY, X1, Y1, X2, Y2, ImageFile
A few things to note here:
A: X2 and Y2 are essentially "absolute" coordinates; in other words, they're not width and height (unlike GUI functions). Sometimes I do dumb things like this:

Code: Select all

ImageSearch, OutputVarX, OutputVarY, 400, 200, 50, 50, ImageFile.png
thinking like it's a GUI function and the other 2 coordinates are width and height. But it's actually a separate point, with another x and y, both of which should be greater than the first x and y. So to correct the example above:

Code: Select all

ImageSearch, OutputVarX, OutputVarY, 400, 200, 450, 250, ImageFile.png
or if this helps, add x to width and y to height to get those coordinates.

B: Expanding the area being searched might change the results. In the example above, I made the mistake of making the search area the same size as the image I was searching for (50 by 50 pixels). This will mess it up almost every time. Instead, the search area should be bigger than the image.

Code: Select all

ImageSearch, OutputVarX, OutputVarY, 300, 100, 600, 600, ImageFile.png
Now the first point (x1 and x2) has been moved up and to the left, while the second point (the bottom right corner of the area being searched, a.k.a. x2 and y2) has been moved down and to the right. So now the function is searching a bigger area, which may give it a better chance of the image being found. The more you mess with these numbers, the more you'll get an idea of where your search area needs to be.

Tip #2: Check for zoom-factor
By "zoom-factor" I mean any combination of the following:
- System settings (screen resolution, font sizes, magnification, transparency etc.)
- Application-specific zooming (like IE, Word etc. have zoom functionality)

I haven't had to deal with this one too much but I've read it can affect the search. Some of that is common sense (can't search an area in the 1000s on an 800x600 screen resolution etc.) and it's not really that common (unless you use a screen magnifier like I do, or share the script with people running a different OS or with different settings.

Tip #3: Find ways to check your coordinates
Sometimes when I don't get why ImageSearch seems to foul up for no good reason, I use things like MsgBox, MouseMove and other AHK functions to figure out exactly where my search area is. What this looks like will depend on your code, but basically the idea is to make sure your search area actually contains the image being searched for. Again, I can be a moron sometimes - and wondering why my ImageSearch fails when the image isn't even in that area to begin with is a typical Miguel move. :lol:

Tip #4: Make sure your image exists
Again, sometimes I can be an ID10T, and this is another goof I'm as guilty of as anyone. Make sure not only that the image exists, but also that it's in the location you're using in your code. Checking this is no biggy:

Code: Select all

^i::
IfNotExist, ImageFile.png
    MsgBox Error: Your file either doesn't exist or isn't in this location.
ImageSearch, OutputVarX, OutputVarY, 400, 200, 600, 600, ImageFile.png
Those first 2 lines say it all. :)

Tip #5: Try modifying the image
On some occasions I've found this helps. If your image is too big, or too complex for the *n property to help, maybe you need a smaller image. Think of "Where's Waldo" (a game where you have to pick out a specific guy out of a crowd): the people around Waldo are never the same way twice, so searching an exact picture of Waldo isn't going to work; but searching for his nose or the stripes on his shirt might.

Tip #6: Refer to the "Docs"
This sounds like a cop-out answer, but sometimes it helps. There's more to ImageSearch than what I've described here, so it might be helpful to check it out. Here's the link: http://ahkscript.org/docs/commands/ImageSearch.htm

Tip #7: If all else fails, "Ask for Help"
There are people on this forum whose mad skillz blow me away. If none of my suggestions help, someone on the forum can.

Tip #8: Check your variable names
This is another classic duh move. :lol: This one happened to me today: I copied a block of code from one script to another and sure enough, ImageSearch failed. Stupid little stuff like this is so easy to overlook; I crack jokes about being dumber than a bag of hammers, but really this kind of thing happens to everyone.

Tip #9: Try some different "Options"
I followed my own advice just now and referred to the docs - and I found some info I had never seen before. There are things in the "options" parameter that we can use to allow for some variation in the image being searched. You can specify things like transparent colors, width and height of the image (again, not to be confused with the X2 and Y2 parameters) and others. You might want to experiment with some of these.

Tip #10: Above all else, don't let it drive you nuts!
I think this was the one that took me the longest to learn. It's important to be patient and not let it get to you. No code is worth the aggravation. Step away from the computer for awhile, or if at work pick another project or function, and when you come back to it you'll have a fresh perspective. This has helped me in many projects, not just at work and not just in AHK. It's also the one I tend to forget the quickest - uh, duh! :lol:

Anyway, I hope these tips & tricks help you make your ImageSearches work, so you can move on with your script and get back to having fun with AHK! :)

Edit: - Today at work, one of the end users of my script pointed out a problem that was ImageSearch-related. The program I'm automating has a section that's divided into "cells" (like Excel), but some cells spanned multiple rows. My script was currently only able to work with the info one row at a time. The guy asked a very simple question that led to a discovery so cool I had to include it here. He asked, "can you have it look for the border?"

That got me thinking. I knew ImageSearch could do whole images, but I needed some way to search for a partial image, a pattern that would let my script know, "this cell spans two rows". Then it hit me: I took a screenshot of the area I wanted to search, replaced most of it with white, and did something like this:

Code: Select all

CellSpans2Rows(){
    ImageSearch, foundX, foundY, x1, y1, x2, y2, *TransWhite %A_ScriptDir%\oneRow.png
    if (foundX == ""){
        return true  ;It didn't find a match for the image with a border, so it spans more than one row
    }else{
        return false    ;it found a match, so the cell only occupies one row.
    }
}
By setting the *TransWhite option in my search, the only thing the ImageSearch needed to find a match for was the black border (in the example above, a border as seen on cells that only span one row). The awesome thing about this is, ImageSearch is fast - faster than I ever thought it would be - so I could throw in a few more to get the exact position in the control.

So the lesson here is, if you're searching for an image that changes often, you can use a transparent color and the *TransColor option to search for locial patterns or partial matches. I used a simple cell border as an example, but the same idea would also work for an animated object in a game or whatever else you're using it for. Now I'm getting beyond just finding images - now we're talking about a script that "sees" things more like we do; very cool! :D
Last edited by Miguel7 on 12 Mar 2015, 22:09, edited 2 times in total.

User avatar
Almost_there
Posts: 404
Joined: 30 Sep 2014, 10:32

Re: ImageSearch Tips & Tricks

Post by Almost_there » 23 Oct 2014, 11:35

Nice post.

Also I can put in a suggestion, typical user error. I always think on coordinates where origo is placed in bottom lect corner. Most graphical programs works this way.
The screen/window coordinate system that AHK uses is turned upside down, the Y-axis start on the top.
OMG how frustrated I was until I some day figured it out - by using MouseMove, and withnessed that search area vas far away from where it was intended to search.

Miguel7
Posts: 186
Joined: 08 Sep 2014, 07:06

Re: ImageSearch Tips & Tricks

Post by Miguel7 » 12 Mar 2015, 21:25

You know I never thought of that; in most 2D game design programs it's just like AHK, but I met a guy doing some kind of math graphing and he said it was "wrong" - I think math gurus do it more like what you're describing, but I've been used to 0, 0 being at the top left since long before I knew AHK.

Another good one for this is CoordMode, Pixel. I goofed on that one this week, telling it to click a button that was technically in another "window" (two embedded browsers in the same application at the same time - madness :lol:). So that can mess things up too.

User avatar
Blackholyman
Posts: 1293
Joined: 29 Sep 2013, 22:57
Location: Denmark
Contact:

Re: ImageSearch Tips & Tricks

Post by Blackholyman » 13 Mar 2015, 01:49

Its not AutoHotkey that's doing the coordinate system wrong, it's a MS windows thing

It's also one of the things seen in this Infographic
Image

Curtesy of: Jszapp.com
Also check out:
Courses on AutoHotkey

My Autohotkey Blog
:dance:

Miguel7
Posts: 186
Joined: 08 Sep 2014, 07:06

Re: ImageSearch Tips & Tricks

Post by Miguel7 » 13 Mar 2015, 09:32

Well that makes sense. I've only ever programmed on Windows (and also the web, but JavaScript/CSS are like that too). Anything else just seems backwards to me. :lol:

fourst4r
Posts: 2
Joined: 31 May 2017, 00:27

Re: ImageSearch Tips & Tricks

Post by fourst4r » 31 May 2017, 00:55

Sorry for the bump, but I think it's important to include the fact that coordinates are relative to the active window, (new to AHK) this one stumped me for a while.
The reason I thought this was important to include is since this thread is the second result when you search "imagesearch ahk" on Google.

User avatar
boiler
Posts: 17706
Joined: 21 Dec 2014, 02:44

Re: ImageSearch Tips & Tricks

Post by boiler » 31 May 2017, 06:43

You can change it to search relative to screen coordinates using the CoordMode command.

NoScript
Posts: 1
Joined: 21 Apr 2019, 14:25

Re: ImageSearch Tips & Tricks

Post by NoScript » 21 Apr 2019, 15:06

I've been trying to use ImageSearch for a HTML game, it works fine if I set the theme to pitch black, but when its any other color it wont detect the images even though they are transparent. I'm guessing I'll have to edit the images I use to compare if I want to use it in the default theme. I downloaded the images right from the website, and they are not scaled in anyway otherwise they wouldn't work with the darker theme.

Nvm instead of using the image from the website I just took a ss and cropped the images I wanted and it worked like magic.

abellos
Posts: 10
Joined: 08 Jul 2018, 03:00

Re: ImageSearch Tips & Tricks

Post by abellos » 24 Mar 2020, 12:51

I just resolve my problem, and want to say here so can help other that can have same.
When u use imagesearch or gdip_imagesearch pay much attention that the file u want search on screen or into an image have the same dpi, because if it is different you will never find anything.

tonospatha
Posts: 5
Joined: 31 Mar 2020, 15:01

Re: ImageSearch Tips & Tricks

Post by tonospatha » 31 Mar 2020, 15:27

I'm on x64 Windows 10

Code: Select all

URLDownloadToFile,http://www.autohotkey.com/docs/images/AutoHotkey_logo.gif,AutoHotkey_logo.gif
Gui,Add,Picture,,AutoHotkey_logo.gif
Gui,Show
Sleep 100
ImageSearch,x,y,0,0,%A_ScreenWidth%,%A_ScreenHeight%,*60 AutoHotkey_logo.gif
MsgBox % X "-" Y
MsgBox % A_AhkVersion " " (A_PtrSize=8 ? "64-bit" : "32-bit")
results in
img.png
img.png (982 Bytes) Viewed 30436 times
result.png
result.png (1.65 KiB) Viewed 30436 times
version.png
version.png (2.17 KiB) Viewed 30436 times
ImageSearch is returning ErrorLevel 2. what gives?

tonospatha
Posts: 5
Joined: 31 Mar 2020, 15:01

Re: ImageSearch Tips & Tricks

Post by tonospatha » 31 Mar 2020, 15:54

Code: Select all

URLDownloadToFile,http://www.autohotkey.com/docs/images/AutoHotkey_logo.gif,C:\Users\admin\Documents\AutoHotkey_logo.gif
Gui,Add,Picture,,C:\Users\admin\Documents\AutoHotkey_logo.gif
Gui,Show
Sleep 100
ImageSearch,x,y,0,0,%A_ScreenWidth%,%A_ScreenHeight%,*60 C:\Users\admin\Documents\AutoHotkey_logo.gif
MsgBox % X "-" Y
MsgBox % A_AhkVersion " " (A_PtrSize=8 ? "64-bit" : "32-bit")
MsgBox ErrorLevel: %ErrorLevel%
ExitApp
Results in the same results plus the new MsgBox
errorlevel.png
errorlevel.png (2.01 KiB) Viewed 30379 times

User avatar
boiler
Posts: 17706
Joined: 21 Dec 2014, 02:44

Re: ImageSearch Tips & Tricks

Post by boiler » 31 Mar 2020, 15:56

Sorry, removed my other post when I realized it was a different issue. That file no longer appears to be good. Need a path to a new image file.

tonospatha
Posts: 5
Joined: 31 Mar 2020, 15:01

Re: ImageSearch Tips & Tricks

Post by tonospatha » 31 Mar 2020, 16:02

If that were the case, how did the image load?

i.e.,
img.png
img.png (982 Bytes) Viewed 30371 times

User avatar
boiler
Posts: 17706
Joined: 21 Dec 2014, 02:44

Re: ImageSearch Tips & Tricks

Post by boiler » 31 Mar 2020, 16:04

That's not the image. That's the icon in the caption bar. There is no image loaded.

User avatar
boiler
Posts: 17706
Joined: 21 Dec 2014, 02:44

Re: ImageSearch Tips & Tricks

Post by boiler » 31 Mar 2020, 16:08

Try this:

Code: Select all

URLDownloadToFile,https://www.drawingtutorials101.com/drawing-tutorials/Others/Computers/computer/tn_how-to-draw-Computer-step-0.png,pc.png
Sleep, 500
Gui,Add,Picture,,pc.png
Gui,Show
Sleep 500
ImageSearch,x,y,0,0,%A_ScreenWidth%,%A_ScreenHeight%,*60 pc.png
MsgBox % X "-" Y

tonospatha
Posts: 5
Joined: 31 Mar 2020, 15:01

Re: ImageSearch Tips & Tricks

Post by tonospatha » 31 Mar 2020, 16:15

Ah thanks!
Your code works like a dream i guess i just need to play with the variation now

Code: Select all

URLDownloadToFile,https://www.autohotkey.com/assets/images/ahk-logo-no-text241x78-180.png,ahk-logo-no-text241x78-180.png
Gui,Add,Picture,,ahk-logo-no-text241x78-180.png
Gui,Show
Sleep 100
ImageSearch,x,y,0,0,%A_ScreenWidth%,%A_ScreenHeight%,*60 ahk-logo-no-text241x78-180.png
MsgBox % X "-" Y
MsgBox % A_AhkVersion " " (A_PtrSize=8 ? "64-bit" : "32-bit")
MsgBox ErrorLevel: %ErrorLevel%
ExitApp
img.png
img.png (7.07 KiB) Viewed 30357 times
error.png
error.png (2.39 KiB) Viewed 30357 times
Attachments
result.png
result.png (2.08 KiB) Viewed 30357 times

User avatar
boiler
Posts: 17706
Joined: 21 Dec 2014, 02:44

Re: ImageSearch Tips & Tricks

Post by boiler » 31 Mar 2020, 16:19

The problem with that AHK image is that it has transparent areas, so it won’t find it. That’s why I found a different image.

tonospatha
Posts: 5
Joined: 31 Mar 2020, 15:01

Re: ImageSearch Tips & Tricks

Post by tonospatha » 31 Mar 2020, 16:27

Ah! Thanks!

go1
Posts: 13
Joined: 17 Sep 2018, 17:14

Re: ImageSearch Tips & Tricks

Post by go1 » 09 Jun 2020, 14:57

It's a great post for understanding anything related to this command.
I also want to add #11 for you if you dont mind.
#11 Debugging + bonus
If you ever find yourself troubled where your error in this command lies here is example code of what you can do to remove mistakes from your command:

Code: Select all

4c:
{
CoordMode, Pixel, Window
ImageSearch, FaX, FaY, 0, 50, 1100, 800, *2 C:\Users\dom\Desktop\Graphic\Bun.png
If ErrorLevel = 0
Tooltip, Found image,this tooltip is in image coords,%Fax%,%Fay%,1
If ErrorLevel
Tooltip,Image has not ben found,changing star option or,200,200,2
Tooltip,image size can help if it was not supposed to happen.,200,215,4
If ErrorLevel = 2
Tooltip,You made error in the command,333,333,3
}
Return
WORTH TO NOTE: I dont know why but this command NEVER WORKED to search text itself if it was form of image on the screen.

User avatar
boiler
Posts: 17706
Joined: 21 Dec 2014, 02:44

Re: ImageSearch Tips & Tricks

Post by boiler » 09 Jun 2020, 15:16

go1 wrote:
09 Jun 2020, 14:57
It's a great post for understanding anything related to this command.
I also want to add #11 for you if you dont mind.
#11 Debugging + bonus
If you ever find yourself troubled where your error in this command lies here is example code of what you can do to remove mistakes from your command:

Code: Select all

4c:
{
CoordMode, Pixel, Window
ImageSearch, FaX, FaY, 0, 50, 1100, 800, *2 C:\Users\dom\Desktop\Graphic\Bun.png
If ErrorLevel = 0
Tooltip, Found image,this tooltip is in image coords,%Fax%,%Fay%,1
If ErrorLevel
Tooltip,Image has not ben found,changing star option or,200,200,2
Tooltip,image size can help if it was not supposed to happen.,200,215,4
If ErrorLevel = 2
Tooltip,You made error in the command,333,333,3
}
Return
WORTH TO NOTE: I dont know why but this command NEVER WORKED to search text itself if it was form of image on the screen.
There are some issues with your code:

The first and seconds ToolTip commands are not formatted correctly. They have extra commas causing parameters to be in the wrong places. If you want a comma not to mark the end of the parameter and the start of a new one, you have to escape it like this: `,.

Your ToolTip starting with "image size can help" will be executed no matter what the value of ErrorLevel is. Perhaps you meant it to be contingent on the If ErrorLevel like the one before it. If so, you need braces { } to mark the code block after the If statement.

Post Reply

Return to “Tips and Tricks (v1)”