FindClick() - ImageSearch, Clicking, & More [Newest Version]

Post your working scripts, libraries and tools for AHK v1.1 and older
WaterZephyre
Posts: 1
Joined: 03 Sep 2018, 16:46

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

03 Sep 2018, 16:52

Can you tell people how to use this? It took me a good hour of searching to find out how to do it, I just started using ahk so I didn't know about the include command. Thank you for this program, it helps a lot.
User avatar
berban
Posts: 97
Joined: 14 Apr 2014, 03:20

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

03 Sep 2018, 18:33

Well I can't walk you through the entire process but I did make a "Quick-Start Guide" chapter in the documentation that you should check out. You should also read the entire chapter on AutoHotkey Functions in the AutoHotkey documentation, as I also point out in the FindClick guide.
JenJen

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

19 Sep 2018, 17:34

Hey berban,

I've been battling for a while now with this, and no matter how I try to define 'r' and give it an active window - it still uses the 'monitor' coordinates and apply them to X/Y.

An example of this is if you dont use fullscreen, and then search an image within your application. If you place your active window in the top right corner (I have 3 monitors though), it'll work fine. It'll get the coordinates on your main monitor and move the mouse to the image.

Issue comes if you move that non-fullsized active window somewhere else on your main monitor. Now, if you move your active window 300px to the right, the 900 coordinate becomes 1200, and obviously since your mouse is moving in the 'active' window, it'll move it to coordinate 1200, which is not where it's supposed to be (900).

I think the 'relative' function is broken, honestly. I've run out of ideas to test this, I've made it as simple as:

1. Open Application with an image in your want to click.
2.

Code: Select all

F5::
{
  FindClick(A_ScriptDir . "\images\icon.png", "r""Application"" n", x, y)
  MouseMove, x, y
  MsgBox, x:%x%, y:%y%
}
3. Move the active application (non full screen) around
4. Watch the mouse move all over the place.

Any help would be appreciated, thanks!
User avatar
berban
Posts: 97
Joined: 14 Apr 2014, 03:20

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

20 Sep 2018, 08:00

Hi JenJen,

Thank you for the thorough investigation of the issue you're having. I'm always appreciative if bugs can be found, and a clear explanation makes identifying them way easier.

I'm a little confused by the exact issue you're having however, although your sample code has given me an idea. If you are primarily using the last two parameters of the function (the byref FoundX and FoundY) to get your image information, you should be aware that those are ALWAYS relative to the screen regardless of the setting of "r". (This also applies to the returned value of the function.) So in your sample code, you’d need Coordmode, Mouse, Screen – either before MouseMove or globally at the start of your script – or else moving the window will have the effect you are talking about. (Coordinates relative to the window is the default for MouseMove.)

I know this isn’t very intuitive, but there was some reason I can’t recall now that caused me to originally code it like that. I was going to change it but never got around to it. At the very least I guess maybe I should make that quirk more clear in the documentation.

Anyway, please let me know if that doesn’t fix your issue. The "r" is really pretty simple and just uses a search area that matches the window coordinates, but even the simplest stuff breaks sometimes I guess.
JenJen

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

20 Sep 2018, 21:58

Thanks for you quick and thorough reply.

That did indeed fix the situation, I was under the impression from thoroughly reading your documentation and AHK information on the official site that relative to window was default unless another CoordMode was specified. I haven't gone back and looked, but I must've really misunderstood something, but your reply definitely cleared it right up. Maybe some clarification in the documentation would be nice, yes.

Thanks for your time and efforts! - it is much appreciated,
-JenJen
JenJen

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

20 Sep 2018, 22:26

Also, forgot to mention. As a little side note, to help people that aren't as well versed in this as perhaps others are - having examples of codes where specific functions and parameters are used are a massive help. They have that on the official site as well, and it's a game changer in understanding how to construct things properly so the code works. Sometimes even multiple examples of code variation :)

I went back and read your documentation and it's descriptions like
The search area becomes the area of the given window instead of the entire screen area. This is useful when scanning for an element which only appears on a particular window
that confuses me. It might make sense to people that know what they're looking at, but I read it as 'r' telling the script to use Window Coordinates instead of Screen coordinates. And when the official AHK on image search page states
The X and Y coordinates of the upper left corner of the rectangle to search, which can be expressions. Coordinates are relative to the active window unless CoordMode was used to change that.
I don't feel like you can fault the uninitiated in misinterpreting these things.

I'm still struggling with trying to make FindClick only search for specific image within a specific area. I've set up a script that I've used to test with, and even through trial and error I don't seem to be able to make it pick up the image when it's in a specific area of the screen - however if I remove parameters and just let it search entire screen it picks the image up just perfectly.

Code: Select all

{
TargetExists := FindClick(A_ScriptDir . "\images\image.png", "n o30 a,1530,700,150,150", xCord, yCord)
  if (TargetExists) {
    MoveMouse(xCord, yCord, 0, 0)
    SleepFor(200, 20) ;Custom function
    MsgBox, 48, Target Found!, Managed to locate image.
    return
  } else if (!TargetExists) {
    MsgBox, 48, No target!, Couldn't locate image.
  }
}
return
Again, appreciate your spare time helping,
-JenJen
JenJen

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

20 Sep 2018, 23:45

Again a post, I'm sorry to whatever moderator have to approve these things. If you can, please feel free to merge them, but this thing keeps blowing my mind over and over. First off, the 'dx' parameter is great, and really helped me narrow in an area in the world, despite the way you've written it making 0 sense to me.

First off,
The what to give x[,y,w,h] under "Search Area Modifications.
The first parameter (X) moves the square area along the Y axis and visa versa. This makes no sense to me, and without a lot of pictures and such to show (I assume this isn't intended behaviour) it's probably not something we can get into over a few posts like these.

Secondly, I don't understand why you'd make the w/h (which I first assumed with width and height?) cut off part of the search area. This seems like such a backwards way to make a search area to me - and very non-intuitive/logical. Again, unless I totally misunderstood something.

Is there a reason you don't just define X Y on the screen, then create a square box to search on based on Height/Width from that point? Would be verrryyy simple to understand.

And back to the original topic, I again ran into issues when applying said CoordMode setting (Coordmode, Mouse, Screen) - Now all my mouse clicks with regular coordinates (Not using FindClick to get X, Y) are off because they're now relative to the screen and not the active window. That made me think, why does it make any sense for FindClick to need Screen setting to work for Relative coords?

There's so many things here that seems backwards and confusing to me. I really would prefer to just use Relative in everything because that's what I need. Something that only runs in the application, and I'd really prefer not having put CoordModes all over the .ahk depending on weather Im using imagesearch or not to find the coordinates.

Hope you understand and can shed some light on this for me, it's honestly very confusing to me and many hours spend looking at the same things.
Thanks!
-JenJen
JenJen

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

21 Sep 2018, 00:27

After another hour or so (oh lord) I figured out it's because you gotta write

Code: Select all

aX, Y, Width, Height
(So it was width and height) and not

Code: Select all

A, X, Y, Width, Height
. I know that's probably me being retarded, but judging from the examples in the documentation that never occurred to me. So when writing the parameters it moved everything one because it assumed the first was empty (Im guessing). But still doesn't help me with the Coordmode issue.

Sorry lol,
-JenJen
User avatar
berban
Posts: 97
Joined: 14 Apr 2014, 03:20

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

21 Sep 2018, 09:22

Hahaha no worries about the repeat posting. It’s clear you are working pretty hard on whatever script you are writing, and it’s the least I can do to give you a small hand with this particular part of it.

That being said there was a lot in there so I might not get to everything. But here’s a few responses.


  1. I definitely do not fault you for your confusion about the screen vs. window coordinates. I agree that the way I made the script wasn’t very intuitive. I also saw that I didn’t make it clear at all in the documentation, so I updated it just now to try to fix that a little bit.
  2. If you change CoordMode within the active thread it won't break other parts of your script. Just don't put it at the top of your script if you want this to happen. For instance, if you have a hotkey you can change the coordmode within that hotkey and it will not affect anything else in the script except for the code following CoordMode within that one hotkey. If there is something later in the hotkey that needs relative coordinates, just reset the coordmode using CoordMode, Mouse, Relative somewhere after the bit with FindClick.
  3. I think most of your confusion about the “a” option is because the options are based on the format of AutoHotkey’s GUI command options. Maybe you haven’t used this feature yet. In this format, there’s no leading comma as shown below:

Code: Select all

Gui, Add, Text, x100 y200 ; textbox is added at x = 100 and y = 200
As you can see, the character immediately following “x” or “y” is the input, which is the same for FindClick.
But in the case of some of the FindClick options, this is complicated by the fact that they can have a few “sub-options”, e.g. the search area has an x, y, width, and height component. So these are then separated from each other. I tend to use commas but I can see why that would be confusing and lead you to think the format was similar to a regular AutoHotkey command like MsgBox which would start with a leading comma. You can also use pipes (|) which might make it easier to understand.
Here are some examples that I hope make it clearer.

Code: Select all

FindClick("image", "a1,2,3,4") ; x=1, y=2, w=3, h=4
FindClick("image", "a1|2|3|4") ; same as above
FindClick("image", "a|2|3|4") ; same as above except in this case x is omitted. This is what you had in your code and is why you thought the x was y and vice versa. If x, y, w, or h are omitted they will be the default. If “r” is set the default x is the left edge of the window, otherwise it’ll be the left edge of the screen.
FindClick("image", "a200,,-200") ; will search in a column-shaped region of the screen starting 200 pixels from the left edge of the screen (x=200) and ending 200 pixels from the right edge of the screen (w=-200), i.e. like a 4:3 movie playing on a widescreen TV. The y and h are omitted so the default is to do the entire height of the screen.
FindClick("image", "r a200,,-200") ; same as above except for the active window: will search within a column shaped region excluding the left and right edges of the active window.
If you’re ever unsure, I’d definitely recommend you use the “dx” option which will shade the area of the screen being searched.

Hope that answers some of your questions but let me know if there’s something else.

By the way if you make an account on the forums you can edit your posts if need be, as well as get email alerts when someone replies.
JenJen
Posts: 4
Joined: 21 Sep 2018, 16:50

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

21 Sep 2018, 17:28

berban wrote:By the way if you make an account on the forums you can edit your posts if need be, as well as get email alerts when someone replies.
Appreciate it, feel pretty silly having a ton of replies, I wasn't expecting to have this deep issues with features. I made an account now :thumbup:
berban wrote: [*]If you change CoordMode within the active thread it won't break other parts of your script. Just don't put it at the top of your script if you want this to happen. For instance, if you have a hotkey you can change the coordmode within that hotkey and it will not affect anything else in the script except for the code following CoordMode within that one hotkey. If there is something later in the hotkey that needs relative coordinates, just reset the coordmode using CoordMode, Mouse, Relative somewhere after the bit with FindClick.
This is what I wanted to avoid; having to do a declaration every time I switch from using FindClick Coords to regular Coords. I don't even grasp at the moment why FindClick needs Screen mode to work in a Relative window (the application)? It seems backwards to me.
berban wrote:I’d definitely recommend you use the “dx” option which will shade the area of the screen being searched.[/indent]
Ye it really messed me up trying to sort out through trial and error what the different fields did, while omitting the first parameter (a,1 instead of a1). The DX parameter was what lead me to finally figure out how it was set up and it stopped being so confusing haha. Definitely a good feature.

There actually is another thing outside of the Coord issue, I can't for the life of me make the oTransBlack work, at all. Even deducting it to the most basic error free example it doesn't seem to work. Would you mind testing if of using

Code: Select all

FindClick(A_ScriptDir . "\images\map.png", "n r""Application"" a1400,800,120,120 oTransBlack,20 dx", xCord, yCord)
something similar to this, can find an image that has an ever changing (like animated background) but where the background has been made black.

I tried it various places, with various images trying to ensure that there are no "pixel changes" on the image I'm comparing, to in-game one on the non-#000000-black pixels. I assume that any pixel on the image that is #000 would be ignored (can be any color) and thus as long as the pixels on the test.png are exactly like the in-game (the part that doesn't change) it should find it.

In-game image (map hotkey)
Image

Searchable image (cut and 100% black in photoshop. Cut any potential borders that might change pixels)
Image

Can't get it to find it no matter what. I've tried multiple times, maybe you can tell me what's wrong? I've checked the search area and it's well within the red area using Dx mode.

Here's blown up pictures of the "transparent black" and the "in-game" image.
https://i.imgur.com/UksjmtH.png
https://i.imgur.com/GIARiWi.png

I can't find a single pixel changing, yet it can't locate it. It's little mysterious to me. Thanks!
-JenJen
User avatar
berban
Posts: 97
Joined: 14 Apr 2014, 03:20

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

22 Sep 2018, 16:50

Well it looks like you’re doing it right. At this point it’s just an issue with the underlying imagesearch technique. By which I guess I mean the image really isn’t matching, even if to your eyes it looks like it should.

Personally I never use the trans- feature, except for maybe once or twice. In this case I would definitely use a smaller image than you have. That would avoid the need for making the black background transparent. I hardly ever use images larger than 10x10 pixels. In this case I’d suggest trying a small region of that shade of red/orange you can see in the middle of the icon. That’s just a guess since it seems like a fairy unique color that wouldn’t be present somewhere else on the screen.

Using an image that’s mostly the same color has the added benefit of making the color variation much more likely to work. Depending on the application an image may render differently each time you run it. When this happens, crisp lines like in text may produce radically different pixels but a solid region of color should be about the same.
FastLearner

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

22 Sep 2018, 19:52

Hello berban

Nice Script you have here and I would love to use it, but I can't seem to make it work...I downloaded the script from github and tried running it but no gui is showing or anything

Is there anything I should do before?

Also, Can it detect images on the background? by specifying win title and what not?

Thanks in advance
JenJen
Posts: 4
Joined: 21 Sep 2018, 16:50

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

22 Sep 2018, 20:53

The o options have definitely proven themselves to be finicky to me.

Any explanation for my confusion with why FindClick requires Screen coordinates to find things in a Relative window?

-JenJen
User avatar
berban
Posts: 97
Joined: 14 Apr 2014, 03:20

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

28 Sep 2018, 06:19

FastLearner wrote:Hello berban

Nice Script you have here and I would love to use it, but I can't seem to make it work...I downloaded the script from github and tried running it but no gui is showing or anything

Is there anything I should do before?
Hi FastLearner,

Yes, FindClick is an AutoHotkey function so you'll need to be familiar with those before starting. Make sure you understand this page (for now, you can stop when you get to the section on "Variadic Functions")
https://autohotkey.com/docs/Functions.htm

You will also need to include the actual code in your script somewhere for it to work. You can copy and paste the raw code, but I'd recommend creating a separate file FindClick.ahk and using the #Include directive

Make sure you understand both of those articles and then get back to me if you still have questions.

FastLearner wrote:Also, Can it detect images on the background? by specifying win title and what not?

Thanks in advance
It can detect anything you can see on screen with your two eyes (with a few rare exceptions). So that means the window can be in the background as long as it's visible and not covered by another window.

Keep in mind though that as with anything in coding we're talking about a machine which requires precision. Even if you think you see the image on-screen, it may actually be slightly different than the one you are searching with, which could cause the search to fail.
User avatar
berban
Posts: 97
Joined: 14 Apr 2014, 03:20

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

28 Sep 2018, 06:23

JenJen wrote:The o options have definitely proven themselves to be finicky to me.

Any explanation for my confusion with why FindClick requires Screen coordinates to find things in a Relative window?

-JenJen
FindClick doesn't use CoordMode at all. It is completely separate.

All you need to know is that the coordinates returned when using FindClick are screen coordinates. It's then up to you to figure out how to use those for whatever it is you are doing elsewhere in your script. If you want to use mouseclick then yeah you'll need to use CoordMode to change the settings for mouseclick. Sorry if that wasn't clear before.
JenJen
Posts: 4
Joined: 21 Sep 2018, 16:50

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

28 Sep 2018, 10:09

berban wrote:
JenJen wrote:The o options have definitely proven themselves to be finicky to me.

Any explanation for my confusion with why FindClick requires Screen coordinates to find things in a Relative window?

-JenJen
FindClick doesn't use CoordMode at all. It is completely separate.

All you need to know is that the coordinates returned when using FindClick are screen coordinates. It's then up to you to figure out how to use those for whatever it is you are doing elsewhere in your script. If you want to use mouseclick then yeah you'll need to use CoordMode to change the settings for mouseclick. Sorry if that wasn't clear before.
Hey,
Would it be complicated for me to edit FindClick.ahk as to return Relative coords instead of Screen ones? I have no usage for screen coordinates and changing CoordMode multiple times in various functions all over the .ahk will just clutter things.

I realize this is probably me lacking understanding, but I still don't understand why I need to add Coordmode, Mouse, Screen for FindClick to return relative coordinates in a specific application window (where I assume only relative coords work?) if FindClick already returns Screen coordinates. Wouldn't it make more sense if I had to add relative?

Code: Select all

If (A_TimeSincePriorHotkey < 300)
		Return
Coordmode, Mouse, Screen
FindClick(A_ScriptDir . "\images\image.png", y, x)
MoveMouse(y,x,10,10,1)
Coordmode, Mouse, Relative
return
This works, but I don't understand why I have to change it to Screen when I'm looking for Relative coordinates (inside of an application window not fullscreen).

Thanks,
-JenJen
dice4321
Posts: 8
Joined: 21 May 2016, 21:44

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

15 Mar 2019, 06:06

Hey berban, I've run into this problem. The FindClick function is able to find the image, but does a control send to the wrong ahk_id. It seems to only happen almost all the android emulators I've tried.

The line I used:

FindClick(A_ScriptDir "\pics\combat", "dx rGirl mc o50 Count1 n1 w30000,50")

imgur.com /AbNiXMI Broken Link for safety
User avatar
berban
Posts: 97
Joined: 14 Apr 2014, 03:20

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

15 Mar 2019, 10:45

Hi dice4321, thanks for pointing this out. Your screenshot was very helpful in demonstrating what you are referring to.

The way the function chooses the window is pretty simple, you can see it on lines 617 through 623. Basically it uses WinGet, List to get a list of all windows and chooses the topmost window at the coordinates where the image was found. It's a pretty crude way of doing it but it should work in most cases.

If it isn't working for you, I'd recommend using the func option to make a function that does the clicking the way you'd prefer.

Code: Select all

FindClick(A_ScriptDir "\pics\combat", "mc funcAndroid_ControlClick")

Android_ControlClick(ImgX, ImgY)
{
	Window := WinExist("MyWin")
	; Findclick uses screen coordinates so we need to offset the window pos
	WinGetPos, WinX, WinY, , , ahk_id %Window%
	ImgX -= WinX, ImgY -= WinY
	ControlClick, x%ImgX% y%ImgY%, ahk_id %Window%, , Left, , NA
}
EDIT

I just updated the function so that if you declare a relative window with the "r" option, it'll skip the whole WinGet, List and just use that window. So that should solve your problem - just download the updated code from github.
dice4321
Posts: 8
Joined: 21 May 2016, 21:44

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

15 Mar 2019, 19:41

berban wrote:
15 Mar 2019, 10:45


I just updated the function so that if you declare a relative window with the "r" option, it'll skip the whole WinGet, List and just use that window. So that should solve your problem - just download the updated code from github.
Holy, thanks for the quick reply, alternate solution, and update to your library! You're the best berban!
Oblituarius
Posts: 13
Joined: 31 Mar 2019, 16:57

Re: FindClick() - ImageSearch, Clicking, & More [Newest Version]

31 Mar 2019, 17:14

Hi berban,
So I just recently found your script and it works wonders!

I'm just having a small issue, which involves me not understanding the documentation.

This is what I have:

Code: Select all

Numpad1::
{
Loop, 1
	{
		Tooltip, % FindClick("F:\Programs\AHK\Shooter\TestHP.png", "oTransBlack,30 r e y30 n1 Sleep60 bottom dx"),100,230,2
	}
return
}
if I remove "bottom" works all works as intended, but I wanted to ad "d" - Direction Of Search, I just don't understand how to place it as a parameter on the function.
I've tried dBottom, dB, db, bottom.

As an adition to the question, I would like to know if there is any way to get the stores X and y coords from the found positions and make sure to click first the one closes to a specified X, Y. (960,540 for example as the center).

Can you enlighten me? Thanks!

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: JoeWinograd, TheNaviator and 82 guests