can't find a reliable imagesearch reference point

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
newcod3r
Posts: 505
Joined: 30 Sep 2021, 02:16

can't find a reliable imagesearch reference point

Post by newcod3r » 21 May 2022, 20:25

My objective is to click into media in the WhatsApp Desktop windows app.

For individuals vs groups, the layout is different. The only reliable point is the green call button, which doesn't exist in groups.
I tried using part of the Media text as the reference point, but it's extremely unreliable.
Can't use fixed coordinates either because it changes depending on how long the group description is.

What else can I do?
image.png
image.png (52.93 KiB) Viewed 1011 times
image.png
image.png (56.05 KiB) Viewed 1011 times

User avatar
Spawnova
Posts: 554
Joined: 08 Jul 2015, 00:12
Contact:

Re: can't find a reliable imagesearch reference point

Post by Spawnova » 22 May 2022, 04:25

There is a lot of ways to do this, I can give you some ideas anyway

1. Find the Media text, I know you've said you tried this but as long as the size doesn't change it should be fine, try using higher color variance

2. If the height of the media block doesn't change you can search for that as it would have a unique image

3. Find the arrow and check to see if there are grey pixels to the left of it, since it seems only the media tab has numbers next to the arrow

4. Assuming the layout is always Group Image > Group Description > Group media you could also just search for the spacers, find the second instance and media would be right below it

Image

If you can find any of those you can offset their position to click exactly where you want to.

Also my class for image/pixel scanning may have some additional functions that you may find useful for this task - https://github.com/Spawnova/ShinsImageScanClass
Specifically ImageArray() and PixelRegion() or PixelCountRegion()

BoBo
Posts: 6564
Joined: 13 May 2014, 17:15

Re: can't find a reliable imagesearch reference point

Post by BoBo » 22 May 2022, 04:29

Would be interested if AccViewer.ahk is detecting elements of that GUI? :think:

User avatar
Smile_
Posts: 858
Joined: 03 May 2020, 00:51

Re: can't find a reliable imagesearch reference point

Post by Smile_ » 22 May 2022, 05:20

This is an example of @Spawnova 4's idea, tried with your screenshot.

Code: Select all

F1::
    ; Search for the first occurrence
    ImageSearch, FoundX, FoundY, 0, 0, A_ScreenWidth, A_ScreenHeight, Needle.png
    If (ErrorLevel ~= "1|2")
        Msgbox, Needle 1 was not found
    Else {
        ; Search for the second occurrence
        ImageSearch, FoundX, FoundY, 0, FoundY + 1, A_ScreenWidth, A_ScreenHeight, Needle.png
        If (ErrorLevel ~= "1|2")
            Msgbox, Needle 2 was not found
        Else {
            MouseClick, Left, FoundX + 40, FoundY + 50,, 0
        }
    }
Return

Esc::ExitApp
Needle:
Needle.png
Needle
Needle.png (149 Bytes) Viewed 948 times

User avatar
boiler
Posts: 16774
Joined: 21 Dec 2014, 02:44

Re: can't find a reliable imagesearch reference point

Post by boiler » 22 May 2022, 06:14

An ImageSearch using a *TransRed option should reliably find this image as the top-left corner of the Media section and none of the others. It includes some of the black separator bar above it, and the width is the entire width, so it only finds the corner. None of the other sections should match the dark sections in between the red (ignored) areas. It doesn't find the others in the reference image, and it's unlikely that other sections would match it unless one were to be named only one very short word and nothing else.
media.png
media.png (1.09 KiB) Viewed 895 times

Code: Select all

CoordMode, Pixel, Screen
CoordMode, Mouse, Screen
ImageSearch, FoundX, FoundY, 0, 0, A_ScreenWidth, A_ScreenHeight, *TransRed media.png
if ErrorLevel
	MsgBox, Not found
else
	Click, % FoundX + 25 "," FoundY + 35

newcod3r
Posts: 505
Joined: 30 Sep 2021, 02:16

Re: can't find a reliable imagesearch reference point

Post by newcod3r » 22 May 2022, 07:22

boiler wrote:
22 May 2022, 06:14
An ImageSearch using a *TransRed option should reliably find this image as the top-left corner of the Media section and none of the others. It includes some of the black separator bar above it, and the width is the entire width, so it only finds the corner. None of the other sections should match the dark sections in between the red (ignored) areas. It doesn't find the others in the reference image, and it's unlikely that other sections would match it unless one were to be named only one very short word and nothing else.

media.png

Code: Select all

CoordMode, Pixel, Screen
CoordMode, Mouse, Screen
ImageSearch, FoundX, FoundY, 0, 0, A_ScreenWidth, A_ScreenHeight, *TransRed media.png
if ErrorLevel
	MsgBox, Not found
else
	Click, % FoundX + 25 "," FoundY + 35
may I check if you got the Red from the rectangle box? That was an overlay I drew just to illustrate my target point 😅

newcod3r
Posts: 505
Joined: 30 Sep 2021, 02:16

Re: can't find a reliable imagesearch reference point

Post by newcod3r » 22 May 2022, 07:36

Spawnova wrote:
22 May 2022, 04:25
There is a lot of ways to do this, I can give you some ideas anyway

1. Find the Media text, I know you've said you tried this but as long as the size doesn't change it should be fine, try using higher color variance
Asking for option 1 - do I use oTransBlack 20 or the exact pixel color of the grey Media text? I usually don't specify the color (E.g. o30) and it works well. not sure why it's not detecting reliably this time.

newcod3r
Posts: 505
Joined: 30 Sep 2021, 02:16

Re: can't find a reliable imagesearch reference point

Post by newcod3r » 22 May 2022, 07:41

Smile_ wrote:
22 May 2022, 05:20
This is an example of @Spawnova 4's idea, tried with your screenshot.

Code: Select all

F1::
    ; Search for the first occurrence
    ImageSearch, FoundX, FoundY, 0, 0, A_ScreenWidth, A_ScreenHeight, Needle.png
    If (ErrorLevel ~= "1|2")
        Msgbox, Needle 1 was not found
    Else {
        ; Search for the second occurrence
        ImageSearch, FoundX, FoundY, 0, FoundY + 1, A_ScreenWidth, A_ScreenHeight, Needle.png
        If (ErrorLevel ~= "1|2")
            Msgbox, Needle 2 was not found
        Else {
            MouseClick, Left, FoundX + 40, FoundY + 50,, 0
        }
    }
Return

Esc::ExitApp
Needle:

Needle.png
will it be better to select just a small square box instead of the entire strip for faster search? 🤔

newcod3r
Posts: 505
Joined: 30 Sep 2021, 02:16

Re: can't find a reliable imagesearch reference point

Post by newcod3r » 22 May 2022, 07:45

BoBo wrote:
22 May 2022, 04:29
Would be interested if AccViewer.ahk is detecting elements of that GUI? :think:
hmm not as familiar with accviewer for now..

User avatar
boiler
Posts: 16774
Joined: 21 Dec 2014, 02:44

Re: can't find a reliable imagesearch reference point

Post by boiler » 22 May 2022, 09:11

newcod3r wrote: may I check if you got the Red from the rectangle box? That was an overlay I drew just to illustrate my target point 😅
Whoosh! Wow, you really think I didn’t understand that the red markings were added by you and are not part of what you’re searching for? You totally didn’t understand anything about my post, apparently.

No, it has nothing to do with the red in your first image. I guess you don’t understand what *TransRed does and/or didn’t see the image I attached in my post that you would use as a reference image. That along with the script I posted clicks exactly on the Media section when used with your second image as an example (the one that contains none of your red markings) and should work in general. Did you even try it, or were you too busy laughing at what you incorrectly thought was my misunderstanding?

newcod3r
Posts: 505
Joined: 30 Sep 2021, 02:16

Re: can't find a reliable imagesearch reference point

Post by newcod3r » 22 May 2022, 16:49

boiler wrote:
22 May 2022, 09:11
newcod3r wrote: may I check if you got the Red from the rectangle box? That was an overlay I drew just to illustrate my target point 😅
Whoosh! Wow, you really think I didn’t understand that the red markings were added by you and are not part of what you’re searching for? You totally didn’t understand anything about my post, apparently.

No, it has nothing to do with the red in your first image. I guess you don’t understand what *TransRed does and/or didn’t see the image I attached in my post that you would use as a reference image. That along with the script I posted clicks exactly on the Media section when used with your second image as an example (the one that contains none of your red markings) and should work in general. Did you even try it, or were you too busy laughing at what you incorrectly thought was my misunderstanding?
I didn't understand because there was no red in my original references and your image had red lines, hence I inferred you may have used my red box as the reference point.

Also mentioned above that I don't really specify other colors when using the oTrans option. The ahk docs didn't really explain it the way you did u 3 years ago, which was way more helpful than your latest reply now..
image.png
image.png (189.97 KiB) Viewed 713 times
and no, I wasn't laughing.

User avatar
boiler
Posts: 16774
Joined: 21 Dec 2014, 02:44

Re: can't find a reliable imagesearch reference point

Post by boiler » 22 May 2022, 20:03

newcod3r wrote: and no, I wasn't laughing.
OK. :thumbup:

You really should learn how to use the *Trans option since you do a good amount of ImageSearch-ing. It can make all the difference in whether a performing a search is possible or not. I didn’t explain how it works again now because I thought you had learned it, so I was only pointing out how to implement it specifically for this case, not to give instruction on its use in general.

Post Reply

Return to “Ask for Help (v1)”