Imagesearch shades of variation for a very small icon Topic is solved
Imagesearch shades of variation for a very small icon
Trying to use imagesearch to determine if an icon has been enabled or not, and if not, go ahead and click to enable. I can't seem to differentiate these two images using the shades of variation option.
Specifically, imagesearch is searching for the 'off'' variant.
*161 - both images are found
*159 - neither image is found
*160 - sometimes both images are found, sometimes neither image is found.
These images were created by taking a screenshot of the entire desktop, then cropping the icons.
Any suggestions? If I click the icon regardless, and it's enabled to begin with, I'd be disabling it, when the goal is to enable it.
Specifically, imagesearch is searching for the 'off'' variant.
*161 - both images are found
*159 - neither image is found
*160 - sometimes both images are found, sometimes neither image is found.
These images were created by taking a screenshot of the entire desktop, then cropping the icons.
Any suggestions? If I click the icon regardless, and it's enabled to begin with, I'd be disabling it, when the goal is to enable it.
Re: Imagesearch shades of variation for a very small icon
Rather than using different levels of variation to see which one is being displayed, you should have two different reference images and search with a very low level of variation (possibly 0) for each and see which one is found.
Re: Imagesearch shades of variation for a very small icon
Not sure I follow, but what I just tried is the following:
Code: Select all
imageFile1 := "D:\\Documents\\AutoHotKey\\AHK v2\\images for imagesearch\keyboard trading off.png"
imageFile2 := "D:\\Documents\\AutoHotKey\\AHK v2\\images for imagesearch\keyboard trading on.png"
found1 := ImageSearch(&foundX, &foundY, x1, y1, x2, y2, "*162 " . imageFile1)
found2 := ImageSearch(&foundX, &foundY, x1, y1, x2, y2, "*162 " . imageFile2)
msgbox("Off " . found1)
msgbox("On " . found2)
So the first image (the 'off' image) requires ~*160 to match, and once there, the result is the same whether icon is enabled or disabled.
With the second image (the 'on' image), *218 is required to match, and at that point, it matches whether icon is enabled or disabled.
Seems strange given the difference is easily discernable to the eye.
Re: Imagesearch shades of variation for a very small icon
If you zoom in on the images, it's very clear why you need so much variation to find them and what you should do about it. The vertical lines are produced by anti-aliasing, which is to place multiple pixels of different colors next to each other to fool the eye into thinking it's another color. That's how text and other images like these can look like they're sharper and have higher resolution than they actually have. You should color over all of those vertical lines (including the ones that are a very light yellow that you might not easily see at first) with a known color such as pure red (FF0000), and then use the *TransRed option. If you do that, it looks like you should be able to find each image with a variation of something like 5.
When coloring over the pixels of your reference images, you need to make sure to do it the right way. You need to use a tool like a "pencil" that paints in pure color pixels, not a "brush" that paints in blended edges. And you need to make sure the color you paint with is exactly what you think it is. If you select a red color, make sure it's FF0000 or make sure you know its exact value for use with the *Trans option.
When coloring over the pixels of your reference images, you need to make sure to do it the right way. You need to use a tool like a "pencil" that paints in pure color pixels, not a "brush" that paints in blended edges. And you need to make sure the color you paint with is exactly what you think it is. If you select a red color, make sure it's FF0000 or make sure you know its exact value for use with the *Trans option.
Re: Imagesearch shades of variation for a very small icon
Wow, I never noticed that until I zoomed in close. Thanks for explaining how to resolve this issue.boiler wrote: ↑11 Apr 2024, 19:26If you zoom in on the images, it's very clear why you need so much variation to find them and what you should do about it. The vertical lines are produced by anti-aliasing, which is to place multiple pixels of different colors next to each other to fool the eye into thinking it's another color. That's how text and other images like these can look like they're sharper and have higher resolution than they actually have. You should color over all of those vertical lines (including the ones that are a very light yellow that you might not easily see at first) with a known color such as pure red (FF0000), and then use the *TransRed option. If you do that, it looks like you should be able to find each image with a variation of something like 5.
When coloring over the pixels of your reference images, you need to make sure to do it the right way. You need to use a tool like a "pencil" that paints in pure color pixels, not a "brush" that paints in blended edges. And you need to make sure the color you paint with is exactly what you think it is. If you select a red color, make sure it's FF0000 or make sure you know its exact value for use with the *Trans option.
Tried MS Paint, since the line tool allows specification of width in pixels. I specified color as 255,0,0.
Doesn't match any better though.
Code: Select all
found1 := ImageSearch(&foundX, &foundY, x1, y1, x2, y2, "*160 *TransFF0000 " . imageFile1)
For the 'on' icon, I had to go to 220 to get a match, and again, it then matches whether the toggle button is enabled or disabled.
Code: Select all
found2 := ImageSearch(&foundX, &foundY, x1, y1, x2, y2, "*220 *TransFF0000 " . imageFile2)
Is there an app or website you'd suggest for this repainting? Hard to really see well even at MS Paint's max zoom, so maybe another app/site would be easier.
Re: Imagesearch shades of variation for a very small icon
Your new pixels are not all pure FF0000. Some are FE0000, FF0100, or FF0001. Any pixel that is not exactly FF0000 will not be skipped at all, so now you are searching for red pixels as well.
Using MS Paint is what I would suggest, but the method you described is suspect. You said you can specify a width. The width applies to brush tools, and I specifically said to use the pencil tool as that is the only one that will draw only in pure pixels of the specified color (and it always paints in a single pixel height/width). You should start again from the base images because if you try just painting over the current red ones, you might miss some since they are visually identical to FF0000.
You might also paste a screenshot of where you are trying to find them. It’s possible something else entirely is the problem in addition to (or instead of) anti-aliasing, such as a differing background being part of the image in some respect.
Using MS Paint is what I would suggest, but the method you described is suspect. You said you can specify a width. The width applies to brush tools, and I specifically said to use the pencil tool as that is the only one that will draw only in pure pixels of the specified color (and it always paints in a single pixel height/width). You should start again from the base images because if you try just painting over the current red ones, you might miss some since they are visually identical to FF0000.
You might also paste a screenshot of where you are trying to find them. It’s possible something else entirely is the problem in addition to (or instead of) anti-aliasing, such as a differing background being part of the image in some respect.
Re: Imagesearch shades of variation for a very small icon
Yes, once I figured out that WindowSpy shows colors, it was easy to see differences that my eyes could not see even with max zoom. MS Paint's line tool is distinct from the brush, but I guess when you go thicker than 1 pixel, it blends. This time I used the pencil, 1 pixel thickness (pencil also allows various thicknesses), along w/ WinSpy. I got both images down to only 2 colors each (I think).boiler wrote: ↑12 Apr 2024, 01:43Your new pixels are not all pure FF0000. Some are FE0000, FF0100, or FF0001. Any pixel that is not exactly FF0000 will not be skipped at all, so now you are searching for red pixels as well.
Using MS Paint is what I would suggest, but the method you described is suspect. You said you can specify a width. The width applies to brush tools, and I specifically said to use the pencil tool as that is the only one that will draw only in pure pixels of the specified color (and it always paints in a single pixel height/width). You should start again from the base images because if you try just painting over the current red ones, you might miss some since they are visually identical to FF0000.
Still no match differentiation. Here are the latest repainted icons.
Here are screenshots of the general area, with the keyboard button toggled on and off.
Last edited by redrum on 12 Apr 2024, 11:50, edited 2 times in total.
Re: Imagesearch shades of variation for a very small icon
Couldn't attach the second screenshot of general area in prior post, so here it is.
Re: Imagesearch shades of variation for a very small icon
If you're trying to locate the position of a button,
I'd advise giving UIA a try. Not all programs respond to it, but it's worth a shot.
Otherwise, it might be worth identifying a consistent image in the vicinity of the button, regardless of its toggle state.
I'd advise giving UIA a try. Not all programs respond to it, but it's worth a shot.
Otherwise, it might be worth identifying a consistent image in the vicinity of the button, regardless of its toggle state.
- English is not my native language. Please forgive any awkward expressions.
- 영어는 제 모국어가 아닙니다. 어색한 표현이 있어도 양해해 주세요.
Re: Imagesearch shades of variation for a very small icon
Well, if you can determine the location of a button with UIA, the story becomes easier.
In this case, not only the position but also the width and height values can be determined, so you can differentiate the toggle state just by searching for a 1-pixel background value.
Perhaps, if you're lucky, you might even be able to determine it just by the attribute element values obtained through UIA.
In this case, not only the position but also the width and height values can be determined, so you can differentiate the toggle state just by searching for a 1-pixel background value.
Perhaps, if you're lucky, you might even be able to determine it just by the attribute element values obtained through UIA.
- English is not my native language. Please forgive any awkward expressions.
- 영어는 제 모국어가 아닙니다. 어색한 표현이 있어도 양해해 주세요.
Re: Imagesearch shades of variation for a very small icon
Is the program you're using free? (Or does it offer a free trial?)
If it's all right, I'd like to check if it's compatible with UIA.
If it's all right, I'd like to check if it's compatible with UIA.
- English is not my native language. Please forgive any awkward expressions.
- 영어는 제 모국어가 아닙니다. 어색한 표현이 있어도 양해해 주세요.
Re: Imagesearch shades of variation for a very small icon
Sure, it's free with basic features, and that covers everything related to this particular issue. https://www.quantower.com/
Re: Imagesearch shades of variation for a very small icon
Discovered something that may be at the root of the problem. When I place the mouse pointer at a particular spot on the toggle button, and then toggle the button on and off with a hotkey, WindowSpy is showing the color at that pixel changes - one color for toggled on, another color for toggled off.
So I thought, great, I'll use `PixelGetColor()` to check for the color at that pixel. But that shows the same color whether toggled on or off - in both cases, it shows 252E37 (toggle off color).
So I thought, great, I'll use `PixelGetColor()` to check for the color at that pixel. But that shows the same color whether toggled on or off - in both cases, it shows 252E37 (toggle off color).
Re: Imagesearch shades of variation for a very small icon
UIA cannot be used in this case.
It seems like the program was designed without considering it.
When I ran it, I found that the GUI windows were much blurrier than expected.
Just to be sure, I checked the value, and DPI_AWARENESS_CONTEXT was UNAWARE.
In this scenario, if the monitor scaling isn't at 100%, the drawn window itself stretches according to the ratio.
This makes it challenging for AHK, which uses SYSTEM_AWARE, to perform image searches.
While it's possible to adjust AHK to UNAWARE to attempt to resolve this, it's not a method I particularly prefer.
The left side is the window captured (considering DPI) with Gdip, and the right side is the actual window.
It demonstrates that the graphics drawn are smaller in reality. Therefore, when performing image searches, using the image obtained through PrintWindow, which is crisper than the actual window considering the enlargement and anti-aliasing, is preferable.
So, by capturing the image directly using PrintWindow and saving it to a file, we can examine the state of the actual drawn graphics.
Our task is simply to extract the starting image for the image search from here.
Sorry, It seems like the program was designed without considering it.
Code: Select all
#Requires AutoHotkey v2.0
#SingleInstance Force
F5::msgbox GetWindowDpiAwarenessContextIgnoringInfoFlag(winExist("A")) ; #define DPI_AWARENESS_CONTEXT_UNAWARE ((DPI_AWARENESS_CONTEXT)-1)
GetWindowDpiAwarenessContextIgnoringInfoFlag(hWnd) {
AreDpiAwarenessContextsEqual(dpiContextA, dpiContextB) {
return dllCall("User32.dll\AreDpiAwarenessContextsEqual", "Ptr",dpiContextA, "Ptr",dpiContextB)
}
GetWindowDpiAwarenessContext(hWnd) {
return dllCall("User32.dll\GetWindowDpiAwarenessContext", "Ptr",hWnd, "Ptr")
}
if (dpiContextA:=getWindowDpiAwarenessContext(hWnd)) {
loop 5 {
dpiContextB:=-1*A_Index
if (areDpiAwarenessContextsEqual(dpiContextA,dpiContextB))
return dpiContextB
}
}
return false
}
Just to be sure, I checked the value, and DPI_AWARENESS_CONTEXT was UNAWARE.
In this scenario, if the monitor scaling isn't at 100%, the drawn window itself stretches according to the ratio.
This makes it challenging for AHK, which uses SYSTEM_AWARE, to perform image searches.
While it's possible to adjust AHK to UNAWARE to attempt to resolve this, it's not a method I particularly prefer.
Code: Select all
/*
The libraries used in this code, excluding Gdip, have not been shared anywhere, so it's not executable.
I plan to share them in the future, but I'm currently too busy to do so.
*/
#Requires AutoHotkey v1.1
#SingleInstance Force
#Include <FunctionGroup_Gdip_Ex-1.54.0-ahk1>
#Include <FunctionGroup_mmikeww.Gdip_All-1.54.0-ahk1>
#Include <FunctionGroup_MonitorExGet-1.0.0-ahk1>
pToken:=Gdip_Startup()
onExit("exitFunc")
exitFunc(_*) {
global pToken
Gdip_Shutdown(pToken)
}
F5::
if (hWnd:=winExist("A")) {
pBitmap:=Gdip_DpiBitmapFromHWND(hWnd) ; PrintWindow
hBitmap:=Gdip_CreateHBITMAPFromBitmap(pBitmap)
splashImage % "HBITMAP:" hBitmap
Gdip_DisposeImage(pBitmap)
}
return
It demonstrates that the graphics drawn are smaller in reality. Therefore, when performing image searches, using the image obtained through PrintWindow, which is crisper than the actual window considering the enlargement and anti-aliasing, is preferable.
Code: Select all
#Requires AutoHotkey v2.0
#SingleInstance Force
#Include <FunctionGroup_buliasz.Gdip_All-1.61.0-ahk2> ; #Include <Gdip_All> on your computer.
; AHKv2-Gdip
; 8960b875c4f1064865b51c72978d61e6648c0343
; https://github.com/buliasz/AHKv2-Gdip
pToken:=Gdip_Startup(), onExit((*)=>Gdip_Shutdown(pToken))
F5:: {
static DPI_AWARENESS_CONTEXT_UNAWARE:=-1
if !(hWnd:=winExist("A"))
return
if (getWindowDpiAwarenessContextIgnoringInfoFlag(hWnd)!==DPI_AWARENESS_CONTEXT_UNAWARE)
return
winGetPos(,, &width, &height, hWnd)
if (windowDpiScale:=dllCall("User32.dll\GetDpiForWindow", "Ptr",hwnd, "UInt")/A_ScreenDPI) ; In fact, I'm aware that there is a better API for this case, but since I'm still studying, I'll stick to the method I've been using so far for now.
width:=round(width*windowDpiScale), height:=round(height*windowDpiScale)
;------------------
hbm:=CreateDIBSection(Width, Height), hdc:=CreateCompatibleDC(), obm:=SelectObject(hdc, hbm)
PrintWindow(hwnd, hdc)
pBitmap:=Gdip_CreateBitmapFromHBITMAP(hbm)
SelectObject(hdc, obm), DeleteObject(hbm), DeleteDC(hdc)
Gdip_SaveBitmapToFile(pBitmap, sOutput:=A_ScriptDir "\ChartImage-" A_Now ".png")
Gdip_DisposeImage(pBitmap)
;------------------
if (fileExist(sOutput))
run(sOutput)
}
; ahk1.1 ahk2.0
GetWindowDpiAwarenessContextIgnoringInfoFlag(hWnd) {
AreDpiAwarenessContextsEqual(dpiContextA, dpiContextB) {
return dllCall("User32.dll\AreDpiAwarenessContextsEqual", "Ptr",dpiContextA, "Ptr",dpiContextB)
}
GetWindowDpiAwarenessContext(hWnd) {
return dllCall("User32.dll\GetWindowDpiAwarenessContext", "Ptr",hWnd, "Ptr")
}
if (dpiContextA:=getWindowDpiAwarenessContext(hWnd)) {
loop 5 {
dpiContextB:=-1*A_Index
if (areDpiAwarenessContextsEqual(dpiContextA,dpiContextB))
return dpiContextB
}
}
return false
}
Our task is simply to extract the starting image for the image search from here.
Last edited by Seven0528 on 12 Apr 2024, 17:08, edited 6 times in total.
- English is not my native language. Please forgive any awkward expressions.
- 영어는 제 모국어가 아닙니다. 어색한 표현이 있어도 양해해 주세요.
Re: Imagesearch shades of variation for a very small icon
If you don't have tools like Photoshop, you'll have to use Paint or something similar to cut out the image yourself.
Keep in mind that not a single pixel should change, so you shouldn't use compressed formats like JPG.
You must use uncompressed formats like PNG.
- English is not my native language. Please forgive any awkward expressions.
- 영어는 제 모국어가 아닙니다. 어색한 표현이 있어도 양해해 주세요.
Re: Imagesearch shades of variation for a very small icon Topic is solved
we'll utilize ImagePut for the image search.
Built-in ImageSearch in AHK might not be suitable for such complex scenarios.
Code: Select all
#Requires AutoHotkey v2.0
#SingleInstance Force
#Include <Class_iseahound.ImagePut-1.10.0-ahk2> ; #Include <ImagePut> on your computer.
; ImagePut
; 4bf78ee724527a150df9407d43e09940e1a378ff
; https://github.com/iseahound/ImagePut
; https://github.com/iseahound/ImagePut/blob/master/ImagePut.ahk
; https://github.com/iseahound/ImagePut/commits/master/ImagePut.ahk
; https://www.autohotkey.com/boards/viewtopic.php?t=76633
#Include <FunctionGroup_buliasz.Gdip_All-1.61.0-ahk2> ; #Include <Gdip_All> on your computer.
; AHKv2-Gdip
; 8960b875c4f1064865b51c72978d61e6648c0343
; https://github.com/buliasz/AHKv2-Gdip
pToken:=Gdip_Startup(), onExit((*)=>Gdip_Shutdown(pToken))
download("https://www.autohotkey.com/boards/download/file.php?id=25636", A_ScriptDir "\ChartImage_ToggleOff.png")
download("https://www.autohotkey.com/boards/download/file.php?id=25635", A_ScriptDir "\ChartImage_ToggleOn.png")
bufNeedleOff:=ImagePutBuffer(A_ScriptDir "\ChartImage_ToggleOff.png")
bufNeedleOn:=ImagePutBuffer(A_ScriptDir "\ChartImage_ToggleOn.png")
F5:: {
static DPI_AWARENESS_CONTEXT_UNAWARE:=-1
if !(hWnd:=winExist("A"))
return
if (getWindowDpiAwarenessContextIgnoringInfoFlag(hWnd)!==DPI_AWARENESS_CONTEXT_UNAWARE)
return
winGetPos(&winX, &winY, &winW, &winH, hWnd)
if (windowDpiScale:=dllCall("User32.dll\GetDpiForWindow", "Ptr",hwnd, "UInt")/A_ScreenDPI) ; In fact, I'm aware that there is a better API for this case, but since I'm still studying, I'll stick to the method I've been using so far for now.
winW:=round(winW*windowDpiScale), winH:=round(winH*windowDpiScale)
;------------------
hbm:=CreateDIBSection(winW, winH), hdc:=CreateCompatibleDC(), obm:=SelectObject(hdc, hbm)
PrintWindow(hwnd, hdc)
pHaystack:=Gdip_CreateBitmapFromHBITMAP(hbm)
SelectObject(hdc, obm), DeleteObject(hbm), DeleteDC(hdc)
bufHaystack:=ImagePutBuffer(pHaystack)
;------------------
state:=-1 ; Unknown
variation:=5 ; I thought a little variation wouldn't be necessary, but it is needed. I'm not sure why.
loop 1 {
if (xys:=bufHaystack.ImageSearchAll(bufNeedleOff,variation)) {
if (xys.length) {
state:=0 ; Off
break
}
}
if (xys:=bufHaystack.ImageSearchAll(bufNeedleOn,variation)) {
if (xys.length) {
state:=1 ; On
break
}
}
}
tooltip "state: " state
;------------------
Gdip_DisposeImage(pHaystack)
}
; ahk1.1 ahk2.0
GetWindowDpiAwarenessContextIgnoringInfoFlag(hWnd) {
AreDpiAwarenessContextsEqual(dpiContextA, dpiContextB) {
return dllCall("User32.dll\AreDpiAwarenessContextsEqual", "Ptr",dpiContextA, "Ptr",dpiContextB)
}
GetWindowDpiAwarenessContext(hWnd) {
return dllCall("User32.dll\GetWindowDpiAwarenessContext", "Ptr",hWnd, "Ptr")
}
if (dpiContextA:=getWindowDpiAwarenessContext(hWnd)) {
loop 5 {
dpiContextB:=-1*A_Index
if (areDpiAwarenessContextsEqual(dpiContextA,dpiContextB))
return dpiContextB
}
}
return false
}
Please refer to the official documentation for how to use #Include.
AHKv2-Gdip: https://github.com/buliasz/AHKv2-Gdip
ImagePut: https://github.com/iseahound/ImagePut
A Tooltip will display either 1 or 0 as the result.
Code: Select all
if !(hWnd:=winExist("Chart ahk_exe Starter.exe"))
return
The required PNG files are designed to be downloaded automatically from the internet, but there's no guarantee they'll work in your environment.
If it doesn't work, please refer to the instructions provided above.
The example script requires the window to be active, but with proper WinTitle configuration, it doesn't necessarily need to be activated.
(PrintWindow doesn't require the window to be activated.)
Additionally, when the cursor hovers over a button, the button image subtly changes.
Considering this aspect would require additional image searching.
Personally, I'm quite exhausted from writing the code up to this point.
If such details are needed, I encourage you to implement them yourself.
@redrum
And finally, YOU'RE WELCOME.
Last edited by Seven0528 on 12 Apr 2024, 18:18, edited 8 times in total.
- English is not my native language. Please forgive any awkward expressions.
- 영어는 제 모국어가 아닙니다. 어색한 표현이 있어도 양해해 주세요.
Re: Imagesearch shades of variation for a very small icon
Those are still not all pure red pixels. It's spiraling into something much more complicated than it needs to be. All that needs to happen is for this to be done correctly. It's easy to see they're not all FF0000 using the Window Spy tool.
Re: Imagesearch shades of variation for a very small icon
I think what's happening is when the forum renders the images inline, it creates some distortion. When I see the images in my post, I can see that not all the red pixels are FF0000. But all the red pixels in the file are FF0000 - they have to be because I painted them with pencil using 1 pixel width. And I went over them just now.
There are only 3 colors in the 'off' image:
252E57
BAC7D7
FF0000
And 3 colors in the 'on' image:
00566C
FFFFFF
FF0000
If you download the files, you should be able to see that. Again, the inline images are being distorted by the forum software.
- Attachments
-
- keyboard trading off - red v3.png (469 Bytes) Viewed 518 times
-
- keyboard trading on - red v3.png (370 Bytes) Viewed 518 times
Re: Imagesearch shades of variation for a very small icon
Wow, even the downloaded images are being modified. Let me see if I can screenshot this. But how would I even create these variations of red? All the visible red was introduced by me as the original icons had no (visible) red.
First screenshot is the original painted toggle off icon. Note the cursor coordinates - 2074,1429 with color FF0000.
Second screenshot is the downloaded (from my prior post) painted toggle off icon. Note the cursor coordinates - 2074,1429 with color FF0100. And just going thru the downloaded image, I can see other shades of red FE0100, FF0001, FF0101, FE0001, etc. Obviously, I'm not sitting here wasting my time and everyone else's by changing the pencil color in paint to these variations of red. I went thru every pixel in my files, and there are only 3 colors in each file, with FF0000 being common to both.
I had no idea all these shenanigans were going on at the pixel level until I came across this particular imagesearch task!
First screenshot is the original painted toggle off icon. Note the cursor coordinates - 2074,1429 with color FF0000.
Second screenshot is the downloaded (from my prior post) painted toggle off icon. Note the cursor coordinates - 2074,1429 with color FF0100. And just going thru the downloaded image, I can see other shades of red FE0100, FF0001, FF0101, FE0001, etc. Obviously, I'm not sitting here wasting my time and everyone else's by changing the pencil color in paint to these variations of red. I went thru every pixel in my files, and there are only 3 colors in each file, with FF0000 being common to both.
I had no idea all these shenanigans were going on at the pixel level until I came across this particular imagesearch task!
Last edited by redrum on 12 Apr 2024, 16:49, edited 3 times in total.