Imagesearch shades of variation for a very small icon Topic is solved

[Seven0528]

temp.png (203 Bytes) Viewed 189 times

In my case, the original file and the downloaded file are the same.

[boiler]

I've never seen such an issue before. It's just a file, so I don't know why uploading and downloading it would change the color values of any of the pixels.

The way it would be possible to create those color variations is if you are actually using one of the brush tools (anything from the Brushes dropdown) rather than the actual Pencil tool. But I guess that's what you're doing, so I don't know what's going on.

[Seven0528]

　By the way, the original image icons seem to have already been anti-aliased before insertion. It gets anti-aliased again by DPI_AWARENESS_CONTEXT_UNAWARE.
Using an image that has been anti-aliased twice for image searching is not appropriate.
ImageSearch itself isn't very efficient to begin with, but requiring a variation value of 200 to function properly indicates that something is seriously wrong.

I understand that you might find it difficult to understand my code.
Nevertheless, there is significance in taking the time to understand examples using Gdip and ImagePut.
It was to utilize images before being anti-aliased by DPI_AWARENESS_CONTEXT.

DPI_AWARENESS_CONTEXT_UNAWARE
DPI unaware. This window does not scale for DPI changes and is always assumed to have a scale factor of 100% (96 DPI). It will be automatically scaled by the system on any other DPI setting.

　If you have time, please install the necessary libraries and test it.
If the code doesn't work in your environment, it's unfortunate, but please at least try.

20240413_071056.png (196.45 KiB) Viewed 175 times

20240413_071047.png (197.17 KiB) Viewed 175 times

[redrum]

　By the way, the original image icons seem to have already been anti-aliased before insertion. It gets anti-aliased again by DPI_AWARENESS_CONTEXT_UNAWARE.
Using an image that has been anti-aliased twice for image searching is not appropriate.
ImageSearch itself isn't very efficient to begin with, but requiring a variation value of 200 to function properly indicates that something is seriously wrong.

I understand that you might find it difficult to understand my code.
Nevertheless, there is significance in taking the time to understand examples using Gdip and ImagePut.
It was to utilize images before being anti-aliased by DPI_AWARENESS_CONTEXT.

DPI_AWARENESS_CONTEXT_UNAWARE
DPI unaware. This window does not scale for DPI changes and is always assumed to have a scale factor of 100% (96 DPI). It will be automatically scaled by the system on any other DPI setting.

If you have time, please install the necessary libraries and test code.
20240413_071056.png
20240413_071047.png

Thank you very much for taking the time to go thru all this. After getting my taxes done (this is tax time for us in the US), I will check out these libraries and go thru your code.

I really wish app developers would show us automation geeks some love and give us keyboard access to everything, and not force us to learn about anti-aliasing, etc.

[Seven0528]

Thank you very much for taking the time to go thru all this. After getting my taxes done (this is tax time for us in the US), I will check out these libraries and go thru your code.

I really wish app developers would show us automation geeks some love and give us keyboard access to everything, and not force us to learn about anti-aliasing, etc.

　@redrum
I provided you with a script that would automatically execute with just a link to the library to include, without requiring programming knowledge.
(The library isn't mine, and it's too extensive to #Include in the original script.)
Furthermore, I didn't force learning on you.
I simply explained why your current ImageSearch would fail without providing too many variations.
The fact that you don't understand doesn't mean it's coercion.

I know that everyone isn't an expert, and I understand that not everyone is a programmer.
So, when introducing such a method, I was concerned. No matter how good the method is, it becomes useless if it's not understood.
You're not wrong. Honestly, there are many times when I don't understand other people's code either.
But the forum isn't just yours. I'm sure someone else will find something useful from my post.
Honestly, I hoped it would be helpful to you, even though I don't even know your name.
Otherwise, why would I spend over two hours of my precious time crafting a response, even writing in a foreign language?

I just felt uncomfortable about not even attempting to try the code I had written.
I'm not fond of your attitude towards the volunteer who helped you,
but I understand it can happen when reality gets busy. (My reality is just as busy.)
I don't live in the United States, but the taxes there have a notorious reputation and are known even overseas.
I hope your tax issue (?) gets resolved well, and you come back to this issue later.
Hopefully, my assistance will be meaningful to you, even if not now.

[Seven0528]

Code: Select all

#Requires AutoHotkey v2.0
#SingleInstance Force
#Include <FunctionGroup_buliasz.Gdip_All-1.61.0-ahk2>   ;  #Include <Gdip_All> on your computer.
#Include <Class_iseahound.ImagePut-1.10.0-ahk2>         ;  #Include <ImagePut> on your computer.
pToken:=Gdip_Startup(), onExit((*)=>Gdip_Shutdown(pToken))

F5::  {
    state := getQuantowerKeybdBtnToggleState()
    tooltip "state: " state
    /*
    -2      The window cannot be found.
    -1      Unknown
    0       Off
    1       On
    */
}


GetQuantowerKeybdBtnToggleState()    {
    static init:=false, bufNeedleOff, bufNeedleOn
        ,DPI_AWARENESS_CONTEXT_UNAWARE:=-1
    getWindowDpiAwarenessContextIgnoringInfoFlag(hWnd)    {
        areDpiAwarenessContextsEqual(dpiContextA, dpiContextB)    {
            return dllCall("User32.dll\AreDpiAwarenessContextsEqual", "Ptr",dpiContextA, "Ptr",dpiContextB)
        }
        getWindowDpiAwarenessContext(hWnd)    {
            return dllCall("User32.dll\GetWindowDpiAwarenessContext", "Ptr",hWnd, "Ptr")
        }
        if (dpiContextA:=getWindowDpiAwarenessContext(hWnd))    {
            loop 5    {
                dpiContextB:=-1*A_Index
                if (areDpiAwarenessContextsEqual(dpiContextA,dpiContextB))
                    return dpiContextB
            }
        }
        return false
    }
    if (!init)    {
        init:=true
        if (!fileExist(pathOff:=A_Temp "\QuantowerKeybdBtnOff.png"))
            download("https://www.autohotkey.com/boards/download/file.php?id=25636", pathOff)
        if (!fileExist(pathOn:=A_Temp "\QuantowerKeybdBtnOn.png"))
            download("https://www.autohotkey.com/boards/download/file.php?id=25635", pathOn)
        bufNeedleOff:=ImagePutBuffer(pathOff)
        bufNeedleOn:=ImagePutBuffer(pathOn)
    }
    prevTMM:=setTitleMatchMode("RegEx")
    hWnd:=winExist("ahk_class ^HwndWrapper\[Starter")
    setTitleMatchMode(prevTMM)
    /*
    Chart
    ahk_class HwndWrapper[Starter;;0868b63c-43fa-412f-9089-77cadc9056b2]
    ahk_exe Starter.exe
    */
    if (!hWnd) || (getWindowDpiAwarenessContextIgnoringInfoFlag(hWnd)!==DPI_AWARENESS_CONTEXT_UNAWARE)
        return -2
    winGetPos(&winX, &winY, &winW, &winH, hWnd)
    if (windowDpiScale:=dllCall("User32.dll\GetDpiForWindow", "Ptr",hwnd, "UInt")/A_ScreenDPI)
        winW:=round(winW*windowDpiScale), winH:=round(winH*windowDpiScale)
    ;------------------
    hbm:=CreateDIBSection(winW, winH), hdc:=CreateCompatibleDC(), obm:=SelectObject(hdc, hbm)
	PrintWindow(hwnd, hdc)
	pHaystack:=Gdip_CreateBitmapFromHBITMAP(hbm)
	SelectObject(hdc, obm), DeleteObject(hbm), DeleteDC(hdc)
    bufHaystack:=ImagePutBuffer(pHaystack)
    loop 1    {
        state:=-1
        variation:=5 ;  I thought a little variation wouldn't be necessary, but it is needed. I'm not sure why.
        if (xys:=bufHaystack.ImageSearchAll(bufNeedleOff,variation))    {
            if (xys.length)    {
                state:=0
                break
            }
        }
        if (xys:=bufHaystack.ImageSearchAll(bufNeedleOn,variation))    {
            if (xys.length)    {
                state:=1
                break
            }
        }
    }
    Gdip_DisposeImage(pHaystack)
    return state
}

AHKv2-Gdip
ImagePut
The only part you need to pay attention to is applying the libraries through #Include.

[boiler]

I'm not fond of your attitude towards the volunteer who helped you

I believe you totally misunderstood redrum’s response. I don’t see anything in his posts that are anything but appreciative of all the help he has received. I suppose something is getting lost in translation.

[Seven0528]

　...Wait, as you know, my English skills are lacking a bit.
But does 'Geek' not have the same derogatory meaning as 'Nerd'?
(I'm currently looking it up, but it seems like it doesn't have a negative meaning.)
I totally interpreted it in a sarcastic tone.
Even the expression 'Thank you very much' seemed like a subtle way of saying 'Wow, you've done something really impressive.'

　@redrum, @boiler
I mostly rely somewhat on translators when writing, but not so much when reading.
I should have been more careful with the interpretation, especially considering it's a foreign language for me.
If I overinterpreted, I sincerely apologize. Please forgive me.
(I made some slight modifications to make the code I wrote easier to use. I sincerely hope it helps.)

[redrum]

I'm not fond of your attitude towards the volunteer who helped you

I believe you totally misunderstood redrum’s response. I don’t see anything in his posts that are anything but appreciative of all the help he has received. I suppose something is getting lost in translation.

Indeed, I think there is some misunderstanding here. Perhaps it was due to this comment:

I really wish app developers would show us automation geeks some love and give us keyboard access to everything, and not force us to learn about anti-aliasing, etc.

@Seven0528 that was not directed at you. That was just me venting frustration at the app developers. You did not force me to learn about anti-aliasing and these other details I'd have never bothered with if it was up to me. That was the app developers who make so many parts of their apps inaccessible via keyboard. You have given me the tools to work around the limitations of the app, and for that I am very grateful. I will be making use of your code because I need a way to hotkey that button on, and you have demonstrated a solution, so thank you for that.

[Seven0528]

　
Ah, upon reconsideration of the intent you mentioned, I think I misunderstood completely.
Thank you for taking the time to correct my misunderstanding. I apologize. Is it evening there now? Have a good night!

[redrum]

　
Ah, upon reconsideration of the intent you mentioned, I think I misunderstood completely.
Thank you for taking the time to correct my misunderstanding. I apologize. Is it evening there now? Have a good night!

No worries. We can deal with the occasional misunderstanding that comes with the gift of easy international communication that the internet has given us!

[redrum]

When I ran it, I found that the GUI windows were much blurrier than expected.
Just to be sure, I checked the value, and DPI_AWARENESS_CONTEXT was UNAWARE.
In this scenario, if the monitor scaling isn't at 100%, the drawn window itself stretches according to the ratio.
This makes it challenging for AHK, which uses SYSTEM_AWARE, to perform image searches.
While it's possible to adjust AHK to UNAWARE to attempt to resolve this, it's not a method I particularly prefer.

If it matters, I'm using a 4k (3840x2160) monitor at native resolution, no scaling.

The left side is the window captured (considering DPI) with Gdip, and the right side is the actual window.
It demonstrates that the graphics drawn are smaller in reality. Therefore, when performing image searches, using the image obtained through PrintWindow, which is crisper than the actual window considering the enlargement and anti-aliasing, is preferable.

I think I understand. The screenshot of the actual window that I cropped the button image from is blurrier than what I'd get from PrintWindow. And that the sharper image from PrintWindow would more easily match the blurrier button on the screen than the blurrier image taken from a screenshot.

Looking at the image you obtained from PrintWindow, wow, that is definitely sharper. Now I can see the yellow that @boiler was referring to.

The script does download the 2 files. But those are low resolution images, similar to the ones I cropped from screenshots. I guess I don't understand why you are using these when you have the much sharper image you obtained using PrintWindow.

Please activate the Quantower program and press F5. Refer to the link below for the necessary libraries.

I downloaded both libraries, and I guess I included them correctly, as the script does run.

I have Quantower activated, meaning it is the active window. When I press F5, I get the following error:

Screenshot 2024-04-12 211038.png (18.04 KiB) Viewed 139 times

[redrum]

　Well, if you can determine the location of a button with UIA, the story becomes easier.
In this case, not only the position but also the width and height values can be determined, so you can differentiate the toggle state just by searching for a 1-pixel background value.

That's what I tried here: viewtopic.php?p=567621#p567621

I don't understand why that doesn't work. WindowSpy can see the color change. Why can't PixelGetColor()?

[redrum]

　Well, if you can determine the location of a button with UIA, the story becomes easier.
In this case, not only the position but also the width and height values can be determined, so you can differentiate the toggle state just by searching for a 1-pixel background value.

That's what I tried here: viewtopic.php?p=567621#p567621

I don't understand why that doesn't work. WindowSpy can see the color change. Why can't PixelGetColor()?

On a similar note, I took a screenshot of desktop, then in MS Paint, cropped just the lower edge of the toggle button (with toggle on). That gave me a little strip that is all color 00566C. But that strip matches toggle OFF at the lowest *N at which it will match toggle ON, which is *53.

If the button is aliased, then why can I crop a strip of it that is pure 00566C? But if the button isn't aliased, then imagesearch should easily be able to differentiate between pure 00566C and a very different color. So confusing!

[Seven0528]

　Wow, this problem is confusing even though I know it. It seems quite difficult.

I guess I don't understand why you are using these when you have the much sharper image you obtained using PrintWindow.

Because regardless of the monitor's resolution or DPI scaling, the rendering window images for windows using UNAWARE are always internally consistent.
This means that image searches use images from a pre-display stage rather than what's actually shown on the monitor.
(This characteristic allows image searching even when the window is covered by other windows.)

In theory, that's how it works, but honestly, I'm not entirely sure.
Windows using UNAWARE are quite rare. (They tend to forcibly expand, appearing blurry.)
Even if they exist, they're mostly found in very old programs.
Personally, I have very little experience dealing with this type of window.

I have Quantower activated, meaning it is the active window. When I press F5, I get the following error:

Surprisingly, the ImagePut library has been updated in real-time...
It seems the creator keeps making continuous adjustments, but aside from that, it seems you've installed a slightly older version.
Curious, I checked version 1.9, but couldn't find the 'variation' parameter in that version.
It seems you might need to update to the latest version (1.10).

However, even if everything seems perfect, there are still existing issues.
I failed to retrieve the coordinates of a button through ImagePut. More precisely, it returns strange values.
I'm unsure if this is an issue with my computer or if it works fine on other computers...
The C code is hardcoded, making it difficult for me to modify at my level.

[Seven0528]

If the button is aliased, then why can I crop a strip of it that is pure 00566C?

　Other elements besides the button also use the color value 00566C. That might be the reason for the confusion.
Let me be absolutely clear: if the capture was truly perfect, then even with a Variation value of 0, it should still match.
There might be some slight variation permissible, but 53 is excessively high. (Considering the color range is only from 0 to 255.)

[Seven0528]

　I'm honestly not sure. Have you checked elements like CoordMode when using the built-in ImageSearch command?
ImageSearch seems quite vulnerable in a multi-monitor environment, and pixel searching is similarly affected. Are you in a multi-monitor environment?
Honestly, I'm not certain. It might be worth trying to change AHK itself to UNAWARE using SetThreadDpiAwarenessContext, although I'm not sure if that would make a difference. Since you've successfully captured the window cleanly, trying Gdip_ImageSearch instead of ImagePut could be a solution. (But I've only seen it in v1; I haven't encountered code translated into v2.)
I mentioned experiences involving DPI_AWARENESS_CONTEXT and image searching between different processes in the past, but understanding DPI-related issues can be quite challenging.
(I'm not fully knowledgeable myself and heavily rely on experience. Even looking at the official Microsoft documentation, some information is missing.)
If not, posting a separate question on how to determine the match between two pBitmaps could also be a viable option.

If it's a single monitor setup, internal ImageSearch could still be a viable option.
Try determining whether success is achievable with very minimal variation values (5 or less), within a very narrow range, using small images.
In this case, capturing the image rendered on the actual monitor with anti-aliasing applied would be necessary for use.
Additionally, it's necessary to verify whether the obtained coordinates are indeed the intended positions. This is to determine if the search has been conducted in the wrong location. Pay attention to CoordMode for this as well.

[redrum]

@Seven0528 I'll read your latest posts tomorrow and try to absorb them and respond (past my bedtime now). Since you have Quantower, something that may spark an idea - if you right click in the chart > Settings > Hotkeys > scroll down to Trading > Toggle keyboard trading - so the app does allow for a hotkey to toggle keyboard trading (the equivalent of mouse clicking the button). Of course since it's a toggle hotkey not a pair of "On" and "Off" hotkeys, it still requires determining the current state. But thought I'd mention it if that sparks an idea for a workaround.

Edit: while mulling this over in bed, one potential workaround is, after making Quantower the active window (which I do with an AHK hotkey), use one of the app's order quantity hotkeys to select an order quantity that I won't use (e.g., 7), then use imagesearch to see if there's a match to the image of 7 in the order entry panel. If true, then keyboard toggle must be set to ON, so revert order quantity to some default. If false, then keyboard toggle must be set to OFF, so toggle it to ON. Kind of a convoluted, inelegant solution, but a potential solution nonetheless. This also assumes that imagesearch won't run into similar issues as it ran into with matching the toggle button.

[redrum]

I have Quantower activated, meaning it is the active window. When I press F5, I get the following error:

Surprisingly, the ImagePut library has been updated in real-time...
It seems the creator keeps making continuous adjustments, but aside from that, it seems you've installed a slightly older version.
Curious, I checked version 1.9, but couldn't find the 'variation' parameter in that version.
It seems you might need to update to the latest version (1.10).

I definitely downloaded the latest. Checked my downloads folder: ImagePut.v1.10.AutoHotkey.v2.zip
And when I look at the ImagePut.ahk file that I placed in my AHKv2\Lib folder:

image.png

(9.75 KiB) Downloaded 44 times

[Seven0528]

　ImagePut tends to receive frequent updates,
but there's a tendency for the version information in the code comments not to be updated properly.
With AutoHotkey, there are several folders that can be used as Lib, so I speculate that there might be a script in one of those folders with higher priority.

Here, there are 8736 lines, 380805 characters, and not a single line of code included with #Include.
Would you like to test it out? (I've attached the license terms to the code.)
GetQuantowerKeybdBtnToggleState
Tutorial.1_GetQuantowerKeybdBtnToggleState-1.0.1-ahk2.ahk