Best way to troubleshoot Gdip_ImageSearch? Topic is solved

Get help with using AutoHotkey and its commands and hotkeys
doubledave22
Posts: 42
Joined: 08 Jun 2019, 17:36

Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 07:23

Ultimately what I am trying to do is find a portion of an image on a part of a window that can be visible or behind other windows. I have read the many threads on this and have gotten very far with the code below. The saved .bmp (is this the best format for imagesearch?) is a small white square and this only appears at certain times on the window in question in the area of question.

The issue is this works beautifully on my Windows 7 system but when I switch it over to my Windows 10 it fails to find the image. What are some possible things to look for when trying to debug the issue? Both systems are 64bit and I have the 64bit latest gdip library. In the below code all I am able to really check is if the positions, path, and inputs are correct (I believe they are?). The return is just always 0 (its not a negative) meaning the image is being searched properly but cannot be found. I tried using color variation but anything above 50 was always found (on my Win10 system) and anything below was never found so some strangeness happening there.

What can I do to work through problems like this?

note: I am not using Winexist("A") in my real code this is just an example.

Code: Select all


#include *i Gdip_All.ahk 

If !pToken := Gdip_Startup()
{
  MsgBox, 48, gdiplus error!, Gdiplus failed to start. Please ensure you have gdiplus on your system
  ExitApp
}

return

^r::

hwnd := WinExist("A")

wingetpos, X, Y, W, H, % "ahk_id " hwnd 

X1 := W - 150
Y1 := Y + 75
X2 := W - 10
Y2 := Y + 150

vPathN := A_Desktop "\MyHaystackImage.bmp"

pBitmap := Gdip_BitmapFromHWND(hwnd)
pNeedle := Gdip_CreateBitmapFromFile(vPathN)

vRet := Gdip_ImageSearch(pBitmap,pNeedle,vPosXY,X1,Y1,X2,Y2)

vPos := InStr(vPosXY, ",")
vPosX := SubStr(vPosXY, 1, vPos-1)
vPosY := SubStr(vPosXY, vPos+1)

Gdip_DisposeImage(pBitmap)
Gdip_DisposeImage(pNeedle)

MsgBox, % vRet " (" vPosXY ")"

return
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 07:50

Is the Win10 machine using a display scaling factor other than 100%?
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 07:58

To help isolate the issue, you should verify that you can find the image using regular ImageSearch when the window is active.
swagfag
Posts: 3972
Joined: 11 Jan 2017, 17:59

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 09:10

have u verified whether u actually get an image back when u run ...FromHWND()? ure probably getting just a black screen. u probably had the DWM disabled on win7, so ur captures were working, but on win10 it cant be disabled. if are, in fact, getting an image back, then dump it to disk, load both images in an image editor, overlay the needle ontop of the haystack(where u think it ought to be found), diff both images. if there are no differences, then the problem is somewhere in ur code(bad coordinates, etc). if there are differences, ur needle image is bad
doubledave22
Posts: 42
Joined: 08 Jun 2019, 17:36

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 10:22

swagfag wrote:
09 Aug 2020, 09:10
have u verified whether u actually get an image back when u run ...FromHWND()? ure probably getting just a black screen. u probably had the DWM disabled on win7, so ur captures were working, but on win10 it cant be disabled. if are, in fact, getting an image back, then dump it to disk, load both images in an image editor, overlay the needle ontop of the haystack(where u think it ought to be found), diff both images. if there are no differences, then the problem is somewhere in ur code(bad coordinates, etc). if there are differences, ur needle image is bad
This is so cool. I will totally try it if my next steps fail. Boiler made a great point and looks like my coordinates are messed up (regular image search revealed this). Will report back, thanks guys
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?  Topic is solved

09 Aug 2020, 10:49

I have had a problem like swagfag has described where I would use Gdip_BitmapFromHWND() and only get a black image depending on the window (Chrome being one of those windows). As malcev pointed out to me, you can change the call to the PrintWindow() function that Gdip_BitmapFromHWND() uses so it can capture images from windows like that which use hardware acceleration for their graphics rendering. In your Gdip_All.ahk file, find the Gdip_BitmapFromHWND() function definition, and if its call to PrintWindow() is like this:

Code: Select all

PrintWindow(hwnd, hdc)
change it to this:

Code: Select all

PrintWindow(hwnd, hdc, "0x2") ; 0x2 = PW_RENDERFULLCONTENT
Once I did that, it was able to grab images from windows like Chrome and others that use hardware acceleration.
doubledave22
Posts: 42
Joined: 08 Jun 2019, 17:36

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 12:15

boiler wrote:
09 Aug 2020, 10:49
I have had a problem like swagfag has described where I would use Gdip_BitmapFromHWND() and only get a black image depending on the window (Chrome being one of those windows). As malcev pointed out to me, you can change the call to the PrintWindow() function that Gdip_BitmapFromHWND() uses so it can capture images from windows like that which use hardware acceleration for their graphics rendering. In your Gdip_All.ahk file, find the Gdip_BitmapFromHWND() function definition, and if its call to PrintWindow() is like this:

Code: Select all

PrintWindow(hwnd, hdc)
change it to this:

Code: Select all

PrintWindow(hwnd, hdc, "0x2") ; 0x2 = PW_RENDERFULLCONTENT
Once I did that, it was able to grab images from windows like Chrome and others that use hardware acceleration.
The window was definitely using some acceleration for the graphics. I never would have solved this on my own, thanks guys!!

I didn't have a chance to run Gdip_SaveBitmapToFile to check if the file was black but given that this worked right away I am going to assume it was.

Can I leave the Gdip_BitmapFromHWND() this way for other non accelerated captures or do I need to double up the function or add an argument to toggle it?
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 12:37

You can leave it as is, and it will work for other windows as well.
doubledave22
Posts: 42
Joined: 08 Jun 2019, 17:36

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 12:50

Good to know, however upon making that change the Win7 system no longer picked up the image. Toggling the "0x2" back off worked right away.

I wrote in an argument to Gdip_BitmapFromHWND() to toggle this "0x2" add-on in this case which is working unless you know of some other reason why adding this would stop a capture from working properly.
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 12:55

No, I’m surprised to hear that. Thanks for pointing it out.
swagfag
Posts: 3972
Joined: 11 Jan 2017, 17:59

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 12:57

because the flag doesn't exist on pre 8.0 systems (iirc)
doubledave22
Posts: 42
Joined: 08 Jun 2019, 17:36

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 13:03

While I have you guys, what types of adjustments are needed if I do run into a user who's changed the scale and layout in windows to say 150%. I know you mentioned that in the beginning of the thread boiler, just curious because I am bound to run into it.
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 13:11

If you're searching for a purely white square, then it can probably handle it. I suggest keeping the reference image as small as possible that would still be unique. The problem is, from what I've seen working with many users of software that required finding a specific image, when the scaling factor is 150% or greater, Windows stretches the image so that it is no longer a pixel-for-pixel representation of the original. It's sort of smudged or smeared, if you will. It looks the same to the naked eye, but if you zoom in, depending on the image, you can see that an ImageSearch would never find it no matter how much variation you allow (unless you allow so much you can also get false positives). It's sort of like how a jpg image can have compression artifacts, although it can be worse. In the cases where those users couldn't or didn't want to reduce their scaling factor, we had to implement a non-ImageSearch-based solution to identifying the location of that image.
doubledave22
Posts: 42
Joined: 08 Jun 2019, 17:36

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 13:16

Could I increase my own scaling to 150% then recapture the same small image, detect if they are zoomed using a_screendpi/96 and use the zoomed image in those cases for comparisons?
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 13:20

You can try, but in my experience that won't work, and I tried several variations on that theme. Even on the same system, if you drag the window over a little bit, the image will be rendered differently when at 150% or higher scaling. The Windows folks decided that pixel-for-pixel repeatable graphics isn't important. It just needs to look visually correct.
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 13:34

Using this instead of the current function call should allow it to handle the Windows version correctly:

Code: Select all

PrintWindow(hwnd, hdc, (SubStr(A_OsVersion, 1, (InStr(A_OsVersion, ".") - 1)) >= 8) ? 2 : 0)
doubledave22
Posts: 42
Joined: 08 Jun 2019, 17:36

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 13:56

boiler wrote:
09 Aug 2020, 13:34
Using this instead of the current function call should allow it to handle the Windows version correctly:

Code: Select all

PrintWindow(hwnd, hdc, (SubStr(A_OsVersion, 1, (InStr(A_OsVersion, ".") - 1)) >= 8) ? 2 : 0)
how does this work?

Code: Select all

msgbox, % (SubStr(A_OsVersion, 1, (InStr(A_OsVersion, ".") - 1)) >= 8) ? 2 : 0
seems to return 2 on both my Win7 and my Win10 systems. I trust your work i just wanted to know how it works and this seemed odd (havent tested it in the function yet)
User avatar
boiler
Posts: 5909
Joined: 21 Dec 2014, 02:44

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 14:09

My bad. I assumed incorrectly that A_OsVersion would return 7.xx.xx just as it returns 10.xx.xx for Win10. But I see it doesn't now, and this will extract the number. Use these two lines in place of the one line:

Code: Select all

RegExMatch(A_OsVersion, "\d+", Version)
PrintWindow(hwnd, hdc, Version >= 8 ? 2 : 0)
doubledave22
Posts: 42
Joined: 08 Jun 2019, 17:36

Re: Best way to troubleshoot Gdip_ImageSearch?

09 Aug 2020, 14:14

Simple enough, glad I'm rewarded for not just copy/pasting code like i do 99.99% of the time.

Thanks for all your help past couple days.

Return to “Ask For Help”

Who is online

Users browsing this forum: Google [Bot] and 35 guests