"Magic wand" to get X1 Y1 X2 Y2

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
viniam
Posts: 5
Joined: 18 Oct 2021, 20:20

"Magic wand" to get X1 Y1 X2 Y2

Post by viniam » 18 Oct 2021, 20:43

Hello there, totally new to AHK macro but I have an idea, if you are able to at least tell me how hard it is to make it possible, it would be incredible.

My problem is that I need to make a rectangular frame above an speech bubble, I want to:
• Click any part white part inside the bubble;
• AHK macro detects it's size (just like Photoshop magic wand);
• After detection, move mouse to the X1Y1, X1Y1 being the upper left part of the bubble;
• Drag to X2Y2 position, X2Y2 being the lower right part of the bubble.

Professional graphical example of my need:
Untitled-1.png
Untitled-1.png (120.36 KiB) Viewed 532 times
Is this achievable with AHK or would I need to venture deeper into image recognition, like python opencv?
Thanks in advance.

User avatar
Hellbent
Posts: 2102
Joined: 23 Sep 2017, 13:34

Re: "Magic wand" to get X1 Y1 X2 Y2

Post by Hellbent » 19 Oct 2021, 07:04

***Untested Concept***

A simple pixel search should do the trick here.

How To:

1. Set up a pixel search (or read the pixel data from the image bitmap, the method of searching for the pixels doesn't really matter)
that will search from the top left to the bottom right looking for your range of trigger colors.


2. When the first position is found (ErrorLevel = 0), mark that as your starting point and apply any offset that you want (when you draw your rectangle do you want it to be exactly on target or offset a bit).

3. Adjust the search area to move over 1px to the right and change the size to something much smaller. The idea is that it should look for your color within a small enough area so that it is still likely the same box but is able to not falsely think it is at the right edge when there is text etc in the search area.
(lets say 30px by 30px for example).

4. Once you no longer find your color in your search area ( ErrorLevel = 1 ), mark the starting position of the search area as your ending top right point

5. Go back to start and repeat steps 3 and 4 but this time move down 1px each time you get a hit.

6. You should now have your starting point (aka top left ), your width ( top right x - top left x = width), and your height ( bottom left y - top left y = height )

7. Apply your offsets and draw your rectangle or w/e


As you should be able to see, the above search pattern wouldn't work in your case as you would need to do something more like this.


find the center of your target and then search up until null, search down until null, search left until null, and search right until null.
ta da you have your box.

User avatar
Hellbent
Posts: 2102
Joined: 23 Sep 2017, 13:34

Re: "Magic wand" to get X1 Y1 X2 Y2

Post by Hellbent » 19 Oct 2021, 07:19

You could also do edge detection by searching the other directions once you have you first hit (full image ) and then failure (1px by 1px).

You would just need to create a pattern that knows where it should be in the shape so that the search area is adjusted correctly (1px by 1px).

this would trace out your shape exactly and then you can look for your x1 y1 x2 y2 in it as the highest and lowest values on the x and y axis.

User avatar
Hellbent
Posts: 2102
Joined: 23 Sep 2017, 13:34

Re: "Magic wand" to get X1 Y1 X2 Y2

Post by Hellbent » 19 Oct 2021, 09:12

Here is a very simple POC search pattern( aka algorithm )

Code: Select all

;****************************************************************************************************************************************************************************
#Include <My Altered GDIP lib> ;GDI+ Lib
#Include <PopUpWindow Class> ;https://www.autohotkey.com/boards/viewtopic.php?f=6&t=94961
;****************************************************************************************************************************************************************************
#SingleInstance, Force
SetBatchlines, -1
#NoEnv
Gdip_Startup()
Gui1 := New PopUpWindow( { WindowName: "1" , WindowOptions: " -DPIScale +AlwaysOnTop " , WindowSmoothing: 2 , X: 0 , Y: 0 , W: A_ScreenWidth , H: A_ScreenHeight } )
return
GuiClose:
GuiContextMenu:
*ESC::ExitApp

NumPad3::PopUpWindow.Helper()

Numpad4::
	SoundBeep, 555
	CoordMode, Mouse, Screen
	CoordMode, Pixel, Screen
	MouseGetPos, sx, sy
	PixelGetColor, SColor, sx, sy, RGB 
	cx := sx, cy := sy
	TrayTip,, Searching left
	Loop, {
		cx -= 30
		PixelGetColor, CColor, cx , cy, RGB
		if( SColor != CColor ){
			nx := cx
			Loop, 30 	{
				nx++
				PixelGetColor, CColor, nx , cy, RGB
				if( SColor = CColor ){
					LeftX := nx
					break 2
				}
			}
			LeftX := cx
			break
		}
	}
	TrayTip,, Searching Right
		
	;Search Right
	cx := sx, cy := sy
	Loop, {
		cx += 30
		PixelGetColor, CColor, cx , cy, RGB
		if( SColor != CColor ){
			nx := cx
			Loop, 30	{
				nx--
				PixelGetColor, CColor, nx , cy, RGB
				if( SColor = CColor ){
					RightX := nx
					break 2
				}
			}
			RightX := Cx
			break
		}
	}
	TrayTip,, Searching Up
	;Search Up
	cx := sx, cy := sy
	Loop, {
		cy -= 30
		PixelGetColor, CColor, cx , cy, RGB
		if( SColor != CColor ){
			ny := cy
			Loop, 30	{
				ny++
				PixelGetColor, CColor, cx , ny, RGB
				if( SColor = CColor ){
					TopY := ny
					break 2
				}
			}
			TopY := Cy
			break
		}
	}
	;Search Down
	TrayTip,, Searching Down
	cx := sx, cy := sy
	Loop, {
		cy += 30
		PixelGetColor, CColor, cx , cy, RGB
		if( SColor != CColor ){
			ny := cy
			Loop, 30	{
				ny--
				PixelGetColor, CColor, cx , ny, RGB
				if( SColor = CColor ){
					BottomY := ny
					break 2
				}
			}
			BottomY := cy
			break
		}
	}
	SoundBeep, 999
	Margin := 20
	Positions := { X: LeftX - Margin , Y: TopY - Margin, W: ( RightX - LeftX ) + ( 2 * Margin ) , H: ( BottomY - TopY ) + ( 2 * Margin ) }
	Gui1.ClearWindow()
	Pen := Gdip_CreatePen( "0xFF00FF00" , 5 ) , Gdip_DrawRectangle( Gui1.G , pen , Positions.X , Positions.Y , Positions.W , Positions.H ) , Gdip_DeletePen( Pen )
	Gui1.UpdateWindow()
	Gui1.ShowWindow()
	return
	

Temp (1).gif
Temp (1).gif (158.77 KiB) Viewed 429 times

Post Reply

Return to “Ask for Help (v1)”