FindText - Capture screen image into text and then find it

[SteveMylo]

@trialAdd i think that’s normal for 200 loops. But i’m not totally sure. There is a wait() function in FINDTEXT which I use instead of (loop) and i can hear my computer ramp up

[trialAdd]

if loop is the main issue then adding sleep would have reduced cpu usage , but spike of cpu usage is still staying high with findtext

[sakamoto]

Hi all, i'm trying to use FINDTEXT to work only in a specific window, is there a way to do it?
i want to split my screen, let FINDTEXT working in the right window and i keep using the left window.
But when i do this, FINDTEXT 'steals' my mouse pointer when running .Click().
Furthermore I think FINDTEXT returns X and Y based on screen coordinates and not on the not active window i want to.
Thanks!

[feiyue]

Updated to 9.4 version - 2023/12/18
1. Add: The image processing window has added a screenshot page,
which makes it more convenient to obtain colors when using multi-color search mode.
2. Add: EditScroll() function is added to Locate the specified row in the edit box.

[MancioDellaVega]

@feiyue
Hi bro,
I am a newbie to Ahk and enthusiastic about your function and I believe it is a must-have for all users of Ahk. Your function was the first one I saw when I started learning ahk and I have been using it since version 6.0, thank you very much.
I use your function with a function of mine that integrates a kind of debug mode, that helps me find out if something goes wrong in the search... I have to complete it, but it is only one idea...

Code: Select all

DrawRectangle(x1:=0, y1:=0, w1:=0, h1:=0, n:=1, bw:=3, cl:="ff0000") 
{ 
/*
    flyingDman, RRR
	Disegna la cornice di un rettangolo con l'uso di gui
*/

Global FirstCall

    if ((w1=0) && (h1=0)) {
        Gui % (n ": Destroy", FirstCall:=[])
        Return
    }
    FirstCall? "": FirstCall:=[], FirstCall[n]? "": (FirstCall[n]:=[], FirstCall[n].1:=1)
  , w1<0? (w1:=Abs(w1), x1:=x1-w1): "", h1<0? (h1:=Abs(h1), y1:=y1-h1): ""
    Gui % n ": +LastFound"
    w2:= w1 - bw, h2:= h1 - bw
    WinSet, Region, 0-0 %w1%-0 %w1%-%h1% 0-%h1% 0-0 %bw%-%bw% %w2%-%bw% %w2%-%h2% %bw%-%h2% %bw%-%bw%
    If (FirstCall[n].1) {
        Gui % n ": -Caption +AlwaysOnTop +ToolWindow -DPIScale +HWNDhwnd +e0x8000088"
        Gui % n ": Color", % cl
        FirstCall[n].1:= 0
    }   
	Gui % n ": Show", NA x%x1% y%y1% w%w1% h%h1% ; performs better then WinMove over some controls
	
}

CercaTesto(Text, ByRef Ok, Premi:=1, MaxCicliAttesa:=5, ShowMessage:=1, Param*)
{
/*
		Text = stringa da ricercare data dalla funzione Findtext
		Premi =  0  NON preme e NON sposta il mouse
				 1  Clicca e sposta il mouse sul testo trovato
				-1  Mostra Mousetip e sposta il mouse alle coordinate trovate
				-2  Mostra solo Mousetip sul testo trovato
				
		MaxCicliAttesa = Massima Attesa prima di dare la ricerca fallita
		
		ShowMessage = 0|1 : Mostra messaggio di Search
					  -1  : Mostra Area di ricerca e messaggio di Search
		
		Return CercaTesto = 0|1 Testo trovato o non trovato
		
		OK = Parametri della immagine trovata
		
========================================================================================================================================

	    If the return variable is set to 'ok', ok[1] is the first result found.
	    ok[1].id is the comment text, which is included in the <> of its parameter.
	    ok[1].1 is the X coordinate of the upper left corner of the found image
	    ok[1].2 is the Y coordinate of the upper left corner of the found image
	    ok[1].3 is the width of the found image
	    ok[1].4 is the height of the found image
	    ok[1].x <==> ok[1].1+ok[1].3//2 ( is the Center X coordinate of the found image )
	    ok[1].y <==> ok[1].2+ok[1].4//2 ( is the Center Y coordinate of the found image )


  returnArray:=FindText(
      OutputX:="" --> The name of the variable used to store the returned X coordinate
    , OutputY:="" --> The name of the variable used to store the returned Y coordinate
    , X1:=0 --> the search scope's upper left corner X coordinates
    , Y1:=0 --> the search scope's upper left corner Y coordinates
    , X2:=0 --> the search scope's lower right corner X coordinates
    , Y2:=0 --> the search scope's lower right corner Y coordinates
    , err1:=0 --> Fault tolerance percentage of text       (0.1=10%)
    , err0:=0 --> Fault tolerance percentage of background (0.1=10%)
    , Text:="" --> can be a lot of text parsed into images, separated by '|'

    , ScreenShot:=1 --> if the value is 0, the last screenshot will be used
    , FindAll:=1 --> if the value is 0, Just find one result and return
    , JoinText:=0 --> if you want to combine find, it can be 1, or an array of words to find
    , offsetX:=20 --> Set the max text offset (X) for combination lookup
    , offsetY:=10 --> Set the max text offset (Y) for combination lookup
    , dir:=1 --> Nine directions for searching: up, down, left, right and center
    , zoomW:=1 --> Zoom percentage of image width  (1.0=100%)
    , zoomH:=1 --> Zoom percentage of image height (1.0=100%)
  )

  The function returns an Array containing all lookup results,
  any result is a object with the following values:
  {1:X, 2:Y, 3:W, 4:H, x:X+W//2, y:Y+H//2, id:Comment}
  If no image is found, the function returns 0.
  All coordinates are relative to Screen, colors are in RGB format

  If the return variable is set to 'ok', ok[1] is the first result found.
  ok[1].id is the comment text, which is included in the <> of its parameter.
  ok[1].1 is the X coordinate of the upper left corner of the found image
  ok[1].2 is the Y coordinate of the upper left corner of the found image
  ok[1].3 is the width of the found image
  ok[1].4 is the height of the found image
  ok[1].x <==> ok[1].1+ok[1].3//2 ( is the Center X coordinate of the found image ),
  ok[1].y <==> ok[1].2+ok[1].4//2 ( is the Center Y coordinate of the found image ),

  If OutputX is equal to 'wait' or 'wait1'(appear), or 'wait0'(disappear)
  it means using a loop to wait for the image to appear or disappear.
  the OutputY is the wait time in seconds, time less than 0 means infinite waiting
  Timeout means failure, return 0, and return other values means success
  If you want to appear and the image is found, return the found array object
  If you want to disappear and the image cannot be found, return 1
  Example 1: FindText(X:='wait', Y:=3, 0,0,0,0,0,0,Text)   ; Wait 3 seconds for appear
  Example 2: FindText(X:='wait0', Y:=-1, 0,0,0,0,0,0,Text) ; Wait indefinitely for disappear

*/
	CoordMode, Tooltip, Screen    

	;~ MsgBox, %   "Count= " . Param.Count() 
			  ;~ . "`nMaxIndex= " . Param.MaxIndex() 
		      ;~ . "`nMinIndex= " . Param.MinIndex() 
		      ;~ . "`nLength= " . Param.length() 


	If (!Param.Count()) {
	
		; Parametri di default della zona di ricerca
		;tollerranza 0 = 0.000001
		
		Param := { x1:0
				  ,y1:0 
				  ,x2:1920
				  ,y2:1080 
				  ,tolleranza:0.1 
				  ,Appear:1}
	
	}else if (Param.Count() < 6) {
	
		Param.Appear := 1
		
	}
	
	pos1:=InStr(Text,"<")
	pos2:=InStr(Text,">")
    Commento:=SubStr(Text, Pos1+1,pos2-pos1-1)
	
	x_tooltip:=Param.x1 
	y_tooltip:= (Param.y1>29) ? (Param.y1-29) : 0
	
    ;MouseMove,0,50,0,R
	
    if(ShowMessage = -1 ) 
	   DrawRectangle(Param.x1, Param.y1, Param.x2-Param.x1, Param.y2-Param.y1, 1, 3, "red") ; Mostra Area di Ricerca
	;ShowArrayInGui(Param)
	
	if(Param.Appear = 1 ) {
	
					Loop {
					
							Switch ShowMessage
							{
							  Case 0: ; NON Mostra niente
							  
							  Case 1,-1: ; Mostra messaggi di stato
										
										;MsgBox,% Param.x1 " " Param.y1
										
										ToolTip,% "Search <" Commento ">... " A_Index "/" MaxCicliAttesa,% x_tooltip,% y_tooltip,20
										;Sleep,5000
							  
							  Default:
							   
							}
						  
							;ok:=FindText(Param.x1, Param.y1, Param.x2, Param.y2, Param.tolleranza, Param.tolleranza, Text)
							ok:=FindText(OutputX,OutputY,Param.x1, Param.y1, Param.x2, Param.y2, Param.tolleranza, Param.tolleranza, Text)
							
							Sleep, 1000
						
					} until (ok) or (A_Index = MaxCicliAttesa)
					
					if(ShowMessage=-1) 
						DrawRectangle(0, 0, 0, 0, 1, 3, "red") ; Cancella Area di Ricerca
					
					if(ShowMessage!=0) 
					  ToolTip,,,,20 ; Cancella Tooltip di ricerca
					
					if (ok) {
							  
								Switch Premi
								{
								  Case -2: ; Mostra Mousetip alle coordinate trovate
										  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.id, X+=W//2, Y+=H//2
										  FindText().MouseTip(ok[1].x, ok[1].y)
								  
								  Case -1: ; Mostra Mousetip e sposta il mouse alle coordinate trovate
										  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.id, X+=W//2, Y+=H//2
										  FindText().MouseTip(ok[1].x, ok[1].y)
										  Click, %X%, %Y%, 0
								  
								  Case 0: ; NON preme e non sposta il mouse
								  
								  Case 1: ; Clicca e sposta il mouse sulle coordinate trovate
										  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.id, X+=W//2, Y+=H//2
										  Click, %X%, %Y%
										  Sleep, 1000
								  
								  Default:
								  
								}
								
								return 1
							  
					}else{
						 
								if(ShowMessage!=0) {
									loop,1
									  SoundBeep
									
									ToolTip,% "<" Commento "> NON TROVATO !!!",% x_tooltip,% y_tooltip,20
									Sleep, 3000
									ToolTip,,,,20
								}
								;MsgBox, % Commento " [ NON trovato!!! ]"
								;ExitApp
								
								return 0
						
					}
					
	}else if(Param.Appear = 0 ) {
	
	
	
	
	}else{
	
	}						

}

[maxkill]

Updated to 9.4 version - 2023/12/18
1. Add: The image processing window has added a screenshot page,
which makes it more convenient to obtain colors when using multi-color search mode.
2. Add: EditScroll() function is added to Locate the specified row in the edit box.

Cool! Will this work with the normal older code in this thread or need to change things? How do I search an object like normal and also inlcude it's color?

I also have a q can't seem to get it to work since I normally only search around the mousepointer.

Q is, if I want search an area say below mouse and to right but insert a gap that is not to be searched between the mouse and the search area, how do I do this?

I tried using the getrange thinking I could search in this rectangle being drawn. But to no avail.

Code: Select all

Mousegetpos, xx, yy

;t1:=A_TickCount, Text:=xx:=yy:=""

Text:="|<>*knajsfdRFjsj3"

if (ok:=FindText(xx+15, yy, 9, 2125, 3803, 2145, 0, 0, Text)) ; 150 can be adjusted by yourself

{
?? ..

[kashmirLZ]

It seems like the "Add FindText()" checkbox isnt changing the text-area below.

[shawn_xwang]

Hello all

I tried the bindwindows, but it seems not working when the target windows is not on top, either I use bind_mode 0, 1, 2, 3, 4, none of them is working, what's wrong? I don't get it, please help here, thanks

Code: Select all

#F12::
t1:=A_TickCount, Text:=X:=Y:=""
WinExistStatus:=WinExist("ahk_class SciTEWindow")
FindText().BindWindow(WinExistStatus,0) ; setting as 0, 1, 2, 3, 4 and tried, could not find the text if the windows is not on top
Text:="|<>*151$71.00000000000000000000000000000000000000000000000800000000000E00000000000U000000000010000000000020yBsDNVXsDMQ16QEln34Mllc0Akn1a60n1aE3tVa3AADa38UQn3A6MNnA6T0Va6MAkm6MAW13AAkNVYAkNY3CMMll7AsllDnwkkxVqDkxVU0000200002000000400004000008M0008M00000T0000T0000000000000000000000000000000000000000000000001"
if (ok:=FindText(X, Y, , , , , 0, 0, Text))
{
  FindText().MouseTip(X, Y)
  FindText().Click(ok[1].x, ok[1].y, "L")
}
else
MsgBox, there in no matching image
FindText().BindWindow(0)
Return