FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 15 Nov 2023, 15:28

@trialAdd i think that’s normal for 200 loops. But i’m not totally sure. There is a wait() function in FINDTEXT which I use instead of (loop) and i can hear my computer ramp up

trialAdd
Posts: 10
Joined: 14 Nov 2023, 04:36

Re: FindText - Capture screen image into text and then find it

Post by trialAdd » 16 Nov 2023, 01:14

if loop is the main issue then adding sleep would have reduced cpu usage , but spike of cpu usage is still staying high with findtext

sakamoto
Posts: 7
Joined: 03 Oct 2022, 19:57

Re: FindText - Capture screen image into text and then find it

Post by sakamoto » 25 Nov 2023, 11:12

Hi all, i'm trying to use FINDTEXT to work only in a specific window, is there a way to do it?
i want to split my screen, let FINDTEXT working in the right window and i keep using the left window.
But when i do this, FINDTEXT 'steals' my mouse pointer when running .Click().
Furthermore I think FINDTEXT returns X and Y based on screen coordinates and not on the not active window i want to.
Thanks!

feiyue
Posts: 358
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 18 Dec 2023, 12:24

Updated to 9.4 version - 2023/12/18 :dance: :beer:
1. Add: The image processing window has added a screenshot page,
which makes it more convenient to obtain colors when using multi-color search mode.
2. Add: EditScroll() function is added to Locate the specified row in the edit box.

MancioDellaVega
Posts: 84
Joined: 16 May 2020, 12:27
Location: Italy

Re: FindText - Capture screen image into text and then find it

Post by MancioDellaVega » 25 Feb 2024, 02:59

@feiyue
Hi bro,
I am a newbie to Ahk and enthusiastic about your function and I believe it is a must-have for all users of Ahk. Your function was the first one I saw when I started learning ahk and I have been using it since version 6.0, thank you very much.
I use your function with a function of mine that integrates a kind of debug mode, that helps me find out if something goes wrong in the search... I have to complete it, but it is only one idea...

Code: Select all


DrawRectangle(x1:=0, y1:=0, w1:=0, h1:=0, n:=1, bw:=3, cl:="ff0000") 
{ 
/*
    flyingDman, RRR
	Disegna la cornice di un rettangolo con l'uso di gui
*/

Global FirstCall

    if ((w1=0) && (h1=0)) {
        Gui % (n ": Destroy", FirstCall:=[])
        Return
    }
    FirstCall? "": FirstCall:=[], FirstCall[n]? "": (FirstCall[n]:=[], FirstCall[n].1:=1)
  , w1<0? (w1:=Abs(w1), x1:=x1-w1): "", h1<0? (h1:=Abs(h1), y1:=y1-h1): ""
    Gui % n ": +LastFound"
    w2:= w1 - bw, h2:= h1 - bw
    WinSet, Region, 0-0 %w1%-0 %w1%-%h1% 0-%h1% 0-0 %bw%-%bw% %w2%-%bw% %w2%-%h2% %bw%-%h2% %bw%-%bw%
    If (FirstCall[n].1) {
        Gui % n ": -Caption +AlwaysOnTop +ToolWindow -DPIScale +HWNDhwnd +e0x8000088"
        Gui % n ": Color", % cl
        FirstCall[n].1:= 0
    }   
	Gui % n ": Show", NA x%x1% y%y1% w%w1% h%h1% ; performs better then WinMove over some controls
	
}

CercaTesto(Text, ByRef Ok, Premi:=1, MaxCicliAttesa:=5, ShowMessage:=1, Param*)
{
/*
		Text = stringa da ricercare data dalla funzione Findtext
		Premi =  0  NON preme e NON sposta il mouse
				 1  Clicca e sposta il mouse sul testo trovato
				-1  Mostra Mousetip e sposta il mouse alle coordinate trovate
				-2  Mostra solo Mousetip sul testo trovato
				
		MaxCicliAttesa = Massima Attesa prima di dare la ricerca fallita
		
		ShowMessage = 0|1 : Mostra messaggio di Search
					  -1  : Mostra Area di ricerca e messaggio di Search
		
		Return CercaTesto = 0|1 Testo trovato o non trovato
		
		OK = Parametri della immagine trovata
		
========================================================================================================================================

	    If the return variable is set to 'ok', ok[1] is the first result found.
	    ok[1].id is the comment text, which is included in the <> of its parameter.
	    ok[1].1 is the X coordinate of the upper left corner of the found image
	    ok[1].2 is the Y coordinate of the upper left corner of the found image
	    ok[1].3 is the width of the found image
	    ok[1].4 is the height of the found image
	    ok[1].x <==> ok[1].1+ok[1].3//2 ( is the Center X coordinate of the found image )
	    ok[1].y <==> ok[1].2+ok[1].4//2 ( is the Center Y coordinate of the found image )


  returnArray:=FindText(
      OutputX:="" --> The name of the variable used to store the returned X coordinate
    , OutputY:="" --> The name of the variable used to store the returned Y coordinate
    , X1:=0 --> the search scope's upper left corner X coordinates
    , Y1:=0 --> the search scope's upper left corner Y coordinates
    , X2:=0 --> the search scope's lower right corner X coordinates
    , Y2:=0 --> the search scope's lower right corner Y coordinates
    , err1:=0 --> Fault tolerance percentage of text       (0.1=10%)
    , err0:=0 --> Fault tolerance percentage of background (0.1=10%)
    , Text:="" --> can be a lot of text parsed into images, separated by '|'

    , ScreenShot:=1 --> if the value is 0, the last screenshot will be used
    , FindAll:=1 --> if the value is 0, Just find one result and return
    , JoinText:=0 --> if you want to combine find, it can be 1, or an array of words to find
    , offsetX:=20 --> Set the max text offset (X) for combination lookup
    , offsetY:=10 --> Set the max text offset (Y) for combination lookup
    , dir:=1 --> Nine directions for searching: up, down, left, right and center
    , zoomW:=1 --> Zoom percentage of image width  (1.0=100%)
    , zoomH:=1 --> Zoom percentage of image height (1.0=100%)
  )

  The function returns an Array containing all lookup results,
  any result is a object with the following values:
  {1:X, 2:Y, 3:W, 4:H, x:X+W//2, y:Y+H//2, id:Comment}
  If no image is found, the function returns 0.
  All coordinates are relative to Screen, colors are in RGB format

  If the return variable is set to 'ok', ok[1] is the first result found.
  ok[1].id is the comment text, which is included in the <> of its parameter.
  ok[1].1 is the X coordinate of the upper left corner of the found image
  ok[1].2 is the Y coordinate of the upper left corner of the found image
  ok[1].3 is the width of the found image
  ok[1].4 is the height of the found image
  ok[1].x <==> ok[1].1+ok[1].3//2 ( is the Center X coordinate of the found image ),
  ok[1].y <==> ok[1].2+ok[1].4//2 ( is the Center Y coordinate of the found image ),

  If OutputX is equal to 'wait' or 'wait1'(appear), or 'wait0'(disappear)
  it means using a loop to wait for the image to appear or disappear.
  the OutputY is the wait time in seconds, time less than 0 means infinite waiting
  Timeout means failure, return 0, and return other values means success
  If you want to appear and the image is found, return the found array object
  If you want to disappear and the image cannot be found, return 1
  Example 1: FindText(X:='wait', Y:=3, 0,0,0,0,0,0,Text)   ; Wait 3 seconds for appear
  Example 2: FindText(X:='wait0', Y:=-1, 0,0,0,0,0,0,Text) ; Wait indefinitely for disappear

*/
	CoordMode, Tooltip, Screen    

	;~ MsgBox, %   "Count= " . Param.Count() 
			  ;~ . "`nMaxIndex= " . Param.MaxIndex() 
		      ;~ . "`nMinIndex= " . Param.MinIndex() 
		      ;~ . "`nLength= " . Param.length() 


	If (!Param.Count()) {
	
		; Parametri di default della zona di ricerca
		;tollerranza 0 = 0.000001
		
		Param := { x1:0
				  ,y1:0 
				  ,x2:1920
				  ,y2:1080 
				  ,tolleranza:0.1 
				  ,Appear:1}
	
	}else if (Param.Count() < 6) {
	
		Param.Appear := 1
		
	}
	
	pos1:=InStr(Text,"<")
	pos2:=InStr(Text,">")
    Commento:=SubStr(Text, Pos1+1,pos2-pos1-1)
	
	x_tooltip:=Param.x1 
	y_tooltip:= (Param.y1>29) ? (Param.y1-29) : 0
	
    ;MouseMove,0,50,0,R
	
    if(ShowMessage = -1 ) 
	   DrawRectangle(Param.x1, Param.y1, Param.x2-Param.x1, Param.y2-Param.y1, 1, 3, "red") ; Mostra Area di Ricerca
	;ShowArrayInGui(Param)
	
	if(Param.Appear = 1 ) {
	
					Loop {
					
							Switch ShowMessage
							{
							  Case 0: ; NON Mostra niente
							  
							  Case 1,-1: ; Mostra messaggi di stato
										
										;MsgBox,% Param.x1 " " Param.y1
										
										ToolTip,% "Search <" Commento ">... " A_Index "/" MaxCicliAttesa,% x_tooltip,% y_tooltip,20
										;Sleep,5000
							  
							  Default:
							   
							}
						  
							;ok:=FindText(Param.x1, Param.y1, Param.x2, Param.y2, Param.tolleranza, Param.tolleranza, Text)
							ok:=FindText(OutputX,OutputY,Param.x1, Param.y1, Param.x2, Param.y2, Param.tolleranza, Param.tolleranza, Text)
							
							Sleep, 1000
						
					} until (ok) or (A_Index = MaxCicliAttesa)
					
					if(ShowMessage=-1) 
						DrawRectangle(0, 0, 0, 0, 1, 3, "red") ; Cancella Area di Ricerca
					
					if(ShowMessage!=0) 
					  ToolTip,,,,20 ; Cancella Tooltip di ricerca
					
					if (ok) {
							  
								Switch Premi
								{
								  Case -2: ; Mostra Mousetip alle coordinate trovate
										  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.id, X+=W//2, Y+=H//2
										  FindText().MouseTip(ok[1].x, ok[1].y)
								  
								  Case -1: ; Mostra Mousetip e sposta il mouse alle coordinate trovate
										  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.id, X+=W//2, Y+=H//2
										  FindText().MouseTip(ok[1].x, ok[1].y)
										  Click, %X%, %Y%, 0
								  
								  Case 0: ; NON preme e non sposta il mouse
								  
								  Case 1: ; Clicca e sposta il mouse sulle coordinate trovate
										  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.id, X+=W//2, Y+=H//2
										  Click, %X%, %Y%
										  Sleep, 1000
								  
								  Default:
								  
								}
								
								return 1
							  
					}else{
						 
								if(ShowMessage!=0) {
									loop,1
									  SoundBeep
									
									ToolTip,% "<" Commento "> NON TROVATO !!!",% x_tooltip,% y_tooltip,20
									Sleep, 3000
									ToolTip,,,,20
								}
								;MsgBox, % Commento " [ NON trovato!!! ]"
								;ExitApp
								
								return 0
						
					}
					
	}else if(Param.Appear = 0 ) {
	
	
	
	
	}else{
	
	}						

}


Courses on AutoHotkey

maxkill
Posts: 158
Joined: 11 Apr 2016, 13:03

Re: FindText - Capture screen image into text and then find it

Post by maxkill » 29 Feb 2024, 21:30

feiyue wrote:
18 Dec 2023, 12:24
Updated to 9.4 version - 2023/12/18 :dance: :beer:
1. Add: The image processing window has added a screenshot page,
which makes it more convenient to obtain colors when using multi-color search mode.
2. Add: EditScroll() function is added to Locate the specified row in the edit box.
Cool! Will this work with the normal older code in this thread or need to change things? How do I search an object like normal and also inlcude it's color?

I also have a q can't seem to get it to work since I normally only search around the mousepointer.

Q is, if I want search an area say below mouse and to right but insert a gap that is not to be searched between the mouse and the search area, how do I do this?

I tried using the getrange thinking I could search in this rectangle being drawn. But to no avail.

Code: Select all

Mousegetpos, xx, yy

;t1:=A_TickCount, Text:=xx:=yy:=""

Text:="|<>*knajsfdRFjsj3"

if (ok:=FindText(xx+15, yy, 9, 2125, 3803, 2145, 0, 0, Text)) ; 150 can be adjusted by yourself

{
?? ..

kashmirLZ
Posts: 58
Joined: 06 Oct 2022, 23:27

Re: FindText - Capture screen image into text and then find it

Post by kashmirLZ » 24 Mar 2024, 02:23

It seems like the "Add FindText()" checkbox isnt changing the text-area below.

shawn_xwang
Posts: 2
Joined: 22 Mar 2024, 19:07

Re: FindText - Capture screen image into text and then find it

Post by shawn_xwang » 25 Mar 2024, 06:34

Hello all

I tried the bindwindows, but it seems not working when the target windows is not on top, either I use bind_mode 0, 1, 2, 3, 4, none of them is working, what's wrong? I don't get it, please help here, thanks

Code: Select all

#F12::
t1:=A_TickCount, Text:=X:=Y:=""
WinExistStatus:=WinExist("ahk_class SciTEWindow")
FindText().BindWindow(WinExistStatus,0) ; setting as 0, 1, 2, 3, 4 and tried, could not find the text if the windows is not on top
Text:="|<>*151$71.00000000000000000000000000000000000000000000000800000000000E00000000000U000000000010000000000020yBsDNVXsDMQ16QEln34Mllc0Akn1a60n1aE3tVa3AADa38UQn3A6MNnA6T0Va6MAkm6MAW13AAkNVYAkNY3CMMll7AsllDnwkkxVqDkxVU0000200002000000400004000008M0008M00000T0000T0000000000000000000000000000000000000000000000001"
if (ok:=FindText(X, Y, , , , , 0, 0, Text))
{
  FindText().MouseTip(X, Y)
  FindText().Click(ok[1].x, ok[1].y, "L")
}
else
MsgBox, there in no matching image
FindText().BindWindow(0)
Return

feiyue
Posts: 358
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 27 Apr 2024, 06:14

Updated to 9.5 version - 2024/04/27 :beer: :dance:
1. Modify: Improved the machine code for searching from the center spiral to the surrounding area.
2. Modify: The color exclusion mode for <FindMultiColor> <FindColor>
can now exclude multiple unwanted colors. Previously, only one color could be excluded.

feiyue
Posts: 358
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 24 May 2024, 18:52

Updated to 9.6 version - 2024/05/25 :beer: :dance:
1. Modify: X:="wait/wait1/wait0" After running once, X will be cleared without affecting subsequent calls.
2. Modify: The color similarity and color difference binarization mode can mix multiple colors for binarization.
Text:="|<>RRGGBB1@0.8/RRGGBB2-DRDGDB2/... $ ..."
3. Add: ColorBlock() function, which can generate color blocks with specified area and color, and limit the number of matching colors.
4. Add: Add the GuiCreate Class in V1 to make Gui related code compatible with V2 and reduce the difficulty of code maintenance.

ozzii
Posts: 487
Joined: 30 Oct 2013, 06:04

Re: FindText - Capture screen image into text and then find it

Post by ozzii » 25 May 2024, 03:06

Thanks feiyue for the update and keeping the v1 alive

MancioDellaVega
Posts: 84
Joined: 16 May 2020, 12:27
Location: Italy

Re: FindText - Capture screen image into text and then find it

Post by MancioDellaVega » 25 May 2024, 11:53

Is there one place where i can find the old versions of the FindText before the 9.6 version?
Courses on AutoHotkey

User avatar
Chunjee
Posts: 1504
Joined: 18 Apr 2014, 19:05
Contact:

Re: FindText - Capture screen image into text and then find it

Post by Chunjee » 04 Jun 2024, 09:07

Very cool new version I am having fun exploring it :clap:

Well done :thumbup:



I noticed .Floor at the very top of the class and I found it to be an odd name choice

Code: Select all

Floor(i)
{
  if i is number
    return i+0
  else return 0
}

Code: Select all

floor("hello")
; => 0
FindTextClass.Floor("hello")
; => 0
floor(10.00001)
; => 10
FindTextClass.Floor(10.00001)
; => 10.000010


As an experiment I changed all this.Floor to ahk's built-in floor and it seems to work the same but I only have 20 tests or so

feiyue
Posts: 358
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 17 Jun 2024, 22:56

Updated to 9.7 version - 2024/06/18 :beer: :dance:
1. Modify: The use of left and right dilation algorithm greatly improves the success rate of searching for slightly deformed images.
It is recommended to upgrade to this new version.
2. Add: Use FindText().GetTextFromScreen() quickly generates Text to the clipboard.

Post Reply

Return to “Scripts and Functions (v1)”