FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
Shanghei
Posts: 28
Joined: 04 Jan 2021, 22:42

Re: FindText - Capture screen image into text and then find it

10 Jun 2021, 20:01

@feiyue
Quick question, I have added three different words I have monitored in my script and I want my script to execute only if all 3 words appear at same time. Any advice? Thank you!
vava
Posts: 3
Joined: 25 May 2021, 22:08

Re: FindText - Capture screen image into text and then find it

12 Jun 2021, 22:21

@feiyue How to use FindText for active windows, and get the X, Y coordinates of found object in active windows only ? instead of searching the whole desktop windows. Appreciate if anyone can help and share. thank!
braddo
Posts: 6
Joined: 05 Jun 2021, 10:28

Re: FindText - Capture screen image into text and then find it

15 Jun 2021, 07:45

@feiyue , thank you for making the updates! I can now see the interface very well.

However, now there is a different problem - no matter what I attempt to capture, it looks like this (not what I'm trying to capture):
image.png
image.png (130.29 KiB) Viewed 4722 times
I'm running Chrome on Windows, on a 4k monitor. Somehow the capture isn't working properly, or at least I'm not using it properly.
emanate22
Posts: 52
Joined: 11 May 2021, 00:03

Re: FindText - Capture screen image into text and then find it

20 Jun 2021, 01:35

May I ask how to use Findtext to identify four digits numbers?
I made a simple script that recognizes two-digit numbers, but using the same method to recognize four digits and three digits numbers should be very complicated. Is there a simpler way to achieve this?

Code: Select all

SetDefaultMouseSpeed,0
SetKeyDelay,-1
SetMouseDelay,-1
SetStoreCapslockMode, off
SetBatchLines -1 
Process, Priority, , High
#Include <FindText>
B::
MouseGetPos,xpos,ypos
MouseMove,1810,28
SLEEP,1
Text:="|<0>*60$8.ktYwTbtyTbtyD9b3U|<1>*60$7.ztkvTjrvxyzTjr0Ts|<2>*60$8.UnnyzjntwwyTDnw0zs|<3>*60$9.zw3jDxzjvsznzjxz9nUzzU|

<4>*60$9.yTnwT/vSPrRvDM0zTvzQ|<5>*60$8.U/yzjsDtzDvyz3a3U|<6>*60$8.sNyzToANT7tyD9b3U|<7>*60$8.0DnxyTjrxyTjvyzjns|

<8>*60$8.UHVyTXm16LlyT1a3U|<9>*60$8.knZwTbswUDvwzTa3U"
if (ok:=FindText(1719, 90, 1760, 112, 0, 0, Text,,,,,,dir:=5))
 {
X1:=ok.1.x, Y1:=ok.1.y, Comment1:=ok.1.id
X2:=ok.2.x, Y2:=ok.2.y, Comment2:=ok.2.id
If (X1 > X2)
tooltip,%Comment2%%Comment1%,817,857,2
If (X1 < X2)
tooltip,%Comment1%%Comment2%,817,857,2
 }
MouseMove,xpos,ypos
return
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

20 Jun 2021, 19:36

That's all you need: FindText.Ocr(ok).Text
Shanghei
Posts: 28
Joined: 04 Jan 2021, 22:42

Re: FindText - Capture screen image into text and then find it

24 Jun 2021, 14:27

I am having an issue with findtext, if I have it watching for a specific keyword I capture, for the example lets say the keyword is "hello" and it was the first word in the sentence when I captured it. It will then only find that word if it is the first word in the sentence going forward.

it will find it in these examples
hello how are you
hello where are you

it will NOT find it in these examples
well hello there
where is hello
can you say hello

How can I get it to get triggered no matter where that word falls in the sentence?

Thank you
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

24 Jun 2021, 20:04

@Shanghei You can do this: :D
1. Search multiple words with subtle differences at one time
Text := "|<hello>*149$30.k02A0k02A0k02A0y7WASl8GAll8GAVlDmAVl82AVl83All7VaSU"
Text .= "|<hello>*149$30.U0680U0680U0680yDa8SWMa8XWEq9VWTq9VWE69VW828WW73ASU"

2. Set the appropriate fault tolerance instead of the default zero.
, 0.3, 0.3, Text
3. Instead of searching the full screen, use the "GetRange" button to select the appropriate search range.
Shanghei
Posts: 28
Joined: 04 Jan 2021, 22:42

Re: FindText - Capture screen image into text and then find it

25 Jun 2021, 08:13

feiyue wrote:
24 Jun 2021, 20:04
@Shanghei You can do this: :D
1. Search multiple words with subtle differences at one time
Text := "|<hello>*149$30.k02A0k02A0k02A0y7WASl8GAll8GAVlDmAVl82AVl83All7VaSU"
Text .= "|<hello>*149$30.U0680U0680U0680yDa8SWMa8XWEq9VWTq9VWE69VW828WW73ASU"

2. Set the appropriate fault tolerance instead of the default zero.
, 0.3, 0.3, Text
3. Instead of searching the full screen, use the "GetRange" button to select the appropriate search range.
I will try that, it seems like I have to add the same word several times because wherever it is on screen provides different results.

Text:="|<>*192$44.60A18001U30E000w0k400091wT9yDmEX8mMaBaMm4a93TYBV9WEoBX8GMYD1MmAa9XUHwT9WDk00000080000006000000T8"
Text.="|<>*183$44.2041g001k10M000I0E60005VwTYi7G8nANAmAW8G6G4VTm4VYVMI4V8N8K51An6G4XkNoTYV7E00000040000003000000Dc"
Text.="|<>*182$44.2041A001k10E000I0E4000B1wTAi7GMXAnAmAW8G4m4VTq4VAVMI4V8H8K518nAm4XkNoRAV7E00000040000003000000Dc"
Text.="|<>*188$43.60A38003061U002k30k0018TXtDVwYMmAaNaX8P6G4VTYB392EsG6lYV8MBXMmEaQ2Tbt8Fw00000020000003000000TE"

I have 4 different results for the same word. In this example "Adding" is the word. If I change tolerance to .30 if a word ends in "ing" like flying, itll count as a match to Adding.
chaoscreater
Posts: 59
Joined: 12 Sep 2019, 21:15

Re: FindText - Capture screen image into text and then find it

26 Jun 2021, 19:36

@feiyue - how does multicolor work? I picked a few colors from an image and tested, it works. However, if I move the image slightly out of position, Findtext will not find the color. The search range is not modified, it searches for everything on the screen.
pv007
Posts: 93
Joined: 20 Jul 2020, 23:50

Re: FindText - Capture screen image into text and then find it

04 Jul 2021, 14:01

@feiyue is possible to convert the findtext code back to an image?

When you have too many text codes, sometimes you forget how the image was captured, if we could see the image again could help.
foxdanger
Posts: 83
Joined: 11 Jun 2019, 13:48

Re: FindText - Capture screen image into text and then find it

10 Jul 2021, 14:44

feiyue wrote:
14 May 2016, 12:10
I wrote a useful function to share with you.
It is used to find text or images on the screen.
Hope you like it ! :)

Detailed usage can refer to:
1、The introduction by ed1chandler.
2、The introduction by c4p (latest).
feiyue I want to say thank you! Your class worked like a charm for my software and it made possible some great things for me. Thanks a lot for this my friend!
byzod
Posts: 87
Joined: 21 Jun 2021, 06:46

Re: FindText - Capture screen image into text and then find it

12 Jul 2021, 07:08

没有字库就用户自己生成字库
这个土味操作有点想法,老哥稳( ´・ω・`)b
pv007
Posts: 93
Joined: 20 Jul 2020, 23:50

Re: FindText - Capture screen image into text and then find it

23 Jul 2021, 21:36

Do someone knows how to convert the findtext code back to an image?
I think its some custom code made on base64? I tried to convert the code back to image utilizing this website:
https://base64.guru/converter/decode/image

The findtext code:

Code: Select all

"|<>*145$36.Dbsyz8DbUQQ0Db6AQE07zAMs0DyANs4DUANsCD7ANsDDDAMsD76AQEDX0AQ0U"
I have added to the site just Dbsyz8DbUQQ0Db6AQE07zAMs0DyANs4DUANsCD7ANsDDDAMsD76AQEDX0AQ0U
But it does not work...
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

25 Jul 2021, 00:37

@pv007
You can use FindText.ASCII() to display images. For example: :dance:

Code: Select all


#Include <FindText>

F1::
if (A_Cursor!="IBeam")
  return
bak:=ClipboardAll
Clipboard:=""
Send {End}+{Home}^c{Ctrl Up}
ClipWait, 2
s:=Clipboard, Clipboard:=bak
if (s="")
  return
;---------------------
s:=FindText.ASCII(s)
Loop, 10
{
  ToolTip(s, 0, 0)
  Sleep, 200
}
ToolTip()
return


ToolTip(s:="", x:="", y:="", num:=1, arg:="")
{
  static ini:=[]
  f:= "ToolTip_" . Round(num)
  if (s="")
  {
    ini.Delete(f)
    Gui, %f%: Destroy
    return
  }
  ;-----------------
  r1:=A_CoordModeToolTip
  r2:=A_CoordModeMouse
  CoordMode, Mouse, Screen
  MouseGetPos, x1, y1
  CoordMode, Mouse, %r1%
  MouseGetPos, x2, y2
  CoordMode, Mouse, %r2%
  x:=Round(x="" ? x1+16 : x+x1-x2)
  y:=Round(y="" ? y1+16 : y+y1-y2)
  ;-----------------
  bgcolor:=arg.bgcolor ? arg.bgcolor : "FAFBFC"
  color:=arg.color ? arg.color : "Black"
  font:=arg.font ? arg.font : "Consolas"
  size:=arg.size ? arg.size : "8"
  bold:=arg.bold ? arg.bold : ""
  ;-----------------
  r:=bgcolor "|" color "|" font "|" size "|" bold "|" s
  if (ini[f]!=r)
  {
    ini[f]:=r
    Gui, %f%: Destroy
    Gui, %f%: +AlwaysOnTop -Caption +ToolWindow +Border
    Gui, %f%: Margin, 2, 2
    Gui, %f%: Color, %bgcolor%
    Gui, %f%: Font, c%color% s%size% %bold%, %font%
    Gui, %f%: Add, Text,, %s%
    Gui, %f%: Show, Hide, %f%
  }
  Gui, %f%: +AlwaysOnTop
  Gui, %f%: Show, NA x%x% y%y%
}

pv007
Posts: 93
Joined: 20 Jul 2020, 23:50

Re: FindText - Capture screen image into text and then find it

25 Jul 2021, 08:42

@feiyue I have captured a screenshot of your name and used the code from FindText.ASCII(), and it returned:
image.png
image.png (3.6 KiB) Viewed 3951 times
What i was asking if is possible to retrieve the original image in which was used to generate the findtext code utilizing only the text below
I mean this:

Code: Select all

Text := "|<feiyue>*177$41.S0Q0000k0s0001U000007XngPaD6RrMrAnAliPCNbNzQqQnyn0tsta1b5lknA33vXVyDU006000000Q000U"
To this:
image.png
image.png (509 Bytes) Viewed 3951 times
byzod
Posts: 87
Joined: 21 Jun 2021, 06:46

Re: FindText - Capture screen image into text and then find it

25 Jul 2021, 22:46

老哥有个问题请教一下
对于有包含关系的找图素材,不希望重复报告匹配结果,有什么设置可以让小图不要匹配大图已经匹配过的区域?

例如下例
匹配过小弟爆伤的区域,不要再返回爆伤的匹配结果
An1.gif
An1.gif (44.17 KiB) Viewed 3922 times


1.ahk:
1.ahk
(1.47 KiB) Downloaded 84 times

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
#Include <FindText> ;Capture screen image into text and then find it

msgbox % "打开1.png,鼠标放在其右下角按num/"

; 打开1.png,鼠标放在其右下角按num/
NumpadDiv::
{
	CoordMode, Mouse, Screen
	TargetTalentText := ""
	TargetTalentText.="|<小弟爆伤>*71$107.zzyzvjU3yzxz23qvrz06AD07Trxzvyprc3jzjhLSvi0C0DrxfjKrS7TORw0RzTrzjukSZ85hyp0vis0zjz0pikPRfLnehk1xrU0yzhRu0vKTzJPzzU3yzxzO3rzqhU6eq03rTRrvypPg7hPTg07zw06vjrxejPjOqzOpw0SqxrTjsqyERpg0zRvymeM0zTranivcPxxwk1uPzxU0jSz1gqU"
	TargetTalentText.="|<爆伤>*72$71.U3yzxz23qvrzTrxzvyprc3jy0C0DrxfjKrS5zTrzjukSZ85c0zjz0pikPRfRrU0yzhRu0vKU3yzxzO3rzqhrTRrvypPg7hM06vjrxejPjOqqxrTjsqyERpWeM0zTranivcOPzxU0jSz1gqU"

	MouseGetPos mX, mY
	talents := FindText(mX - 300, mY - 200, mX + 50, mY + 50, 0, 0, TargetTalentText)
	if(talents){
		rStr := ""
		
		for i,talent in talents{
			rStr .= "#" . i . ": [" . talent.id . "]`n"
			Tooltip % rStr
			FindText.MouseTip(talent.x, talent.y, talent.3/2, talent.4/2)
		}
	} else {
		Tooltip % "没找到"
		RectTip(mX - 300, mY - 200, mX + 50, mY + 50)
	}
	
	RectTip(x1, y1, x2, y2, d:=1){
		if(FindText && FindText.MouseTip){
			FindText.MouseTip((x2+x1)/2, (y2+y1)/2, (x2-x1)/2, (y2-y1)/2, d)
		}
	}
	return
}

1.png:
1.png
1.png (6.55 KiB) Viewed 3922 times

https://translate.google.com/

Brother have a question for advice
For image-finding materials that have inclusion relationships, and do not want to report the matching results repeatedly, is there any setting to prevent the small image from matching the area of the large image?

For example, the following example
Match the explosive injury area of the younger brother, do not return the explosive injury matching result
byzod
Posts: 87
Joined: 21 Jun 2021, 06:46

Re: FindText - Capture screen image into text and then find it

25 Jul 2021, 23:46

好像发现了个bug

我试图让小图多截取2个像素,这样在句中的小图就不再会匹配了
t1.png
t1.png (9.35 KiB) Viewed 3915 times
t2.png
t2.png (9.57 KiB) Viewed 3915 times

单独的测试证明这个策略是有效的,但是实际用起来又出现问题
仔细一看,只要text包含的搜索目标不止一个时,无论其他的目标是否真的能找到,这个包含额外像素的小图都会错误地匹配到本不应该匹配的地方
只要不包含其他目标,带额外像素的小图就能匹配到正确结果
An2.gif
An2.gif (56.49 KiB) Viewed 3915 times



1.png:(还是用楼上的图不变)

2.ahk

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
#Include <FindText> ;Capture screen image into text and then find it

CoordMode, Mouse, Screen
CoordMode, Tooltip, Screen

global TargetTalentText := ""
global TargetTalentTextBase := "|<爆伤>*80$73.s0zjzTkUxixzxzTrzjvLSUCzy0C0DrxfjKrS7Trxzvyg7dG1PU3yzw3Kv1hqhxrU0yzhRu0vKs0zjzTqUxzxfTRxrTjvJikSpg06vjrxejPjOrhjRrvyBjY7RP+dU3xzSPCviVuPzxU0jSz1gqU"
global TargetTalentTextAdd := "|<不存在的图>*197$13.zzk0wUSED07zzzzzzw0SzDDbU3zzk"

TargetTalentText := TargetTalentTextBase . TargetTalentTextAdd

msgbox % "打开1.png,按num*切换搜索内容,鼠标放在其右下角按num/"

; 更改搜索内容
NumpadMult::
{
	MouseGetPos mX, mY
	name := ""
	if(TargetTalentText != TargetTalentTextBase){
		TargetTalentText := TargetTalentTextBase
		name := "<爆伤>"
	} else {
		TargetTalentText := TargetTalentTextBase . TargetTalentTextAdd
		name := "<爆伤> + <不存在的图>"
	}
	Tooltip % "当前text:`n" . name, % mX, % (mY - 100), 2
	return
}


; 打开1.png,鼠标放在其右下角按num/
NumpadDiv::
{
	CoordMode, Mouse, Screen
	CoordMode, Tooltip, Screen

	MouseGetPos mX, mY
	talents := FindText(mX - 300, mY - 200, mX + 50, mY + 50, 0, 0, TargetTalentText)
	
	if(talents){
		rStr := ""
		
		for i,talent in talents{
			rStr .= "#" . i . ": [" . talent.id . "]`n"
			Tooltip % rStr
			FindText.MouseTip(talent.x, talent.y, talent.3/2, talent.4/2)
		}
	} else {
		Tooltip % "没找到"
		RectTip(mX - 300, mY - 200, mX + 50, mY + 50)
	}
	
	RectTip(x1, y1, x2, y2, d:=1){
		if(FindText && FindText.MouseTip){
			FindText.MouseTip((x2+x1)/2, (y2+y1)/2, (x2-x1)/2, (y2-y1)/2, d)
		}
	}
	return
}

https://translate.google.com/

Seems to have found a bug

I tried to make the small picture intercept 2 more pixels, so that the small picture in the sentence will no longer match

t1.png
t1.png (9.35 KiB) Not viewed yet

t2.png
t2.png (9.57 KiB) Not viewed yet


A separate test proves that this strategy is effective, but there are problems in actual use
Take a closer look, as long as the text contains more than one search target, regardless of whether the other targets can be found, the small image containing extra pixels will be incorrectly matched to the place that should not be matched.
As long as no other targets are included, the small image with extra pixels can be matched to the correct result
thebbandit
Posts: 45
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

27 Jul 2021, 16:15

@feiyue
My users have found a very obscure bug which only affects people with multiple monitors with non-identical refresh rates.
The script ends up crashing, or going into a memory overflow. Variables all become blank, and all GUI become blank.
Once users disable the non-conforming monitors, the issue is resolved.
The captures all occur inside the range of one monitor, so I am a little perplexed why the other monitors are affecting the performance.

The users who reported this issue had setups like so:

(1) 144hz monitor, (2) 60hz monitors
(1) 59hz monitor, (1) 60hz monitor

And both confirmed that disabling the extra screens resolves the issue.

Using version 8.5
User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

28 Jul 2021, 02:35

STOPING a LOOP doesn't seem to work. **EDIT- I figured it out see my reply :bravo:
I know how to use FindText, it's geat but I have this Loop Problem.

I do a loop search with a normal 'imagesearch' and it works Perfect, It BEEPS in a LOOP when the image is found in a loop continuously Forever and STOPS FOREVER when image NOT found, which is what I want.
BUT..... when I use FindText for an image search, the BEEPING continues everytime the image is FOUND again even though it's the exact same script.

Here is a normal imagesearch script below that BEEPS when image is FOUND and then STOPS beeping 'forever' when image NOT FOUND. The script stops which is good.

Code: Select all

Loop
{
    CoordMode, Pixel, Screen
    ImageSearch, FoundX, FoundY, 0, 0, 1920, 1080, C:\Users\Mylo\Documents\AutoHotkey\SCRIPTS_STARTUP\DavinciPIXELs\Screenshots\Screen_20210728160804.png
    If ErrorLevel = 0
    	SoundBeep
    If ErrorLevel
    	Return
}
Until ErrorLevel
So...The above code DOESN'T beep again if the image is found again (which is what I want) cause of the Return Value. But the RETURN value doesn't seem to work when I replace the imagesearch feature with FindText Feature.

Code: Select all

#Include C:\Users\Mylo\Documents\AutoHotkey\lib\Functions\FindText.ahk      

Loop
{   
     Text:="|<hist>#[email protected]$22.TzzvzzzzzzzzzzzzzzzzzzzzzzzzzzxrzzrTzzfzzyDzzxzzzrzzzTzzxzzzzzzzzzzzzzzzzzzzzTzzu"
     if (ok:=FindText(1298-150000, 732-150000, 1298+150000, 732+150000, 0, 0, Text))
     ;If ErrorLevel = 0
    	SoundBeep
    If ErrorLevel
     return
}
As mentioned, the BEEPING continues again every time the image is FOUND 'again' even though it's the exact same script. It's like the RETURN isn't working.

I have tried replacing the #include with the normal text function, still doesn't get what I want.
I have tried replacing and mixing and swapping -
if (ok:=FindText(1298-150000, 732-150000, 1298+150000, 732+150000, 0, 0, Text))
If ErrorLevel = 0
Using one with the other, or without, It still keeps BEEPING when the image is found again.

Any help would be great. :-)
Last edited by SteveMylo on 31 Jul 2021, 06:46, edited 1 time in total.
Tommi[Teo]
Posts: 1
Joined: 25 Jun 2021, 22:37

Re: FindText - Capture screen image into text and then find it

28 Jul 2021, 03:05

Hi, Feiyu, thank you very much
this is the first time an image finder returns me results instantly what i want, it's really fast, great job :):)

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: No registered users and 114 guests