FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

01 Apr 2019, 09:30

snowiscold1 wrote:
01 Apr 2019, 08:14
Hi feiyue,

I am new to autohotkey and I found out about your script a month ago. I have been working on my games in Bluestack, but I have 1 issue. I am unable to pass the parameter of EmulatorArea variable to the function. It gives me this error. I have tried with and without percentage sign. Please help me thank you.

test.png

Code: Select all

EmulatorArea=145,645,244,751
if (ok:=FindText(EmulatorArea, 0, 0, Text))
look not really different to version i using:

Code: Select all

 if ok:=FindText(22  ,76  ,150000,150000,0,0,buttonStartWindows10_Text)
if (ok:=FindText(145,645,244     ,751      ,0,0, Text)) ; <=  yours

please use this variable differently:
EmulatorArea

please write your values ​​directly in without variable
snowiscold1
Posts: 2
Joined: 01 Apr 2019, 08:05

Re: FindText - Capture screen image into text and then find it

01 Apr 2019, 12:25

Hi,
Thank you for your quick reply. I have no problem when I write the value directly to the function. But I need to use variable because I don't want to change the values at other lines as well. My script has 300++ lines , and it'll be easier for me to just change the value at EmulatorArea:="x,y,h,w" . Is there any ways I can achieve this? For some reasons, I think the way FindText() interpret my variable is like this . It only replaces the value of x.
EmulatoraArea:="x,y,h,w"

if (ok:=FindText( [x,y,h,w],y,h,w, 0, 0, Text))
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

01 Apr 2019, 12:38

snowiscold1 wrote:
01 Apr 2019, 12:25
Hi,
Thank you for your quick reply. I have no problem when I write the value directly to the function. But I need to use variable because I don't want to change the values at other lines as well. My script has 300++ lines , and it'll be easier for me to just change the value at EmulatorArea:="x,y,h,w" . Is there any ways I can achieve this? For some reasons, I think the way FindText() interpret my variable is like this . It only replaces the value of x.
EmulatoraArea:="x,y,h,w"

if (ok:=FindText( [x,y,h,w],y,h,w, 0, 0, Text))
no there is no other option unless you change the function itself.
or you write a small wrapper function which then calls FindText.
Please read the documentation topics array or/and objects
feiyue
Posts: 239
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

07 Apr 2019, 18:41

Updated to 6.5 version - 2019/04/07
.... 1. Add Gray Difference Mode, so there are four modes.
.... 2. Fixed SplitAdd/AllAdd buttons doesn't work(broken by v6.4).
burque505
Posts: 1396
Joined: 22 Jan 2017, 19:37

Re: FindText - Capture screen image into text and then find it

07 Apr 2019, 18:47

@feiyue, thanks again for all your great work on this script. I just tested it on Win7 Pro 64-bit, AHK_L 1.1.30.01 (haven't upgraded yet), and it seems faster, and works great.
EDIT: Also fine with AHK_L 1.1.30.03.
Regards,
burque505
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

08 Apr 2019, 10:45

i have problem with this very great script (thanks) if i using:
if (ok:=FindText(wX-150, wY-150, wW+150, wH+150, 0, 0, Text)) ; <=== this dont find it
what i doing wrong?

Code: Select all

CoordMode, Caret
CoordMode, Mouse
CoordMode, Pixel
CoordMode, Menu
Text:="|<>**50$18.TzzU00jzzc00c00cQ0dW0e20dW0cW0cW0cW0cW0dX0e1Udz0c00U"
SetTitleMatchMode,2
; WinGetPos,wX,wY,wW,wH,ahk_class AutoHotkeyGUI ahk_exe AutoHotkey.exe ahk_id 0x100b50
WinGetPos,wX,wY,wW,wH,ahk_class AutoHotkeyGUI ahk_exe AutoHotkey.exe 
ToolTip,%wX% %wY% (19-04-08_17-30)
if (ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, Text)) ; <=== this works
; if (ok:=FindText(wX-150, wY-150, wW+150, wH+150, 0, 0, Text)) ; <=== this dont find it
attached the complete script
Attachments
findFirstEntry_by_using_imageSearch.ahk
(41.79 KiB) Downloaded 60 times
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

08 Apr 2019, 10:57

update: i have still a problem :

updated 19-04-08_20-54:
i think i need to use the half ... i test it as soon i have time again:
youtu.be/wiOljkG3ERc https://www.youtube.com/watch?v=pXAB58tpmIE (updated)

updated 19-04-08_19:

Code: Select all

CoordMode, Caret,Screen
CoordMode, Mouse,Screen
CoordMode, Pixel,Screen
CoordMode, Menu,Screen
Text:="|<>**50$18.TzzU00jzzc00c00cQ0dW0e20dW0cW0cW0cW0cW0dX0e1Udz0c00U"
SetTitleMatchMode,2
needle=Action List Appears Here ahk_class AutoHotkeyGUI ; mouseWindowTitle=0x100b50  ; 
WinGetPos,wX,wY,wW,wH,% needle
MsgBox,%wX% %wY% %g_Helper_Id% ; <= this gives the correct coordinates
; Reload ; ToolTip,%wX% %wY% (19-04-08_17-30)
; if (ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, Text)) ; <=== this works
if (ok:=FindText(wX, wY, wW+150, wH+150, 0, 0, Text)) ; <=== this dont find it
{
using a douple while loop and the reulsts looks really strange to me:

yyy is here 98
2019-04-08 19_00_06-FindText - Capture screen image into text and then find it - AutoHotkey Communit.png
2019-04-08 19_00_06-FindText - Capture screen image into text and then find it - AutoHotkey Communit.png (51.53 KiB) Viewed 3851 times
how is that x, y working?

; text bellow is outdated:
feiyue
Posts: 239
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

08 Apr 2019, 14:50

Novice may think that the first four parameters are the range parameters of AHK: x1, y1, x2, y2
However, the function uses the range parameters of WinAPI: x, y, w, h
In fact, only one search can solve the problem, but you need to extract the left + top results by yourself:

Code: Select all

WinGetPos, wX, wY, wW, wH, % needle
ok:=FindText(wX, wY, wW, wH, 0, 0, Text)
x:=y:=""
;-- Get the leftmost coordinate value
For k, v in ok
  if (x="" or v.1<x)
    x:=v.1
;-- Get the topmost coordinate value
For k, v in ok
  if (v.1=x) and (y="" or v.2<y)
    y:=v.2
MsgBox, %x%  %y%
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

08 Apr 2019, 16:06

feiyue wrote:
08 Apr 2019, 14:50
Novice may think that the first four parameters are the range parameters of AHK: x1, y1, x2, y2
no: https://autohotkey.com/docs/commands/WinGetPos.htm
feiyue wrote:
08 Apr 2019, 14:50
... However, the function uses the range parameters of WinAPI: x, y, w, h ...
what is the difference from parameters get from

WinGetPos, wX, wY, wW, wH, % needle

and your

parameters of WinAPI: x, y, w, h

I have never heard of it before.

is it perhaps due to my two monitors?

As you can see in the YouTube (update 19-04-08_23-36: sorry wrong monitor was recordet, now update => https://youtu.be/pXAB58tpmIE) videos it is about factor 2.
therfore i actually using this (what better solution you have?).

I think your source code looks like mine at the beginning, right?

I just do not understand why you use two loops again below?

Is not it possible to just take the coordinates with Is not it possible to just take the coordinates over and look for them? and search inside them?
SL5 wrote:
08 Apr 2019, 10:57

Code: Select all

needle=Action List Appears Here ahk_class AutoHotkeyGUI ; mouseWindowTitle=0x100b50  ; 
WinGetPos,wX,wY,wW,wH,% needle
; if (ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, Text)) ; <=== this works
if (ok:=FindText(wX, wY, wW+150, wH+150, 0, 0, Text)) ; <=== this dont find it
i reused your example this way:

Code: Select all

needle=Action List Appears Here ahk_class AutoHotkeyGUI ; mouseWindowTitle=0x100b50  ; 
WinGetPos, wX, wY, wW, wH, % needle
if(ok:=FindText(wX, wY, wW, wH, 0, 0, Text))
	msgbox,found
Clipboard = found nothing in %wX%`, %wY%`, %wW%`, %wH%
msgbox, % Clipboard
Reload
and get: found nothing in 1480, 752, 792, 164
whats wrong? as i said befor it will be found if i use ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, Text)) ; <=== this works

feiyue wrote:
08 Apr 2019, 14:50
Novice may think that the first four parameters are the range parameters of AHK: x1, y1, x2, y2
However, the function uses the range parameters of WinAPI: x, y, w, h
In fact, only one search can solve the problem, but you need to extract the left + top results by yourself:

Code: Select all

WinGetPos, wX, wY, wW, wH, % needle
ok:=FindText(wX, wY, wW, wH, 0, 0, Text)
x:=y:=""
;-- Get the leftmost coordinate value
For k, v in ok
  if (x="" or v.1<x)
    x:=v.1
;-- Get the topmost coordinate value
For k, v in ok
  if (v.1=x) and (y="" or v.2<y)
    y:=v.2
MsgBox, %x%  %y%
feiyue
Posts: 239
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

09 Apr 2019, 00:30

:D I'm sorry, I'm from China, so I can't go to www.youtube.com or https://youtu.be

Full screen can be found and window range can not be found, indicating that window range acquisition error.
Your problem can only be debugged by yourself, others can not help, because it is difficult to reproduce your situation.

Maybe you have multiple window matching conditions, but WinGetPos operated an inappropriate window.
You can test how many windows match your query conditions:
WinGet, list, List, % needle
MsgBox, % list

-------------------------------

@snowiscold1 , Here's a small wrapper function that calls FindText.

MyFindText(EmulatorArea, arg*) {
a:=StrSplit(EmulatorArea, ",")
return, FindText(a[1], a[2], a[3], a[4], arg*)
}

EmulatoraArea:="x,y,w,h"
ok:=MyFindText(EmulatorArea, 0, 0, Text)
Last edited by feiyue on 09 Apr 2019, 16:05, edited 2 times in total.
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

09 Apr 2019, 01:45

feiyue wrote:
09 Apr 2019, 00:30
:D I'm sorry, I'm from China, so I can't go to www.youtube.com or https://youtu.be
ups. @feiyue, thanks for answering and again for all your great work on this script.(now inserted it as gif without audio now).
feiyue wrote:
09 Apr 2019, 00:30
Full screen can be found and window range can not be found, indicating that window range acquisition error.
Your problem can only be debugged by yourself, others can not help, because it is difficult to reproduce your situation.
in other words:
How would you do it if you wrote that yourself?
1. So first search for the window and save the area. how?
2. then use this area in FindText. how?
whey is facotr 1.88 nedet? (see examples below). do you use factor 1.88? is everybody using factor 1.88? only me? How would you do it if you did not use a factor?


Code: Select all

needle=Action List Appears Here ahk_class AutoHotkeyGUI ; mouseWindowTitle=0x100b50  ; 
WinGetPos, wX, wY, wW, wH, % needle
xxx:=floor(wX/1.88)
yyy:=floor(wY/1.88)
if(ok:=FindText(xxx,yyy,xxx+wW,yyy+wH, 0, 0, Text)){
	X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
	MouseMove,% X+50,% Y
	msgbox,found
}else{
	Clipboard = found nothing  in %wX%`, %wY%`, %wW%`, %wH% (19-04-09_01-29)
	msgbox, % Clipboard
}
feiyue wrote:
09 Apr 2019, 00:30
Maybe you have multiple window matching conditions, but WinGet return an inappropriate window.
at least that sounds like I'm not wrong, not totally wrong. that helps me a lot already! thanks for the feedback.
feiyue wrote:
09 Apr 2019, 00:30
You can check how many window matching conditions:
WinGet, list, List, % needle
MsgBox, % list
i will check that.
feiyue wrote:
09 Apr 2019, 00:30
snowiscold1 , Here's a small wrapper function that calls FindText.

Code: Select all

MyFindText(EmulatorArea, arg*) {
  a:=IsObject(EmulatorArea) ? EmulatorArea : StrSplit(EmulatorArea, ",")
  return, FindText(a[1]+0, a[2]+0, a[3]+0, a[4]+0, arg*)
}
EmulatoraArea:="x,y,w,h"  ...
^---- I think it has nothing to do with my question but thanks anyway (i dont use a string)
Attachments
findNotepadPPtopLeft_by_using_imageSearch.ahk
(42 KiB) Downloaded 53 times
findFirstEntry_by_using_imageSearch.ahk
(43.87 KiB) Downloaded 47 times
problem_with_findText19-04-09_08-56.gif
problem_with_findText19-04-09_08-56.gif (197.64 KiB) Viewed 3760 times
feiyue
Posts: 239
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

09 Apr 2019, 10:16

Looking at your code, I don't know what you're going to do.
Can't you use full screen search? You said full screen search was successful.

Or do you want to search in the window range?
It's also very simple (just find the right window):
WinGetPos, wX, wY, wW, wH, %WindowsTitle%
ok:=FindText(wX, wY, wW, wH, 0, 0, Text)

I can't figure out what you're doing to make the range parameters so confusing.
Generally, the search width and height are determined, independent of the upper-left coordinates.
I reminded beginners not to think of W and H parameters as coordinates x2 and Y2,
But you still used it incorrectly. :beer:
Last edited by feiyue on 09 Apr 2019, 16:11, edited 1 time in total.
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

09 Apr 2019, 11:20

feiyue wrote:
09 Apr 2019, 10:16
Looking at your code, I don't know what you're going to do.
Can't you use full screen search? You said full screen search was successful.

Or do you want to search in the window range?
It's also very simple (just find the right window):
WinGetPos, wX, wY, wW, wH, %WindowsTitle%
ok:=FindText(wX, wY, wW, wH, 0, 0, Text)

I can't figure out what you're doing to make the range parameters so confusing.
Generally, the search width and height are determined, independent of the upper-left coordinates.
I reminded beginners not to think of W and H parameters as coordinates x2 and Y2, but you are still wrong.
That's not simple with my computer because I need a Faktor between this two lines from abaut one 1.88.
First I tried to do it as you descripe above . probably it has something to do with my computer screen. I just use this command very often in other Scripts with no problem only with this script. Strange

and I'm not wrong I have used this comand about handle 1000 times and nearly every day (*1). i just need this factor and it works at every postiton of all of my monitors. your here what i am saiing? i said that so often below.

if somebody have the same problem, he can use the two why while-loops below to find the factor.

*1 this ony from one script i programming this year:
feiyue
Posts: 239
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

09 Apr 2019, 15:16

@SL5 , You may not need the special number 1.88, :D
You can calculate the width and height according to the
upper-left coordinate (x1, y1) and lower-right coordinate (x2, y2) of the search range:
----------------------
x1, y1 ==> xxx, yyy ==> wX-150, wY-150 (instead of Floor(wX/1.88), Floor(wY/1.88) )
x2, y2 ==> wX+wW-1, wY+wH-1 (this is unchangeable)
width ==> x2-x1+1 ==> wX+wW-xxx (instead of xxx+wW) ==>wW+150
height ==> y2-y1+1 ==> wY+wH-yyy (instead of yyy+wH) ==>wH+150
----------------------
ok:=FindText(wX-150, wY-150, wW+150, wH+150, 0, 0, Text)
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

10 Apr 2019, 05:20

feiyue wrote:
09 Apr 2019, 15:16
SL5 , You may not need the special number 1.88, :D
You can calculate the width and height according to the
upper-left coordinate (x1, y1) and lower-right coordinate (x2, y2) of the search range:
----------------------
x1, y1 ==> xxx, yyy ==> wX-150, wY-150 (instead of Floor(wX/1.88), Floor(wY/1.88) )
x2, y2 ==> wX+wW-1, wY+wH-1 (this is unchangeable)
width ==> x2-x1+1 ==> wX+wW-xxx (instead of xxx+wW) ==>wW+150
height ==> y2-y1+1 ==> wY+wH-yyy (instead of yyy+wH) ==>wH+150
----------------------
ok:=FindText(wX-150, wY-150, wW+150, wH+150, 0, 0, Text)
you still does not understand it. :beer:
factor ~2 means the area starts for e.g. at x 2000 instead of x 1000
if i search a smal icon with about15 pixel with and hight.
=> this is not about 150 pix rectancle:
2019-04-10 12_21_02-Action List Appears Here.png
2019-04-10 12_21_02-Action List Appears Here.png (319 Bytes) Viewed 3630 times
=> with all of my other tests with other pics it always results in the factor 1.88 all over my monitors. could you hear me?

thanks for your calculations but it doesent matter here. i understand your misundertanding. the reason has to be somwhere else of this problem.

Nevertheless, I have tried your example of your last posing and it is of course no match:
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

10 Apr 2019, 07:27

After a long time of not understanding each other, I have to report the following.
I have described in great detail and youtube video and gif video several times the factor 1.88 problem and everytime with sample source code.
I think my posts was simply not read completly(?). This caused a lot of misunderstandings and it took longer to clarify them. but thanks for answering anyway!

Now i have the nutritional procedure (remember:https://www.youtube.com/watch?v=pXAB58tpmIE) run again and now get a factor of 1.
So I've tried the little lines of source code from @feiyue again and get a hit this time.

the hit msgbox:

Code: Select all

RESULT2: x=1683 y=652 w=18 h=17 
 xxx=1584 ?= 1674(expected)=1.056818 
 yyy=656 ?= 644(expected)=0.981707
FindText(wX-wW//2, wY-wH//2, wW, wH, 0, 0, Text)

short report of a difficulty at the beginning:

becouse the little icon is inside the window I also expected the perfect search area inside the window.
ok:=FindText(wX, wY, wW, wH, 0, 0, Text)
but I have to use an offset of minimum -2 pixels :arrow:

Code: Select all

while(a_index<200){
	ok:=FindText(wX-a_index, wY-a_index, wW, wH, 0, 0, Text) ; matching by index 2 (19-04-10_14-56)
	X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
	msg =
	(
a_index=%a_index%
wW=%wW%
wH=%wH%
	)
	if(ok){
		MouseTip(X, Y)	
		MsgBox,262208,% msg footer:="`n:)`n`n(" A_ThisFunc ":" A_LineNumber " " RegExReplace(A_LineFile, ".*\\") ")" ,% msg footer
		Reload
	}
}
marat271
Posts: 5
Joined: 01 Sep 2017, 10:23

Re: FindText - Capture screen image into text and then find it

10 Apr 2019, 13:24

I'm sorry this has been asked before, but does this function support ErrorLevel, like the search image function?

So if the image I search for is detected, it does one thing but if it isn't detected it does something else?
User avatar
SL5
Posts: 879
Joined: 12 May 2015, 02:10
GitHub: sl5net
Contact:

Re: FindText - Capture screen image into text and then find it

10 Apr 2019, 16:19

marat271 wrote:
10 Apr 2019, 13:24
I'm sorry this has been asked before, but does this function support ErrorLevel, like the search image function?

So if the image I search for is detected, it does one thing but if it isn't detected it does something else?
if(FindText( ...){
; its found
}else{
; its not found
}

you mean this?

FindText(x, y, w, h, err1, err0, text, ScreenShot=1)
marat271
Posts: 5
Joined: 01 Sep 2017, 10:23

Re: FindText - Capture screen image into text and then find it

11 Apr 2019, 00:02

SL5 wrote:
10 Apr 2019, 16:19
marat271 wrote:
10 Apr 2019, 13:24
if(FindText( ...){
; its found
}else{
; its not found
}

you mean this?
Yes, thank you very much for the help ^^
feiyue
Posts: 239
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

11 Apr 2019, 19:08

Updated to 6.6 version - 2019/04/11 :dance:
.... 1. FindText(), FindText2() share the same machine code, reduces the code size.

For a single text or image, the effects of these two functions are similar,
but for multiple text or images, FindText() loops through each text or image,
while FindText2() looks at them as a continuous combination of text for a one-time search.

Sometimes the text of a web page will be deformed and the spacing will be increased.
You can automatically cut a text into more than one text with PicX() and use FindText2() to find it.
FindText(..., Text ) ==> FindText2(..., PicX(Text) )

Return to “Scripts and Functions”

Who is online

Users browsing this forum: Eik, jetrotal and 17 guests