FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
itz4sking
Posts: 8
Joined: 17 Dec 2018, 12:33

Re: FindText - Capture screen image into text and then find it

18 Dec 2018, 14:40

feiyue wrote:
18 Dec 2018, 12:52
itz4sking wrote: I've actually figured out my problem. For some reason GetKeyState() doesn't work through teamviewer.... anyone have a work around?
@itz4sking, I also encountered this problem when I was remotely controlling the computer.
Now I have upgraded the script to solve this problem. :dance:

Updated to 6.3 version - 2018/12/18
.... 1. Changed the way to capture, Now first click the right mouse button,
.... you can move the mouse, and then click the right mouse button again.
.... 2. Add MouseTip(). It is used to prompt mouse position in remote assistance.
Awesome! Thank you. I'm also having an issue with MouseMove, MouseClick, and PixelSearch when remotely controlling the computer. Have you found the solution to this problem, too?
feiyue
Posts: 162
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

18 Dec 2018, 17:40

MouseMove, MouseClick command execution should be no problem,
but we can not see the mouse trajectory when remote control,
so I added MouseTip() function for script debugging to prompt the mouse position.
What is the PixelSearch problem ?
itz4sking
Posts: 8
Joined: 17 Dec 2018, 12:33

Re: FindText - Capture screen image into text and then find it

19 Dec 2018, 07:23

Hmm it appears remotely controlling is not actually my issue. It has something to do with settings macros before a loop that causes the loop to never occur. My code is below, it works if i put the macros after the loop but not if i have them before. Weird....

^g::
CoordMode, mouse, window
MouseGetPos, X, Y
MsgBox, %X%, %Y%
sleep, 1000
return

^h::
CoordMode, Mouse, window
CoordMode, pixel, window
MouseGetPos, a, b
PixelGetColor, color, %a%, %b%
MsgBox, %color%
sleep, 1000
return


Loop
{

MouseClick, Left
MouseClick, Left
sleep, 5000
}
Lin
Posts: 1
Joined: 27 Dec 2018, 03:48

Re: FindText - Capture screen image into text and then find it

27 Dec 2018, 04:02

反馈个小bug,英文版6.3,弹出测试结果的时候不点击确定,过几秒窗口自己消失,弹出主界面,然后那个高亮提示方框就在主界面闪烁。正常情况应该是不点击确定不进入下一步动作?
中文版5.6无此问题,要点击确定才会进入下一个动作。
win7sp1,x64,AHK版本1.1.30.01

另,楼主中文版的还更新吗?英文版看着头好大,要是中文版能继续更新就太棒了!感恩。
evo88215
Posts: 6
Joined: 31 Dec 2018, 18:03

Re: FindText - Capture screen image into text and then find it

01 Jan 2019, 22:18

@feiyue great work, I have a question with multiple text/pic search. I'm trying to use your script along with the multi pic search. So I want it to write a user's input in an excel file for text(A) when found, write to a specific row(1) and in 3 columns(A1, B1, C1). save and close excel. here is code:

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.

F1::

Gui, New
Gui, Font, s11, Calibri            
Gui, Add, Text, , Password:
Gui, Add, Text, , Eng:            
Gui, Add, Text, , Heater:
Gui, Add, Text, , Air:
Gui, Add, Edit, w100 y5	vpassword Password
Gui, Add, Edit, w100 	veng,
Gui, Add, Edit, w100    vheat,
Gui, Add, Edit, w100    vair,
Gui, Add, Button, gButtonOK Default, OK    
Gui, Show,, EQSO ver. 4
Return

ButtonOK:
Gui, Submit, NoHide

FilePath := "C:\Users\Desktop\test.xlsx"
oExcel := ComObjCreate("Excel.Application")
oExcel.DisplayAlerts := False
oWorkBook := oExcel.Workbooks.Open(FilePath)
oExcel.Visible := False


; This function import text images through the comments in < >
/*Pic(comments, add_new=0) {
  static Lib:=[]
  if (add_new)
  {
    re:="<([^>]*)>[^$]+\$\d+\.[\w+/]{3,}"
    Loop, Parse, comments, |
      if RegExMatch(A_LoopField,re,r)
        Lib[Trim(r1)]:=r
  }
  else
  {
    text:=""
    Loop, Parse, comments, |
      text.="|" . Lib[Trim(A_LoopField)]
    return, text
  }
}
*/

; Usage For example:

; You can put the text library at the beginning of the script
CoordMode, Mouse, Screen
Text:="|<606>*159$64.D900l001lktW403Y00A8Y42z8+Hn0UWkk94UZNA7m/t0YI2J40Ncca+Ek8qH0aWmD9n0XDA3lls0080000000001U0000000U"
Text:="|<607>*159$64.D900l001lntW403Y00A8UY2z8+Hn0UW6k94UZNA7m8F0YI2J40NcW6+Ek8qH0aW8D9n0XDA3ll00080000000001U0000000U"
Text:="|<608>*158$64.D900l001lltW403Y00A8cY2z8+Hn0UWmk94UZNA7m9l0YI2J40Ncca+Ek8qH0aWWD9n0XDA3lls0080000000001U0000000U"

;-- Use Pic(Text,1) to add the text library to Pic()'s Lib
Pic(606,1)
Pic(607,1)
Pic(608,1)

;.....................

;.....................

;.....................

;-- Find one picture at a time
/*if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1"))
{
  X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, X+=W//2, Y+=H//2
  ;------------------------------
  Click, X, Y
}
*/

;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("606|607|608"))
{
	Loop, % ok.MaxIndex()//5
	{
		i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
		W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
    ;------------------------------
		if (Comment="606")
		{
			oExcel.Range("B33").Value := eng
			oExcel.Range("C33").Value := heat
			oExcel.Range("D33").Value := air
			oExcel.Save()
			oExcel.Quit()
			oExcel := ""
			
		}
		else if (Comment="607")
		{
			oExcel.Range("B34").Value := eng
			oExcel.Range("C34").Value := heat
			oExcel.Range("D34").Value := air
			oExcel.Save()
			oExcel.Quit()
			oExcel := ""
			
		}
		else if (Comment="608")
		{
			oExcel.Range("B35").Value := eng
			oExcel.Range("C35").Value := heat
			oExcel.Range("D35").Value := air
			oExcel.Save()
			oExcel.Quit()
			oExcel := ""
			
		}
	}
}
feiyue
Posts: 162
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

04 Jan 2019, 21:33

@Lin,已修复了那个小问题。中文论坛总是沦陷,而且没有人气,所以中文版也没有兴趣更新了,目前没有改进的思路。

@evo88215, You need to fix it:

Code: Select all

Text:="|<606>*159$64.D900l001lktW403Y00A8Y42z8+Hn0UWkk94UZNA7m/t0YI2J40Ncca+Ek8qH0aWmD9n0XDA3lls0080000000001U0000000U"
Text.="|<607>*159$64.D900l001lntW403Y00A8UY2z8+Hn0UW6k94UZNA7m8F0YI2J40NcW6+Ek8qH0aW8D9n0XDA3ll00080000000001U0000000U"
Text.="|<608>*158$64.D900l001lltW403Y00A8cY2z8+Hn0UWmk94UZNA7m9l0YI2J40Ncca+Ek8qH0aWWD9n0XDA3lls0080000000001U0000000U"
;-- Use Pic(Text,1) to add the text library to Pic()'s Lib
Pic(Text,1)

OR:

Text:="|<606>*159$64.D900l001lktW403Y00A8Y42z8+Hn0UWkk94UZNA7m/t0YI2J40Ncca+Ek8qH0aWmD9n0XDA3lls0080000000001U0000000U"
Pic(Text,1)
Text:="|<607>*159$64.D900l001lntW403Y00A8UY2z8+Hn0UW6k94UZNA7m8F0YI2J40NcW6+Ek8qH0aW8D9n0XDA3ll00080000000001U0000000U"
Pic(Text,1)
Text:="|<608>*158$64.D900l001lltW403Y00A8cY2z8+Hn0UWmk94UZNA7m9l0YI2J40Ncca+Ek8qH0aWWD9n0XDA3lls0080000000001U0000000U"
Pic(Text,1)
paulpma
Posts: 19
Joined: 08 Sep 2018, 22:05

Re: FindText - Capture screen image into text and then find it

05 Jan 2019, 06:49

Hello feiyue,

Thank you for amazing script, have been working with it for few months now. I have been using FindTextOCR() function to OCR simple text. Works great. I am trying to work with multiple text lines, seems this function does not support multiple lines of text? Is this something possible to do with this function?

Code: Select all

FindTextOCR(nX, nY, nW, nH, err1, err0, Text, Interval=20)
{
  OCR:="", RightX:=nX+nW-1
  While (ok:=FindText(nX, nY, nW, nH, err1, err0, Text))
  {
    ; For multi text search, This is the number of text images found
    For k,v in ok
    {
      ; X is the X coordinates of the upper left corner
      ; and W is the width of the image have been found
      x:=v.1, y:=v.2, w:=v.3, h:=v.4, comment:=v.5
      ; We need the leftmost X coordinates
      if (A_Index=1 or x<LeftX)
        LeftX:=x, LeftY:=y, LeftW:=w, LeftH:=h, LeftOCR:=comment
      else if (x=LeftX)
      {
        Loop, 100
        {
          err:=A_Index/100
          if FindText(x, y, w, h, err, err, Text)
          {
            LeftX:=x, LeftY:=y, LeftW:=w, LeftH:=h, LeftOCR:=comment
            Break
          }
          if FindText(LeftX, LeftY, LeftW, LeftH, err, err, Text)
            Break
        }
      }
    }
    ; If the interval exceeds the set value, add "*" to the result
    OCR.=(A_Index>1 and LeftX-nX-1>Interval ? "*":"") . LeftOCR
    ; Update nX and nW for next search
    nX:=LeftX+LeftW-1, nW:=RightX-nX+1
  }
  return, OCR
}
Thank you .

Paul.
evo88215
Posts: 6
Joined: 31 Dec 2018, 18:03

Re: FindText - Capture screen image into text and then find it

06 Jan 2019, 19:19

feiyue wrote:
04 Jan 2019, 21:33
@Lin,已修复了那个小问题。中文论坛总是沦陷,而且没有人气,所以中文版也没有兴趣更新了,目前没有改进的思路。

@evo88215, You need to fix it:

Code: Select all

Text:="|<606>*159$64.D900l001lktW403Y00A8Y42z8+Hn0UWkk94UZNA7m/t0YI2J40Ncca+Ek8qH0aWmD9n0XDA3lls0080000000001U0000000U"
Text.="|<607>*159$64.D900l001lntW403Y00A8UY2z8+Hn0UW6k94UZNA7m8F0YI2J40NcW6+Ek8qH0aW8D9n0XDA3ll00080000000001U0000000U"
Text.="|<608>*158$64.D900l001lltW403Y00A8cY2z8+Hn0UWmk94UZNA7m9l0YI2J40Ncca+Ek8qH0aWWD9n0XDA3lls0080000000001U0000000U"
;-- Use Pic(Text,1) to add the text library to Pic()'s Lib
Pic(Text,1)

OR:

Text:="|<606>*159$64.D900l001lktW403Y00A8Y42z8+Hn0UWkk94UZNA7m/t0YI2J40Ncca+Ek8qH0aWmD9n0XDA3lls0080000000001U0000000U"
Pic(Text,1)
Text:="|<607>*159$64.D900l001lntW403Y00A8UY2z8+Hn0UW6k94UZNA7m8F0YI2J40NcW6+Ek8qH0aW8D9n0XDA3ll00080000000001U0000000U"
Pic(Text,1)
Text:="|<608>*158$64.D900l001lltW403Y00A8cY2z8+Hn0UWmk94UZNA7m9l0YI2J40Ncca+Ek8qH0aWWD9n0XDA3lls0080000000001U0000000U"
Pic(Text,1)

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.

F1::

Gui, New
Gui, Font, s11, Calibri            
Gui, Add, Text, , Password:
Gui, Add, Text, , Eng:            
Gui, Add, Text, , Heater:
Gui, Add, Text, , Air:
Gui, Add, Edit, w100 y5	vpassword Password
Gui, Add, Edit, w100 	veng,
Gui, Add, Edit, w100    vheat,
Gui, Add, Edit, w100    vair,
Gui, Add, Button, gButtonOK Default, OK    
Gui, Show,, EQSO ver. 4
Return

ButtonOK:
Gui, Submit, NoHide

FilePath := "C:\Users\Desktop\test.xlsx"
oExcel := ComObjCreate("Excel.Application")
oExcel.DisplayAlerts := False
oWorkBook := oExcel.Workbooks.Open(FilePath)
oExcel.Visible := False


; This function import text images through the comments in < >
/*Pic(comments, add_new=0) {
  static Lib:=[]
  if (add_new)
  {
    re:="<([^>]*)>[^$]+\$\d+\.[\w+/]{3,}"
    Loop, Parse, comments, |
      if RegExMatch(A_LoopField,re,r)
        Lib[Trim(r1)]:=r
  }
  else
  {
    text:=""
    Loop, Parse, comments, |
      text.="|" . Lib[Trim(A_LoopField)]
    return, text
  }
}
*/

; Usage For example:

; You can put the text library at the beginning of the script
CoordMode, Mouse, Screen
Text:="|<606>*159$64.D900l001lktW403Y00A8Y42z8+Hn0UWkk94UZNA7m/t0YI2J40Ncca+Ek8qH0aWmD9n0XDA3lls0080000000001U0000000U"
Pic(Text,1)
Text:="|<607>*159$64.D900l001lntW403Y00A8UY2z8+Hn0UW6k94UZNA7m8F0YI2J40NcW6+Ek8qH0aW8D9n0XDA3ll00080000000001U0000000U"
Pic(Text,1)
Text:="|<608>*158$64.D900l001lltW403Y00A8cY2z8+Hn0UWmk94UZNA7m9l0YI2J40Ncca+Ek8qH0aWWD9n0XDA3lls0080000000001U0000000U"
Pic(Text,1)

;-- Use Pic(Text,1) to add the text library to Pic()'s Lib


;.....................

;.....................

;.....................

;-- Find one picture at a time
/*if ok:=FindText(27,1061,150000,150000,0,0,Pic("comment1"))
{
  X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, X+=W//2, Y+=H//2
  ;------------------------------
  Click, X, Y
}
*/

;-- Find more pictures at a time
if ok:=FindText(27,1061,150000,150000,0,0,Pic("606|607|608"))
{
	Loop, % ok.MaxIndex()//5
	{
		i:=(A_Index-1)*5, X:=ok[i+1], Y:=ok[i+2]
		W:=ok[i+3], H:=ok[i+4], Comment:=ok[i+5], X+=W//2, Y+=H//2
    ;------------------------------
		if (Comment="606")
		{
			oExcel.Range("B33").Value := eng
			oExcel.Range("C33").Value := heat
			oExcel.Range("D33").Value := air
			oExcel.Save()
			oExcel.Quit()
			oExcel := ""
			
		}
		else if (Comment="607")
		{
			oExcel.Range("B34").Value := eng
			oExcel.Range("C34").Value := heat
			oExcel.Range("D34").Value := air
			oExcel.Save()
			oExcel.Quit()
			oExcel := ""
			
		}
		else if (Comment="608")
		{
			oExcel.Range("B35").Value := eng
			oExcel.Range("C35").Value := heat
			oExcel.Range("D35").Value := air
			oExcel.Save()
			oExcel.Quit()
			oExcel := ""
			
		}
	}
}
This didn't work, not sure if I changed some stuff incorrectly or not
rickytan
Posts: 1
Joined: 11 Jan 2019, 05:25

Re: FindText - Capture screen image into text and then find it

11 Jan 2019, 05:32

Thank you for ur script, it's very interesting!! :clap: :dance: :superhappy:
pedroabs
Posts: 9
Joined: 04 Oct 2017, 23:09

Re: FindText - Capture screen image into text and then find it

29 Jan 2019, 06:50

To: All

Re: Suggestion regarding MouseTip().

What I have done is declared a global variable X and Y in my script.
global X:=""
global Y:=""

Then in MouseTip() {, I just removed the first 2 lines which gets the mouse coordinate. X and Y will be the coordinate of the image found less 10 on X and Y. So, when the image is found, no need to move the mouse to that location, get the current mouse coordinate then flash the red/blue indicator. This is useful when you just want to know the location of the image without moving the mouse for whatever reason.
chris93
Posts: 1
Joined: 29 Jan 2019, 12:17

Re: FindText - Capture screen image into text and then find it

29 Jan 2019, 12:22

Hola amigos soy nuevo en esto y no se si podrían ayudarme.
Como puedo modificar el FindText o como podría hacer, para capturar el texto de una imagen, dándole los valores de x, y, h, w y guarde lo capturado en el portapeles.
feiyue
Posts: 162
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

04 Feb 2019, 06:45

Updated to 6.4 version - 2019/02/04
.... 1. Change the label name and function name to make it easier
.... for “#Include” to be integrated into other scripts as libraries.
.... 2. You can even open the tool directly by calling “Goto, ft_Start”.

Note: If you want to compile and use it, you should put the
“#Include FindText.ahk” statement on the last line of the script.
Last edited by feiyue on 07 Feb 2019, 06:11, edited 11 times in total.
burque505
Posts: 1005
Joined: 22 Jan 2017, 19:37

Re: FindText - Capture screen image into text and then find it

04 Feb 2019, 18:15

@feiyue, thank you very much for this great script and your continued work on it.
Regards,
burque505
klong
Posts: 1
Joined: 04 Feb 2019, 19:15

Re: FindText - Capture screen image into text and then find it

04 Feb 2019, 19:27

every tre sti tutut ltulal , goyto and sub
pedroabs
Posts: 9
Joined: 04 Oct 2017, 23:09

Re: FindText - Capture screen image into text and then find it

14 Feb 2019, 12:44

Hi feiyue,

Sorry for an out of topic comment here but this is regarding another awesome script that you have written called AHK Encryptor v2.7. I cannot comment on the thread because the site throws a "You are not authorised to read this forum." error. I was able to view it if I am not logged in to the site.

The said script does not support FileInstall. I tried to compile a script that has this line/command and was encoded by your script but the file was not included in the exe generated. A workaround I have done is to put the FileInstall lines in the encoded script (after encrypting my script using your script).

It is not a major issue since only few lines will not be encrypted but is there a better way to do it so the FileInstall lines will also be encrypted?

Thank you in advance and thanks for the FindText() script. It is really very useful and I continue to use it on daily basis over imagesearch.
nou
Posts: 25
Joined: 24 Feb 2019, 21:21

Re: FindText - Capture screen image into text and then find it

24 Feb 2019, 21:51

Hey Feiyue,

I stumbled upon FindText() not too long ago (a week or two), and it's amazing! Props to such a good work!

One addition I would like to throw in is the MouseTip. More often than not, just the flashing itself is enough to show me where the location is, and I don't need to move the mouse at all. I've modified the MouseTip() to be as followed (as oppose to what pedroabs suggested by setting the "x" and "y" variable as "global" ) : Removed the 2 dll calls to grab the cursor position, and put a "X" and "Y" parameter in the function. The info from the parameters will be from the search

Code: Select all

MouseTipNoMove(x,y) {
  x:=Round(x-10), y:=Round(y-10), w:=h:=2*10+1
  ;-------------------------
  Gui, _MouseTipNoMove_: +AlwaysOnTop -Caption +ToolWindow +Hwndmyid +E0x08000000
  Gui, _MouseTipNoMove_: Show, Hide w%w% h%h%
  ;-------------------------
  dhw:=A_DetectHiddenWindows
  DetectHiddenWindows, On
  d:=4, i:=w-d, j:=h-d
  s=0-0 %w%-0 %w%-%h% 0-%h% 0-0
  s=%s%  %d%-%d% %i%-%d% %i%-%j% %d%-%j% %d%-%d%
  WinSet, Region, %s%, ahk_id %myid%
  DetectHiddenWindows, %dhw%
  ;-------------------------
  Gui, _MouseTipNoMove_: Show, NA x%x% y%y%
  Loop, 4 {
    Gui, _MouseTipNoMove_: Color, % A_Index & 1 ? "Red" : "Blue"
    Sleep, 500
  }
  Gui, _MouseTipNoMove_: Destroy
}
Then, during the search, you can pass the "X" and the "Y" coordinates as follow:

Code: Select all

if (ok:=FindText(268-150000//2, 107-150000//2, 150000, 150000, 0, 0, Text))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
  MouseTipNoMove(x,y) ; passes the location "X" and "Y" into the function
  }

hackslash
Posts: 1
Joined: 05 Mar 2019, 17:33

Re: FindText - Capture screen image into text and then find it

05 Mar 2019, 17:39

Thanks very much for all your hard work!
canga
Posts: 8
Joined: 21 Apr 2018, 11:35

Re: FindText - Capture screen image into text and then find it

28 Mar 2019, 12:48

Is it possible to use this code for Pulover's Macro Creator? If yes, how can I convert?
snowiscold1
Posts: 2
Joined: 01 Apr 2019, 08:05

Re: FindText - Capture screen image into text and then find it

01 Apr 2019, 08:14

Hi feiyue,

I am new to autohotkey and I found out about your script a month ago. I have been working on my games in Bluestack, but I have 1 issue. I am unable to pass the parameter of EmulatorArea variable to the function. It gives me this error. I have tried with and without percentage sign. Please help me thank you.
test.png
test.png (23.92 KiB) Viewed 2021 times

Code: Select all

#SingleInstance Force
EmulatorArea=145,645,244,751

t1:=A_TickCount


Text:="|<>*122$41.zVzzzzzytzzzzzvvzzzzzrq3X4DzjgquTTzTPg5szyyrPvhzytanrPzyD3ki7zzyzzzzzzxzzzzzzvzzzzzzzzzzzzzzzzzzzzzzzzsTzzzzzrTzzzzzizzzzzzRX6vcsk7BqmLiZqvhJb1PhrOfayrPil7YxVrXrQQ/U"

if (ok:=FindText(EmulatorArea, 0, 0, Text))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
}

MsgBox, 4096,, % "Time:`t" (A_TickCount-t1) " ms`n`n"
  . "Pos:`t" X ", " Y "`n`n"
  . "Result:`t" (ok ? "Success !":"Failed !"), 3
MouseMove, X, Y
(ok) && MouseTip()



Return to “Scripts and Functions”

Who is online

Users browsing this forum: BruceO, SpeedMaster and 25 guests