FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
bautai
Posts: 17
Joined: 29 Nov 2020, 02:54

Re: FindText - Capture screen image into text and then find it

17 Dec 2020, 07:38

hi @feiyue , can you add a button to save the screenshot?
because when I tried the SaveFile function, it didn't work very well. It will produce a screenshot that is far from coordinate.

Code: Select all

Text:="|<>*157$26.xTs1zTy03nz80wzW0CSL43bbGUtv0kCSE6XD00ynk0wsw04+TU02b4077U01prk0BTKU3LUU1ZsA4uS1ESz2E/DlYDrs3E3s0k0y0Bsz0STzkEDzw1bTz3rzzXzzzXzxvlzUxszmTwO"
; file := 3.png
 if (ok:=FindText(299-150000, 362-150000, 299+150000, 362+150000, 0, 0, Text))
 {
   
	 WinGetPos, x1, y1, w1, h1, A
   X:=(ok.1.1)-x1, Y:=(ok.1.2)-y1, W:=(ok.1.3)-w1, H:=(ok.1.4)-h1, Comment:=ok.1.id
	 CoordMode, Mouse, Window
	 ; FindText_Gui("ShowScreenShot")
	FindText.SavePic("a.bmp", X, Y, W, H)
   Click, %X%, %Y%
 }
 
feiyue
Posts: 261
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

17 Dec 2020, 11:43

feiyue wrote:
12 Dec 2020, 06:09
The SavePic() function, similar to the CaptureScreen() function,
is used to capture screen shots and save to a file,
but it can only be saved as a BMP file. :dance: :beer:

Code: Select all


F1::
r:=FindText_Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic("c:\a.bmp", x1, y1, x2, y2)
Run, c:\a.bmp
return

@bautai
Note: FindText.SavePic(file, x1, y1, x2, y2) uses two coordinates
to determine the range instead of one plus width and height.
You can use it like this: :beer:

Code: Select all

Text:="|<>*157$26.xTs1zTy03nz80wzW0CSL43bbGUtv0kCSE6XD00ynk0wsw04+TU02b4077U01prk0BTKU3LUU1ZsA4uS1ESz2E/DlYDrs3E3s0k0y0Bsz0STzkEDzw1bTz3rzzXzzzXzxvlzUxszmTwO"
; file := 3.png
 if (ok:=FindText(299-150000, 362-150000, 299+150000, 362+150000, 0, 0, Text))
 {
   X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4
   FindText.SavePic("a.bmp", X, Y, X+W-1, Y+H-1, ScreenShot:=0)
   CoordMode, Mouse
   Click, % ok.1.x  ","  ok.1.y
}
This is just a fun feature, it has little to do with the main features,
so I'm not going to add it to the main interface.
feiyue
Posts: 261
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

17 Dec 2020, 12:12

metaprogram wrote:
16 Dec 2020, 16:38
Hi @feiyue , thanks for your excellent script. I'm trying to port the refScan algorithm to C#, so instead of the mcode I want to just use the regular C code that is in the comments at the bottom, and port that code to C#. But I notice the C function signature takes an extra argument Stride that you don't seem to pass in when calling the mcode.

Was the mcode compiled from the same exact source code that's in the comments?
@metaprogram DllCall and C language function are one-to-one corresponding:

Code: Select all

DllCall(&MyFunc, "int",mode, "uint",color
    , "uint",n, "int",offsetX, "int",offsetY, Ptr,bits.Scan0
    , "int",bits.Stride, "int",sx, "int",sy, "int",sw, "int",sh
    , Ptr,&gs, Ptr,&ss, "AStr",text, Ptr,&s1, Ptr,&s0
    , Ptr,&input, "int",num, Ptr,&allpos, "int",allpos_max)
 ----------------------------
 int __attribute__((__stdcall__)) PicFind(
  int mode, unsigned int c, unsigned int n
  , int offsetX, int offsetY, unsigned char * Bmp
  , int Stride, int sx, int sy, int sw, int sh
  , unsigned char * gs, char * ss, char * text
  , int * s1, int * s0, int * input, int num
  , unsigned int * allpos, int allpos_max )
metaprogram
Posts: 4
Joined: 16 Dec 2020, 16:31

Re: FindText - Capture screen image into text and then find it

17 Dec 2020, 12:25

Oh silly me, I have an old version of the code where it passes bits.1 and bits.2 and I didn't trace this back to realize that bits.2 was Stride :D
feiyue
Posts: 261
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

17 Dec 2020, 12:59

@keytrap
This function has no OCR effect and can only find the text in the text library.
You can wait for the text to be prepared. For example: :beer:

Code: Select all


WaitPic(time, x1, y1, x2, y2, Text, click:=1, offsetX:=0, offsetY:=0, WaitNotExist:=0)
{
  timeout:=A_TickCount+Round(time*1000), found:=0
  Loop {
    ok:=FindText(x1, y1, x2, y2, 0, 0, Text)
    if (!found and ok)
    {
      found:=1
      if (click)
      {
        CoordMode, Mouse, % ("Screen", cmm:=A_CoordModeMouse)
        Click, % (ok.1.x+offsetX) "," (ok.1.y+offsetY)
        Sleep, 100
        CoordMode, Mouse, %cmm%
      }
      if (!WaitNotExist)
        return ok
    }
    if (found and WaitNotExist and !ok)
      return 1
    if (time!="" and A_TickCount>=timeout)
      return 0
    Sleep, 10
  }
}

metaprogram
Posts: 4
Joined: 16 Dec 2020, 16:31

Re: FindText - Capture screen image into text and then find it

18 Dec 2020, 14:30

@feiyue given a text string that has been converted from an image, is there any way to go the reverse direction? Re create the original bmp from text and save it?
feiyue
Posts: 261
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

19 Dec 2020, 21:07

Updated to 7.8 version - 2020/12/20 :D
.... 1. Modify: FindText_Gui() function is moved to the FindText class, reducing the extra functions.
.... 2. The text of the interface is read from one language library, which is easy to translate into other languages.
srooo
Posts: 4
Joined: 05 Jun 2019, 14:50

Re: FindText - Capture screen image into text and then find it

29 Dec 2020, 02:04

feiyue wrote:
17 Dec 2020, 11:43
feiyue wrote:
12 Dec 2020, 06:09
The SavePic() function, similar to the CaptureScreen() function,
is used to capture screen shots and save to a file,
but it can only be saved as a BMP file. :dance: :beer:

Code: Select all


F1::
r:=FindText_Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic("c:\a.bmp", x1, y1, x2, y2)
Run, c:\a.bmp
return

@bautai
Note: FindText.SavePic(file, x1, y1, x2, y2) uses two coordinates
to determine the range instead of one plus width and height.
You can use it like this: :beer:

Code: Select all

Text:="|<>*157$26.xTs1zTy03nz80wzW0CSL43bbGUtv0kCSE6XD00ynk0wsw04+TU02b4077U01prk0BTKU3LUU1ZsA4uS1ESz2E/DlYDrs3E3s0k0y0Bsz0STzkEDzw1bTz3rzzXzzzXzxvlzUxszmTwO"
; file := 3.png
 if (ok:=FindText(299-150000, 362-150000, 299+150000, 362+150000, 0, 0, Text))
 {
   X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4
   FindText.SavePic("a.bmp", X, Y, X+W-1, Y+H-1, ScreenShot:=0)
   CoordMode, Mouse
   Click, % ok.1.x  ","  ok.1.y
}
This is just a fun feature, it has little to do with the main features,
so I'm not going to add it to the main interface.
Can I easily convert the saved bmp file to text for future use?
Thanks again
feiyue
Posts: 261
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

29 Dec 2020, 15:53

@srooo You can try this: :beer: :dance:

Code: Select all


#Include <FindText>

F1::
;-----------
r:=FindText.Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic(file:="c:\a.bmp", x1, y1, x2, y2)
Text1:=FindText.GetTextFromScreen(x1, y1, x2, y2)
;-----------
Gui, New
Gui, +LastFound +AlwaysOnTop -Caption +ToolWindow -DPIScale
Gui, Margin, 0, 0
Gui, Add, Picture,, % file:="c:\a.bmp"
Gui, Show
DetectHiddenWindows, Off
WinWait, % "ahk_id " WinExist()
WinGetPos, x, y, w, h
x1:=x, y1:=y, x2:=x+w-1, y2:=y+h-1
Clipboard:=Text2:=FindText.GetTextFromScreen(x1, y1, x2, y2)
Gui, Destroy
MsgBox, 4096,, % "Text1 = Text2  -->  " (Text1=Text2)
;-----------
Gui, New
Gui, +LastFound +AlwaysOnTop
Gui, Font, s9, Verdana
Gui, Add, Edit, w800 h600 -Wrap +HScroll, % FindText.ASCII(Text2)
Gui, Show,, Text2 is in Clipboard !
DetectHiddenWindows, Off
WinWaitClose, % "ahk_id " WinExist()
Gui, Destroy
return

srooo
Posts: 4
Joined: 05 Jun 2019, 14:50

Re: FindText - Capture screen image into text and then find it

29 Dec 2020, 19:50

feiyue wrote:
29 Dec 2020, 15:53
@srooo You can try this: :beer: :dance:

Code: Select all


#Include <FindText>

F1::
;-----------
r:=FindText.Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic(file:="c:\a.bmp", x1, y1, x2, y2)
Text1:=FindText.GetTextFromScreen(x1, y1, x2, y2)
;-----------
Gui, New
Gui, +LastFound +AlwaysOnTop -Caption +ToolWindow -DPIScale
Gui, Margin, 0, 0
Gui, Add, Picture,, % file:="c:\a.bmp"
Gui, Show
DetectHiddenWindows, Off
WinWait, % "ahk_id " WinExist()
WinGetPos, x, y, w, h
x1:=x, y1:=y, x2:=x+w-1, y2:=y+h-1
Clipboard:=Text2:=FindText.GetTextFromScreen(x1, y1, x2, y2)
Gui, Destroy
MsgBox, 4096,, % "Text1 = Text2  -->  " (Text1=Text2)
;-----------
Gui, New
Gui, +LastFound +AlwaysOnTop
Gui, Font, s9, Verdana
Gui, Add, Edit, w800 h600 -Wrap +HScroll, % FindText.ASCII(Text2)
Gui, Show,, Text2 is in Clipboard !
DetectHiddenWindows, Off
WinWaitClose, % "ahk_id " WinExist()
Gui, Destroy
return

Thank you for your great work, it is interesting and practical. :thumbup: :bravo:
Dillweed
Posts: 3
Joined: 10 Oct 2020, 12:33

Re: FindText - Capture screen image into text and then find it

30 Dec 2020, 17:41

@feiyue
Hey there. Sorry the repeat question, but didn't receive an answer. I have the following simple code

Code: Select all

Text:="|<>*146$6.0TAT0U"

Loop {
    ok:=FindText(0, 0, 150000, 150000, 0, 0, Text)
    CoordMode, Mouse
    X:=ok.1.x, Y:=ok.1.y
    ;Click, %X%, %Y%, 0
} until (x != "")
I've found that if I use FindText then the script doesn't exit after doing code and I have to manually right-click on the systemtray icon and select exit.

If I do the following then the script will cleanly exit.

Code: Select all

Loop {
	ImageSearch, StarX, StarY, 0, 0, %DWidth%, %DHeight%, %StarImage%
} until (ErrorLevel = 0)
I guess I could use a Exitapp, but I would think that after FindText does it's business then it would cleanly exit. Does this make sense? Do you know why ahk scripts stay open?

Thanks for the help!
feiyue
Posts: 261
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

31 Dec 2020, 06:16

#Persistent

Keeps a script permanently running (that is, until the user closes it or ExitApp is encountered).

If this directive is present anywhere in the script, that script will stay running after the auto-execute section (top part of the script) completes.
This is useful in cases where a script contains timers and/or custom menu items but not hotkeys, hotstrings, or any use of OnMessage() or Gui.

Once a script has at least one hotkey, it becomes persistent, meaning that ExitApp rather than Exit should be used to terminate it.
Hotkey scripts are also automatically #SingleInstance unless #SingleInstance Off has been specified.

Any script that calls OnMessage anywhere is automatically persistent.
It is also single-instance unless #SingleInstance has been used to override that.

Any script that uses the GUI command anywhere is automatically persistent (even if the GUI command is never actually executed).
It is also single-instance unless the #SingleInstance directive has been used to override that.

The code for FindText function contains GUI instructions.
Dillweed
Posts: 3
Joined: 10 Oct 2020, 12:33

Re: FindText - Capture screen image into text and then find it

31 Dec 2020, 10:13

Hey, thanks for the reply. I understand what you are saying, but those lines from the previous message are the entire script and nothing else. If I use built in imagesearch then the script starts and then exits. If I use FindText then the starts and stays running. Do you know why FindText would cause the script to stay running?

edit: Ah, nevermind. I took out all the GUI stuff in the FindText script and now my scripts exit cleanly.
Last edited by Dillweed on 31 Dec 2020, 17:34, edited 1 time in total.
shenanigans
Posts: 3
Joined: 31 Dec 2020, 11:31

Re: FindText - Capture screen image into text and then find it

31 Dec 2020, 11:37

I just downloaded the script and am having what is likely a simple issue that I'm overlooking. When running the script, I press the capture button and get an outlined box that blinks red/blue. I can't seem to select any text or screen area when this happens. I end up having to start the script over again. Am I doing something dumb?

edit 1: I should mention that I am using remote desktop to access this windows pc. So while right click seems right, I'm afraid it's not what AHK is looking for.
edit 2: It because I'm going through remote desktop, the right clicks just aren't the same. I went to the PC I'm controlling, right clicked and FindText worked just fine. If anyone knows how I can use it through a remote connections, please let me know.
feiyue
Posts: 261
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

31 Dec 2020, 13:40

Your remote control can't simulate the physical state of keys,
It may only simulate the logic state of the key.
You can try to replace All GetKeyState("RButton","P") with GetKeyState("RButton") in the code.
Or try the CTRL key instead of the right key to capture the picture.
shenanigans
Posts: 3
Joined: 31 Dec 2020, 11:31

Re: FindText - Capture screen image into text and then find it

31 Dec 2020, 14:44

feiyue wrote:
31 Dec 2020, 13:40
Your remote control can't simulate the physical state of keys,
It may only simulate the logic state of the key.
You can try to replace All GetKeyState("RButton","P") with GetKeyState("RButton") in the code.
Or try the CTRL key instead of the right key to capture the picture.
The CTRL key works like a charm. Thanks. Problem Solved!
necomerx
Posts: 30
Joined: 01 Jan 2021, 09:09

Re: FindText - Capture screen image into text and then find it

05 Jan 2021, 17:07

Thanks you feiyue for your awesome script!
Qhimin
Posts: 15
Joined: 30 Nov 2020, 19:24

Re: FindText - Capture screen image into text and then find it

06 Jan 2021, 09:18

How can I get the Gray2Two Text through code using GetTextFromScreen() ?

Thanks!
Qhimin
Posts: 15
Joined: 30 Nov 2020, 19:24

Re: FindText - Capture screen image into text and then find it

11 Jan 2021, 20:16

feiyue wrote:
08 Jan 2021, 01:43
@Qhimin
Just update it ! :beer:
Thank you very much! U r amazing :D

Return to “Scripts and Functions”

Who is online

Users browsing this forum: boiler and 19 guests