FindText - Capture screen image into text and then find it

[bautai]

hi @feiyue , can you add a button to save the screenshot?
because when I tried the SaveFile function, it didn't work very well. It will produce a screenshot that is far from coordinate.

Code: Select all

Text:="|<>*157$26.xTs1zTy03nz80wzW0CSL43bbGUtv0kCSE6XD00ynk0wsw04+TU02b4077U01prk0BTKU3LUU1ZsA4uS1ESz2E/DlYDrs3E3s0k0y0Bsz0STzkEDzw1bTz3rzzXzzzXzxvlzUxszmTwO"
; file := 3.png
 if (ok:=FindText(299-150000, 362-150000, 299+150000, 362+150000, 0, 0, Text))
 {
   
	 WinGetPos, x1, y1, w1, h1, A
   X:=(ok.1.1)-x1, Y:=(ok.1.2)-y1, W:=(ok.1.3)-w1, H:=(ok.1.4)-h1, Comment:=ok.1.id
	 CoordMode, Mouse, Window
	 ; FindText_Gui("ShowScreenShot")
	FindText.SavePic("a.bmp", X, Y, W, H)
   Click, %X%, %Y%
 }
 

[feiyue]

The SavePic() function, similar to the CaptureScreen() function,
is used to capture screen shots and save to a file,
but it can only be saved as a BMP file.

Code: Select all

F1::
r:=FindText_Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic("c:\a.bmp", x1, y1, x2, y2)
Run, c:\a.bmp
return

@bautai
Note: FindText.SavePic(file, x1, y1, x2, y2) uses two coordinates
to determine the range instead of one plus width and height.
You can use it like this:

Code: Select all

Text:="|<>*157$26.xTs1zTy03nz80wzW0CSL43bbGUtv0kCSE6XD00ynk0wsw04+TU02b4077U01prk0BTKU3LUU1ZsA4uS1ESz2E/DlYDrs3E3s0k0y0Bsz0STzkEDzw1bTz3rzzXzzzXzxvlzUxszmTwO"
; file := 3.png
 if (ok:=FindText(299-150000, 362-150000, 299+150000, 362+150000, 0, 0, Text))
 {
   X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4
   FindText.SavePic("a.bmp", X, Y, X+W-1, Y+H-1, ScreenShot:=0)
   CoordMode, Mouse
   Click, % ok.1.x  ","  ok.1.y
}

This is just a fun feature, it has little to do with the main features,
so I'm not going to add it to the main interface.

[feiyue]

Hi @feiyue , thanks for your excellent script. I'm trying to port the refScan algorithm to C#, so instead of the mcode I want to just use the regular C code that is in the comments at the bottom, and port that code to C#. But I notice the C function signature takes an extra argument Stride that you don't seem to pass in when calling the mcode.

Was the mcode compiled from the same exact source code that's in the comments?

@metaprogram DllCall and C language function are one-to-one corresponding:

Code: Select all

DllCall(&MyFunc, "int",mode, "uint",color
    , "uint",n, "int",offsetX, "int",offsetY, Ptr,bits.Scan0
    , "int",bits.Stride, "int",sx, "int",sy, "int",sw, "int",sh
    , Ptr,&gs, Ptr,&ss, "AStr",text, Ptr,&s1, Ptr,&s0
    , Ptr,&input, "int",num, Ptr,&allpos, "int",allpos_max)
 ----------------------------
 int __attribute__((__stdcall__)) PicFind(
  int mode, unsigned int c, unsigned int n
  , int offsetX, int offsetY, unsigned char * Bmp
  , int Stride, int sx, int sy, int sw, int sh
  , unsigned char * gs, char * ss, char * text
  , int * s1, int * s0, int * input, int num
  , unsigned int * allpos, int allpos_max )

[metaprogram]

Oh silly me, I have an old version of the code where it passes bits.1 and bits.2 and I didn't trace this back to realize that bits.2 was Stride

[feiyue]

@keytrap
This function has no OCR effect and can only find the text in the text library.
You can wait for the text to be prepared. For example:

Code: Select all

WaitPic(time, x1, y1, x2, y2, Text, click:=1, offsetX:=0, offsetY:=0, WaitNotExist:=0)
{
  timeout:=A_TickCount+Round(time*1000), found:=0
  Loop {
    ok:=FindText(x1, y1, x2, y2, 0, 0, Text)
    if (!found and ok)
    {
      found:=1
      if (click)
      {
        CoordMode, Mouse, % ("Screen", cmm:=A_CoordModeMouse)
        Click, % (ok.1.x+offsetX) "," (ok.1.y+offsetY)
        Sleep, 100
        CoordMode, Mouse, %cmm%
      }
      if (!WaitNotExist)
        return ok
    }
    if (found and WaitNotExist and !ok)
      return 1
    if (time!="" and A_TickCount>=timeout)
      return 0
    Sleep, 10
  }
}

[metaprogram]

@feiyue given a text string that has been converted from an image, is there any way to go the reverse direction? Re create the original bmp from text and save it?

[feiyue]

Updated to 7.8 version - 2020/12/20
.... 1. Modify: FindText_Gui() function is moved to the FindText class, reducing the extra functions.
.... 2. The text of the interface is read from one language library, which is easy to translate into other languages.

[srooo]

The SavePic() function, similar to the CaptureScreen() function,
is used to capture screen shots and save to a file,
but it can only be saved as a BMP file.

Code: Select all

F1::
r:=FindText_Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic("c:\a.bmp", x1, y1, x2, y2)
Run, c:\a.bmp
return

@bautai
Note: FindText.SavePic(file, x1, y1, x2, y2) uses two coordinates
to determine the range instead of one plus width and height.
You can use it like this:

Code: Select all

Text:="|<>*157$26.xTs1zTy03nz80wzW0CSL43bbGUtv0kCSE6XD00ynk0wsw04+TU02b4077U01prk0BTKU3LUU1ZsA4uS1ESz2E/DlYDrs3E3s0k0y0Bsz0STzkEDzw1bTz3rzzXzzzXzxvlzUxszmTwO"
; file := 3.png
 if (ok:=FindText(299-150000, 362-150000, 299+150000, 362+150000, 0, 0, Text))
 {
   X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4
   FindText.SavePic("a.bmp", X, Y, X+W-1, Y+H-1, ScreenShot:=0)
   CoordMode, Mouse
   Click, % ok.1.x  ","  ok.1.y
}

This is just a fun feature, it has little to do with the main features,
so I'm not going to add it to the main interface.

Can I easily convert the saved bmp file to text for future use?
Thanks again

[feiyue]

@srooo You can try this：

Code: Select all

#Include <FindText>

F1::
;-----------
r:=FindText.Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic(file:="c:\a.bmp", x1, y1, x2, y2)
Text1:=FindText.GetTextFromScreen(x1, y1, x2, y2)
;-----------
Gui, New
Gui, +LastFound +AlwaysOnTop -Caption +ToolWindow -DPIScale
Gui, Margin, 0, 0
Gui, Add, Picture,, % file:="c:\a.bmp"
Gui, Show
DetectHiddenWindows, Off
WinWait, % "ahk_id " WinExist()
WinGetPos, x, y, w, h
x1:=x, y1:=y, x2:=x+w-1, y2:=y+h-1
Clipboard:=Text2:=FindText.GetTextFromScreen(x1, y1, x2, y2)
Gui, Destroy
MsgBox, 4096,, % "Text1 = Text2  -->  " (Text1=Text2)
;-----------
Gui, New
Gui, +LastFound +AlwaysOnTop
Gui, Font, s9, Verdana
Gui, Add, Edit, w800 h600 -Wrap +HScroll, % FindText.ASCII(Text2)
Gui, Show,, Text2 is in Clipboard !
DetectHiddenWindows, Off
WinWaitClose, % "ahk_id " WinExist()
Gui, Destroy
return

[srooo]

@srooo You can try this：

Code: Select all

#Include <FindText>

F1::
;-----------
r:=FindText.Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic(file:="c:\a.bmp", x1, y1, x2, y2)
Text1:=FindText.GetTextFromScreen(x1, y1, x2, y2)
;-----------
Gui, New
Gui, +LastFound +AlwaysOnTop -Caption +ToolWindow -DPIScale
Gui, Margin, 0, 0
Gui, Add, Picture,, % file:="c:\a.bmp"
Gui, Show
DetectHiddenWindows, Off
WinWait, % "ahk_id " WinExist()
WinGetPos, x, y, w, h
x1:=x, y1:=y, x2:=x+w-1, y2:=y+h-1
Clipboard:=Text2:=FindText.GetTextFromScreen(x1, y1, x2, y2)
Gui, Destroy
MsgBox, 4096,, % "Text1 = Text2  -->  " (Text1=Text2)
;-----------
Gui, New
Gui, +LastFound +AlwaysOnTop
Gui, Font, s9, Verdana
Gui, Add, Edit, w800 h600 -Wrap +HScroll, % FindText.ASCII(Text2)
Gui, Show,, Text2 is in Clipboard !
DetectHiddenWindows, Off
WinWaitClose, % "ahk_id " WinExist()
Gui, Destroy
return

Thank you for your great work, it is interesting and practical.

[Dillweed]

@feiyue
Hey there. Sorry the repeat question, but didn't receive an answer. I have the following simple code

Code: Select all

Text:="|<>*146$6.0TAT0U"

Loop {
    ok:=FindText(0, 0, 150000, 150000, 0, 0, Text)
    CoordMode, Mouse
    X:=ok.1.x, Y:=ok.1.y
    ;Click, %X%, %Y%, 0
} until (x != "")

I've found that if I use FindText then the script doesn't exit after doing code and I have to manually right-click on the systemtray icon and select exit.

If I do the following then the script will cleanly exit.

Code: Select all

Loop {
	ImageSearch, StarX, StarY, 0, 0, %DWidth%, %DHeight%, %StarImage%
} until (ErrorLevel = 0)

I guess I could use a Exitapp, but I would think that after FindText does it's business then it would cleanly exit. Does this make sense? Do you know why ahk scripts stay open?

Thanks for the help!

[feiyue]

#Persistent

Keeps a script permanently running (that is, until the user closes it or ExitApp is encountered).

If this directive is present anywhere in the script, that script will stay running after the auto-execute section (top part of the script) completes.
This is useful in cases where a script contains timers and/or custom menu items but not hotkeys, hotstrings, or any use of OnMessage() or Gui.

Once a script has at least one hotkey, it becomes persistent, meaning that ExitApp rather than Exit should be used to terminate it.
Hotkey scripts are also automatically #SingleInstance unless #SingleInstance Off has been specified.

Any script that calls OnMessage anywhere is automatically persistent.
It is also single-instance unless #SingleInstance has been used to override that.

Any script that uses the GUI command anywhere is automatically persistent (even if the GUI command is never actually executed).
It is also single-instance unless the #SingleInstance directive has been used to override that.

---

The code for FindText function contains GUI instructions.

[Dillweed]

Hey, thanks for the reply. I understand what you are saying, but those lines from the previous message are the entire script and nothing else. If I use built in imagesearch then the script starts and then exits. If I use FindText then the starts and stays running. Do you know why FindText would cause the script to stay running?

edit: Ah, nevermind. I took out all the GUI stuff in the FindText script and now my scripts exit cleanly.

[shenanigans]

I just downloaded the script and am having what is likely a simple issue that I'm overlooking. When running the script, I press the capture button and get an outlined box that blinks red/blue. I can't seem to select any text or screen area when this happens. I end up having to start the script over again. Am I doing something dumb?

edit 1: I should mention that I am using remote desktop to access this windows pc. So while right click seems right, I'm afraid it's not what AHK is looking for.
edit 2: It because I'm going through remote desktop, the right clicks just aren't the same. I went to the PC I'm controlling, right clicked and FindText worked just fine. If anyone knows how I can use it through a remote connections, please let me know.

[feiyue]

Your remote control can't simulate the physical state of keys,
It may only simulate the logic state of the key.
You can try to replace All GetKeyState("RButton","P") with GetKeyState("RButton") in the code.
Or try the CTRL key instead of the right key to capture the picture.

[shenanigans]

Your remote control can't simulate the physical state of keys,
It may only simulate the logic state of the key.
You can try to replace All GetKeyState("RButton","P") with GetKeyState("RButton") in the code.
Or try the CTRL key instead of the right key to capture the picture.

The CTRL key works like a charm. Thanks. Problem Solved!

[necomerx]

Thanks you feiyue for your awesome script!

[Qhimin]

How can I get the Gray2Two Text through code using GetTextFromScreen() ?

Thanks!

[feiyue]

@Qhimin
Just update it !

[Qhimin]

@Qhimin
Just update it !

Thank you very much! U r amazing