FindText - Capture screen image into text and then find it

[JavierBO]

@JavierBO You can do this:

Code: Select all

#Include <FindText>

^RButton::
w:=10, h:=6
CoordMode, Mouse
MouseGetPos, x, y
KeyWait, RButton
s:=GetTextFromScreen(x-w,y-h,x+w,y+h)
Clipboard=if (ok:=FindText(0,0,150000,150000,0,0,"%s%"))
MsgBox, 4096,, Copy OK !, 0.5
return

Sorry for the time it took me to answer to your reply.
It doen't seem to be doing anything for me.
MsgBox never shows up

[feiyue]

Sorry, GetTextFromScreen() has been changed to FindText_GetTextFromScreen() in the new version.
If you can't use this combination hotkey (^RButton),
It can be changed to a simple hotkey, such as F1.

[JavierBO]

Sorry, GetTextFromScreen() has been changed to FindText_GetTextFromScreen() in the new version.
If you can't use this combination hotkey (^RButton),
It can be changed to a simple hotkey, such as F1.

Thanks a lot, FindText_GetTextFromScreen() works perfectly.
How can i make it show the red box around the w and h that i set up?
Since i don't have those guide lines to show me exactly where i'm at it kinda messed my x and y positioning

[feiyue]

Updated to 7.6 version - 2020/11/09
.... 1. Modify: The functions are packaged into the FindText class,
....... which can generate multiple objects, bind different windows,
....... and store different screenshots. However, this creates some incompatibility.
....... For example: MouseTip() must be changed to FindText.MouseTip()
.... 2. Modify: Renamed SortOK, SortOK2, OcrOK, ScreenShot_GetColor to Sort, Sort2, Ocr, GetColor.

[c7aesa7r]

@feiyue

Is possible to FindText work on windows that are currently transparent?

Example:

Code: Select all

WinSet,Transparent,0,Firefox
WinGet,ID,ID,Firefox
FindText_BindWindow(ID)

When the window is transparent FindText did not work.
I did a test trying to capture a image of a transparent window like:

Code: Select all

#include gdip.ahk
WinSet,Transparent,0,Firefox
WinGet,ID,ID,Firefox
pBitmap  := Gdip_BitmapFromHWND(ID)
gdip_SaveBitmapToFile(pBitmap, "" A_ScriptDir "\teste.png", 100)	

And it captured the image correctly even with the window full transparent.
Would like to ask if could be possible to FindText read transparent windows

----EDIT----

I have changed:

Code: Select all

  if (bits.hBM) and !(w<1 or h<1)
  {
    win:=DllCall("GetDesktopWindow", Ptr)
    hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
    mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
    oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,bits.hBM, Ptr)

to

Code: Select all

  if (bits.hBM) and !(w<1 or h<1)
  {
    ;win:=DllCall("GetDesktopWindow", Ptr)
    ;hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
    hDC:=DllCall("GetDC","UInt",hWnd)
    mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
    oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,bits.hBM, Ptr)

Now it can capture the window transparent, but my script went from 1.5~% use of cpu to 10~11%. and my mouse pointer keeps flickering the whole time

[joshatt]

Hi,
I'm trying to use findtext to grab the last one of a target in a webpage.
I ran into some issue, so I did a simplified experiment: use a random webpage, grab an area that contains a repeated (two or three times) word and save as jpeg.
With this sample, I used findtext's "test" function, and I tried some different values within Gray, Graydiff, Color, Colordiff......
I'm confused with the result - "Found" should be two or three, but all I got was always 0,1, or thousands of them, instead. For example, with any value from 50 to 254, found is 1, while with value 255, found is 10000.

Any idea?

[tank]

Brilliant Great work

[fe_frtz]

I'm having issue with FindText on my secondary monitor. FindText fail to find whenever my active window is at my secondary monitor.
Secondary monitor is on my left. Tried using negative coordinate as suggested here but still not able to make it work.
On my primary monitor no issue.

Code: Select all

CoordMode, Pixel
CoordMode, Mouse
SysGet, XX2, 78
SysGet, YY2, 79
SysGet, XX1, 76
SysGet, YY1, 77
XX2 := XX1 + XX2
YY2 := YY1 + YY2

Text:="|<3dots>*106$21.nsyQC3VVkQAC3VnsyQ"

if (ok:=FindText(XX1, YY1, XX2, YY2, 0.1, 0.1, Text))
{
	X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
	Click, %X%, %Y%
}

[tpitera]

I have some code improvement suggestions
1. fix the width and height adjusters for the capture screen
A. As a side note I have adjusted these manually in the code and it does increase the capture box size but does not capture the updated box, just the old box
2. Make Capture Get Range which allows you to click and drag box over text you want
3. a button for bringing in your own image (not very necessary but i think would be nice

Really really like this tool, would love it with these additional features

[joshatt]

I think I nailed it - to grab the last one of some repeated text targets in a browser. Problem here is: these texts are created by the browser, and background is pure and blank without image, so the targets should be the same, but they are just not.

For anyone who wants to know, here's my steps:

findtoolstepsformultitargets.jpg (29.89 KiB) Viewed 3620 times

1, capture target text and use GrayDiff 50 or so......I don't know why, but this number can not be too high or too low.
2, add code of line 12, to get the last one of the target texts.
3, use ", ,1" to find multi targets. I tried ",0,1" and it failed.
4, do some real test of value, "0.1,0.1"....."0.2,0.2"...."0.3,0.3", until we find threshold that could help us grab the supposed last target.

[feiyue]

In addition to adjusting the fault tolerance threshold,
I recommend using multiple similarity texts to search together, for example:

Code: Select all

Text:="|<abc_1>......"
Text.="|<abc_2>......"
Text.="|<abc_3>......"
if (ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0.1, 0.1, Text))

[joshatt]

In addition to adjusting the fault tolerance threshold,
I recommend using multiple similarity texts to search together, for example:

Code: Select all

Text:="|<abc_1>......"
Text.="|<abc_2>......"
Text.="|<abc_3>......"
if (ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0.1, 0.1, Text))

Sounds great! I'll try.

[gibbons6546]

Hello,

I am on a three multi-monitor system setup. I'm trying to have FindText locate text strings across all three monitors. However, FindText only locates text strings on the main monitor. I followed instructions in this post and also this post, but neither has fixed the problem. I can't seem to find any other help in this forum thread about this issue.

Can anyone please help me get FindText to search across my entire desktop across all three monitors for a text string? I am using the latest version of FindText.

Thank you very much

[feiyue]

Updated to 7.7 version - 2020/12/10
.... 1. Modify: Images can already be found correctly on multiple displays.
.... 2. Add: the SavePic() function is added to save the screenshot for debugging.
.... 3. Some buttons are added in the main interface to adjust the result Text in the code.

For a long time, I haven't tested the problem of finding pictures with multiple monitors.
Today, I connected a second monitor for testing, and the problem of multiple monitors was solved immediately.

[gibbons6546]

Hi feiyue,

Your update fixed the problem!

Thank you!

Updated to 7.7 version - 2020/12/10
.... 1. Modify: Images can already be found correctly on multiple displays.

For a long time, I haven't tested the problem of finding pictures with multiple monitors.
Today, I connected a second monitor for testing, and the problem of multiple monitors was solved immediately.

[feiyue]

The SavePic() function, similar to the CaptureScreen() function,
is used to capture screen shots and save to a file,
but it can only be saved as a BMP file.

Code: Select all

F1::
r:=FindText_Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic("c:\a.bmp", x1, y1, x2, y2)
Run, c:\a.bmp
return

[fe_frtz]

Great!
Thanks alot for fixing this
Its now working for me.

Updated to 7.7 version - 2020/12/10
.... 1. Modify: Images can already be found correctly on multiple displays.
.... 2. Add: the SavePic() function is added to save the screenshot for debugging.

For a long time, I haven't tested the problem of finding pictures with multiple monitors.
Today, I connected a second monitor for testing, and the problem of multiple monitors was solved immediately.

[Jinpachi]

@feiyue


Hello,
I want to operate with SCRCPY on android but it doesn't work.

[keytrap]

Hey,

I struggled with all the explanations but finally got it to work.

I did a Gif for people who have difficulties to make it work.

[Warning, this is not AutoIt, this is AutoHotKey, you must install the program https://www.autohotkey.com/download/]

1) Copy the whole script and save it as "FindText.ahk"
2) Then, you should be able to open it with AutoHotkey program by double clicking on "FindText.ahk"
3) You should have a Window which appears. Follow my steps :

https i.imgur.com /5zOc7co.gif (delete spaces in link)

Now I have 1 question for @feiyue ? This is perfect when you know what text you are looking for.
Is there any way of making this work when you don't know what text is going to be searched ? I only know the text at runtime (searching for Character name of a game)
How can I generate this (i know the font and the fontsize) :

image.png (12.54 KiB) Viewed 2607 times

[metaprogram]

Hi @feiyue , thanks for your excellent script. I'm trying to port the refScan algorithm to C#, so instead of the mcode I want to just use the regular C code that is in the comments at the bottom, and port that code to C#. But I notice the C function signature takes an extra argument Stride that you don't seem to pass in when calling the mcode.

Was the mcode compiled from the same exact source code that's in the comments?