FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
JavierBO
Posts: 14
Joined: 08 Jul 2019, 05:43

Re: FindText - Capture screen image into text and then find it

04 Nov 2020, 12:30

feiyue wrote:
14 Oct 2020, 05:49
@JavierBO You can do this:

Code: Select all

#Include <FindText>

^RButton::
w:=10, h:=6
CoordMode, Mouse
MouseGetPos, x, y
KeyWait, RButton
s:=GetTextFromScreen(x-w,y-h,x+w,y+h)
Clipboard=if (ok:=FindText(0,0,150000,150000,0,0,"%s%"))
MsgBox, 4096,, Copy OK !, 0.5
return
Sorry for the time it took me to answer to your reply.
It doen't seem to be doing anything for me.
MsgBox never shows up
feiyue
Posts: 260
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

04 Nov 2020, 14:34

Sorry, GetTextFromScreen() has been changed to FindText_GetTextFromScreen() in the new version.
If you can't use this combination hotkey (^RButton),
It can be changed to a simple hotkey, such as F1.
JavierBO
Posts: 14
Joined: 08 Jul 2019, 05:43

Re: FindText - Capture screen image into text and then find it

05 Nov 2020, 06:18

feiyue wrote:
04 Nov 2020, 14:34
Sorry, GetTextFromScreen() has been changed to FindText_GetTextFromScreen() in the new version.
If you can't use this combination hotkey (^RButton),
It can be changed to a simple hotkey, such as F1.
Thanks a lot, FindText_GetTextFromScreen() works perfectly.
How can i make it show the red box around the w and h that i set up?
Since i don't have those guide lines to show me exactly where i'm at it kinda messed my x and y positioning
feiyue
Posts: 260
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

08 Nov 2020, 19:03

Updated to 7.6 version - 2020/11/09 :dance: :beer:
.... 1. Modify: The functions are packaged into the FindText class,
....... which can generate multiple objects, bind different windows,
....... and store different screenshots. However, this creates some incompatibility.
....... For example: MouseTip() must be changed to FindText.MouseTip()
.... 2. Modify: Renamed SortOK, SortOK2, OcrOK, ScreenShot_GetColor to Sort, Sort2, Ocr, GetColor.
c7aesa7r
Posts: 108
Joined: 02 Jun 2016, 21:09

Re: FindText - Capture screen image into text and then find it

25 Nov 2020, 18:33

@feiyue

Is possible to FindText work on windows that are currently transparent?

Example:

Code: Select all

WinSet,Transparent,0,Firefox
WinGet,ID,ID,Firefox
FindText_BindWindow(ID)
When the window is transparent FindText did not work.
I did a test trying to capture a image of a transparent window like:

Code: Select all

#include gdip.ahk
WinSet,Transparent,0,Firefox
WinGet,ID,ID,Firefox
pBitmap  := Gdip_BitmapFromHWND(ID)
gdip_SaveBitmapToFile(pBitmap, "" A_ScriptDir "\teste.png", 100)	
And it captured the image correctly even with the window full transparent.
Would like to ask if could be possible to FindText read transparent windows :salute: :thumbup:

----EDIT----

I have changed:

Code: Select all

  if (bits.hBM) and !(w<1 or h<1)
  {
    win:=DllCall("GetDesktopWindow", Ptr)
    hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
    mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
    oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,bits.hBM, Ptr)
to

Code: Select all

  if (bits.hBM) and !(w<1 or h<1)
  {
    ;win:=DllCall("GetDesktopWindow", Ptr)
    ;hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
    hDC:=DllCall("GetDC","UInt",hWnd)
    mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
    oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,bits.hBM, Ptr)
Now it can capture the window transparent, but my script went from 1.5~% use of cpu to 10~11%. and my mouse pointer keeps flickering the whole time :facepalm:
joshatt
Posts: 121
Joined: 07 Dec 2014, 08:41
Facebook: joshatt
Google: joshatt

Re: FindText - Capture screen image into text and then find it

29 Nov 2020, 08:15

Hi,
I'm trying to use findtext to grab the last one of a target in a webpage.
I ran into some issue, so I did a simplified experiment: use a random webpage, grab an area that contains a repeated (two or three times) word and save as jpeg.
With this sample, I used findtext's "test" function, and I tried some different values within Gray, Graydiff, Color, Colordiff......
I'm confused with the result - "Found" should be two or three, but all I got was always 0,1, or thousands of them, instead. For example, with any value from 50 to 254, found is 1, while with value 255, found is 10000.

Any idea?
User avatar
tank
Posts: 2855
Joined: 28 Sep 2013, 22:15
Facebook: charlie.simmons.7334
Google: ttnnkkrr
GitHub: ttnnkkrr
Location: Irving TX
Contact:

Re: FindText - Capture screen image into text and then find it

29 Nov 2020, 14:07

Brilliant Great work
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
https://www.facebook.com/ahkscript.org
If you have forum suggestions please submit a pull request
Check Out WebWriter
Thanks Tank :thumbup:
fe_frtz
Posts: 2
Joined: 01 Dec 2020, 04:19

Re: FindText - Capture screen image into text and then find it

01 Dec 2020, 04:31

I'm having issue with FindText on my secondary monitor. FindText fail to find whenever my active window is at my secondary monitor.
Secondary monitor is on my left. Tried using negative coordinate as suggested here but still not able to make it work.
On my primary monitor no issue.

Code: Select all

CoordMode, Pixel
CoordMode, Mouse
SysGet, XX2, 78
SysGet, YY2, 79
SysGet, XX1, 76
SysGet, YY1, 77
XX2 := XX1 + XX2
YY2 := YY1 + YY2

Text:="|<3dots>*106$21.nsyQC3VVkQAC3VnsyQ"

if (ok:=FindText(XX1, YY1, XX2, YY2, 0.1, 0.1, Text))
{
	X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
	Click, %X%, %Y%
}
tpitera
Posts: 13
Joined: 27 Oct 2020, 15:56

Re: FindText - Capture screen image into text and then find it

04 Dec 2020, 09:16

I have some code improvement suggestions
1. fix the width and height adjusters for the capture screen
A. As a side note I have adjusted these manually in the code and it does increase the capture box size but does not capture the updated box, just the old box
2. Make Capture Get Range which allows you to click and drag box over text you want
3. a button for bringing in your own image (not very necessary but i think would be nice

Really really like this tool, would love it with these additional features
joshatt
Posts: 121
Joined: 07 Dec 2014, 08:41
Facebook: joshatt
Google: joshatt

Re: FindText - Capture screen image into text and then find it

06 Dec 2020, 00:20

I think I nailed it - to grab the last one of some repeated text targets in a browser. Problem here is: these texts are created by the browser, and background is pure and blank without image, so the targets should be the same, but they are just not.

For anyone who wants to know, here's my steps:
findtoolstepsformultitargets.jpg
findtoolstepsformultitargets.jpg (29.89 KiB) Viewed 1221 times
1, capture target text and use GrayDiff 50 or so......I don't know why, but this number can not be too high or too low.
2, add code of line 12, to get the last one of the target texts.
3, use ", ,1" to find multi targets. I tried ",0,1" and it failed.
4, do some real test of value, "0.1,0.1"....."0.2,0.2"...."0.3,0.3", until we find threshold that could help us grab the supposed last target.
feiyue
Posts: 260
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

06 Dec 2020, 01:06

In addition to adjusting the fault tolerance threshold,
I recommend using multiple similarity texts to search together, for example:

Code: Select all

Text:="|<abc_1>......"
Text.="|<abc_2>......"
Text.="|<abc_3>......"
if (ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0.1, 0.1, Text))

joshatt
Posts: 121
Joined: 07 Dec 2014, 08:41
Facebook: joshatt
Google: joshatt

Re: FindText - Capture screen image into text and then find it

08 Dec 2020, 07:22

feiyue wrote:
06 Dec 2020, 01:06
In addition to adjusting the fault tolerance threshold,
I recommend using multiple similarity texts to search together, for example:

Code: Select all

Text:="|<abc_1>......"
Text.="|<abc_2>......"
Text.="|<abc_3>......"
if (ok:=FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0.1, 0.1, Text))

Sounds great! I'll try.
gibbons6546
Posts: 27
Joined: 07 Apr 2020, 11:53

Re: FindText - Capture screen image into text and then find it

09 Dec 2020, 19:34

Hello,

I am on a three multi-monitor system setup. I'm trying to have FindText locate text strings across all three monitors. However, FindText only locates text strings on the main monitor. I followed instructions in this post and also this post, but neither has fixed the problem. I can't seem to find any other help in this forum thread about this issue.

Can anyone please help me get FindText to search across my entire desktop across all three monitors for a text string? I am using the latest version of FindText.

Thank you very much
feiyue
Posts: 260
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

10 Dec 2020, 09:00

Updated to 7.7 version - 2020/12/10
.... 1. Modify: Images can already be found correctly on multiple displays.
.... 2. Add: the SavePic() function is added to save the screenshot for debugging.
.... 3. Some buttons are added in the main interface to adjust the result Text in the code.

For a long time, I haven't tested the problem of finding pictures with multiple monitors.
Today, I connected a second monitor for testing, and the problem of multiple monitors was solved immediately. :D
Last edited by feiyue on 12 Dec 2020, 22:40, edited 2 times in total.
gibbons6546
Posts: 27
Joined: 07 Apr 2020, 11:53

Re: FindText - Capture screen image into text and then find it

10 Dec 2020, 10:13

Hi feiyue,

Your update fixed the problem!

Thank you!
feiyue wrote:
10 Dec 2020, 09:00
Updated to 7.7 version - 2020/12/10
.... 1. Modify: Images can already be found correctly on multiple displays.

For a long time, I haven't tested the problem of finding pictures with multiple monitors.
Today, I connected a second monitor for testing, and the problem of multiple monitors was solved immediately. :D
feiyue
Posts: 260
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

12 Dec 2020, 06:09

The SavePic() function, similar to the CaptureScreen() function,
is used to capture screen shots and save to a file,
but it can only be saved as a BMP file. :dance: :beer:

Code: Select all


F1::
r:=FindText_Gui("GetRange"), x1:=r.1, y1:=r.2, x2:=r.3, y2:=r.4
FindText.SavePic("c:\a.bmp", x1, y1, x2, y2)
Run, c:\a.bmp
return

Last edited by feiyue on 13 Dec 2020, 17:33, edited 3 times in total.
fe_frtz
Posts: 2
Joined: 01 Dec 2020, 04:19

Re: FindText - Capture screen image into text and then find it

12 Dec 2020, 06:14

Great!
Thanks alot for fixing this
Its now working for me.
feiyue wrote:
10 Dec 2020, 09:00
Updated to 7.7 version - 2020/12/10
.... 1. Modify: Images can already be found correctly on multiple displays.
.... 2. Add: the SavePic() function is added to save the screenshot for debugging.

For a long time, I haven't tested the problem of finding pictures with multiple monitors.
Today, I connected a second monitor for testing, and the problem of multiple monitors was solved immediately. :D
Jinpachi
Posts: 2
Joined: 25 Oct 2019, 09:27

Re: FindText - Capture screen image into text and then find it

13 Dec 2020, 15:29

@feiyue


Hello,
I want to operate with SCRCPY on android but it doesn't work.
keytrap
Posts: 1
Joined: 15 Dec 2020, 15:53

Re: FindText - Capture screen image into text and then find it

15 Dec 2020, 16:27

Hey,

I struggled with all the explanations but finally got it to work.

I did a Gif for people who have difficulties to make it work.

[Warning, this is not AutoIt, this is AutoHotKey, you must install the program https://www.autohotkey.com/download/]

1) Copy the whole script and save it as "FindText.ahk"
2) Then, you should be able to open it with AutoHotkey program by double clicking on "FindText.ahk"
3) You should have a Window which appears. Follow my steps :

https i.imgur.com /5zOc7co.gif (delete spaces in link)

Now I have 1 question for @feiyue ? This is perfect when you know what text you are looking for.
Is there any way of making this work when you don't know what text is going to be searched ? I only know the text at runtime (searching for Character name of a game)
How can I generate this (i know the font and the fontsize) :
image.png
image.png (12.54 KiB) Viewed 208 times
metaprogram
Posts: 4
Joined: 16 Dec 2020, 16:31

Re: FindText - Capture screen image into text and then find it

16 Dec 2020, 16:38

Hi @feiyue , thanks for your excellent script. I'm trying to port the refScan algorithm to C#, so instead of the mcode I want to just use the regular C code that is in the comments at the bottom, and port that code to C#. But I notice the C function signature takes an extra argument Stride that you don't seem to pass in when calling the mcode.

Was the mcode compiled from the same exact source code that's in the comments?

Return to “Scripts and Functions”

Who is online

Users browsing this forum: pgeugene and 25 guests