FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

14 Dec 2019, 15:18

feiyue wrote:
30 Nov 2019, 00:33
You can sort it with SortOK(). For example:
ok:=SortOK( FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, Text) )
I have provided examples, which should be easy to understand.
After sorting, if you want to click the second image, you can do this.

Code: Select all

Click, % ok.2.x, % ok.2.y
ArkuS wrote:
14 Dec 2019, 11:26
My problem is two the same words, which causes FindText to go crazy and choose none of them.
You can test whether two images are found in the tool first.
If you can't find any images, you should re capture them to create a text library.

My advice is to find the only text near the target text, then click it with the offset coordinates.
thebbandit
Posts: 45
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

14 Dec 2019, 23:57

I was playing with this again, and built a wrapper to automatically create a few squares from a hex color. I imagine you would be able to create something MUCH MUCH more robust than this. This function accepts either individual hex colors, or an array of colors. It then allows for the colors to be RGB or BGR, and lets you set the variance. It also allows to create a single pixel square or a 5 pixel square.
Hex2FindText
It would be awesome to allow for both width and height to be adjustable, but I do not have the understanding to do that.
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

15 Dec 2019, 00:59

You can do it use bit2base64().
eg. v:=bit2base64("1111111111111111")
or:
varsetcapacity(s, w*h, chr("1"))
v:=bit2base64(strget(&s, w*h, 0))
thebbandit
Posts: 45
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

15 Dec 2019, 04:32

@feiyue Thanks for the reply, I will play with that and see if I can make something more flexible :)

I found a critical bug as well, related to using findtext function.

If I place two back to back findtext expressions it seems to randomly crash my script.

if (ok:=FindText(x,y,xx,yy,0,0,text))
{
}
else if (Ok:=FindText(x,y,xx,yy,0,0,text))
{
}

(The expressions are done in a loop, so it could be from being too rapidly called)
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

15 Dec 2019, 05:49

Crash is a serious error, which generally means that the AHK process exits without prompt and only keeps the tray icon.

Please distinguish your situation:

1. Did the process exit automatically? This could be an error caused by dllcall.

2. Just can't find the image? This may be the wrong parameter you are using.
thebbandit
Posts: 45
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

15 Dec 2019, 17:25

the process exits and the icon shows in the tray, until i mouse over it.

The image is being found, but possibly too rapidly in sequence from the previous search? If I combine the two searches into one by appending them onto the end of it, it will work. But if I try do the same search in a new Else If FindText() then it crashes

I also found the same to be true when using many ScreenShot() commands too quickly, it seems to make the mouse cursor flicker and I had issues with stability.
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

15 Dec 2019, 23:19

thebbandit wrote:
15 Dec 2019, 17:25
I also found the same to be true when using many ScreenShot() commands too quickly
@thebbandit I think I found out why. :dance: :headwall:

Because we usually use FindText() on a single thread,
But you use a Timer and ScreenShot(), which is multithreaded.
Both FindText() and ScreenShot() call the GetBitsFromScreen() function,
GetBitsFromScreen() uses static variables internally, so it interferes with each other.
If the Timer interrupt occurs while GetBitsFromScreen() is running internally,
the static variable will be modified midway, which causes DllCall() to crash.

I will solve the problem as soon as possible.
Thank you for your feedback. :beer:
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

16 Dec 2019, 16:38

Updated to 7.2 version - 2019/12/16
.... 1. Improvement: the GetBitsFromScreen() function was redesigned.
....... Because some static variables are used in this function to retain
....... the last screenshot data, if multiple threads interrupt each other,
....... the later thread may modify the static variables of this function,
....... which may cause fatal errors after the previous thread resumes execution.
....... Now this problem is solved.
thebbandit
Posts: 45
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

16 Dec 2019, 22:28

Thank you for fixing that issue :bravo: , I can confirm that has resolved the crash that was occuring. I recreated the same if, else if situation and it now works. :dance:

The issue remains though regarding the mouse flickering with the screenshot() function. When doing FindText() rapidly you will not see any flicker, but when you use screenshot it will cause the cursor to flicker no matter how fast. I have found that the flicker becomes more intense the larger of an area it is capturing. When constraining the area of the capture to one screen its marginally noticable, but when capturing both of my screens it causes a very noticable flicker when at a rapid pace.
Demonstration of issue
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

16 Dec 2019, 23:27

If you want to avoid mouse flicker during quick screenshots,
you can modify it as follows. The disadvantage is that some transparent windows cannot be captured.

Code: Select all

0x00CC0020|0x40000000  --> 0x00CC0020
thebbandit
Posts: 45
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

17 Dec 2019, 11:51

@feiyue OMG THANK YOU that fixes the flicker, even at 15ms refresh.
Paul H 81
Posts: 9
Joined: 11 Dec 2019, 22:19

Re: FindText - Capture screen image into text and then find it

17 Dec 2019, 15:27

@thebbandit


Thank you!! I really appreciate the help :)
geeza
Posts: 1
Joined: 18 Dec 2019, 05:12

Re: FindText - Capture screen image into text and then find it

18 Dec 2019, 06:08

EDIT: The issue with my script crashing does not seem to be related to find text it seems to be just a coincidence that it has crashed exactly on findtext multiple times during testing. I was able to switch FindText with PixelSearch and my script still crashes after the script runs over the exact same lines of codes multiple times successfully. Removed the comment because it's irrelevant to this thread.
Hotte
Posts: 32
Joined: 03 Jan 2018, 14:39

Re: FindText - Capture screen image into text and then find it

19 Dec 2019, 12:02

Hi I just stumbled overFindText() - absolutely fantastic! Thank you.

A suggestion: I think FindText() does not find items, wenn they are momentarily covered by other windows -right ? I was discussing FindText() in another thread and i was given this technical following technical advice. Maybe this of interest for you ? (I cannot comment the technical aspects, since I am not an expert and just quoting:
i'm sure findText can be modified to work with windows that are behind others. it depends on how you get the source graphics. with the gdip lib, you can use Gdip_BitmapFromHWND which will use PrintWindow (which is a little slower), or you can use Gdip_BitmapFromScreen("Hwnd:" hwnd) and that will grab the graphics straight from the GetDC, which requires the DWM to be running (always running on win8.1+, needs Aero theme on Win7). i'm sure findText can be modified to use one of these
Is this considerable ?
guest3456
Posts: 3454
Joined: 09 Oct 2013, 10:31

Re: FindText - Capture screen image into text and then find it

20 Dec 2019, 10:11

Hotte wrote:
19 Dec 2019, 12:02
Hi I just stumbled overFindText() - absolutely fantastic! Thank you.

A suggestion: I think FindText() does not find items, wenn they are momentarily covered by other windows -right ? I was discussing FindText() in another thread and i was given this technical following technical advice. Maybe this of interest for you ? (I cannot comment the technical aspects, since I am not an expert and just quoting:
i'm sure findText can be modified to work with windows that are behind others. it depends on how you get the source graphics. with the gdip lib, you can use Gdip_BitmapFromHWND which will use PrintWindow (which is a little slower), or you can use Gdip_BitmapFromScreen("Hwnd:" hwnd) and that will grab the graphics straight from the GetDC, which requires the DWM to be running (always running on win8.1+, needs Aero theme on Win7). i'm sure findText can be modified to use one of these
Is this considerable ?
that was my quote

it looks like FindText() is just using the desktop's DC in GetBitsFromScreen(). you'd have to modify that func to accept a hwnd, and use GetDCex and looks like you probably need to pass in the w/h so do WinGetPos before

here look at this example:

i just used the findtext script to capture the "UTF-8" text in the Notepad status bar at the bottom of the window. then it gave me some test code to copy. and i modified the test code to work when Notepad is obscured. see the diff here:

https://gist.github.com/mmikeww/f61a74bd3518e0b3f8b020e11655eb54/revisions

of course this was just quickly hacked together. you'd have to modify it for your needs

feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

20 Dec 2019, 12:20

Updated to 7.3 version - 2019/12/21
.... 1. Add: the BindWindow() function is added so that the window image can be found in the background.
....... It is a state that affects subsequent FindText() and ScreenShot() functions.

I've implemented the background search window image since a long time ago,
But in win7, the background screenshots of many windows are black,
I think it's not very general, so I canceled this function.

Recently, I found that in win7 and above systems,
Most windows can be screenshots in the background,
Since many friends need it, now add this function. :dance: :beer:
Hotte
Posts: 32
Joined: 03 Jan 2018, 14:39

Re: FindText - Capture screen image into text and then find it

20 Dec 2019, 21:38

Hi, thank you for your kind support!

I have just test-read captured text with V7.3, but it does not find the text, when it is covered by another window (I've got Wndows 10).
Did I get it something wrong oder how does this work ?
feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

21 Dec 2019, 02:11

Let me talk about how to use background search: :dance:

1. After grabbing an image in a window, click bind window button,
and the enlarged image will change dynamically with the movement of the mouse.
At this time, you can switch other windows to block the window
and see if you can perspective the image of the window. (If not, it's bad luck :D )
Right click anywhere to complete this binding window,
Then quickly click the Gray2Two and OK button,
the background search code can be generated.

2. The code of background search starts with the BindWindow(id),
which is equivalent to setting a state.
Subsequent searches will see this window at the top of the screen.
You can cancel this state with the BindWindow(0).

3. Once the BindWindow(id) code is generated,
subsequent capture images can be performed in a normal way,
You don't have to click the bind window button again.
Hotte
Posts: 32
Joined: 03 Jan 2018, 14:39

Re: FindText - Capture screen image into text and then find it

23 Dec 2019, 05:57

Being on holidays at the moment, I shall be looking into this as soon as I am back. Thanks and happy Christmas all around.
dzint
Posts: 2
Joined: 02 Dec 2019, 20:03

Re: FindText - Capture screen image into text and then find it

24 Dec 2019, 17:31

How do you use the fault tolerant feature?

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: charlie89, gwarble, peter_ahk and 121 guests