FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 23 May 2022, 09:37

@AHKharry, this is untested but should work:

Code: Select all

If ok:=FindText(outX, outY,574,1092,1107,1387,,,Text)
{
	X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
	FindText().ScreenToWindow(newX, newY, outX, outY, WinExist("A"))
	ControlClick,x%newX% y%newY%, A,,,,NA
}

AHKharry
Posts: 7
Joined: 15 May 2022, 07:40

Re: FindText - Capture screen image into text and then find it

Post by AHKharry » 23 May 2022, 12:01

@Descolada, perfect! Thank you.

AlexNLeo
Posts: 1
Joined: 24 May 2022, 04:03

Re: FindText - Capture screen image into text and then find it

Post by AlexNLeo » 24 May 2022, 04:10

I thank the author for the excellent script that he created. Also, I would like to personally thank Descolada for the excellent tutorial that helped me to use this script. The script works great and I have used it in several of my projects.
At the moment I want to port one of the projects to Python.
Can anyone suggest a similar script for Python or how can this script be used from Python?

feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 27 May 2022, 23:40

Updated to 8.9 version - 2022/05/28 :dance: :beer:
1. Modify: FindText() adds two parameters (zoomW, zoomH) at the end to scale the original image.
2. Modify: Fixed the problem that adding two parameters in v8.6 resulted in incompatible v8.5 code.
3. Modify: Optimize whether BitBlt requires CAPTUREBLT flag. ---- thanks Descolada

User avatar
Automata
Posts: 4
Joined: 30 May 2022, 05:06

Re: FindText - Capture screen image into text and then find it

Post by Automata » 30 May 2022, 05:13

How to reduce delay of wait1? When text appear it takes around +10 seconds to actually detect that it appeared.

Code: Select all

	Text:="|<>*85$30.s1skwztsswztsswzsswtz1swNQ9syNMtsyPNtsy3Ntsz7slsz7U"
	if (ok:=FindText(X:="wait1", Y:=30, 0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, Text))
	{
	  FindText().Click(X, Y, "L")
	}
If I simply slap FindText into loop with 100ms sleep then it finds inside 200-300ms. Would be much more convenient if wait1, wait 0 worked faster.

Thanks

Descolada
Posts: 1098
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 30 May 2022, 07:36

Automata wrote:
30 May 2022, 05:13
How to reduce delay of wait1? When text appear it takes around +10 seconds to actually detect that it appeared.

Code: Select all

	Text:="|<>*85$30.s1skwztsswztsswzsswtz1swNQ9syNMtsyPNtsy3Ntsz7slsz7U"
	if (ok:=FindText(X:="wait1", Y:=30, 0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, Text))
	{
	  FindText().Click(X, Y, "L")
	}
If I simply slap FindText into loop with 100ms sleep then it finds inside 200-300ms. Would be much more convenient if wait1, wait 0 worked faster.

Thanks
"wait1" simply slaps FindText into a loop with 50ms sleep, and it works fine with my test scripts. Perhaps something in your full script is interfering with it somehow?

User avatar
Automata
Posts: 4
Joined: 30 May 2022, 05:06

Re: FindText - Capture screen image into text and then find it

Post by Automata » 30 May 2022, 08:26

Descolada wrote:
30 May 2022, 07:36
"wait1" simply slaps FindText into a loop with 50ms sleep, and it works fine with my test scripts. Perhaps something in your full script is interfering with it somehow?
My script is simple as it gets, what I get directly from findtext gui. Basically by default it finds my text in 60-300ms but if I add X:="wait1", Y:=30 then it takes +10 sec to find or many times even reaches timeout of 30sec.
At some occasion it will find fast if text is already visible on screen. I cant figure out why its so inconsistent.

schuldig
Posts: 2
Joined: 05 Mar 2021, 18:29

Re: FindText - Capture screen image into text and then find it

Post by schuldig » 30 May 2022, 13:33

Hi, I've been working on my script these days and was wondering if there's a possibility to exclude a region on the screen from the search results.
Currently I have four vertical rows that I would like to search but one of them results in false results thus making the script obsolete.

Thank you for your hard work and great library!

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 30 May 2022, 19:23

@schuldigHave you not used the 'get range' to select the search area?
Image
Attachments
image.png
image.png (3.6 KiB) Viewed 4075 times

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 30 May 2022, 19:32

Automata wrote:
30 May 2022, 08:26
I cant figure out why its so inconsistent.
Try writing it like this. I use this script (with my image) and it works instantly.... let me know if it works.

Code: Select all

Text:="|<>*85$30.s1skwztsswztsswzsswtz1swNQ9syNMtsyPNtsy3Ntsz7slsz7U"
FindText(X:="wait", Y:=30, 0,0,A_ScreenWidth,A_ScreenHeight, 0,0,Text)
if (ok:=FindText(X, Y, 0,0,A_ScreenWidth,A_ScreenHeight, 0, 0, Text))
{
	FindText().Click(X, Y, "L")
}

return
~Esc::ExitApp

schuldig
Posts: 2
Joined: 05 Mar 2021, 18:29

Re: FindText - Capture screen image into text and then find it

Post by schuldig » 30 May 2022, 22:49

@Stevmylo,

I have yes but using 2 different selected areas in a search with multiple images didn't work for me... yet

User avatar
Automata
Posts: 4
Joined: 30 May 2022, 05:06

Re: FindText - Capture screen image into text and then find it

Post by Automata » 31 May 2022, 03:37

SteveMylo wrote:
30 May 2022, 19:32
Try writing it like this. I use this script (with my image) and it works instantly.... let me know if it works.
Still the same, taking enormous time to find.
Image
However if I remove the wait line FindText(X:="wait", Y:=30, 0,0,A_ScreenWidth,A_ScreenHeight, 0,0,Text) it finds text instantly if its visible.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 31 May 2022, 03:41

@Automataok have you tried it on a normal simple image somewhere on your computer? see if the 'wait' works. The issue may be your image. Especially text.

feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 31 May 2022, 04:08

@Automata If you look at the following source code, you will know:
The wait mode will intelligently wait for the image to be stable before returning,
which prevents the function from returning when the page is still changing dynamically,
resulting in subsequent click errors. However, the side effect is that it is not applicable to the constantly changing dynamic images.

Code: Select all

      ; Wait for the image to remain stable
      While (ok:=this.FindText(OutputX, OutputY
        , x1, y1, x2, y2, err1, err0, text, ScreenShot, FindAll
        , JoinText, offsetX, offsetY, dir, zoomW, zoomH))
        and (found)
      {
        v:=ok[1], x:=v[1], y:=v[2], w:=v[3], h:=v[4]
        hash:=this.GetPicHash(x, y, x+w-1, y+h-1, 0)
        Sleep, 10
        if (hash=this.GetPicHash(x, y, x+w-1, y+h-1, 1))
          return ok
      }

User avatar
Automata
Posts: 4
Joined: 30 May 2022, 05:06

Re: FindText - Capture screen image into text and then find it

Post by Automata » 31 May 2022, 04:23

@feiyue Yeah thats it, there is slightly shifting pixels of the background color at the side of screen but text area is always the same and FindText by itself never had problems finding it. I suppose simply removing GetPicHash comparison fixes the problem

feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 31 May 2022, 05:56

@Automata I updated the following code to ignore slight background changes while waiting for the image to stabilize.

Code: Select all

      ; Wait for the image to remain stable
      While (ok:=this.FindText(OutputX, OutputY
        , x1, y1, x2, y2, err1, err0, text, ScreenShot, FindAll
        , JoinText, offsetX, offsetY, dir, zoomW, zoomH))
        and (found)
      {
        v:=ok[1], x:=v[1], y:=v[2], w:=v[3], h:=v[4]
        Sleep, 10
        if this.FindText(0, 0, x, y, x+w-1, y+h-1
        , err1, err0, text, ScreenShot, FindAll
        , JoinText, offsetX, offsetY, dir, zoomW, zoomH)
          return (this.ok:=ok)
      }

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 31 May 2022, 20:52

schuldig wrote:
30 May 2022, 22:49
I have yes but using 2 different selected areas in a search with multiple images didn't work for me... yet
Correct me if I'm wrong, but FindText works so fast, wouldn't you just do multiple scripts one after the other in different search regions?
Also, *note that if you have identical images, but you only want it to find the 1st or 2nd or 3rd etc instance , then change the ok[1].x, ok[1].y to the particular instance ... e.g. ok[2].x, ok[2].y or you can change the search direction.

Code: Select all

CoordMode, Mouse, Screen
x::
Text:="|<>*170$12.DwTyTzzzzzTzDyDsC0E0U"
if (ok:=FindText(X, Y, 87, 144, 169, 212, 0, 0, Text,,,,,,1))   ; small search range
{
  FindText().Click(ok[1].x, ok[1].y, "L")
  return   ;; remove the 'return' if you want it to click on the next image too
}

Text:="|<>*165$11.D0zX1a382EAkNlVzUtU1U1U"
if (ok:=FindText(X, Y, 157, 205, 345, 305, 0, 0, Text,,,,,,1))   ;; 2nd small search range
{
  FindText().Click(ok[1].x, ok[1].y, "L")
  return
}

return
~Esc::ExitApp

newbie007
Posts: 20
Joined: 21 Jan 2022, 06:34

Re: FindText - Capture screen image into text and then find it

Post by newbie007 » 01 Jun 2022, 00:40

Does anyone else get the error:

Error at line XXX.
Line Text: local
Error: This line does not contain a recognized action.

The program will exit.

What do I do wrong?

PS: I didn't change anything in the script

gregster
Posts: 8916
Joined: 30 Sep 2013, 06:48

Re: FindText - Capture screen image into text and then find it

Post by gregster » 01 Jun 2022, 01:00

newbie007 wrote:
01 Jun 2022, 00:40
Error at line XXX.
Line Text: local
Error: This line does not contain a recognized action.
Probably you are using a wrong (eg AHK v2 beta) or outdated (probably older than v1.1.27) AHK version.

newbie007
Posts: 20
Joined: 21 Jan 2022, 06:34

Re: FindText - Capture screen image into text and then find it

Post by newbie007 » 01 Jun 2022, 01:27

gregster wrote:
01 Jun 2022, 01:00
newbie007 wrote:
01 Jun 2022, 00:40
Error at line XXX.
Line Text: local
Error: This line does not contain a recognized action.
Probably you are using a wrong (eg AHK v2 beta) or outdated (probably older than v1.1.27) AHK version.
This explains alot, I'm on my job pc.. this is probably why.
I will try at home. Thanks mate!

Post Reply

Return to “Scripts and Functions (v1)”