FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 15 Nov 2021, 22:40

Shikster wrote:
15 Nov 2021, 17:31
I want to store this value and use it for something else later. What would I have to do in order to achieve that?
Good Question, do you mean in the same script? Because I think it has to be.

Shikster
Posts: 3
Joined: 15 Nov 2021, 17:22

Re: FindText - Capture screen image into text and then find it

Post by Shikster » 17 Nov 2021, 11:00

Yep, same script. I'm already some steps ahead now. I saved the X and Y coordinates as variables and that seemed to do the trick.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 17 Nov 2021, 16:12

Shikster wrote:
17 Nov 2021, 11:00
I saved the X and Y coordinates as variables and that seemed to do the trick.
great. Can you kindly show me how you did that?
I've been trying to save finding different X Y coordinates on a loop multiple and saving them to different clipboard variables..... (which seems impossible) and your code involving saving the X, Y as a variable will help. Many thanks. :clap:

Shikster
Posts: 3
Joined: 15 Nov 2021, 17:22

Re: FindText - Capture screen image into text and then find it

Post by Shikster » 18 Nov 2021, 14:14

@SteveMylo


Code: Select all

t1:=A_TickCount, X:=Y:=""

Text:="|
<Shikii>*131$45.zzzzzzzzzzzzzzzy7nzAzaT0CTtbwnlsnzwzzyTaEtbAnnzk3AlaT3yANYwnw3nnADaTwCSNVwnzwnnAbaSTaSNaQnlsnnAnaT0CSNbAny3nnAsaTzzzzzzzzzzzzzzw"

if (ok:=FindText(X, Y, 1379-150000, 821-150000, 1379+150000, 821+150000, 0, 0, Text))

{
  ;FindText().Click(X, Y, "L")
}
;MsgBox, % "" X ", " Y

VarX1:= % X
VarY1:= % Y



;MsgBox, % "Shiki's position:`n" VarX1 ", " VarY1 

;MsgBox, 4096, Tip, % "Found:`t" Round(ok.Length()) . "`n`nPos:`t" X ", " Y


t1:=A_TickCount, X:=Y:=""

Text:="|<Xanafalgue>*136$93.zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzsszzzzzsTyTzzzzzbDzzzzy3znzzzzzyHzzzzznzyTzzzzzkTUt3sA71nlAwwDz7s30C0Uk6Q1bb0zsySMtbaQwn6Awtbz7z3DDkny6NtbaSTkz0Ntk6S0nDAwk3yHknDAAnVaNtba0TWCSNtbaQwnDAwnzwtnXDAsnb6Mlb6CTDa0NtU6Q0nUA0s7lwMHDC4nkaS9kbVzzzzzzzzzzzzDzzzzzzzzzzzzztlzzzzzzzzzzzzzz0Tzzzzzzzzzzzzzy7zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzw"

if (ok:=FindText(X, Y, 635-150000, 566-150000, 635+150000, 566+150000, 0, 0, Text))
{
  ; FindText().Click(X, Y, "L")
}

;MsgBox, 4096, Tip, % "Found:`t" Round(ok.Length()) . "`n`nPos:`t" X ", " Y

VarX2:= % X
VarY2:= % Y
This just works for me.

If I call VarX1 it spits out the X from finding 'Shikii'

If I call VarX2 it spits out the X from 'Xanafalgue'

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 18 Nov 2021, 22:30

Great thanks. I didn’t think of splitting the X & Y.

That helps

Humbug
Posts: 18
Joined: 14 Jul 2017, 14:11

Re: FindText - Capture screen image into text and then find it

Post by Humbug » 20 Nov 2021, 18:21

Hello,
I have an app that has a list in it.
On a screen, a bunch of data is displayed and I can flip to the next page that refreshes the data to the next set.
The problem is, the list "rolls over" infinitely, so to know I went though it once, I would have to check a specific set of data on screen.
This is of course, painful, hence the idea.
Take a "shot" of the data set and signal to me when the same set is on the screen again.

Is it possible to generate the 'Text' element dynamically (i.e. in runtime) somehow?
The data is always in the same position on the screen (the area can be hardcoded), but I do need to use the hotkey to signal the starting point, wherever that may be.

Here's the pseudocode of what I am trying to achieve:
1. Hotkey is hit.
2. Use FindText to get a specific area of the screen for reference - e.g. x1:=660, y1:=80, x2:=1230, y2:=190, and generate the 'Text' for FindText from that area.
3. Set up a loop to use FindText and determine whether screen (well, the specific area we are checking with FindText) has changed.
4a. If screen has changed, break loop (proceed to step 5).
4b. If screen has not changed, Sleep 200 and continue loop.
As you can see, 4a and 4b so far just waits for the first time for the screen to change to prevent early termination.
5. Set up a loop to use FindText and determine whether screen (well, the specific area we are checking with FindText) has returned to the original.
6a. If screen has returned to the original, move the mouse to a specific area of the screen, click, and break the loop (job done).
6b. If screen has not changed, Sleep 200 and continue loop.

Any thoughts how this - well, step 2, to be specific - can be done?
If there is a better way to go on about it then FindText, please let me know.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 29 Nov 2021, 09:02

jsong55 wrote:
28 Apr 2021, 23:05
I solved it lel
I can't get this to work.
Can you kindly give us an example script of a simple FINDTEXT search where you can choose an occourance.
many thanks

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 29 Nov 2021, 16:46

GregTree wrote:
15 May 2019, 14:59
When a screen is scaled, the capture function does not work as expected.
Did you figure this out? I don't think it works as it's stated in the rules of FINDTEXT that all screens need to be matched. I got it working for just one screen, and changing the Scale, and just using the AllAdd button after capture so it searches all SCALES.
But doesn't work if two monitors are at different Scales/resolutions.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 02 Dec 2021, 01:42

*I deleted my Question as it would have been misleading
Last edited by SteveMylo on 05 Mar 2022, 03:02, edited 4 times in total.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 03 Dec 2021, 18:08

*Deleted
Last edited by SteveMylo on 05 Mar 2022, 03:03, edited 1 time in total.

sleepypig
Posts: 1
Joined: 03 Dec 2021, 22:32

Re: FindText - Capture screen image into text and then find it

Post by sleepypig » 03 Dec 2021, 22:38

I have two images for the text "Restart"

"|<>*133$56.0zzzzzzzzk7zzzzzzzwwzzzzzzzzDDzzyzzzxnnsT31kwW01w3UUM7000yStwytntn7U6DDyQyQss1lny7DbDCTzAyNntnlbztDaQyQyQ1UM83DkDnUwC74nw8"

______00000000000000000000000000000000000000000000000000
_______0000000000000000000000000000000000000000000000000
__0000__000000000000000000000000000000000000000000000000
__0000__000000000000000000000_0000000000000000000000_000
__0000__00000____00000____00_____000____0000__0___0_____
_______00000______000_____0______00______000____________
______00000__0000_000__00000__00000_000__000__00000__000
__00___0000________00___0000__000000000__000__00000__000
__000___000________000___000__0000000____000__00000__000
__0000__000__00000000000__00__00000__00__000__00000__000
__0000___00__000000000000__0__00000__00__000__00000__000
__00000__000_______00______00_____0_______00__000000____
__000000__000_____0000____000____000___0__00__000000____

"<>*150$69.0Tzzzzzzzzzs1zzzzzzzzzzD7zzzzzzzzztwzzzzrzzzzrDbzzzwzzzzwtwz1w70w7tX0D7k70E70T8001wQNvbtlsvb0Tbn7wzzD7wtlw0QTbzltzbD7U3kwzkDDwtwwzz3bwttzbDXbzyQzDDDwtyATPnbsltzbDtk30Q307Dw1z7Uw7kw9tzkU"

_______00000000000000000000000000000000000000000000000000000000000000
________0000000000000000000000000000000000000000000000000000000000000
__0000___000000000000000000000000000000000000000000000000000000000000
__00000__00000000000000000000000000_00000000000000000000000000000_000
__00000__0000000000000000000000000__0000000000000000000000000000__000
__00000__000000_____00000_____000______0000_____000000__00___00______
__0000___00000_______000_______0_______000_______00000__0____________
________00000___000___00__0000_000__000000__000___0000___000_000__000
_______000000__00000__00___0000000__000000000000__0000___0000000__000
__000___00000_________000___000000__00000000000___0000__00000000__000
__0000___0000_________0000____0000__00000000______0000__00000000__000
__00000__0000__000000000000____000__0000000__000__0000__00000000__000
__00000___000__00000000000000__000__000000__0000__0000__00000000__000
__000000___00___00000_00_0000__000__000000___00___0000__00000000__000
__0000000__000________00_______000______00_________000__00000000_____
__0000000___0000_____0000_____00000____0000____0__0000__000000000____

They are varied by the font size.
Is there any way we can scale the image search in FindText so that I dont have to maintain two image capture?

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 03 Dec 2021, 22:58

@sleepypig one page one in the rules, no you can’t, all has to be same size
Resolution and Scaling absolutely has to be the same. You could get FindText to search for different scales if you like.

Example ....
set you screen scale to 100% and perform the capture you like.
Then set your scale to 150% for example, exit FINDTEXT GUI , reload it and perform the capture again.
Then combine both text strings (which are at different scales) in the one script.
So when you change scales, your findtext script still works.
For example:

Code: Select all

Text:="|<S>[email protected]$6.TUUkT11VyU"

Text.="|<B>[email protected]$8.zAv6njn6lwPwU"  ; <=== make sure your second Text string has a period dot  (.)  after the word [u]Text[/u] and not a semi-colon   (:) which is the same as clicking ' AllAdd ' after your 1st search. 

if (ok:=FindText(X, Y, 849-150000, 826-150000, 849+150000, 826+150000, 0, 0, Text))
{
   FindText().Click(X, Y, "L")
}
Last edited by SteveMylo on 05 Dec 2021, 05:07, edited 1 time in total.

bobelyk
Posts: 64
Joined: 26 Jun 2021, 16:55

Re: FindText - Capture screen image into text and then find it

Post by bobelyk » 04 Dec 2021, 21:38

Hello, when i capture my blue letter text, there is no way to export it and keep it blue, it always goes black on white, i think i exhausted the options

Also, when using on a different pc with same resolution it won't work, is scalation a thing here?

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 05 Dec 2021, 05:03

bobelyk wrote:
04 Dec 2021, 21:38
when using on a different pc with the same resolution it won't work, is scalation a thing here?=
, Not sure what you mean saving blue text? Save what? the text string or image? The function saves the color ID so it'll know it's blue.

Also yes, resolution and Scaling absolutely has to be the same. You could get FindText to search for different scales if you like.

Example ....
  • set you screen scale to 100% and perform the capture you like.
  • Then set your scale to 150% for example, exit FINDTEXT GUI , reload it and perform the capture again.
Then combine both text strings (which are at different scales) in the one script.
So when you change scales, your findtext script still works.
For example:

Code: Select all

Text:="|<S>[email protected]$6.TUUkT11VyU"

Text.="|<B>[email protected]$8.zAv6njn6lwPwU"  ; <=== make sure your second Text string has a period dot  (.)  after the word [u]Text[/u] and not a semi-colon   (:) which is the same as clicking ' AllAdd ' after your 1st search. 

if (ok:=FindText(X, Y, 849-150000, 826-150000, 849+150000, 826+150000, 0, 0, Text))
{
   FindText().Click(X, Y, "L")
}

chaoscreater
Posts: 59
Joined: 12 Sep 2019, 21:15

Re: FindText - Capture screen image into text and then find it

Post by chaoscreater » 07 Dec 2021, 16:26

I'm having trouble with a specific text/image that I'm trying to capture using FindText. I'm trying to capture an empty tickbox with the label "Cc:" next to it. I have attached files for reference. One shows the CC tickbox ticked and the other shows the CC tickbox unticked.

FindText is able to find the CC tickbox unticked - which is good. However, the problem here is that if the CC tickbox is ticked, FindText will find the Contact tickbox (unticked) as a valid result - which is not what I want. I only care about the CC tickbox. Just run the code below against the 2 attached images and you'll see what I mean.

I DON'T want to use GetRange to limit the search to a specific area. This is because the attached images are not always presented in the same spot and may be on different screen resolutions. Rather than updating the code to work for different screen resolutions and different screen scaling, I just want to search for everything on the screen.

Code: Select all

 t1:=A_TickCount, X:=Y:=""

Text:="|<>**50$41.40807U080E0NU0E0U0VCEU1010q102021020404+04080Aqk80E0D78E0U0000Ty00000000000000000000000000000002"

 if (ok:=FindText(-85-150000, -855-150000, -85+150000, -855+150000, 0, 0, Text))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   ; Click, %X%, %Y%
 }

 MsgBox, 4096, Tip, % "Found:`t" Round(ok.MaxIndex())
   . "`n`nTime:`t" (A_TickCount-t1) " ms"
   . "`n`nPos:`t" X ", " Y
   . "`n`nResult:`t" (ok ? "Success !" : "Failed !")

 for i,v in ok
   if (i<=2)
     FindText.MouseTip(ok[i].x, ok[i].y)
Attachments
Capture2.PNG
Capture2.PNG (3.89 KiB) Viewed 4523 times
Capture1.PNG
Capture1.PNG (3.73 KiB) Viewed 4523 times

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 07 Dec 2021, 19:37

chaoscreater wrote:
07 Dec 2021, 16:26
Rather than updating the code to work for different screen resolutions and different screen scaling, I just want to search for everything on the screen.
The last two posts are about me saying that you need to have different codes for different screen resolutions and scales. I have given examples above, please read. Good thing is, you can combine many resolutions in the one search. You just have to capture all the ones you want. Follow my instructions above.

as for your Tickbox search. This is easily done. But please tell me what you want exactly?
Also, I chose ColorPos as a capture method and it worked. (see script below)

You have to play around with different captures. MultiColor is my favourite and the most precise. ColorPos means everything is B&W so different colours don't matter. Color is good too. I hardly use Grey and GreyDiff anymore, as you saw, it can grab other stuff.

Do you want to automate a click on the box if it's missing? Or vice versa?
I recommend you upgrade to the latest version although it's not totally necessary.

Code: Select all

#Include <FindText>
s::

Text:="|<NoTick>#[email protected]$37.Ty0000E0U00080E0D04080Ak204049n02021gU1010UE0U0VE80E0NgY0807Xb0A0000zw0000U"

Text.="|<Tick>#[email protected]$35.zTzwDzwLznTzkjzjsp3TzTikCzyzTUxzxyzXvzthrbrzsQO"

 if (ok:=FindText(0, 0, 0, 0, 0, 0, Text))  ; all 0's in range means search entire screen. At least in the new Version.

 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   	X := X - 14
	Y := Y + 0
   Click, %X%, %Y%
 }
return

;========================================================;

;~  You don't need all the other code below in a running script  e.g.

;~   t1:=A_TickCount, X:=Y:=""


;~ MsgBox, 4096, Tip, % "Found:`t" Round(ok.Length())
  ;~ . "`n`nTime:`t" (A_TickCount-t1) " ms"
  ;~ . "`n`nPos:`t" X ", " Y
  ;~ . "`n`nResult:`t<" (Comment:=ok[1].id) ">"

;~ for i,v in ok
  ;~ if (i<=4)
    ;~ FindText().MouseTip(ok[i].x, ok[i].y)

yesorno
Posts: 2
Joined: 28 Jul 2019, 03:43

Re: FindText - Capture screen image into text and then find it

Post by yesorno » 09 Dec 2021, 08:29

请问如何指定某个图片文件查找坐标呢,而不是在当前屏幕查找



[Mod edit: DeepL translation added. Please use english in the general forums. Thank you!]
How can I specify the coordinates of an image file search, instead of searching on the current screen?

chaoscreater
Posts: 59
Joined: 12 Sep 2019, 21:15

Re: FindText - Capture screen image into text and then find it

Post by chaoscreater » 11 Dec 2021, 19:00

SteveMylo wrote:
07 Dec 2021, 19:37
chaoscreater wrote:
07 Dec 2021, 16:26
Rather than updating the code to work for different screen resolutions and different screen scaling, I just want to search for everything on the screen.
The last two posts are about me saying that you need to have different codes for different screen resolutions and scales. I have given examples above, please read. Good thing is, you can combine many resolutions in the one search. You just have to capture all the ones you want. Follow my instructions above.

as for your Tickbox search. This is easily done. But please tell me what you want exactly?
Also, I chose ColorPos as a capture method and it worked. (see script below)

You have to play around with different captures. MultiColor is my favourite and the most precise. ColorPos means everything is B&W so different colours don't matter. Color is good too. I hardly use Grey and GreyDiff anymore, as you saw, it can grab other stuff.

Do you want to automate a click on the box if it's missing? Or vice versa?
I recommend you upgrade to the latest version although it's not totally necessary.

Code: Select all

#Include <FindText>
s::

Text:="|<NoTick>#[email protected]$37.Ty0000E0U00080E0D04080Ak204049n02021gU1010UE0U0VE80E0NgY0807Xb0A0000zw0000U"

Text.="|<Tick>#[email protected]$35.zTzwDzwLznTzkjzjsp3TzTikCzyzTUxzxyzXvzthrbrzsQO"

 if (ok:=FindText(0, 0, 0, 0, 0, 0, Text))  ; all 0's in range means search entire screen. At least in the new Version.

 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   	X := X - 14
	Y := Y + 0
   Click, %X%, %Y%
 }
return

;========================================================;

;~  You don't need all the other code below in a running script  e.g.

;~   t1:=A_TickCount, X:=Y:=""


;~ MsgBox, 4096, Tip, % "Found:`t" Round(ok.Length())
  ;~ . "`n`nTime:`t" (A_TickCount-t1) " ms"
  ;~ . "`n`nPos:`t" X ", " Y
  ;~ . "`n`nResult:`t<" (Comment:=ok[1].id) ">"

;~ for i,v in ok
  ;~ if (i<=4)
    ;~ FindText().MouseTip(ok[i].x, ok[i].y)

I do understand what you're saying about updating the code for different screen resolutions and scaling, but that's not very efficient. There are too many different resolutions and scaling options and the image shared in my previous post is not always located in the same spot. This means that there's no point to account for different resolutions and scaling, as you can just get the range for the entire screen.

As for trying different captures, I've already tried that. The code you've shared still doesn't work for me. It captures the "Contact" tickbox when the "CC" tickbox is ticked. The goal is to capture when the CC box is unticked. It should NOT care about the other fields.

Again, the issue is:

- When CC box is unticked - capture is able to successfully detect that CC box is unticked.
- When CC box is ticked - capture should return a failed result. However, capture returns a successful result, because it finds that the Contact box is unticked.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 12 Dec 2021, 04:53

[I tried my code again, I deleted the TICK CODE it does exactly what you want. It only finds the 'Unticked CC' and that's it. It never goes to Contact. ever

So just use the no tick string then.

Code: Select all

Text:="|<NoTick>#[email protected]$37.Ty0000E0U00080E0D04080Ak204049n02021gU1010UE0U0VE80E0NgY0807Xb0A0000zw0000U"
Can I ask what's your screen resolution and scale? Mine is 1080 and scale is 100%

chaoscreater
Posts: 59
Joined: 12 Sep 2019, 21:15

Re: FindText - Capture screen image into text and then find it

Post by chaoscreater » 12 Dec 2021, 16:14

SteveMylo wrote: [I tried my code again, I deleted the TICK CODE it does exactly what you want. It only finds the 'Unticked CC' and that's it. It never goes to Contact. ever

So just use the no tick string then.

Code: Select all

Text:="|<NoTick>#[email protected]$37.Ty0000E0U00080E0D04080Ak204049n02021gU1010UE0U0VE80E0NgY0807Xb0A0000zw0000U"
Can I ask what's your screen resolution and scale? Mine is 1080 and scale is 100%
Tried that, same issue.
image.png
image.png (14.6 KiB) Viewed 4133 times
I'm using 1920x1080 and also another monitor on 2048x1080 and scale is 100%. However, it doesn't matter what I use. Obviously, we're both getting different results and it's because we use different screen resolution and scaling settings. This code needs to run on different PCs and I have no way of knowing what resolution and scaling other people use and the tickboxes could appear on anywhere in the screen (it's not an app I've created).

Post Reply

Return to “Scripts and Functions (v1)”