FindText tutorial

[cgx5871]

Code: Select all

~LButton::
	if ((A_PriorHotkey <> "~LButton") or A_TimeSincePriorHotkey > 400)
	FindText(X:="wait", Y:=0.4, X0-offset, Y0-offset, X0+offset, Y0+offset, 0.000001, 0.000001, Text,1,0)

This "wait" is in conflict with the double-click of the mouse judgment.
Can it be changed to "Settimer" mode?

[Descolada]

@cgx5871, no, you would need to implement it yourself. What are you trying to achieve exactly? Perhaps there is a better way than SetTimer...

[dostroll]

Is there a way to describe it in one line?

work(DISCORD click)

Code: Select all

Text:="|<>*115$58.UQsD3kQ1kA0n0M60k30E3AH2Ml0A07AlwT74QnWQn0lwQFXCBnC17ll0CszAz4T741zXsngFYAF7w03A10M16A10QkC3kAMEC"

if (ok:=FindText(X, Y, 869-150000, 545-150000, 869+150000, 545+150000, 0, 0.000001, Text))
{
  FindText().Click(X, Y, "R")
}

It's not working.(DISCORD click)

Code: Select all

FindText(X, Y, 869-150000, 545-150000, 869+150000, 545+150000, 0, 0.000001, "|<>*115$58.UQsD3kQ1kA0n0M60k30E3AH2Ml0A07AlwT74QnWQn0lwQFXCBnC17ll0CszAz4T741zXsngFYAF7w03A10M16A10QkC3kAMEC").Click(X, Y, "R")

[Descolada]

@dostroll, the FindText function returns either and object containing all the search results, or 0 if nothing was found. Since the object does not contain the Click method then your example unfortunately can't work. You could remove the "if" and just separate the commands by a comma, but that would be unwise since if FindText fails then the script will still click coordinates (0;0) since X and Y will be set to 0.

Your best bet would probably be to create your own FindText-and-click function, something like

Code: Select all

FindTextClick(869-150000, 545-150000, 869+150000, 545+150000, 0, 0.000001, Text)

FindTextClick(args*) {
	if FindText(X, Y, args*) {
		FindText().Click(X, Y, "L")
		return 1
	}
	return 0
}

[dostroll]

@Descolada
Thanks for the sound advice. I learned a lot:)
My next task is to figure out what happens when the Return 0 process fails.

I know this may be a basic mistake, but let me know if you'd like to know.
Declaring vX, vY, vW, vH variables globally didn't work.
The following did not work.

Code: Select all

WinGetPos, vX, vY, vW, vH, ahk_exe chrome.exe
FindTextClick(X, Y, vX, vY, vX+vW, vY+vH, 0.000001, 0, Text)

[Descolada]

@Descolada
Thanks for the sound advice. I learned a lot:)
My next task is to figure out what happens when the Return 0 process fails.

I don't understand what you mean by that. Return 0 cannot fail

I know this may be a basic mistake, but let me know if you'd like to know.
Declaring vX, vY, vW, vH variables globally didn't work.
The following did not work.

Code: Select all

WinGetPos, vX, vY, vW, vH, ahk_exe chrome.exe
FindTextClick(X, Y, vX, vY, vX+vW, vY+vH, 0.000001, 0, Text)

The FindTextClick was just an example to give you an idea of what I meant, and should be customized for your specific needs. For example, FindTextClick(X, Y, vX, vY, vX+vW, vY+vH, 0.000001, 0, Text) won't work because FindTextClick will not return anything with the X and Y variables. So you would need to modify it to FindTextClick(vX, vY, vX+vW, vY+vH, 0.000001, 0, Text), or change the FindTextClick function to return X and Y as well:

Code: Select all

WinGetPos, vX, vY, vW, vH, ahk_exe chrome.exe
FindTextClick(X, Y, vX, vY, vX+vW, vY+vH, 0.000001, 0, Text)

FindTextClick(ByRef X:="", ByRef Y:="", args*) {
	if (ok := FindText(X, Y, args*))
		FindText().Click(X, Y, "L")
	return ok
}

[dostroll]

@Descolada
Thanks, I learned so so so much.
And thanks to Descolada's kind heart, our original goal was achieved!!

[John1]

@Descolada

hello,

sometimes i have images that are already there and sometimes images that will take like 0.1 second to appear.
Can i just have always:

Code: Select all

FindText(X:="wait", Y:=1, 0,0,0,0,0,0,Text)

;________________________
instead sometimes

Code: Select all

FindText(X,Y, 0,0,0,0,0,0,Text)

and sometimes

Code: Select all

FindText(X:="wait", Y:=1, 0,0,0,0,0,0,Text)

Would it make any difference in speed?

thank you!

[Descolada]

@John1, there shouldn't be a speed difference between calling FindText with or without "wait", so you can use the "wait" version. Note that if the image never appears for some reason, then "wait" will still wait the whole second before continuing on with the code, so make sure to not use it in cases where you just need to check whether the Text exists or not, rather than waiting for it.

[John1]

@Descolada Ok sure. Thank you a lot.

[John1]

@Descolada

Hello,

Code: Select all

if (ok := FindText(X,Y,0,0,0,0,0.000001,0.000001, Text,,1,["One","Two","Three"]))
{
    MouseMove, ok[1][1]+15, ok[1][2]+5, 0
    gosub GoSub_Klick
    MouseMove, ok[2][1]+15, ok[2][2]+5, 0
    gosub GoSub_Klick
}

It finds and clicks "One" 2 times, but for "Two" and "Three" it does nothing.
How can i click 2 times all three?

Thank you!

[Descolada]

@John1, you are trying to use OCR to detect three different words? If "One" is composed of multiple Text (eg for characters "O", "n", "e"), then the same word might be found multiple times depending on how close of a match each of the characters were. This is because characters have very small Text and can easily match multiple times. For example, Text

Code: Select all

_o
_o
_o

will match

Code: Select all

_oo_
_oo_
_oo_
_oo_

twice, first for the top half and then one pixel down as well. If using OCR then that will double the amount of results you get.

Try highlighting all the results of your FindText call to see what matches you get and where. If getting more than one match then you would need to filter the results by id/comment.

[John1]

@Descolada

Thank you for your reply.
I know i have "One" 2 times, "Two" 2 times "Three" 2 times on the screen.

Code: Select all

if (ok := FindText(X,Y,0,0,0,0,0.000001,0.000001, Text,,1,["One","Two","Three"]))
{
    MouseMove, ok[1][1]+15, ok[1][2]+5, 0
    gosub GoSub_Klick
    MouseMove, ok[2][1]+15, ok[2][2]+5, 0
    gosub GoSub_Klick
}

Here it clicks 2 times "One", but only "One"

Code: Select all

if (ok := FindText(X,Y,0,0,0,0,0.000001,0.000001, Text,,1,["Two","One","Three"]))
{
    MouseMove, ok[1][1]+15, ok[1][2]+5, 0
    gosub GoSub_Klick
    MouseMove, ok[2][1]+15, ok[2][2]+5, 0
    gosub GoSub_Klick
}

Here it clicks 2 times "Two", but only "Two"

Code: Select all

if (ok := FindText(X,Y,0,0,0,0,0.000001,0.000001, Text,,1,["Three","One","Two"]))
{
    MouseMove, ok[1][1]+15, ok[1][2]+5, 0
    gosub GoSub_Klick
    MouseMove, ok[2][1]+15, ok[2][2]+5, 0
    gosub GoSub_Klick
}

Here it clicks 2 times "Three", but only "Three"

What do i have to change to click 2 times all of the 3 words?

Thank you!

[Descolada]

@John1, as I recommended before, try to figure out which words FindText is finding for you and where, something like this:

Code: Select all

ok := FindText(X,Y,0,0,0,0,0.000001,0.000001, Text,,1,["Three","One","Two"]
for k, v in ok {
    ToolTip, % "Found Text id: " v.id "`nX: " v.x " Y: " v.y
    FindText().RangeTip(v[1], v[2], v[3], v[4])
    Sleep, 2000
}

[hellen_dorandt89]

@Descolada
Hey thanks for this excellent write up. I am looking to this library fill up some some gaps that your UIAutomation cannot fill.

You mentioned in this tutorial that Findtext can find graphics on window behind another, this is excellent as it is one of my concerns.
I would like to ask for my last concern. Does this library find graphics on a window that is off monitor? For example, if I have only one monitor and the top half of the active window is off monitor, can I still find its graphic is on somewhere on its top half area?

Thanks allot for all your amazing work.

[Descolada]

@hellen_dorandt89, yes, it should find the graphics even in off-screen windows, especially if you use FindText().BindWindow(). In some cases you can also interact with off-screen windows: regular Click doesn't work, but ControlClick does work in some windows (though you have to convert the coordinates from Screen to Window for ControlClick to work). For example I've tried this with Chrome, and BindWindow with mode 4 + sending clicks with ControlClick does work.

As a side-note, UIAutomation can usually also detect elements if a windows top half is off-screen - the window has to be at least partially visible.

[Ralf_Reddings200244]

@Descolada

Code: Select all

characterset := "|<0>[b]50$6.SGHnnnHGSU|<1>[/b]50$3.D999A|<2>[b]50$6.SH336AMEzU|<3>[/b]50$6.SG22C33nSU|<4>[b]50$6.66CCKqz66U|<5>[/b]50$6.SEESH33mSU|<6>[b]50$6.CPEkynHHCU|<7>[/b]50$6.z12264A88U|<8>*205$6.SznSSnnnSU|<9>[b]50$6.SGnHT33KSU"

FindText().PicLib(characterset, 1)
searchPhrase := "101"
if (ok:=FindText(X, Y,1095, 1131,1620, 1388, 0, 0,FindText().PicN(searchPhrase)))
    {
    FindText().MouseTip(X, Y)
    }
    return

I followed your notepad example, without modifying it, it works as expected. so I took it as inspiration and transformed it to suit my needs. I am trying to find a number/string in the interface of a window.
My test string is 101, the script will find it successfully at every try.

The problem I am having is that, it will also match/find all the following instances of the window showing:

• 1

• 10

• 01

• 101

• 111

• etc etc

Is it possible for FindText to only confirm a match if the window is displaying 101 and get it to fail to match all other variations?

By the way I just wanted to thank you and appreciate the time put into writing this very detailed article. For a long time I found FindText to be impenetrable. I am actually using it now, thanks to you!

[Descolada]

@Ralf_Reddings200244, PicN() simply fetches Text from string, converting "101" into |<1>[/b]50$3.D999A|<0>[/b]50$6.SGHnnnHGSU|<1>[/b]50$3.D999A (the forums have messed something up there though, I'm not sure what [/b] was supposed to be). That Text will simply search for any of the provided Text, matching all occurrences of both "1" and "0", which is why you are getting so many false results. Instead take a look at section 8 "FindText combination lookup". FindText(X, Y,1095, 1131,1620, 1388, 0, 0,FindText().PicN(searchPhrase),,,1) should do what you want I think.

By the way I just wanted to thank you and appreciate the time put into writing this very detailed article. For a long time I found FindText to be impenetrable. I am actually using it now, thanks to you!

Thanks for the kind words

[Ralf_Reddings200244]

It looks like my bold [\b] tags got mixed into the code.

Instead take a look at section 8 "FindText combination lookup". FindText(X, Y,1095, 1131,1620, 1388, 0, 0,FindText().PicN(searchPhrase),,,1) should do what you want I think.

Ahh, perfect. For prosperity, I will go over this article again as soon as I can. Thanks allot Descolada.

[Ralf_Reddings200244]

How come the ocr function will not find symbols such as square brackets?



(The yellow colour is just for the screenshot)


I am trying to build a library of letters symbols (such as [ or _ ) only for this specific drop down panel.
I have built all the characters (a-z, A-Z) and find text finds them and consistently returns the string in ocr.text but for the life of me I haven not had any success with any symbols.

I have redone the capture to asci text process for these symbols three times but I still cannot get the FindText().OCR function to match/return any of them in ocr.text .

Take the symbol [ for example, when I capture and covert it to a black and white asci text, then test it the GUI. FindText finds it.
But in my own script it will fail, though in my script I am using the FindText().OCR function, here is my script:

Code: Select all

FindText().PicLib("|<[>*125$4.00N4F4F4FU2", 1)  ;<=== In this example, only left bracket is present in library


TopX := 666
TopY := 5
BottomX := 1271
BottomY := 261


ok:=FindText(X, Y, TopX, TopY, BottomX, BottomY, 0, 0, FindText().PicN("["),,,1)

if (ocr := FindText().OCR(ok, 3, 3)){
    MsgBox, % ocr.text
}

Any help would be greatly appreciated!