FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
Humbug
Posts: 18
Joined: 14 Jul 2017, 14:11

Re: FindText - Capture screen image into text and then find it

Post by Humbug » 04 Sep 2021, 04:33

feiyue wrote:
02 Sep 2021, 20:16
@Humbug :beer:
1. The search image should be as small as possible, so it is faster and more reliable.
2. For images that can switch the foreground and background colors,
I suggest capturing the image twice, and then connecting it to find it at one time (Text := Text1 . Text2).
3. For the default fault tolerance (0,0), you often need to adjust it yourself in actual use, such as (0.1,0.1).
@feiyue, :thumbup:
1. I understand, I cut it as small as I could, even cut off a few useful pixels to make it even smaller
2. Not sure I understand. The image is white text on dark background (as shown above). In the earlier versions of FindText, there was an invert feature, I am thinking that might be useful here except this feature no longer exists.
3. Yes, I found it, thanks! I went from 0.0 to 0.3 using increments of 0.05 but nothing happened. After a while, I got a lot a of false positives, but still not the right one :-(

diamondfinger
Posts: 3
Joined: 09 Sep 2021, 22:29

Re: FindText - Capture screen image into text and then find it

Post by diamondfinger » 10 Sep 2021, 22:23

Hi guys, I am new to AHK and scripting. Right now, I am using ImageSearch to run a loop for an emulator on my PC (Bluestacks) but I realized its really slow and inefficient. There is this android based game called Final Fantasy Record Keeper, which has this play mode that takes around 5 hours to complete per run manually. Some players make 2 runs per day, but its just a complete waste of time. So someone in the reddit community was kind enough to share their AHK script. Although it works, it works for some and for some people it doesn't. The owner of the script said he's a beginner so I'm assuming this script has a lot of flaws. A video of how it works well (https://www.reddit.com/r/FFRecordKeeper/comments/phkslk/lab_running_ahk_script_and_team_sample/) looks perfect in my eyes, but this second script uses pixelsearch instead of imagesearch, and is highly configured for his own computer, thereby making it difficult for other players to copy and paste.
Getting to my request, can someone be kind enough to take a look at the script and give me some guidance or make revisions? Also, I really want to use FindText instead of ImageSearch, since it takes around 2-5 seconds between clicks, and I feel its looking for the image in the whole screen instead of just the left corner (around 15% of my screen). My screen resolution is 3840 x 2160 with 150% scaling.
I tried searching through forums to understand FindText, but i just don't understand how to use a function. I tried creating a lib folder under the AHK program files, and created the FindText script as instructed, but from there, I got lost. I have around 40 images in the image folder for my script that needs to be clicked. But using FindText, my script just failed to work with error codes. I spent the past 4 days trying to improve the script and getting FindText to work, but I am out of my depth. Please help, thanks!
Attachments
FFRK Labyrinth auto.ahk
(12.98 KiB) Downloaded 207 times

User avatar
Chunjee
Posts: 1400
Joined: 18 Apr 2014, 19:05
Contact:

Re: FindText - Capture screen image into text and then find it

Post by Chunjee » 10 Sep 2021, 23:12

diamondfinger wrote:
10 Sep 2021, 22:23
So someone in the reddit community was kind enough to share their AHK script. Although it works, it works for some and for some people it doesn't.
If it was modified to use FindText instead of pixelsearch it would still be highly configured for one resolution size. Since I do not have a 3840 x 2160 monitor; I'm unable to generate the queries needed to make it work on your screen size.

User avatar
Xtra
Posts: 2744
Joined: 02 Oct 2015, 12:15

Re: FindText - Capture screen image into text and then find it

Post by Xtra » 11 Sep 2021, 01:06

diamondfinger wrote:
10 Sep 2021, 22:23
This will speed up the script some:

Code: Select all

SetBatchLines, -1    ; Put this at top of script (under SendMode Input for example)
Im assuming your window is on the left side of the screen? When it is found the speed is fine,
the issue is when the image is not found in the window. It will search the whole screen trying.

You could do: ( Put this at top of script)

Code: Select all

WinGetPos, winX, winY, winW, winH, YOURGAMEWINTITLEGOESHERE    ; <------------
Then instead of using:

Code: Select all

ImageSearch, FoundX, FoundY, 0, 0, A_ScreenWidth, A_ScreenHeight, *20 %A_ScriptDir%\images\labyrinth_blue.png
You would use: (for all the ImageSearch coords)

Code: Select all

ImageSearch, FoundX, FoundY, winX, winY, winX+winW, winY+winH, *20 %A_ScriptDir%\images\labyrinth_blue.png
Note: if you only need to search part of your window you could tighten each search area more with an +or- offset from the window pos top left corner and do it for each ImageSearch coords of course.
(Think of it as making the search area even smaller in the window search area, something you can try later)

You could also use ControlClick and not worry about moving the mouse back to starting position. (something you can try later)

Instead of getting ahead of yourself trying new things like findtext or gdip etc try to fix the real issues you are having.
I will bet this is more than fast enough for what you want to do.
Take your time and think it through.

HTH

diamondfinger
Posts: 3
Joined: 09 Sep 2021, 22:29

Re: FindText - Capture screen image into text and then find it

Post by diamondfinger » 12 Sep 2021, 01:58

@Xtra Thanks for your detailed response and solution. I worked on it yesterday and it works like a charm. I will work on the part where it will search only part of the window, but need a bit of time. ControlClick is definitely the next step for me! This is a completely new language for me and I need to learn the basics instead of just copy and pasting. Thanks for your help and spending the time to teach me! Appreciate it :-)

feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 12 Sep 2021, 16:18

Updated to 8.6 version - 2021/09/12 :dance: :beer:

1. Modify: FindText() adds two parameters that output X,Y coordinates as the first two parameters of the function,
This is more friendly to novices without adding additional calculation coordinate statements.
However, this will cause some compatibility problems. The previous code needs to add two parameters: FindText(0,0, ......).

2. Add: The first two newly added parameters for returning coordinates are also parameters for starting circular search.
If the first parameter is equal to the string "wait", you can start circular search. See the function description for details.

3. Add: FindText.ToolTip() has been added, which can be displayed in an equal width font, or the font size and color can be modified.
FindText.Click() has been added, which can click absolute coordinates without changing "CoordMode, Mouse" settings.

4. Add the "GetOffset" function in the main window to obtain the offset coordinates in FindText.Click(X+0,Y+0),
It automatically calculates the offset value based on the found image as the origin.

5. Modify: Now the contents of the edit box of the main window will be restored when restarting,
In addition, clicking the "CaptureS" button no longer requires at least one picture in the screenshot directory.

6. Modify: There are some small changes to move closer to v2. Of course, it is still difficult to change to v2.
The class name of V2 cannot have the same name as the function, so the class name of FindText is changed to FindTextClass,
This causes some compatibility problems. The previous "FindText.Method()" needs to be replaced with "FindText().Method()"

deten
Posts: 4
Joined: 22 Sep 2019, 03:33

Re: FindText - Capture screen image into text and then find it

Post by deten » 05 Oct 2021, 14:30

What are some tips to improve the time it takes to find the text?

Also what exactly is the highlighted and underlined items below. From reading it is the upperleft and lower right boundaries for search... but why is it subtraction and addition?

(ok:=FindText(X, Y, 817-150000, 482-150000, 817+150000, 482+150000, 0, 0, TextHooked)

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 06 Oct 2021, 17:42

feiyue wrote:
12 Sep 2021, 16:18
Updated to 8.6 version - 2021/09/12
Thanks for this! Is there any way I can send you money (a small tip) for your hard work?
Can I ask what does the "L" mean in the line ....... FindText().Click(X, Y, "L") ?

Also If I wanted to do a mouse-move instead, Should I just write a normal code e.g. MouseMove, X, Y, 1 .....Or.... Is there a benefit for adding the function ==> FindText(). in it like you did with the FindText().Click ?
Last edited by SteveMylo on 24 Oct 2021, 17:26, edited 2 times in total.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 06 Oct 2021, 17:49

deten wrote:
05 Oct 2021, 14:30
What are some tips to improve the time it takes to find the text?
I find clicking on GetRange speeds things up and you should always use it, even if it's just your entire screen.
Also the shorter the FIND TEXT code the better. Using MultiColour can shorten it considerably. I always use the multicolour method if the colours are always consistent. It goes off how far apart each coloured pixel is to each other.
So for example,...... if you want to find the Bigger yellow circle out of two... just click on two outer edge points only of the bigger yellow circle. That way it will only find the bigger yellow circle very quick cause of the bigger distance apart compared to the smaller yellow circle. This also means it is only two pixels to find thus a very very short FINDTEXT code which works extremely quick.

tpitera
Posts: 31
Joined: 27 Oct 2020, 15:56

Re: FindText - Capture screen image into text and then find it

Post by tpitera » 14 Oct 2021, 13:41

Experiencing a particularly annoying issue
I am running my code, once it hits the #Include the findtext gui pops up. My program does not run. I try closing it while program is running, nothing
This is strange because i have run the program many times and this was not happening. Anyone have this happen to you?? Please help

I have tried deleting gui show in find text, didnt help

Code: Select all

#Include C:\Users\tpitera\Downloads\AHK\Chrome.ahk-master\Chrome.ahk
#Include C:\Users\tpitera\Downloads\AHK\FindText.ahk
;#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
;connect to chrome
FileCreateDir, ChromeProfile
ChromeInst := new Chrome( A_ScriptDir "\ChromeProfile")

; --- Connect to the page ---
if !(PageInst := ChromeInst.GetPage())
{
	MsgBox, Could not retrieve page!
	ChromeInst.Kill()
}
else
{
	; --- Navigate to the desired URL ---
	PageInst.Call("Page.navigate", {"url": "https://erp.i-techcloud.com/MCS/Frames/Login.aspx?ReturnUrl=%2fMCS%2fMain%3fScreenId%3dGI000113&CompanyID=MCS"})
	PageInst.WaitForLoad()
	;; login to Acumatica
	PageInst.Evaluate("document.getElementById('txtUser').value = 'tap';")
	PageInst.Evaluate("document.getElementById('txtPass').value = 'Nwfi9].|';")
	PageInst.Evaluate("document.getElementById('cmbCompany').value = 'MCS';")	
	PageInst.Evaluate("document.getElementById('btnLogin').click();")
	Sleep, 5000  ;sometimes computer lags
	;verify if on data views tab, release cases
	Text:="|<>*152$71.000000000000Ts00E0000000UQ00U00000010M01000000020E02000000040Uw47UT1w3k833A8NVb6AQkTwAAFVa288UUVUM8W304M31V1UUFA60sS6321VzmTwTkDDy43204E1UU3M083628Um344EN"
	if !(ok:=FindText(X, Y, 259-150000, 139-150000, 259+150000, 139+150000, 0, 0, Text))
	{
  		FindText().Click(X, Y, "L")  ;checks for word releases
	}  
	Sleep, 2000
	Text:="|<>*191$47.3k0003TUMk0006lUk7XVkBV109hakP320DQ8Uq640qSz1gAA3ACU3MEMqP9g6lUD7nlkBy000000008"
	if (ok:=FindText(X, Y, 956-150000, 542-150000, 956+150000, 542+150000, 0, 0, Text))
	{
  		FindText().Click(X, Y, "L")
	}
	MouseClick, Left, 412, 339   ;click sort decending
	Sleep, 5000
	Text:="|<>*152$35.0000000000000000000000000000000000007U0000EU0003zk00080E000E0U000U101z10201w20401k40800080E000E7U000UH00010g00021k0003z000000000000000000000000001"
	if (ok:=FindText(X, Y, 959-150000, 543-150000, 959+150000, 543+150000, 0, 0, Text))
	{
  		FindText().Click(X, Y, "L")  ;click copy symbol
	}
	Sleep, 2000
	Text:="|<>*189$59.000000000000000000000000000000000000000000000000000000000000S0000000012003k0000Dz00Ak0000U100kk0001020107Xn42040209aGM408040l8oU80E081WFh0E0U0MP4XC0U100MWNYM10S00S7Xkk21A000041U42k00008208700000EA0Dw00000000000000000000000000000000000000000000001"
	if (ok:=FindText(X, Y, 957-150000, 542-150000, 957+150000, 542+150000, 0, 0, Text))
	{
  		FindText().Click(X, Y, "L")  ;click copy in dropdown
	}
	Sleep, 2000
	Text:="|<>*165$26.000000000000001U000M00060001U000M0006001zzU0Tzs0060001U000M00060001U000M00000000008"
	if (ok:=FindText(X, Y, 960-150000, 543-150000, 960+150000, 543+150000, 0, 0, Text))
	{
  		FindText().Click(X, Y, "L")  ;click plus (Add new record)
	}
	Sleep, 2000
	Text:="|<>*152$35.0000000000000000000000000000000000007U0000EU0003zk00080E000E0U000U101z10201w20401k40800080E000E7U000UH00010g00021k0003z000000000000000000000000001"
	if (ok:=FindText(X, Y, 959-150000, 543-150000, 959+150000, 543+150000, 0, 0, Text))
	{
  		FindText().Click(X, Y, "L")  ;click copy symbol again
	}
	Sleep, 2000
	Text:="|<>*179$59.0s00000000zTU00000018900z00002Tm01300204040220040808044QSQsE0E08N4YG8U0U0TU90YF0zU0U7llDm110108UGE422020H8YW844040uSAwE880000000UEE0000001zUU00000001z00000008"
	if (ok:=FindText(X, Y, 960-150000, 543-150000, 960+150000, 543+150000, 0, 0, Text))
	{
  		FindText().Click(X, Y, "L")  ;click paste
	}
	Sleep, 2000
***	MouseClick, Left, 514, 346  ;click subject textbox
	Send ^a   ;select all
	Send Magnum SW V%NewVersionNum%
	MouseClick, Left, 45, 468  ;click details box
	Send %VersionCtrlText%
***	MouseClick, Left, 298, 397  ;Click Attributes
	if(ReleaseChoice = "Beta")
	{
		MouseClick, Left, 243, 496  ;Click Release checkbo to uncheck
		MouseClick, Left, 245, 526  ;Click Beta
	}
	MouseClick, Left, 258, 581, 2  ;double click to change version
	Send %NewVersionNum%
	Text:="|<>*141$20.000000000000000000000000Ty0+6k2Ve0c2E/zY2010U0E8042010byE90Y2E90Y2E7zs00000000000008"
	if (ok:=FindText(X, Y, 960-150000, 543-150000, 960+150000, 543+150000, 0, 0, Text))
	{
  		FindText().Click(X, Y, "L")  ;find and click save
	}
	MouseClick, Left, 416, 388     ;click activities
	MouseClick, Left, 294, 434     ;click add email
}

Shanghei
Posts: 28
Joined: 04 Jan 2021, 22:42

Re: FindText - Capture screen image into text and then find it

Post by Shanghei » 19 Oct 2021, 09:31

Hello, I have find text setup to trigger if a certain word is posted, I was wondering if there was a way to setup a script to copy the word immediately following the trigger word.

So for example, lets say i have findtext setup to find the world "Hello"

and the phrase Hello world comes up
I want the script to copy the word "world"

thanks for help

User avatar
Chunjee
Posts: 1400
Joined: 18 Apr 2014, 19:05
Contact:

Re: FindText - Capture screen image into text and then find it

Post by Chunjee » 20 Oct 2021, 14:01

Shanghei wrote:
19 Oct 2021, 09:31
Hello, I have find text setup to trigger if a certain word is posted, I was wondering if there was a way to setup a script to copy the word immediately following the trigger word.

So for example, lets say i have findtext setup to find the world "Hello"

and the phrase Hello world comes up
I want the script to copy the word "world"

thanks for help
How about double clicking 50 pixels to the right of the found word?

User avatar
PiecefulPoem
Posts: 6
Joined: 30 Oct 2021, 10:50

Re: FindText - Capture screen image into text and then find it

Post by PiecefulPoem » 30 Oct 2021, 11:06

Hello,

I've been using AutoHotkey for years on a very basic level and recently I came across a task that is begging for some automation. The task has led me to this function/topic.

I'm looking to use a hotkey to start the FindText tool selection box, select the target text, search an excel column for a match, copy the contents of different column but same row to clipboard

Will my Excel file need to be the exact font/size of my target text?

Is this generally something this function is suited for?

Avastgard
Posts: 133
Joined: 30 Sep 2016, 21:54

Re: FindText - Capture screen image into text and then find it

Post by Avastgard » 30 Oct 2021, 16:34

I believe not, since FindText needs the image you are looking for to be visible on the screen.

User avatar
PiecefulPoem
Posts: 6
Joined: 30 Oct 2021, 10:50

Re: FindText - Capture screen image into text and then find it

Post by PiecefulPoem » 31 Oct 2021, 11:10

Yeah, I think I'm going to need to go with OCR.

gregster
Posts: 8916
Joined: 30 Sep 2013, 06:48

Re: FindText - Capture screen image into text and then find it

Post by gregster » 31 Oct 2021, 11:18

PiecefulPoem wrote:
31 Oct 2021, 11:10
Yeah, I think I'm going to need to go with OCR.
For interacting with Excel, you can use the much more reliable COM automation interface. No need to OCR.

User avatar
PiecefulPoem
Posts: 6
Joined: 30 Oct 2021, 10:50

Re: FindText - Capture screen image into text and then find it

Post by PiecefulPoem » 31 Oct 2021, 12:00

For my purpose the window I'm looking at isn't interactable at all and isn't excel. I need to OCR text from it and search in an excel file for a match. I think its going to take a combo of OCR (I'm looking at adapting Vis2 at the moment) and COM.

gregster
Posts: 8916
Joined: 30 Sep 2013, 06:48

Re: FindText - Capture screen image into text and then find it

Post by gregster » 31 Oct 2021, 12:05

PiecefulPoem wrote:
31 Oct 2021, 12:00
For my purpose the window I'm looking at isn't interactable at all and isn't excel.
In ths case, I would also try the accessibility library - Acc.ahk


User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 01 Nov 2021, 20:39

tpitera wrote:
14 Oct 2021, 13:41
I have tried deleting gui show in find text, didnt help
I don't really know but have you tried adding a ' Return ' at the end to close the script?

Post Reply

Return to “Scripts and Functions (v1)”