[v2] FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools.
feiyue
Posts: 358
Joined: 08 Aug 2014, 04:08

Re: [v2] FindText - Capture screen image into text and then find it

Post by feiyue » 14 Aug 2023, 15:42

@hisrRB57 Thank you for your feedback. I didn't read the help file carefully. Please download the script again to fix this bug.

manafiqster
Posts: 1
Joined: 04 Sep 2023, 06:58

Re: [v2] FindText - Capture screen image into text and then find it

Post by manafiqster » 04 Sep 2023, 07:00

@feiyue Can u pls tell me how i can get rid safely of buttons from "MakeMainWindow" i want to keep just the capture button

hisrRB57
Posts: 69
Joined: 13 Jan 2019, 11:43

Re: [v2] FindText - Capture screen image into text and then find it

Post by hisrRB57 » 11 Oct 2023, 01:44

@feiyue I tried this code:

Code: Select all

xHelp := FindText().help()
But that gives a blank string. I tried a lot of variations, but I think the code in findtext.ahk should be:

Code: Select all

help()
{
return "
(
;--------------------------------
;  FindText - Capture screen image into text and then find it
;  Version : 9.1  (2023-07-30)
;--------------------------------
;  returnArray:=FindText(
;      &OutputX --> The name of the variable used to store the returned X coordinate
;    , &OutputY --> The name of the variable used to store the returned Y coordinate
;    , X1 --> the search scope's upper left corner X coordinates
;    , Y1 --> the search scope's upper left corner Y coordinates
;    , X2 --> the search scope's lower right corner X coordinates
;    , Y2 --> the search scope's lower right corner Y coordinates
;    , err1 --> Fault tolerance percentage of text       (0.1=10%)
;    , err0 --> Fault tolerance percentage of background (0.1=10%)
;    , Text --> can be a lot of text parsed into images, separated by '|'
;    , ScreenShot --> if the value is 0, the last screenshot will be used
;    , FindAll --> if the value is 0, Just find one result and return
;    , JoinText --> if you want to combine find, it can be 1, or an array of words to find
;    , offsetX --> Set the max text offset (X) for combination lookup
;    , offsetY --> Set the max text offset (Y) for combination lookup
;    , dir --> Nine directions for searching: up, down, left, right and center
;    , zoomW --> Zoom percentage of image width  (1.0=100%)
;    , zoomH --> Zoom percentage of image height (1.0=100%)
;  )
;
;  The function returns an Array containing all lookup results,
;  any result is a object with the following values:
;  {1:X, 2:Y, 3:W, 4:H, x:X+W//2, y:Y+H//2, id:Comment}
;  If no image is found, the function returns 0.
;  All coordinates are relative to Screen, colors are in RGB format
;
;  If the return variable is set to 'ok', ok[1] is the first result found.
;  ok[1].1, ok[1].2 is the X, Y coordinate of the upper left corner of the found image,
;  ok[1].3 is the width of the found image, and ok[1].4 is the height of the found image,
;  ok[1].x <==> ok[1].1+ok[1].3//2 ( is the Center X coordinate of the found image ),
;  ok[1].y <==> ok[1].2+ok[1].4//2 ( is the Center Y coordinate of the found image ),
;  ok[1].id is the comment text, which is included in the <> of its parameter.
;
;  If OutputX is equal to 'wait' or 'wait1'(appear), or 'wait0'(disappear)
;  it means using a loop to wait for the image to appear or disappear.
;  the OutputY is the wait time in seconds, time less than 0 means infinite waiting
;  Timeout means failure, return 0, and return other values means success
;  If you want to appear and the image is found, return the found array object
;  If you want to disappear and the image cannot be found, return 1
;  Example 1: FindText(&X:='wait', &Y:=3, 0,0,0,0,0,0,Text)   ; Wait 3 seconds for appear
;  Example 2: FindText(&X:='wait0', &Y:=-1, 0,0,0,0,0,0,Text) ; Wait indefinitely for disappear
;--------------------------------
)"
}
In that case the function returns the help text. Please let me know your thoughts.

feiyue
Posts: 358
Joined: 08 Aug 2014, 04:08

Re: [v2] FindText - Capture screen image into text and then find it

Post by feiyue » 11 Oct 2023, 03:17

The latest version already returns the help text. :D

Because the script generated by this tool contains all the source code of the script itself,
It is convenient for everyone to paste the source code of the function directly into their own script without using '#Include <FindText>'.
For compiled programs, I extract the source code of functions from the program's resources,
However, the extracted compiled source code will remove all comments,
So I assign annotation strings to variables, so that the compiler retains these annotations.

qomph
Posts: 4
Joined: 11 Aug 2023, 22:48
Contact:

Re: [v2] FindText - Capture screen image into text and then find it

Post by qomph » 29 Oct 2023, 22:40

I haven't upgraded to v9.2, which I may do to try out your new screen clipper function.

Here's a diff showing how I added support to change LButton and RButton , using instance methods SetCaptureKey1 and SetCaptureKey2. This is because I had trouble with the default GUI Capture, because my application acted upon RButton, changing the screen. By changing to NumPadAdd, I resolved my problem.

Initially, I tried to hide RButton from the application, but short of PSSuspend or changing the keyboard driver (Intercept, see https://github.com/evilC/AutoHotInterception ), I don't see how. PSSuspend would make window management quite difficult. But I didn't know how to use the Binding buttons when I first did my screenshots, so that might have solved my problem.

(You can consider this a suggestion for adding this "feature" of configuring the bindings, as well as asking if I did it the right way. I suspect not; most configuration is done by passing additional arguments to FindText(), but I think that gets hard to manage fast. And I suspect you're probably trying to remain "almost" compatible with FindText_v1, for now.

However I am a very naive user of FindText; eg. I still don't understand the hotkey configuration at the top of the GUI dialog.)

https://qomph.com/GTA/findtext_v2_difftext_RButton.html

Descolada
Posts: 1206
Joined: 23 Dec 2021, 02:30

Re: [v2] FindText - Capture screen image into text and then find it

Post by Descolada » 02 Dec 2023, 11:43

The following script highlights all the found results at the same time for debugging purposes. Press Esc to exit.

Code: Select all

#Requires AutoHotkey v2
; #Include <FindText>

t1:=A_TickCount, Text:=X:=Y:=""

Text:="|<>**50$19.zzzk00E000003s7l4282041020U10EEU/sE6080040020010DkU80E4082041021U17UDU003"

if (ok:=FindText(&X, &Y, 115-150000, 659-150000, 115+150000, 659+150000, 0, 0, Text))
{
  ; FindText().Click(X, Y, "L")
}

; ok:=FindText(&X:="wait", &Y:=3, 0,0,0,0,0,0,Text)  ; Wait 3 seconds for appear
; ok:=FindText(&X:="wait0", &Y:=-1, 0,0,0,0,0,0,Text)  ; Wait indefinitely for disappear

MsgBox("Found:`t" (IsObject(ok)?ok.Length:ok)
  . "`n`nTime:`t" (A_TickCount-t1) " ms"
  . "`n`nPos:`t" X ", " Y
  . "`n`nResult:`t<" (IsObject(ok)?ok[1].id:"") ">", "Tip", 4096)

try For i,v in ok  ; ok value can be get from ok:=FindText().ok
    Highlight(v.1, v.2, v.3, v.4, 0)
catch
    ExitApp

Esc::ExitApp

/**
 * Highlights an area with a colorful border. If called without arguments then all highlightings
 * are removed. This function also supports named parameters.
 * @param x Screen X-coordinate of the top left corner of the highlight
 * @param y Screen Y-coordinate of the top left corner of the highlight
 * @param w Width of the highlight
 * @param h Height of the highlight
 * @param showTime Can be one of the following:
 * * Unset - if highlighting exists then removes the highlighting, otherwise highlights for 2 seconds. This is the default value.
 * * 0 - Indefinite highlighting 
 * * Positive integer (eg 2000) - will highlight and pause for the specified amount of time in ms
 * * Negative integer - will highlight for the specified amount of time in ms, but script execution will continue
 * * "clear" - removes the highlighting unconditionally
 * @param color The color of the highlighting. Default is red.
 * @param d The border thickness of the highlighting in pixels. Default is 2.
 */
Highlight(x?, y?, w?, h?, showTime?, color:="Red", d:=2) {
	static guis := Map(), timers := Map()
	if IsSet(x) { ; if x is set then check whether a highlight already exists at those coords
		if IsObject(x) {
			d := x.HasOwnProp("d") ? x.d : d, color := x.HasOwnProp("color") ? x.color : color, showTime := x.HasOwnProp("showTime") ? x.showTime : showTime
			, h := x.HasOwnProp("h") ? x.h : h, w := x.HasOwnProp("w") ? x.w : h, y := x.HasOwnProp("y") ? x.y : y, x := x.HasOwnProp("x") ? x.x : unset
		}
		if !(IsSet(x) && IsSet(y) && IsSet(w) && IsSet(h))
			throw ValueError("x, y, w and h arguments must all be provided for a highlight", -1)
		for k, v in guis {
			if k.x = x && k.y = y && k.w = w && k.h = h { ; highlight exists, so either remove it, or update
				if !IsSet(showTime) || (IsSet(showTime) && showTime = "clear")
					TryRemoveTimer(k), TryDeleteGui(k)
				else if showTime = 0
					TryRemoveTimer(k)
				else if IsInteger(showTime) {
					if showTime < 0 {
						if !timers.Has(k)
							timers[k] := Highlight.Bind(x,y,w,h)
						SetTimer(timers[k], showTime)
					} else {
						TryRemoveTimer(k)
						Sleep showTime
						TryDeleteGui(k)
					}
				} else
					throw ValueError('Invalid showTime value "' (!IsSet(showTime) ? "unset" : IsObject(showTime) ? "{Object}" : showTime) '"', -1)
				return
			}
		}
	} else { ; if x is not set (eg Highlight()) then delete all highlights
		for k, v in timers
			SetTimer(v, 0)
		for k, v in guis
			v.Destroy()
		guis := Map(), timers := Map()
		return
	}
	
	if (showTime := showTime ?? 2000) = "clear"
		return
	else if !IsInteger(showTime)
		throw ValueError('Invalid showTime value "' (!IsSet(showTime) ? "unset" : IsObject(showTime) ? "{Object}" : showTime) '"', -1)

	; Otherwise this is a new highlight
	loc := {x:x, y:y, w:w, h:h}
	guis[loc] := Gui("+AlwaysOnTop -Caption +ToolWindow -DPIScale +E0x08000000")
	GuiObj := guis[loc]
	GuiObj.BackColor := color
	iw:= w+d, ih:= h+d, w:=w+d*2, h:=h+d*2, x:=x-d, y:=y-d
	WinSetRegion("0-0 " w "-0 " w "-" h " 0-" h " 0-0 " d "-" d " " iw "-" d " " iw "-" ih " " d "-" ih " " d "-" d, GuiObj.Hwnd)
	GuiObj.Show("NA x" . x . " y" . y . " w" . w . " h" . h)

	if showTime > 0 {
		Sleep(showTime)
		TryDeleteGui(loc)
	} else if showTime < 0
		SetTimer(timers[loc] := Highlight.Bind(loc.x,loc.y,loc.w,loc.h), showTime)

	TryRemoveTimer(key) {
		if timers.Has(key)
			SetTimer(timers[key], 0), timers.Delete(key)
	}
	TryDeleteGui(key) {
		if guis.Has(key)
			guis[key].Destroy(), guis.Delete(key)
	}
}

thebbandit
Posts: 45
Joined: 02 Jul 2019, 11:34

Re: [v2] FindText - Capture screen image into text and then find it

Post by thebbandit » 11 Dec 2023, 17:18

Its here! :D This library is a complete blessing for the AHK v2 community! I felt like I could not move to v2 without losing the best lib it had, now we can have our cake and eat it too!
Glad to see you still have the same fire for this library as always. :clap: :salute:

andersonepk
Posts: 1
Joined: 13 Feb 2024, 18:39

Re: [v2] FindText - Capture screen image into text and then find it

Post by andersonepk » 13 Feb 2024, 18:44

; , zoomW --> Zoom percentage of image width (1.0=100%)
; , zoomH --> Zoom percentage of image height (1.0=100%)


What are zoomw and zoomh for? How do you apply this to a code? Would this work in theory with an image at different resolutions? I tried looking on YouTube for someone explaining it but I didn't find anything. Thanks

User avatar
yfjuu6
Posts: 125
Joined: 28 Apr 2023, 15:28

Re: [v2] FindText - Capture screen image into text and then find it

Post by yfjuu6 » 23 Mar 2024, 14:42

@feiyue
I have a problem with this script :
when the option Ul Acces is checked An empty white window (Always on top) that It does not contain anything, covers a part of the screen, and It is located in the left corner as showen in the image.
But If the option Ul Acces is not checked that window doesn't appear.
Note that the script is running alone there are no others script running with it at the same time.
How to get rid of that empty white window when the option Ul Acces is checked Because I want it to be checked in order to always run Any AHK script as admin.? Any help please!
Attachments
AHK Dash.png
AHK Dash.png (37.69 KiB) Viewed 1601 times
problem whita window.png
problem whita window.png (289.29 KiB) Viewed 1601 times

feiyue
Posts: 358
Joined: 08 Aug 2014, 04:08

Re: [v2] FindText - Capture screen image into text and then find it

Post by feiyue » 27 Mar 2024, 03:42

The blank window in the upper left corner of the screen comes from the following code, This window was originally a sub-window of the main window.
Programs with the uia suffix may cause child windows to not be able to be bound to the main window, This is specifically mentioned in the help file:

Parent: Use +Parent immediately followed by the HWND of any window or control to use it as the parent of this window. To convert the GUI back into a top-level window, use -Parent. This option works even after the window is created. Known limitations:

Running with UI access prevents the +Parent option from working on an existing window if the new parent is always-on-top and the child window is not.
The +Parent option may fail during GUI creation if the parent window is external, but may work after the GUI is created. This is due to differences in how styles are applied.

Code: Select all

    Try FindText_SubPic.Destroy()
    FindText_SubPic:=_Gui:=Gui()
    _Gui.Opt "+Parent" parent_id " +AlwaysOnTop -Caption +ToolWindow -DPIScale"
    _Gui.MarginX:=0, _Gui.MarginY:=0
    _Gui.BackColor:="White"
    id:=_Gui.Add("Pic", "x0 y0 w500 h500"), sub_hpic:=id.Hwnd
    _Gui.Title:="SubPic"
    _Gui.Show "NA x0 y0"
You can try removing "+AlwaysOnTop", and my Win7 test never had this issue.

User avatar
yfjuu6
Posts: 125
Joined: 28 Apr 2023, 15:28

Re: [v2] FindText - Capture screen image into text and then find it

Post by yfjuu6 » 01 Apr 2024, 13:38

@feiyue
Yes, when removed "+AlwaysOnTop", that blank window dissapeared, Thanks !
I want to check whether a specific text appears in a window or not.
The problem is when resize the window, It doesn't find the specified text.
Should the window dimensions be fixed to find the specified text?

feiyue
Posts: 358
Joined: 08 Aug 2014, 04:08

Re: [v2] FindText - Capture screen image into text and then find it

Post by feiyue » 27 Apr 2024, 06:24

Updated to 9.5 version - 2024/04/27 :beer: :dance:
1. Modify: Improved the machine code for searching from the center spiral to the surrounding area.
2. Modify: The color exclusion mode for <FindMultiColor> <FindColor>
can now exclude multiple unwanted colors. Previously, only one color could be excluded.

cgx5871
Posts: 319
Joined: 26 Jul 2018, 14:02

Re: [v2] FindText - Capture screen image into text and then find it

Post by cgx5871 » 17 May 2024, 05:41

Code: Select all

#Include <FindText>
class a {
	static x:=0,y:=0
}

Text:="|<>*86$13.zzlzuTxryxzTTjnrwvzBzqy7RjgrlNxqzvTyTzzU"
if (ok:=FindText(&a.X, &a.Y, 23-150000, 125-150000, 23+150000, 125+150000, 0, 0, Text))
{
}
error : ==> "&" requires a variable.

redrum
Posts: 101
Joined: 25 Jan 2015, 22:57

Re: [v2] FindText - Capture screen image into text and then find it

Post by redrum » 18 May 2024, 21:30

How is the hotkey screenshot function of the FindText GUI supposed to work? I'm trying to capture an image that is only available in a right click menu (which goes away as soon as I click on 'Capture' in the FindText GUI), so I assign a hotkey in the FindText GUI, right click in my app to expose the menu, and hit the hotkey to take screenshot (see 'Success' popup briefly flash on screen). But I don't see a screenshot. What am I doing wrong?
findtext gui.png
findtext gui.png (86.87 KiB) Viewed 945 times

Also, is this functionally the same as right clicking in my app to expose the menu, using Windows PrintScreen, opening the png file and using FindText GUI to capture the portion of the png image I want?
Edit: I guess the answer to this question is yes. The captures I'm doing from the Windows PrintScreen png are matching when running FindText().

Great tool! I only wish I knew about it earlier - would have saved me time and frustration of working with regular imagesearch.
Last edited by redrum on 19 May 2024, 17:29, edited 1 time in total.

redrum
Posts: 101
Joined: 25 Jan 2015, 22:57

Re: [v2] FindText - Capture screen image into text and then find it

Post by redrum » 19 May 2024, 08:49

GetRange doesn't seem to work. Am I using it incorrectly? I capture my image, click Gray2Two to convert to grayscale, click Auto to crop, then click OK to return to main GUI, then click GetRange.

Code: Select all

Error: Parameter #5 of FindTextClass.Prototype.xywh2xywh requires a variable reference, but received an Integer.
get range Screenshot 2024-05-19 093941.png
get range Screenshot 2024-05-19 093941.png (59.6 KiB) Viewed 880 times
getrange error Screenshot 2024-05-19 094030.png
getrange error Screenshot 2024-05-19 094030.png (23.54 KiB) Viewed 880 times

Descolada
Posts: 1206
Joined: 23 Dec 2021, 02:30

Re: [v2] FindText - Capture screen image into text and then find it

Post by Descolada » 19 May 2024, 22:24

@redrum with the v2 tool you first capture the screenshot, then press CaptureS button, select the correct screenshot from right-side list view, and click on the screenshot at the place you want to get the Text from.

What do you want use use GetRange for? Once you've Captured the Text you don't need to do anything else, just copy it from the textbox.

redrum
Posts: 101
Joined: 25 Jan 2015, 22:57

Re: [v2] FindText - Capture screen image into text and then find it

Post by redrum » 20 May 2024, 12:04

Descolada wrote:
19 May 2024, 22:24
@redrum with the v2 tool you first capture the screenshot, then press CaptureS button, select the correct screenshot from right-side list view, and click on the screenshot at the place you want to get the Text from.
Thanks for the correct sequence, works great.
Descolada wrote:
19 May 2024, 22:24
What do you want use use GetRange for? Once you've Captured the Text you don't need to do anything else, just copy it from the textbox.
Seems like GetRange would be a very quick and easy way to define a small search area, rather than having to look at WindowSpy and write down coordinates.

I am loving FindText. Converting all my hotkeys that were using either hardcoded coordinates or regular imagesearch (no longer need to manage imagesearch files!). Thanks to @feiyue and @Descolada!

redrum
Posts: 101
Joined: 25 Jan 2015, 22:57

Re: [v2] FindText - Capture screen image into text and then find it

Post by redrum » 23 May 2024, 00:57

Probably a dumb question, but what is the purpose of PicLib? I have 25 text 'images' near the top of my daily driver script. How would PicLib be of use in my case? If it's not useful in my use case, what use cases is it designed for?

Descolada
Posts: 1206
Joined: 23 Dec 2021, 02:30

Re: [v2] FindText - Capture screen image into text and then find it

Post by Descolada » 23 May 2024, 03:09

@redrum it probably wouldn't be much of use in that case, perhaps only for organisatory purposes (instead of using multiple global variables, you put them in PicLib and access them via their comment name). I primarily use it to do OCR: I capture images of individual characters and add them into a PicLib, then I can use FindText().PicN to construct a combination of those characters for a combination search. Eg I add Text of characters "h", "o", and "t" into the PicLib, and then I can use FindText().PicN("tho") in a combination search to search the screen for the word "tho".

redrum
Posts: 101
Joined: 25 Jan 2015, 22:57

Re: [v2] FindText - Capture screen image into text and then find it

Post by redrum » 23 May 2024, 10:48

Descolada wrote:
23 May 2024, 03:09
@redrum it probably wouldn't be much of use in that case, perhaps only for organisatory purposes (instead of using multiple global variables, you put them in PicLib and access them via their comment name). I primarily use it to do OCR: I capture images of individual characters and add them into a PicLib, then I can use FindText().PicN to construct a combination of those characters for a combination search. Eg I add Text of characters "h", "o", and "t" into the PicLib, and then I can use FindText().PicN("tho") in a combination search to search the screen for the word "tho".
Thanks for the explanation @Descolada. I can see how it would be useful for OCR.

Post Reply

Return to “Scripts and Functions (v2)”