FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
User avatar
tobsto occupied
Posts: 120
Joined: 20 Feb 2020, 09:34
GitHub: washingleavesandthen

Re: FindText - Capture screen image into text and then find it

24 Feb 2020, 14:13

as said still need some help here to make this work, these are my coords>
x>1655 z>881 w>530 h>357 for active window
and these are for the mouse position aka pointed on the play button> x>428 y>45 w>89 h>29
rommmcek
Posts: 1057
Joined: 15 Aug 2014, 15:18

Re: FindText - Capture screen image into text and then find it

24 Feb 2020, 18:19

Try:

Code: Select all

#SingleInstance, force
SetBatchLines, -1

#Include node_modules
#Include graphicsearch.ahk\export.ahk
oGraphicSearch := new graphicsearch()

; ~ variables ~
; searchable graphics
launchGraphic := "|<Play>**25$63.k0DyD1lzs7y01ztwSDzUzk0zzDXlzyTy07XtwSDbnnk0wTDXlwyyS00TtwSD7rUk03zDXlsyw607ztwSD7bUk0zzDXlsyw607ttwSD7rUk1sDDXlsyw60D3twSD7nns3wzjblsyTzzrzwzyD7nzzyzrXzlsS7zznysDyC1UzU"

#p::
CoordMode, Mouse, Screen
result := oGraphicSearch.find(1655, 881, 1655+530, 881+357, 0, 0, launchGraphic)
if (result) {
    ; click all found images
    for i, obj in result {
        MouseClick,, obj.x, obj.y
        sleep, 300
    }
}
return
User avatar
tobsto occupied
Posts: 120
Joined: 20 Feb 2020, 09:34
GitHub: washingleavesandthen

Re: FindText - Capture screen image into text and then find it

26 Feb 2020, 05:36

didnt work , heres the launcher including the windows spy window for info, btw it would be so much easier if windows spy could simply update the script since it has all the info required for which button and its position , would make life a ton easier for sure
Attachments
Play button example.png
Play button example.png (89.51 KiB) Viewed 3493 times
rommmcek
Posts: 1057
Joined: 15 Aug 2014, 15:18

Re: FindText - Capture screen image into text and then find it

26 Feb 2020, 10:59

It cannot work like this! It seems you found some code and copied it into your script. All you did was edit the human readable lable (which has no effect) and trying to adjust the coordinats (btw. wrongly).
I think I altered the coordinats for you correctly.
Now you have to capture "Play" picture in Max Payne window with the "Capture Image To Text" tool (run the code in the first post of this thread - result should be like in attached pic.) and replace

Code: Select all

launchGraphic := "|<Play>**25$63.k0DyD1lzs7y01ztwSDzUzk0zzDXlzyTy07XtwSDbnnk0wTDXlwyyS00TtwSD7rUk03zDXlsyw607ztwSD7bUk0zzDXlsyw607ttwSD7rUk1sDDXlsyw60D3twSD7nns3wzjblsyTzzrzwzyD7nzzyzrXzlsS7zznysDyC1UzU"
with (I did an attempt for you):

Code: Select all

launchGraphic := "|<Play>*115$20.1jzrfzxuzzSgPEPupykhTffLuvBykrzzxzzwy"
Edit: New snippet based on picture/correct browser zoom.
But note that if screen DPI settings (for my and your PC differ), it won't work (then you have to capture the "Play" snippet by yourself).

And after all, try this two simpler alternatives:

Code: Select all

#p:: ControlClick, Button1, Max Payne v1.05
or if the play button is always in the same position in the Max Payne window:

Code: Select all

CoordMode, Mouse, Window
#IfWinActive, Max Payne v1.05
#p:: Click, 454, 56
#IfWinActive
Good luck!
Attachments
Play button example 2.png
Play button example 2.png (55.34 KiB) Viewed 3438 times
Last edited by rommmcek on 26 Feb 2020, 13:50, edited 3 times in total.
User avatar
Chunjee
Posts: 671
Joined: 18 Apr 2014, 19:05
GitHub: Chunjee

Re: FindText - Capture screen image into text and then find it

26 Feb 2020, 11:57

Is it possible to get and save the screencapture? I see that GetBitsFromScreen returns a variable named "bits" but this seems to be an array of only 6 numbers. Hardly an entire screenworth of data.
[108855296, 51200, 0, 0, 12800, 2160] if useful
User avatar
Chunjee
Posts: 671
Joined: 18 Apr 2014, 19:05
GitHub: Chunjee

Re: FindText - Capture screen image into text and then find it

26 Feb 2020, 14:51

feiyue wrote:
19 Jan 2020, 05:49
Updated to 7.4 version - 2020/01/20
.... 1. Add: the GetTextFromScreen() function and ft_Gui("Capture_NoGui") is added.
....... So you can quickly generate the final code without GUI interface. :dance:
I ran GetTextFromScreen(0, 0, A_ScreenWidth, A_ScreenHeight) and it returns "|<>*0$1025.U" so I don't understand the instruction "So you can quickly generate the final code without GUI interface" because it should be returning a much longer string for the whole screen? no?
Last edited by Chunjee on 26 Feb 2020, 15:29, edited 1 time in total.
feiyue
Posts: 230
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

26 Feb 2020, 15:15

You can easily view the last screenshot (if you only capture a small range of images,
because it will be superimposed on the old image, so the full screen image is still displayed here):

Code: Select all

F2:: ft_ShowScreenShot(onoff:=!onoff)
Here is a quick way to generate the FindText call script:

Code: Select all

F1::
r:=ft_Gui("Capture_NoGui"), x:=r.1, y:=r.2, w:=r.3, h:=r.4
x1:=x-w, y1:=y-h, x2:=x+w, y2:=y+h, s:=GetTextFromScreen(x1,y1,x2,y2)
msgbox % Clipboard:="ok:=FindText(" x1 "," y1 "," x2 "," y2 ",0,0,""" s """)"
return
rommmcek
Posts: 1057
Joined: 15 Aug 2014, 15:18

Re: FindText - Capture screen image into text and then find it

26 Feb 2020, 21:48

@Chunjee: Thanks for the last question (I didn't noticed the 01/19 update to 7.4)
@feiyue: Thanks for the answer + example! And OMG you automated the whole capturing process! Awesome!

Edit: In the mode "NoGui" the capturing area is diminished. I wanted it to be resizable, took a look in the code and noticed it was already done (by hitting left/right/top/bottom edge of the screen). But once the code read I proceeded adding resize via hotkey:
Horizontal: f - extend, s - shrink; Vertical: e - extend, d - shrink.
Spoiler
Last edited by rommmcek on 27 Feb 2020, 00:39, edited 1 time in total.
User avatar
Chunjee
Posts: 671
Joined: 18 Apr 2014, 19:05
GitHub: Chunjee

Re: FindText - Capture screen image into text and then find it

26 Feb 2020, 23:36

feiyue wrote:
26 Feb 2020, 15:15
You can easily view the last screenshot (if you only capture a small range of images,
because it will be superimposed on the old image, so the full screen image is still displayed here):

Code: Select all

ft_ShowScreenShot(onoff:=!onoff)

Code: Select all

GetTextFromScreen(0,0,A_ScreenWidth,A_ScreenHeight)
; => "|<>*83$1366.0000000000Tzzzzzzzz..."
Thank you, I will experiment and study. 谢谢, 我会尝试学习.
User avatar
tobsto occupied
Posts: 120
Joined: 20 Feb 2020, 09:34
GitHub: washingleavesandthen

Re: FindText - Capture screen image into text and then find it

27 Feb 2020, 10:04

ok so , am i correctly understanding that you guys have now automated the image capture process , if so , id appreciate it if you would be able to help me out with my current issue thats known , still dont understand what to do to get this to work , help would be appreciated as usual
feiyue
Posts: 230
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

27 Feb 2020, 22:06

@tobsto occupied
For using FindText() to find an image, the first is to make sure that the Text string of the image is correct,
The second is to make sure that the image is included in the lookup scope.

For novices, full screen search is enough.
For example: FindText(0, 0, 150000, 150000,...)

If you want to use a specific scope for finding, understand the following two points:
1. The input parameter coordinates and the returned coordinates used by FindText are all screen coordinates.
2. FindText uses the upper left and lower right coordinates, two points to determine a rectangular range for searching.

Some commands, such as WinGetPos, can get the coordinates of the upper left corner of the window (X, Y),
And the width (W) and height (H) of the window, In this case,
the coordinates of the lower right corner are easy to calculate: (X+W, Y+H),
So the form of searching only in the window scope is: FindText(X, Y, X+W, Y+H,...)
User avatar
tobsto occupied
Posts: 120
Joined: 20 Feb 2020, 09:34
GitHub: washingleavesandthen

Re: FindText - Capture screen image into text and then find it

28 Feb 2020, 16:45

well, apparently i don't need FindText for my problem, because it works with basic
AutoHotkey commands, according to rommmcek, he helped me out to take care of it , thanks again guys love yall
itadakimas
Posts: 3
Joined: 01 Jan 2020, 16:33

Re: FindText - Capture screen image into text and then find it

06 Mar 2020, 20:41

Hello, does anyone know to make the FindText function search relative to the active window rather than the screen?
rommmcek
Posts: 1057
Joined: 15 Aug 2014, 15:18

Re: FindText - Capture screen image into text and then find it

06 Mar 2020, 22:00

Two posts back the author beautiful explained it. If you still have doubts:
Spoiler
rzr68402
Posts: 1
Joined: 22 Mar 2020, 13:00

Re: FindText - Capture screen image into text and then find it

22 Mar 2020, 13:17

Hi there!

Seems to be an issue with the FindText library- from the binding of the window of a game in the FindText library for people using VM/bootcamp.

The issue is with samples showing as blank, when using the binding of the window in the FindText library.

When added the ability for the script to function in the background while running a game, it requires the script to bind the captured image to only the game. So it lets windows obscure the game window without causing false trigger. But it seems in very fringe cases that it causes an issue, mainly with people using VM to run the script.

This issue is temporarily solved when the window binding is removed, however can this be addressed for future VM/bootcamp users to be able to use window binding?

Thanks you!
feiyue
Posts: 230
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

10 Apr 2020, 12:28

Updated to 7.5 version - 2020/04/13 :dance:
.... 1. Add: the button to get the range has been added,
....... You can select a range on the screen with the left mouse button,
....... The range coordinates are copied to the clipboard,
....... And the search scope of FindText() function in the code will be updated.
.... 2. Modify: in order to reduce the conflict of function names,
....... Use "FindText_" as the function prefix. However, this creates some incompatibility.
....... For example: MouseTip() must be changed to FindText_MouseTip()
OMGmarc
Posts: 4
Joined: 05 Feb 2020, 23:46

Re: FindText - Capture screen image into text and then find it

18 Apr 2020, 13:52

feiyue wrote:
10 Apr 2020, 12:28
Updated to 7.5 version - 2020/04/13 :dance:
Excellent!

I was wondering...do you happen to have this hosted on Github or similar, too? Would make it easier for people like me to always have the latest version. :)
Philharmonic
Posts: 32
Joined: 22 May 2017, 13:51

Re: FindText - Capture screen image into text and then find it

18 Apr 2020, 14:05

Hi, tried the script and it seems very useful for what I'm trying to apply it to. I noticed once I run the script, it finds the text but the mouse does not move to that location. Is that not suppose to happen?

When I click on "test" a message box prompts up that says success, I press enter and then it highlight the text that I want. However, the mouse does not move to that position/location. It just stays at my current location. I thought the mouse suppose to move to the text that it found, unless I'm mistaken. Any input would be greatly appreciated. Thanks in advance! :D
OMGmarc
Posts: 4
Joined: 05 Feb 2020, 23:46

Re: FindText - Capture screen image into text and then find it

18 Apr 2020, 17:59

Philharmonic wrote:
18 Apr 2020, 14:05
I noticed once I run the script, it finds the text but the mouse does not move to that location. Is that not suppose to happen?
No, it's not. "Out of the box," if you just run the sample code from the FindText.ahk script's output window (where it shows some sample code alongside the generated search string), all it will do is search for the string you've generated. If it finds any, it will display a message displaying some information regarding the search it just performed as well as highlight the center of each match (up to 2).

The function itself returns the x and y coordinates of each match. You can retrieve them via their respective index values.

For example:

Code: Select all

; First match
X1 := ok.1.x
Y1 := ok.1.y

; Second match
X2 := ok.2.x
Y2 := ok.2.y

Click, %X1%, %Y1%
Sleep, 1000
Click, %X2%, %Y2%
Philharmonic
Posts: 32
Joined: 22 May 2017, 13:51

Re: FindText - Capture screen image into text and then find it

18 Apr 2020, 18:32

OMGmarc wrote:
18 Apr 2020, 17:59
Philharmonic wrote:
18 Apr 2020, 14:05
I noticed once I run the script, it finds the text but the mouse does not move to that location. Is that not suppose to happen?
No, it's not. "Out of the box," if you just run the sample code from the FindText.ahk script's output window (where it shows some sample code alongside the generated search string), all it will do is search for the string you've generated. If it finds any, it will display a message displaying some information regarding the search it just performed as well as highlight the center of each match (up to 2).

The function itself returns the x and y coordinates of each match. You can retrieve them via their respective index values.

For example:

Code: Select all

; First match
X1 := ok.1.x
Y1 := ok.1.y

; Second match
X2 := ok.2.x
Y2 := ok.2.y

Click, %X1%, %Y1%
Sleep, 1000
Click, %X2%, %Y2%

Thanks!

Return to “Scripts and Functions”

Who is online

Users browsing this forum: Aceful, Bartimus and 23 guests