FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
WevertonDev
Posts: 2
Joined: 08 Apr 2021, 18:26

Re: FindText - Capture screen image into text and then find it

Post by WevertonDev » 08 Apr 2021, 18:39

I'm sorry for the doubt, I'm trying to use the script but I would like to use it several times in a code where I need to identify an image and click on a specific text / image, can someone give me a light? I can pay for the help, I have been studying for days and without success :headwall:

basically i need the script to identify the image of a car and click on the correct option, only with the imageSearch function, I cannot identify the option, I tried with the findtext and also without success

Image https://imgur.com/gPDE9Lz

Code: Select all

#SingleInstance Force
CoordMode, Pixel, Screen

Loop
{
	ImageSearch, x, y, 623, 254, 1089, 725, C:\Users\User\Pictures\Screenshots\Captura de Tela (221).png  ;; [b]here would be the image of the car[/b]

	if (ErrorLevel = 0)
	{
		Click, %x%, %y%
	}
	ImageSearch, x, y, 623, 254, 1089, 725, C:\Users\User\Pictures\Screenshots\Captura de Tela (263).png ;; [b]and here the image of the option to be clicked[/b]

	if (ErrorLevel = 0)
	{
		Click, %x%, %y%
	}
	
}

return
maleko48
Posts: 17
Joined: 19 Jul 2019, 13:35

Re: FindText - Capture screen image into text and then find it

Post by maleko48 » 08 Apr 2021, 19:57

WevertonDev wrote:
08 Apr 2021, 18:39
I'm sorry for the doubt, I'm trying to use the script but I would like to use it several times in a code where I need to identify an image and click on a specific text / image, can someone give me a light? I can pay for the help, I have been studying for days and without success :headwall:

basically i need the script to identify the image of a car and click on the correct option, only with the imageSearch function, I cannot identify the option, I tried with the findtext and also without success

Image https://imgur.com/gPDE9Lz

Code: Select all

#SingleInstance Force
CoordMode, Pixel, Screen

Loop
{
	ImageSearch, x, y, 623, 254, 1089, 725, C:\Users\User\Pictures\Screenshots\Captura de Tela (221).png  ;; [b]here would be the image of the car[/b]

	if (ErrorLevel = 0)
	{
		Click, %x%, %y%
	}
	ImageSearch, x, y, 623, 254, 1089, 725, C:\Users\User\Pictures\Screenshots\Captura de Tela (263).png ;; [b]and here the image of the option to be clicked[/b]

	if (ErrorLevel = 0)
	{
		Click, %x%, %y%
	}
	
}

return
You can use FindText and my two functions from a couple posts back:
https://www.autohotkey.com/boards/viewtopic.php?style=7&f=6&t=17834&start=660#p392457

You can call it as much as you need to and use it like this to click on a Text target generated by FindText. Only WinName and Target are required, the other settings are optional.

Code: Select all

		SYS_WinClickTarget("Google Chrome","|<RESTORE_WINDOW_BUTTON>**50$12.Dz81vxU5jpcJcJcJcFjrU4zwU")
chaoscreater
Posts: 32
Joined: 12 Sep 2019, 21:15

Re: FindText - Capture screen image into text and then find it

Post by chaoscreater » 08 Apr 2021, 20:42

maleko48 wrote:
08 Apr 2021, 09:25
chaoscreater wrote:
07 Apr 2021, 22:24
Sorry, I'm not sure I understand what you mean. I wase talking talking about getting a Findtext image adjusted to work on machines with different resolutions. Let's say you have 2 machines, one with a super low resolution and another with an ultra high resolution. The same image will be displayed differently on both machines. The size of the image is different and the coordinate on the screen is different.

image.png

Clearly, if you take a Findtext image on either of the machine, it won't be matched on another, because the image is either too big or too small. Furthermore, the Findtext captured area coordinates do not translate across the machines.
What exactly is it that you are trying to accomplish?

Aside from resolution settings, custom fonts or even Windows' ClearText technology could also be enough to break compatibility and scalability across a wider range of hardware. If that's what you are up against, this may be the wrong tool for the job.

If you were really set on it, you could conceivably write an algorithm that slices and dices a master "calibration image" taken from a target machine meant to dynamically re-create its OCR library of objects to match against (ie. A thru Z and 0 thru 9 along with some common symbols and such).

I'm trying to use FindText to capture an image, generate the code for me and use that code across different machines regardless of their screen resolution. I think this is explained clearly now.

If 2 machines have the same screen resolution configured, then the code will work just fine.

If 2 machines have different screen resolution configured, then one of the machine will not be able to use that code. The screenshot I provided in my previous post should be very clear to understand.

I'm not a software programmer and I don't have the skills to expand on this to make this work, hence why I am asking or requesting if FindText can be improved upon to scale up/down to work on machines with different resolution. Ideally, the FindText GUI would provide options to generate code for different screen resolutions.
maleko48
Posts: 17
Joined: 19 Jul 2019, 13:35

Re: FindText - Capture screen image into text and then find it

Post by maleko48 » 08 Apr 2021, 22:04

chaoscreater wrote:
08 Apr 2021, 20:42

I'm trying to use FindText to capture an image, generate the code for me and use that code across different machines regardless of their screen resolution. I think this is explained clearly now.

If 2 machines have the same screen resolution configured, then the code will work just fine.

If 2 machines have different screen resolution configured, then one of the machine will not be able to use that code. The screenshot I provided in my previous post should be very clear to understand.

I'm not a software programmer and I don't have the skills to expand on this to make this work, hence why I am asking or requesting if FindText can be improved upon to scale up/down to work on machines with different resolution. Ideally, the FindText GUI would provide options to generate code for different screen resolutions.
I know what you want to do, but good luck with that attitude around here. You haven't even given enough details for anyone to help you.

What size is the image you are trying to match?
What is the image being displayed or rendered in?
Is it an image you are ultimately trying to find or font/text within the image.
Do you want to programmatically search an image in the form of a file or is this program intended to operate on your desktop's current output in realtime?
How often does the viewport change, or do you have a folder of files you intend to parse and search?
Are there GUI anchor points you can work from using relative measures?
What scaling percentage are the machines running at despite their differing resolutions?
etc
WevertonDev
Posts: 2
Joined: 08 Apr 2021, 18:26

Re: FindText - Capture screen image into text and then find it

Post by WevertonDev » 09 Apr 2021, 19:47

maleko48 wrote:
08 Apr 2021, 19:57
WevertonDev wrote:
08 Apr 2021, 18:39
I'm sorry for the doubt, I'm trying to use the script but I would like to use it several times in a code where I need to identify an image and click on a specific text / image, can someone give me a light? I can pay for the help, I have been studying for days and without success :headwall:

basically i need the script to identify the image of a car and click on the correct option, only with the imageSearch function, I cannot identify the option, I tried with the findtext and also without success

Image https://imgur.com/gPDE9Lz

Code: Select all

#SingleInstance Force
CoordMode, Pixel, Screen

Loop
{
	ImageSearch, x, y, 623, 254, 1089, 725, C:\Users\User\Pictures\Screenshots\Captura de Tela (221).png  ;; [b]here would be the image of the car[/b]

	if (ErrorLevel = 0)
	{
		Click, %x%, %y%
	}
	ImageSearch, x, y, 623, 254, 1089, 725, C:\Users\User\Pictures\Screenshots\Captura de Tela (263).png ;; [b]and here the image of the option to be clicked[/b]

	if (ErrorLevel = 0)
	{
		Click, %x%, %y%
	}
	
}

return
You can use FindText and my two functions from a couple posts back:
https://www.autohotkey.com/boards/viewtopic.php?style=7&f=6&t=17834&start=660#p392457

You can call it as much as you need to and use it like this to click on a Text target generated by FindText. Only WinName and Target are required, the other settings are optional.

Code: Select all

		SYS_WinClickTarget("Google Chrome","|<RESTORE_WINDOW_BUTTON>**50$12.Dz81vxU5jpcJcJcJcFjrU4zwU")
Thanks, I will try, I am not able to find the script of the latest version to download
chaoscreater
Posts: 32
Joined: 12 Sep 2019, 21:15

Re: FindText - Capture screen image into text and then find it

Post by chaoscreater » 11 Apr 2021, 16:19

maleko48 wrote:
08 Apr 2021, 22:04
chaoscreater wrote:
08 Apr 2021, 20:42

I'm trying to use FindText to capture an image, generate the code for me and use that code across different machines regardless of their screen resolution. I think this is explained clearly now.

If 2 machines have the same screen resolution configured, then the code will work just fine.

If 2 machines have different screen resolution configured, then one of the machine will not be able to use that code. The screenshot I provided in my previous post should be very clear to understand.

I'm not a software programmer and I don't have the skills to expand on this to make this work, hence why I am asking or requesting if FindText can be improved upon to scale up/down to work on machines with different resolution. Ideally, the FindText GUI would provide options to generate code for different screen resolutions.
I know what you want to do, but good luck with that attitude around here. You haven't even given enough details for anyone to help you.

What size is the image you are trying to match?
What is the image being displayed or rendered in?
Is it an image you are ultimately trying to find or font/text within the image.
Do you want to programmatically search an image in the form of a file or is this program intended to operate on your desktop's current output in realtime?
How often does the viewport change, or do you have a folder of files you intend to parse and search?
Are there GUI anchor points you can work from using relative measures?
What scaling percentage are the machines running at despite their differing resolutions?
etc

I could say the same about you. I've clearly explained what I am trying to do and clearly explained my request in the beginning. I've been patient with you asking the same questions multiple times (except your last post). I've also provided enough details already.

You run FindText on a machine with e.g. 1920 x 1080 resolution. You do a capture and FindText generates the code for you. My request was to extend this generated code to be scaled up or down to apply to different screen resolutions. It's like if you take a photo of a certain resolution and you shrink or enlarge it using photo editors. Nobody needs to ask the questions you have asked. The photo editors clearly can shrink and enlarge the photo, regardless of what photo editor program you use, regardless if you're editing the photo on a phone, tablet, PC, whatever. It's that simple.

Now, to answer your other questions:

What size is the image you are trying to match? - Doesn't matter and irrelevant. Any image captured by FindText should then be able to shrink or enlarge to be detected on another machine using another resolution.
What is the image being displayed or rendered in? - It's just a black and white image. using Gray2Two function.
image.png
image.png (11.65 KiB) Viewed 504 times
Is it an image you are ultimately trying to find or font/text within the image. - Image I'm trying to find
Do you want to programmatically search an image in the form of a file or is this program intended to operate on your desktop's current output in realtime? - desktop's current output in realtime
How often does the viewport change, or do you have a folder of files you intend to parse and search? - changes frequently
Are there GUI anchor points you can work from using relative measures? - don't understand your question. I only know the very basics of how to use FindText to do an image capture and use its generated code.
What scaling percentage are the machines running at despite their differing resolutions? - no scaling. It's at 100%.
Attachments
image.png
image.png (11.41 KiB) Viewed 504 times
chaoscreater
Posts: 32
Joined: 12 Sep 2019, 21:15

Re: FindText - Capture screen image into text and then find it

Post by chaoscreater » 12 Apr 2021, 16:35

Is it possible to do a percentage match of the image (just a Gray2Two image). E.g. a 80% match?
Avastgard
Posts: 61
Joined: 30 Sep 2016, 21:54

Re: FindText - Capture screen image into text and then find it

Post by Avastgard » 18 Apr 2021, 15:36

I am running into a problem involving #Include and #IncludeAgain:

I have a master script that I use to execute several commands and run other scripts. In this master script I have 3 commands that open a certain website, fill in forms and click on a checkbox. To click on the checkbox, I used an #Include to include another script generated by FindText.

The problem is that the #Include was only working for the first command in the code. The ones below it would execute all the other actions, except click on the checkbox in the end. I then did some research and found out about #IncludeAgain. I tried using it on all the commands, but when I try to reload the master script, I get this error message:
---------------------------
Host.ahk
---------------------------
Error at line 63 in #include file "C:\Users\username\Desktop\Autohotkey\FindText\Findtext - Click Checkbox.ahk".

Line Text: FindText(args*)
Error: Duplicate function definition.

The script was not reloaded; the old version will remain in effect.
---------------------------
OK
---------------------------
From what I understand from this message, the problem is that in my master script I have 3 blocks of code that are pointing to the same FindText script. Since the FindText script has functions in it and it is included three times in the master script, it's as if the same function appeared three times.

Is my assessment correct? How can I #Include the same FindText script on the master script more than once without incurring in the duplicate function definition issue?

EDIT: Even if I manage to solve this problem regarding #Include or #IncludeAgain, what if I create other FindText scripts and want to #Include them in my master script? I assume FindText will generate functions with the same names and I will run into the same problem, won't I?
feiyue
Posts: 295
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 18 Apr 2021, 20:04

@Avastgard
You just need to write this line in the main script: #Include <FindText>
You don't need to include this line in other scripts. For example, the following script:
#include C:\Users\username\Desktop\Autohotkey\FindText\Findtext - Click Checkbox.ahk
Avastgard
Posts: 61
Joined: 30 Sep 2016, 21:54

Re: FindText - Capture screen image into text and then find it

Post by Avastgard » 20 Apr 2021, 06:47

feiyue wrote:
18 Apr 2021, 20:04
@Avastgard
You just need to write this line in the main script: #Include <FindText>
You don't need to include this line in other scripts. For example, the following script:
#include C:\Users\username\Desktop\Autohotkey\FindText\Findtext - Click Checkbox.ahk
I tried, but I received a "library not found" error message. Switching <FindText> for the actual path where FindText.ahk is located yields the same duplicate function/class error message.
feiyue
Posts: 295
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 20 Apr 2021, 11:15

@Avastgard
1. Don't use #IncludeAgain. Use function or Gosub.
How can I #Include the same FindText script on the master script more than once without incurring in the duplicate function definition issue?
2. Generate the "Lib" subdirectory in the main program directory, and then copy FindText.ahk to the "Lib" subdirectory. Now you can use #Include <FindText>.
c7aesa7r
Posts: 123
Joined: 02 Jun 2016, 21:09

Re: FindText - Capture screen image into text and then find it

Post by c7aesa7r » 20 Apr 2021, 18:41

@feiyue is possible to save a copy of the image after editing it in the "capture image to text gui"?

for example:

I captured this area:
image.png
image.png (1.74 KiB) Viewed 323 times

Then I edited it to this:
image.png
image.png (3.59 KiB) Viewed 323 times

Is possible to save a copy of this new edited image? The copy would be something like:
image.png
image.png (665 Bytes) Viewed 323 times
feiyue
Posts: 295
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 21 Apr 2021, 17:19

Updated to 8.5 version - 2021/04/22 :dance: :beer:
.... 1. Add: Since you can search for pictures now, the function of saving pictures has been added in CaptureS.
....... You can select a small range of pictures from the ScreenShots to save as pictures.
.... 2. Modify: Now when capturing the image, it no longer judges the physical state of the right mouse button,
....... but uses the logical state, which can be used in remote control.

@c7aesa7r The new version can meet your needs.
laozhhaiJohn
Posts: 25
Joined: 20 Mar 2021, 18:18

Re: FindText - Capture screen image into text and then find it

Post by laozhhaiJohn » 21 Apr 2021, 22:20

非常好!飞跃!我在国内看到过您的中文版FindText帖子,但是看完后,仍然对FindText中的子函数不太了解,比如FindText.Ocr函数,不知道哪里有一些例子可以参考。在这个论坛中本帖子首页和后面某页中,可以找到您发表的Ocr例子,但是都已经随着版本的更新不能使用了。
————————————————————————————————————————————————————————————————————————————————————————
very nice!飞跃Feiyue! I have seen your Chinese version of FindText post in China, but after reading it, I still don't know much about the sub functions in FindText, such as FindText.Ocr Function, do not know where there are some examples to refer to. In this forum, you can also find OCR examples on the front page of this post and a post behind it, but they can't be used with the update of the version.

_________________________________________________________________
20210423:感谢飞跃的热心回复!!
_________________________________________________________________
20210423: Thank you Feiyue for your reply!!
Last edited by laozhhaiJohn on 22 Apr 2021, 23:07, edited 2 times in total.
feiyue
Posts: 295
Joined: 08 Aug 2014, 04:08

Re: FindText - Catch screen image into text and then find it

Post by feiyue » 21 Apr 2021, 23:14

feiyue wrote:
19 May 2016, 09:43
In addition to searching for screen text and images,
This function can also be used for simple text recognition.
Here is an example of a simple identification Numbers.
FindText.OCR() function can also be used to identify simple verification code.

Code: Select all


#Include <FindText>

F2::    ; To identify the number near the mouse

; For identification, we need to create a text library,
; Of course, the following text library is not strong enough,
; Perhaps in other computers, other screen resolutions,
; Other browser magnification, different fonts, need to regenerate.
; You can add the newly generated to the following existing text library,
; To enhance the generality of this text library.

Text:="|<0>*147$6.SnVVVVVnSU"
Text.="|<1>*147$5.9kV248Hw"
Text.="|<2>*148$6.yX11248EzU"
Text.="|<3>*149$6.yX13C11XyU"
Text.="|<4>*149$6.66+GmWz22U"
Text.="|<5>*149$6.TEES311XyU"
Text.="|<6>*148$6.CEUyXVVnSU"
Text.="|<7>*149$6.z1224A8MEU"
Text.="|<8>*148$6.SnVnSVVnSU"
Text.="|<9>*148$6.SnVVlT12QU"

CoordMode, Mouse
MouseGetPos, x, y
t1:=A_TickCount
;------------------------------
ok:=FindText(x-150, y-20, x+150, y+20, 0.2, 0.1, Text)
arr:=FindText.OCR(ok), OCR:=arr.text
;------------------------------
t1:=A_TickCount-t1
MsgBox, 4096, OCR, OCR Result: [%OCR%] in %t1% ms.
Return

c7aesa7r
Posts: 123
Joined: 02 Jun 2016, 21:09

Re: FindText - Capture screen image into text and then find it

Post by c7aesa7r » 22 Apr 2021, 04:52

feiyue wrote:
21 Apr 2021, 17:19
Updated to 8.5 version - 2021/04/22 :dance: :beer:
.... 1. Add: Since you can search for pictures now, the function of saving pictures has been added in CaptureS.
....... You can select a small range of pictures from the ScreenShots to save as pictures.
.... 2. Modify: Now when capturing the image, it no longer judges the physical state of the right mouse button,
....... but uses the logical state, which can be used in remote control.

@c7aesa7r The new version can meet your needs.
@feiyue When i click on CaptureS there's only pictures of the entire screen.

I mean to save a copy of the image after editing her in the gui:

1) Captured:
Spoiler
2) Edited:
Spoiler
Save a copy of the edited image (the image inside the red square only)
feiyue
Posts: 295
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 22 Apr 2021, 10:45

You can try the "SavePic" button in the CaptureS interface. This way of saving small pictures is more flexible.
This method needs to set the screen capture hotkey and use the hotkey to capture the screen first.
Last edited by feiyue on 22 Apr 2021, 11:26, edited 1 time in total.
c7aesa7r
Posts: 123
Joined: 02 Jun 2016, 21:09

Re: FindText - Capture screen image into text and then find it

Post by c7aesa7r » 22 Apr 2021, 10:49

This way you need to capture the image 2x which can happen to the images be a little different, and after you edit it in the GUI, cutting edges, for example, will not be the same as you have captured before.
feiyue
Posts: 295
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 22 Apr 2021, 11:23

@c7aesa7r :beer:
I updated it a little bit and realized your requirements.
Please download the latest version for test.
Save the image to the open directory of the CaptureS interface.
c7aesa7r
Posts: 123
Joined: 02 Jun 2016, 21:09

Re: FindText - Capture screen image into text and then find it

Post by c7aesa7r » 22 Apr 2021, 11:43

Thank you feiyue! Question: is possible to load an image and get the text from it? or load an image and get it to this window:
Spoiler
Post Reply

Return to “Scripts and Functions”