FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
songdg
Posts: 494
Joined: 04 Oct 2017, 20:04

Re: FindText - Capture screen image into text and then find it

08 Oct 2017, 02:41

Thanks for sharing,this is an awesome tool!I wonder how to turn off the always on top feature,it cause a little bit inconvenient,thanks again.
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

22 Oct 2017, 06:42

I've made some minor improvements, and the generated code is easier to copy and use.
VaVa

Re: FindText - Capture screen image into text and then find it

06 Nov 2017, 09:52

Hi Feiyue,

Thank you for sharing this wonderful script. LOVE IT !

Here is my last question...
The script works perfectly on full desktop screen. How to fix it to work on 'Active Window, only ?

Thank you much!
leosouza85
Posts: 90
Joined: 22 Jul 2016, 16:28

Re: FindText - Capture screen image into text and then find it

06 Nov 2017, 19:00

Why not do this:

When the script capture the screen, the script automaticaly zoom in and out to simulate various resolutions, and font sharpness too if possible... maybe 5 resolutions samples and 5 samples with different font sharpening setting...
The code generated to paste will be 10 times bigger but maybe will turn computer settings proof.

Best regards!
User avatar
boiler
Posts: 16705
Joined: 21 Dec 2014, 02:44

Re: FindText - Capture screen image into text and then find it

06 Nov 2017, 19:30

Searching for images does not depend on screen resolution. On a smaller resolution screen, the same image will appear larger, but pixel-for-pixel, it's the same (the pixels themselves are larger, not any details about the image). An image search, whether using this tool or the ImageSearch command, finds it in any resolution. The problem becomes when different Windows scaling factors are used. This is because Windows sort of smudges images when it stretches them, so it's not a faithful representation of the original, and it can't really be predicted accurately because it could change depending on where it is on the screen, as is the case with anti-aliased letters.
leosouza85
Posts: 90
Joined: 22 Jul 2016, 16:28

Re: FindText - Capture screen image into text and then find it

07 Nov 2017, 19:47

boiler wrote:Searching for images does not depend on screen resolution. On a smaller resolution screen, the same image will appear larger, but pixel-for-pixel, it's the same (the pixels themselves are larger, not any details about the image). An image search, whether using this tool or the ImageSearch command, finds it in any resolution. The problem becomes when different Windows scaling factors are used. This is because Windows sort of smudges images when it stretches them, so it's not a faithful representation of the original, and it can't really be predicted accurately because it could change depending on where it is on the screen, as is the case with anti-aliased letters.
Thanks for the answer :-)
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: FindText - Capture screen image into text and then find it

07 Nov 2017, 20:01

@boiler: Could you say a bit more about anti-aliased letters, is that for images/text or both? Is something changing how text appears to prevent aliasing, what does this, and what does it do?
Also, do you think it could have some relevance for this:
fonts: text to image (pixels don't quite match) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=37937
Thanks.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
boiler
Posts: 16705
Joined: 21 Dec 2014, 02:44

Re: FindText - Capture screen image into text and then find it

07 Nov 2017, 20:28

I believe it is mainly an issue with letters. Although icons and other graphics are often created using anti-aliasing techniques to make them smoother and such, they are the same no matter where they are drawn if they are pixel-based graphics. Vector graphics get drawn differently depending on various factors like size and orientation. For letters, it seems that the text rendering engine in Windows will draw letters differently when anti-aliased depending on where it shows up, which may be because they are vector-based objects. For example, if you were to look at a black letter "I" magnified, it's weight may need to be 1.5 pixels wide, and it is simulated by drawing a black vertical line next to either a grey or some other color line. Depending on where the "I" is located, the black line could be on the left and the lighter color on the right, or vice-versa. It could even be two lines sort of in between.

One way to attempt to eliminate the issue is to turn off anti-aliasing in Windows, but sometimes you don't have control over that (like if it's for some other user's machine). Also sometimes it doesn't quite work because software may follow its own rules. I've seen where turning off anti-aliasing just changed it from being multiple colors (even black letters are made up of multiple colors if you zoom in on them) to shades of grey, still subject to the same problems of being rendered differently depending on location.

I believe that could very well be the problem in what you linked, although one would hope that a good OCR algorithm would be able to handle those differences since they typically handle different fonts, sizes, colors, etc.
User avatar
jeeswg
Posts: 6902
Joined: 19 Dec 2016, 01:58
Location: UK

Re: FindText - Capture screen image into text and then find it

07 Nov 2017, 20:49

Thanks very much boiler.

Re. 'turn off anti-aliasing in Windows'. Did you have a particular option/setting in mind? I got some interesting results searching for: explorer turn off anti-aliasing, explorer disable anti-aliasing.

This had an interesting list of suggestions.
Disable all font smoothing in Windows 7 (ie: including - Microsoft Community
https://answers.microsoft.com/en-us/win ... aadaecc2d2

I.e. I could see if the text's appearance differed if I switched off anti-aliasing. I'm actually only interested in the text in Edit controls at present. Thanks.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
boiler
Posts: 16705
Joined: 21 Dec 2014, 02:44

Re: FindText - Capture screen image into text and then find it

07 Nov 2017, 21:24

jeeswg wrote:Re. 'turn off anti-aliasing in Windows'. Did you have a particular option/setting in mind? I got some interesting results searching for: explorer turn off anti-aliasing, explorer disable anti-aliasing.
I have only done it in RDP settings to affect a remote desktop, not my local one. I did see that it was possible, but you probably have already learned more about changing that setting than I know about it.
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

23 Nov 2017, 10:30

Updated to the v5.5 version, in order to identify a variety of color verification code, I improved the color model, now it can adapt to various colors. :D
burque505
Posts: 1731
Joined: 22 Jan 2017, 19:37

Re: FindText - Capture screen image into text and then find it

23 Nov 2017, 10:42

Thank you, feiyue, for this script and all your continued work on it.
Regards,
burque505
MikeTheNumbers
Posts: 2
Joined: 24 Nov 2017, 00:22

Re: FindText - Capture screen image into text and then find it

24 Nov 2017, 05:49

Hi Feiyue.
I'm new to ATH, and have no experience with coding what so ever.
But I have been looking for a way to use AutoHotkey as an Anti-AFK for a game.

I made a post in the ask for help part of the forum https://autohotkey.com/boards/viewtopic ... 18&t=40432

I've now been doing a little digging since that post, and I stumbled upon your work, and I actually think It might just work for the purpose I need It for, with a few adjustment, adjustment I'm not able to make myself.
I was wondering if you had the time, if you could check my request out, and if possible, maybe help me out.

/Mike
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

06 Dec 2017, 04:51

hal123 wrote:
Would it somehow be possible to make it work on dynamic text (Taking the text to find from a file)

My idea is somehow to generate the image to find. First I capture all the letters in the alphabet, based on the text and captured letters I then generate the image.

Code: Select all

Pseudo code:
LetterA.="|<A>*144$10.60Q1EBUW28Tl1A6kO"
LetterB.="|<B>*144$8.z8O2Vjm6Uc+6z8"
LetterC.="|<C>*144$10.7UX44E1040E112A7W"

TextToFindDynamic = "ABC"

dynImage = generateImage(TextToFindDynamic)

ok:=FindText(0,0,150000,150000,0,0,dynImage)

generateImage(txt)
{
	return = LetterA + LetterB + LetterC
}
How can I merge the letters into an "image word" that I can search for?
Updated to the v5.6 version, Add a new Function: FindText2()
This function is used for combination lookup,
for example, a 0-9 text library has been set up, and any ID number can be found.
Use Pic(Text,1) and PicN(number) when using. The simple code:

Code: Select all

Text:=""
Text.="|<1>*127$3.KGGGw"
Text.="|<2>*127$5.R68F4F3w"
Text.="|<3>*127$5.R49UV35o"
Text.="|<4>*127$6.26+GGWS27U"
Text.="|<5>*127$5.z27cVX5o"
Text.="|<6>*127$5.Ca5glX5o"
Text.="|<7>*127$5.z8EW48EY"
Text.="|<8>*127$5.R6+bFX5o"
Text.="|<9>*127$5.R6ANh3/Y"
Text.="|<0>*127$5.R6AMlX5o"

Pic(Text,1)

if (ok:=FindText2(492, 68, 150000, 150000, 0, 0, PicN("201712")))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  MouseMove, X, Y
}
r2997790
Posts: 71
Joined: 02 Feb 2017, 02:46

Re: FindText - Capture screen image into text and then find it

14 Dec 2017, 04:23

Thank you for the FindText() script! It is amazing.

I am trying to make a loop from it, to do this:

Load URL
FindText

LoadURL
FindText

etc

It works the first time, but not more than once. I can find multiple text/images on one page BUT as soon as I try to load a new URL the function stops working when I call it.

Does anyone have any idea how to fix this?

Many thanks
R
SOTE
Posts: 1426
Joined: 15 Jun 2015, 06:21

Re: FindText - Capture screen image into text and then find it

19 Dec 2017, 01:20

Looks like this would be good to use with desktop icons, so an automation script could always find and click them. Another use is in browsers, where it could click a particular image that is linked to a URL. The advantage being that it can be used as an alternative way to interact with programs, bypassing more complex methods that use COM, Chrome, Java, etc... I had something that was a bit like this, but you have created a far better script.

For screen resolution and browser magnification, seems like that could be put into the script to check if they match before starting the search. If it doesn't match, it won't start the search and give you an error right away. If it does match, then proceed with the search.
robodesign
Posts: 932
Joined: 30 Sep 2017, 03:59
Location: Romania
Contact:

Re: FindText - Capture screen image into text and then find it

26 Jan 2018, 15:53

Hi!

It seems very nice... Can this script be used to detect the text cursor/caret on screen? Thank you.
Best regards, Marius.
-------------------------
KeyPress OSD v4: GitHub or forum. (presentation video)
Quick Picto Viewer: GitHub or forum.
AHK GDI+ expanded / compilation library (on GitHub)
My home page.
User avatar
ivill
Posts: 124
Joined: 13 May 2016, 02:23

Re: FindText - Capture screen image into text and then find it

01 Feb 2018, 06:01

大神你好,打扰了,请问你的薇信多少,我有个很简单的脚本问题想请教一下您,方便的话可以加下我吗ivill66
Redeemed548
Posts: 2
Joined: 08 Feb 2018, 20:09

Re: FindText - Capture screen image into text and then find it

08 Feb 2018, 20:30

Thank you for your wonderful function feiyue!! This has solved something I've been desiring for years.

I would like to be able differentiate between black text on a yellow background as opposed to black text on a white background. Would it be possible to add the option for color dependent pattern recognition? For example, using the two-color mode when searching for patterns only, and multi-color for color contextual patterns. I was thinking of a color depth of 8-bit (256 colors) or less.

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: furqan, kashmirLZ and 78 guests