FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
feiyue
Posts: 259
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 14 Jan 2021, 14:13

Updated to 7.9 version - 2021/01/15 :dance: :beer:
.... 1. Modify: At present, it can support 101 × 101 pixel capture.
.... The previous sub window will generate 10201 controls, the number of controls exceeds the limit,
.... now do not use the sub window to generate enlarged image.
.... 2. Modify: At present, the combination search can return the exact width.
.... Previously, only an approximate width can be estimated.

arahat
Posts: 1
Joined: 16 Jan 2021, 06:29

Re: FindText - Capture screen image into text and then find it

Post by arahat » 16 Jan 2021, 06:44

feiyue!
Your work is fantastic. I'm using your code very well.
You are my Hero!
Thank you very much. :bravo:

necomerx
Posts: 30
Joined: 01 Jan 2021, 09:09

Re: FindText - Capture screen image into text and then find it

Post by necomerx » 16 Jan 2021, 19:45

Hi feiyue,
How to merge all my found image into my one script? I explain:

I have my antivirus sometimes oppening two windows after a scan succes that i want to close automatically. I write the script so the windows can be automatically closed when they appear. The first window loop work fine but the second is not closed until i place it's lines code at the top. Here is the script:

Code: Select all

 #Include FindText (by feiyue).ahk

 t1:=A_TickCount, X:=Y:=""


Text:="|<Avast>*86$71.zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz7zzzzzzzzzzw3zzzzzzzzzzk7zzzzzzzzzz07zzzzzzzzzw0Tzzzzzzzzzk1zzzzzzzzzz07zzzzzzzzzw0Tzzzzzzzzzk1zzzzzzzzzz07zzzzzzzzzw0Tzzzzzzzzzk1zzzzzzzzzz07zzzzzzzzzw0Tzzzzzzzzzk1zzzzzzzzzz07zzzzDzzzzw0TzzzwDzzzzk1zzzzkDzzzz07zzzz0Dzzzw0Tzzzy0Dzzzk1zzzzw0Dzzz07zzzzw0Dzzw0Tzzzzw0Dzzk1zzzzzw0Dzz07zzzzzw0Dzw0Tzzzzzw0Dzk1zzzzzzw0Dz07zzzzzzw0Dw0Tzzzzzzw0Dk1zzzzzzzw0D07zzzzzzzw000Tzzzzzzzw001zzzzzzzzw007zzzzzzzzw00Tzzzzzzzzw01zzzzzzzzzw07zzzzzzzzzy0Tzzzzzzzzzy1zzzzzzzzzzy7zzzzzzzzzzyTzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzk"

loop
 if (ok:=FindText(919, 393, 1115, 591, 0, 0, Text))
 
 {
 WinClose, ahk_class AvastCefWindow

 }

 MsgBox, 4096, Tip, % "Found:`t" Round(ok.MaxIndex())
   . "`n`nTime:`t" (A_TickCount-t1) " ms"
   . "`n`nPos:`t" X ", " Y
   . "`n`nResult:`t" (ok ? "Success !" : "Failed !")

 for i,v in ok
   if (i<=2)
     FindText.MouseTip(ok[i].x, ok[i].y)
	 
	 

	 Text:="|<Avast>*85$71.zzztzzztzzzzzzzDzzzwzzzzzztzzzzyTzzzzzbzzzzyTzzzzyTzzzzyTzzzzvzzzzzzTzzzzjzzzzzzTzzzyzzzzzzyTzzztzzzzzzyTzzzbzzzzzzyTzzzTzzzzzzyzzzxzzzzzzzyzzzvzzzzzzzxzzzjzzzzzzzxzzzTzzzzzzzvzzxzzzzzzlzvzzvzzzzzz1zrzzbzzzzzw3zbzzTzzzzzkDzjzyzzzzzz0zzTzxzzzzzw3zyzzvzzzzzkDzxzzbzzzzz0zztzzTzzzzw3zzvzyzzjzzkDzzrzxzyDzz0zzzjzvzsDzw3zzzTzrzsDzkDzzyzzjzsDz0zzzxzzDzsDw3zzznzzTzs7kDzzzjzyzzs70zzzzTzxzzs43zzzyzzvzzs0Dzzzxzzvzzs0zzzzrzzrzzs3zzzzjzzjzzsDzzzzTzzjzzszzzzxzzzTzzzzzzzvzzzTzzzzzzzjzzyzzzzzzzzTzzyzzzzzzzxzzzwzzzzzzznzzzwzzzzzzzDzzzwzzzzzzyzzzzxzzzzzzvzzzzxzzzzzzjzzzzwzzzzzwzzzzzwzzzzznzzzzzwzzzzzDzzzzzyTzzztzzzzzzzDzzzDzzzzzzzXzzlzzzzk"

 if (ok:=FindText(1692, 875, 1784, 965, 0, 0, Text))
 {
  WinClose, ahk_class AvastCefToaster
 }

 MsgBox, 4096, Tip, % "Found:`t" Round(ok.MaxIndex())
   . "`n`nTime:`t" (A_TickCount-t1) " ms"
   . "`n`nPos:`t" X ", " Y
   . "`n`nResult:`t" (ok ? "Success !" : "Failed !")

 for i,v in ok
   if (i<=2)
     FindText.MouseTip(ok[i].x, ok[i].y)
	 return


For the moment, for workaround, i simply winclose the second window when i winclose the first one, but i plan in the future to add other captured images/text and wrote somes lines for them (for example capture the youtube annoying message for login and click "NO Thanks" automatically) and i also want them to run in loop or persitently All in the only script that i run. I only want one script icon on my tray. Can you help me with this problem?


EDIT: I've found the solution for those who have that probelm in the future: The loop need to have brackets from the start of were i want to loop until the end of the line it need to stop. Sorry I'm still learning. Writting scripts is new for me.
Last edited by gregster on 16 Jan 2021, 19:46, edited 1 time in total.
Reason: [Code] tags added.

thebbandit
Posts: 36
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

Post by thebbandit » 22 Jan 2021, 05:45

Amazing rebase! I am looking forward to using the new class based style! Just have to say, WOW the new editor is looking very nice with the ability to edit after the capture! You are amazing.

Have you considered making a version that works in AHK v2? It is one of the things stopping me from even trying to use all the new object based GUI changes he made, since I cannot use this library with it.

MrMan2101
Posts: 3
Joined: 15 Apr 2017, 22:49

Re: FindText - Capture screen image into text and then find it

Post by MrMan2101 » 23 Jan 2021, 03:58

Forgive me for not knowing, but how do I actually download/ copy this code? Whenever I paste it into an AHK new file, it says

Error at line 1347

Line Text: Switch cmd
Error: This line does not contain a recognized action

The program will exit.

User avatar
JoeWinograd
Posts: 1648
Joined: 10 Feb 2014, 20:00
Location: U.S. Central Time Zone

Re: FindText - Capture screen image into text and then find it

Post by JoeWinograd » 23 Jan 2021, 04:15

MrMan2101 wrote:Line Text: Switch cmd
Error: This line does not contain a recognized action
That's because you're using an old version of AutoHotkey (Switch requires v1.1.31+). Download the latest AHK version and that error message will go away. Regards, Joe

thebbandit
Posts: 36
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

Post by thebbandit » 28 Jan 2021, 01:32

I gave this my very best effort in making the conversion of this library over to AHK v2 and this was as far as I was able to make it.
FindText_v2_attempt.ahk
GUI does not load
(110.17 KiB) Downloaded 19 times
I am really hoping this is something of interest? I have done the majority of the tedious work of making conversions, but I am not sure if I did the & symbols correctly in the dll calls. And there are some issues with how values are declared because of the GUI system and how you were using them in the old style.

Really hoping you can pick up where I have been able to get, and make this work with v2! I made duplicates of most of the lines that were changed except for a few small ones here and there, so all the reference lines are there to look at.

In my opinion this library is mandatory for anything heavily processing the screen image, since it allows you to recycle the image data several times. I cannot move forward with the project I am trying to create without this working in V2.

feiyue
Posts: 259
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 30 Jan 2021, 18:47

Updated to 8.0 version - 2021/01/31 :dance: :beer:
.... 1. Add a parameter ( dir ), search direction, up, down, left, right four kinds.
.... 2. Modify: Now, the combination search can mix different mode.
.... 3. Add: the FindMultColor mode is added, it's fun, but not visual enough.
Last edited by feiyue on 03 Feb 2021, 14:01, edited 1 time in total.

wrekk
Posts: 3
Joined: 11 May 2020, 15:20

Re: FindText - Capture screen image into text and then find it

Post by wrekk » 01 Feb 2021, 10:47

When running this script from AutoHotkey.dll by HotkeyIt (using the addscript function in the dll), it just gives me an error at line 94. :(
Do you have any ideas as to why this happens? I am using the latest version of the script (8.0).

EDIT: Seems to be the "local" keyword that is causing the error.
EDIT2: Removed all the "local" keywords, but then I get an error with "StrReplace" not existing. I guess that's an issue in the .dll though. Will investigate further.
image.png
image.png (61.49 KiB) Viewed 972 times

feiyue
Posts: 259
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 01 Feb 2021, 14:30

; Usage: (required AHK v1.1.31+)

Also required Autohotkey.dll v1.1.31+


@thebbandit
In v2, StrPtr(a) <=> &a

wrekk
Posts: 3
Joined: 11 May 2020, 15:20

Re: FindText - Capture screen image into text and then find it

Post by wrekk » 01 Feb 2021, 14:34

feiyue wrote:
01 Feb 2021, 14:30
; Usage: (required AHK v1.1.31+)

Also required Autohotkey.dll v1.1.31+
Ah, I guess the .dll might be outdated then. I'll try updating it, thanks!

thebbandit
Posts: 36
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

Post by thebbandit » 01 Feb 2021, 23:21

feiyue wrote:
30 Jan 2021, 18:47
Updated to 8.0 version - 2021/01/31 :dance: :beer:
.... 1. Add a parameter ( dir ), search direction, up, down, left, right four kinds.
.... 2. Modify: Now, the combination search can mix different mode.
WOW you have been busy! :D that is an amazing new feature! really making this library shine!

Edit:
Would you consider a property in the class which enables searching both 0x00CC0020|0x40000000 or just using 0x00CC0020, so I can just set that within my script instead of needing to modify the class

A simple property assigned to the class which can be set to true or false, and determines to include the extra 0x40000000 layer.

Because it causes a mouse flicker when you keep doing rapid shots, this should be disabled by default unless the person actually needs to capture that layer.


And one other suggestion, now that the class contains the bit data inside itself, would you consider adding a method for setting the screen bits from a bits object.

Code: Select all

Class Findtext{
  SetBits(bits){
    If IsObject(bits)
      this.bits := bits
  }
}
I found a way to do the threading I need within v1, so I am going to be having multiple workers sharing one master class object, and would like to share the bit data across the workers instead of them all requiring seperate screengrabs.

feiyue
Posts: 259
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 05 Feb 2021, 20:01

Add the FindMultColor mode, which is interesting but not intuitive.
Find multi color mode is often used in games. :dance: :beer:

srooo
Posts: 4
Joined: 05 Jun 2019, 14:50

Re: FindText - Capture screen image into text and then find it

Post by srooo » 06 Feb 2021, 20:12

feiyue wrote:
05 Feb 2021, 20:01
Add the FindMultColor mode, which is interesting but not intuitive.
Find multi color mode is often used in games. :dance: :beer:
It would be great if it could be as easy to use as FindText.GetColor(x,y). EX: FindText.MultGetColor

need4speed
Posts: 137
Joined: 22 Apr 2016, 06:50

Re: FindText - Capture screen image into text and then find it

Post by need4speed » 06 Feb 2021, 21:43

I use more or less this simple code (only censored the text)

Code: Select all

#Include FindText.ahk
#SingleInstance, Force

Loop {
Sleep 1000

Text.="|<OK>*93$44.zzzzzzzzzzzzzzCensoredzzzzzzzzzzzzzzs"

 if (ok:=FindText(952, 660, 1070, 696, 0, 0, Text))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   Click, %X%, %Y%
 }

Text.="|<OK>*79$23.zzzzzzzzCensoredzzzzzzzz"

 if (ok:=FindText(1088, 562, 1192, 600, 0, 0, Text))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   Click, %X%, %Y%
 }

}
After a few seconds I get into this:

Critical Error in #include file "D:\AutoHotkey\FindText.ahk":
Function recursion limit exceeded.

Line#
130: if !(err1=0 && err0=0 && !arr.MaxIndex())
131: Break
132: }
133: SetBatchLines,%bch%
134: Return,arr.MaxIndex() ? arr:0
135: }
141: {
---> 144: if (!MyFunc)
145: {
146: x32 := "" . "5557565383EC548B9424880000008B8C24900000008B84248C00000001D13B8C" . "24800000000F4F8C248000000089CF894C242C31C985D20F48D189D689542424" . "8B94249400000001C23B9424840000000F4F94248400000085C00F49C889CB89" . "4C242889F929F18954243829DA85C9894C24048954241C0F8E8602000085D20F" . "8E7E020000837C2468050F84810200008B9C24B000000085DB0F8EDB07000031" . "FFC744241000000000C744240C0000000031EDC744241800000000897C241490" . "8B4C24148BB424AC0000008B5C24188B54241001CE89C829CB8B8C24AC000000"
245: x64 := "" . "4157415641554154555756534883EC688B8424F8000000488BB4242801000089" . "5424204589C68B9424F0000000448B84240001000044898C24C80000004101D0" . "443B8424E0000000440F4F8424E00000004589C7448944242C4531C085D2410F" . "48D089D7895424248B94240801000001C23B9424E80000000F4F9424E8000000" . "85C0440F49C04129FF4489442428895424304429C24585FF8954241C0F8ECD02" . "000085D20F8EC502000083F9050F84D3020000448B9424400100004585D20F8E" . "CC08000044897C24104489B424C00000004531ED4C8BB42420010000448BBC24"
358: this.MCode(MyFunc, A_PtrSize=8 ? x64:x32)
359: }
360: num:=info.MaxIndex(), j:=info[index], text:=j.1, w:=j.2, h:=j.3, mode:=j.8, color:=j.9, n:=j.10, comment:=j.11, e1:=(err1 && !j.12 ? Round(j.4*err1) : j.6), e0:=(err0 && !j.12 ? Round(j.5*err0) : j.7), sx:=in.sx, sy:=in.sy, sw:=in.sw, sh:=in.sh, Scan0:=bits.Scan0, Stride:=bits.Stride
367: if (mode=3)

The program is now unstable and will exit.

feiyue
Posts: 259
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 06 Feb 2021, 23:04

@srooo Multiple colors can only be captured by hand.

@need4speed
Loop {
Text .= "|<OK>*93$44.zzzzzzzzzzzzzzCensoredzzzzzzzzzzzzzzs"
}
---->
Loop {
Text := "|<OK>*93$44.zzzzzzzzzzzzzzCensoredzzzzzzzzzzzzzzs"
}

need4speed
Posts: 137
Joined: 22 Apr 2016, 06:50

Re: FindText - Capture screen image into text and then find it

Post by need4speed » 09 Feb 2021, 11:19

i don't run into the critical error, running the code on a different computer. I don't understand what's wrong with my loop definition, why this would lead to a critical issue.

thebbandit
Posts: 36
Joined: 02 Jul 2019, 11:34

Re: FindText - Capture screen image into text and then find it

Post by thebbandit » 16 Feb 2021, 00:20

@feiyue
I am trying to understand how to use the new MultColor, it looks like a very interesting new addition for myself.

Every other match mode has a button to apply the settings for that mode to the image capture. But in MultColor tab, it has
R/G/B [1-255] [check] FindMultColor [Undo]

But I click on the colors inside the edit window, and its building a list in the tooltip, but how do I apply it?


Also, I have been meaning to mention this, but a member of my discord server brought up using HSV (Hue, Saturation, Value) as color values instead of RGB. I did some research, and it would make a rather large difference in dealing with hue differences of one color.

Here is a link to a library for HSL and HSV

feiyue
Posts: 259
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 17 Feb 2021, 02:49

@thebbandit
After selecting multiple colors, just click the OK button.

There are defects in using HSV color to calculate color similarity,
so I use a more scientific color similarity calculation method.

chaoscreater
Posts: 23
Joined: 12 Sep 2019, 21:15

Re: FindText - Capture screen image into text and then find it

Post by chaoscreater » 17 Feb 2021, 16:29

@feiyue

Is it possible to make FindText use CoordMode Window instead of Screen?

Reason is I'm trying to combine Findtext with ControlClick. ControlClick can only use Window coordmode, but Findtext gets the X & Y coordinates based on Screen.

Post Reply

Return to “Scripts and Functions”