FindText - Capture screen image into text and then find it

[feiyue]

Updated to 7.9 version - 2021/01/15
.... 1. Modify: At present, it can support 101 × 101 pixel capture.
.... The previous sub window will generate 10201 controls, the number of controls exceeds the limit,
.... now do not use the sub window to generate enlarged image.
.... 2. Modify: At present, the combination search can return the exact width.
.... Previously, only an approximate width can be estimated.

[arahat]

feiyue!
Your work is fantastic. I'm using your code very well.
You are my Hero!
Thank you very much.

[necomerx]

Hi feiyue,
How to merge all my found image into my one script? I explain:

I have my antivirus sometimes oppening two windows after a scan succes that i want to close automatically. I write the script so the windows can be automatically closed when they appear. The first window loop work fine but the second is not closed until i place it's lines code at the top. Here is the script:

Code: Select all

 #Include FindText (by feiyue).ahk

 t1:=A_TickCount, X:=Y:=""


Text:="|<Avast>*86$71.zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz7zzzzzzzzzzw3zzzzzzzzzzk7zzzzzzzzzz07zzzzzzzzzw0Tzzzzzzzzzk1zzzzzzzzzz07zzzzzzzzzw0Tzzzzzzzzzk1zzzzzzzzzz07zzzzzzzzzw0Tzzzzzzzzzk1zzzzzzzzzz07zzzzzzzzzw0Tzzzzzzzzzk1zzzzzzzzzz07zzzzDzzzzw0TzzzwDzzzzk1zzzzkDzzzz07zzzz0Dzzzw0Tzzzy0Dzzzk1zzzzw0Dzzz07zzzzw0Dzzw0Tzzzzw0Dzzk1zzzzzw0Dzz07zzzzzw0Dzw0Tzzzzzw0Dzk1zzzzzzw0Dz07zzzzzzw0Dw0Tzzzzzzw0Dk1zzzzzzzw0D07zzzzzzzw000Tzzzzzzzw001zzzzzzzzw007zzzzzzzzw00Tzzzzzzzzw01zzzzzzzzzw07zzzzzzzzzy0Tzzzzzzzzzy1zzzzzzzzzzy7zzzzzzzzzzyTzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzk"

loop
 if (ok:=FindText(919, 393, 1115, 591, 0, 0, Text))
 
 {
 WinClose, ahk_class AvastCefWindow

 }

 MsgBox, 4096, Tip, % "Found:`t" Round(ok.MaxIndex())
   . "`n`nTime:`t" (A_TickCount-t1) " ms"
   . "`n`nPos:`t" X ", " Y
   . "`n`nResult:`t" (ok ? "Success !" : "Failed !")

 for i,v in ok
   if (i<=2)
     FindText.MouseTip(ok[i].x, ok[i].y)
	 
	 

	 Text:="|<Avast>*85$71.zzztzzztzzzzzzzDzzzwzzzzzztzzzzyTzzzzzbzzzzyTzzzzyTzzzzyTzzzzvzzzzzzTzzzzjzzzzzzTzzzyzzzzzzyTzzztzzzzzzyTzzzbzzzzzzyTzzzTzzzzzzyzzzxzzzzzzzyzzzvzzzzzzzxzzzjzzzzzzzxzzzTzzzzzzzvzzxzzzzzzlzvzzvzzzzzz1zrzzbzzzzzw3zbzzTzzzzzkDzjzyzzzzzz0zzTzxzzzzzw3zyzzvzzzzzkDzxzzbzzzzz0zztzzTzzzzw3zzvzyzzjzzkDzzrzxzyDzz0zzzjzvzsDzw3zzzTzrzsDzkDzzyzzjzsDz0zzzxzzDzsDw3zzznzzTzs7kDzzzjzyzzs70zzzzTzxzzs43zzzyzzvzzs0Dzzzxzzvzzs0zzzzrzzrzzs3zzzzjzzjzzsDzzzzTzzjzzszzzzxzzzTzzzzzzzvzzzTzzzzzzzjzzyzzzzzzzzTzzyzzzzzzzxzzzwzzzzzzznzzzwzzzzzzzDzzzwzzzzzzyzzzzxzzzzzzvzzzzxzzzzzzjzzzzwzzzzzwzzzzzwzzzzznzzzzzwzzzzzDzzzzzyTzzztzzzzzzzDzzzDzzzzzzzXzzlzzzzk"

 if (ok:=FindText(1692, 875, 1784, 965, 0, 0, Text))
 {
  WinClose, ahk_class AvastCefToaster
 }

 MsgBox, 4096, Tip, % "Found:`t" Round(ok.MaxIndex())
   . "`n`nTime:`t" (A_TickCount-t1) " ms"
   . "`n`nPos:`t" X ", " Y
   . "`n`nResult:`t" (ok ? "Success !" : "Failed !")

 for i,v in ok
   if (i<=2)
     FindText.MouseTip(ok[i].x, ok[i].y)
	 return

For the moment, for workaround, i simply winclose the second window when i winclose the first one, but i plan in the future to add other captured images/text and wrote somes lines for them (for example capture the youtube annoying message for login and click "NO Thanks" automatically) and i also want them to run in loop or persitently All in the only script that i run. I only want one script icon on my tray. Can you help me with this problem?


EDIT: I've found the solution for those who have that probelm in the future: The loop need to have brackets from the start of were i want to loop until the end of the line it need to stop. Sorry I'm still learning. Writting scripts is new for me.

[thebbandit]

Amazing rebase! I am looking forward to using the new class based style! Just have to say, WOW the new editor is looking very nice with the ability to edit after the capture! You are amazing.

Have you considered making a version that works in AHK v2? It is one of the things stopping me from even trying to use all the new object based GUI changes he made, since I cannot use this library with it.

[MrMan2101]

Forgive me for not knowing, but how do I actually download/ copy this code? Whenever I paste it into an AHK new file, it says

Error at line 1347

Line Text: Switch cmd
Error: This line does not contain a recognized action

The program will exit.

[JoeWinograd]

Line Text: Switch cmd
Error: This line does not contain a recognized action

That's because you're using an old version of AutoHotkey (Switch requires v1.1.31+). Download the latest AHK version and that error message will go away. Regards, Joe

[thebbandit]

I gave this my very best effort in making the conversion of this library over to AHK v2 and this was as far as I was able to make it.

FindText_v2_attempt.ahk

GUI does not load

(110.17 KiB) Downloaded 134 times

I am really hoping this is something of interest? I have done the majority of the tedious work of making conversions, but I am not sure if I did the & symbols correctly in the dll calls. And there are some issues with how values are declared because of the GUI system and how you were using them in the old style.

Really hoping you can pick up where I have been able to get, and make this work with v2! I made duplicates of most of the lines that were changed except for a few small ones here and there, so all the reference lines are there to look at.

In my opinion this library is mandatory for anything heavily processing the screen image, since it allows you to recycle the image data several times. I cannot move forward with the project I am trying to create without this working in V2.

[feiyue]

Updated to 8.0 version - 2021/01/31
.... 1. Add a parameter ( dir ), search direction, up, down, left, right four kinds.
.... 2. Modify: Now, the combination search can mix different mode.
.... 3. Add: the FindMultColor mode is added, it's fun, but not visual enough.

[wrekk]

When running this script from AutoHotkey.dll by HotkeyIt (using the addscript function in the dll), it just gives me an error at line 94.
Do you have any ideas as to why this happens? I am using the latest version of the script (8.0).

EDIT: Seems to be the "local" keyword that is causing the error.
EDIT2: Removed all the "local" keywords, but then I get an error with "StrReplace" not existing. I guess that's an issue in the .dll though. Will investigate further.

image.png (61.49 KiB) Viewed 3289 times

[feiyue]

; Usage: (required AHK v1.1.31+)

Also required Autohotkey.dll v1.1.31+


@thebbandit
In v2, StrPtr(a) <=> &a

[wrekk]

; Usage: (required AHK v1.1.31+)

Also required Autohotkey.dll v1.1.31+

Ah, I guess the .dll might be outdated then. I'll try updating it, thanks!

[thebbandit]

Updated to 8.0 version - 2021/01/31
.... 1. Add a parameter ( dir ), search direction, up, down, left, right four kinds.
.... 2. Modify: Now, the combination search can mix different mode.

WOW you have been busy! that is an amazing new feature! really making this library shine!

Edit:
Would you consider a property in the class which enables searching both 0x00CC0020|0x40000000 or just using 0x00CC0020, so I can just set that within my script instead of needing to modify the class

A simple property assigned to the class which can be set to true or false, and determines to include the extra 0x40000000 layer.

Because it causes a mouse flicker when you keep doing rapid shots, this should be disabled by default unless the person actually needs to capture that layer.


And one other suggestion, now that the class contains the bit data inside itself, would you consider adding a method for setting the screen bits from a bits object.

Code: Select all

Class Findtext{
  SetBits(bits){
    If IsObject(bits)
      this.bits := bits
  }
}

I found a way to do the threading I need within v1, so I am going to be having multiple workers sharing one master class object, and would like to share the bit data across the workers instead of them all requiring seperate screengrabs.

[feiyue]

Add the FindMultColor mode, which is interesting but not intuitive.
Find multi color mode is often used in games.

[srooo]

Add the FindMultColor mode, which is interesting but not intuitive.
Find multi color mode is often used in games.

It would be great if it could be as easy to use as FindText.GetColor(x,y). EX: FindText.MultGetColor

[need4speed]

I use more or less this simple code (only censored the text)

Code: Select all

#Include FindText.ahk
#SingleInstance, Force

Loop {
Sleep 1000

Text.="|<OK>*93$44.zzzzzzzzzzzzzzCensoredzzzzzzzzzzzzzzs"

 if (ok:=FindText(952, 660, 1070, 696, 0, 0, Text))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   Click, %X%, %Y%
 }

Text.="|<OK>*79$23.zzzzzzzzCensoredzzzzzzzz"

 if (ok:=FindText(1088, 562, 1192, 600, 0, 0, Text))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   Click, %X%, %Y%
 }

}

After a few seconds I get into this:

Critical Error in #include file "D:\AutoHotkey\FindText.ahk":
Function recursion limit exceeded.

Line#
130: if !(err1=0 && err0=0 && !arr.MaxIndex())
131: Break
132: }
133: SetBatchLines,%bch%
134: Return,arr.MaxIndex() ? arr:0
135: }
141: {
---> 144: if (!MyFunc)
145: {
146: x32 := "" . "5557565383EC548B9424880000008B8C24900000008B84248C00000001D13B8C" . "24800000000F4F8C248000000089CF894C242C31C985D20F48D189D689542424" . "8B94249400000001C23B9424840000000F4F94248400000085C00F49C889CB89" . "4C242889F929F18954243829DA85C9894C24048954241C0F8E8602000085D20F" . "8E7E020000837C2468050F84810200008B9C24B000000085DB0F8EDB07000031" . "FFC744241000000000C744240C0000000031EDC744241800000000897C241490" . "8B4C24148BB424AC0000008B5C24188B54241001CE89C829CB8B8C24AC000000"
245: x64 := "" . "4157415641554154555756534883EC688B8424F8000000488BB4242801000089" . "5424204589C68B9424F0000000448B84240001000044898C24C80000004101D0" . "443B8424E0000000440F4F8424E00000004589C7448944242C4531C085D2410F" . "48D089D7895424248B94240801000001C23B9424E80000000F4F9424E8000000" . "85C0440F49C04129FF4489442428895424304429C24585FF8954241C0F8ECD02" . "000085D20F8EC502000083F9050F84D3020000448B9424400100004585D20F8E" . "CC08000044897C24104489B424C00000004531ED4C8BB42420010000448BBC24"
358: this.MCode(MyFunc, A_PtrSize=8 ? x64:x32)
359: }
360: num:=info.MaxIndex(), j:=info[index], text:=j.1, w:=j.2, h:=j.3, mode:=j.8, color:=j.9, n:=j.10, comment:=j.11, e1:=(err1 && !j.12 ? Round(j.4*err1) : j.6), e0:=(err0 && !j.12 ? Round(j.5*err0) : j.7), sx:=in.sx, sy:=in.sy, sw:=in.sw, sh:=in.sh, Scan0:=bits.Scan0, Stride:=bits.Stride
367: if (mode=3)

The program is now unstable and will exit.

[feiyue]

@srooo Multiple colors can only be captured by hand.

@need4speed
Loop {
Text .= "|<OK>*93$44.zzzzzzzzzzzzzzCensoredzzzzzzzzzzzzzzs"
}
---->
Loop {
Text := "|<OK>*93$44.zzzzzzzzzzzzzzCensoredzzzzzzzzzzzzzzs"
}

[need4speed]

i don't run into the critical error, running the code on a different computer. I don't understand what's wrong with my loop definition, why this would lead to a critical issue.

[thebbandit]

@feiyue
I am trying to understand how to use the new MultColor, it looks like a very interesting new addition for myself.

Every other match mode has a button to apply the settings for that mode to the image capture. But in MultColor tab, it has
R/G/B [1-255] [check] FindMultColor [Undo]

But I click on the colors inside the edit window, and its building a list in the tooltip, but how do I apply it?


Also, I have been meaning to mention this, but a member of my discord server brought up using HSV (Hue, Saturation, Value) as color values instead of RGB. I did some research, and it would make a rather large difference in dealing with hue differences of one color.

Here is a link to a library for HSL and HSV

[feiyue]

@thebbandit
After selecting multiple colors, just click the OK button.

There are defects in using HSV color to calculate color similarity,
so I use a more scientific color similarity calculation method.

[chaoscreater]

@feiyue

Is it possible to make FindText use CoordMode Window instead of Screen?

Reason is I'm trying to combine Findtext with ControlClick. ControlClick can only use Window coordmode, but Findtext gets the X & Y coordinates based on Screen.