FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 22 Mar 2022, 22:30

@NoxDied great! The good thing is the actual FindText search is a small bit of code that you paste anywhere and do tests.

Code: Select all

Text:="|<>**50$69.000000000000000000000000000000000000000000000000000000000000000TzzTDy001s021UC/Uk00tw0EA0lk600Dzk2TXaADk03030HwQFXS00nrM21XaAk00C2dUEA0l600Djbi2TUCAk03k00sH4TFXS0M0032TW2ACk3zzzsE4EFk6042MA20W2/0k0j2yUTzkTDw019Ko0000000090q00000000Dzzw000000000000000000000000000004"

if (ok:=FindText(X, Y, 0, 0, 0, 0, 0, 0, Text))  ; Four (0's) = entire screen
{
  FindText().Click(X, Y, "L")   ; you can change this to MouseClick, Left, X, Y, 1, 1 if you like
}

thaihoa3189
Posts: 35
Joined: 23 Mar 2022, 22:10

Re: FindText - Capture screen image into text and then find it

Post by thaihoa3189 » 23 Mar 2022, 22:24

Can someone make a full function youtube tutorial video for this FindText class please? :thumbup:

Descolada
Posts: 1099
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 26 Mar 2022, 11:54

feiyue wrote:
21 Mar 2022, 17:15
Descolada wrote:
19 Mar 2022, 04:04
What I am trying to find out is whether I could do something like this:

Code: Select all

Text.="|<S>*150$5.003cEM88T002*150$4.00S8kl7U08" ; I combined both "S" characters into one
FindText().PicLib(Text, 1)
ok := FindText(X,Y,0,0,0,0,0,0, FindText().PicN("SSS"),,0,1) ; And now this would first look for S1, then check if it is followed by S1 or S2 and so on until it finds a complete match.
The latest update has fulfilled your wish. :dance: :beer:

Code: Select all

Text:=""
Text.="|<o>*150$5.003cEM88T002" ; I combined both "S" characters into one
Text.="|<o>*150$5.003cEM88T002" ; I combined both "S" characters into one
Text.="|<k>*150$5.003cEM88T002" ; I combined both "S" characters into one
Text.="|<k>*150$5.003cEM88T002" ; I combined both "S" characters into one
Text.="|<k>*150$5.003cEM88T002" ; I combined both "S" characters into one
ok := FindText(X,Y,0,0,0,0,0,0, Text,,0,["ok","ko"]) ; And now this would first look for S1, then check if it is followed by S1 or S2 and so on until it finds a complete match.
Thank you for a quick reply. :) I tried to use your new functionality, but I cannot seem to get it working.
Capture.JPG
Capture.JPG (11.51 KiB) Viewed 5141 times
Objective: find all 3 occurrences of the word "Asa"

This code does not find anything:

Code: Select all

Text:= ""
Text.="|<A>*162$7.472VF4WzkMA"
Text.="|<A>*165$8.61UI92NWTYD1U"
Text.="|<s>**50$5.Tb3klz"
Text.="|<s>**50$5.Tb3klx"
Text.="|<a>**50$5.xA/snz"
Text.="|<a>**50$6.SH3TnnTU"

ok:=FindText(X, Y, 258-150000, 336-150000, 258+150000, 336+150000, 0.1, 0.1, Text,, 1,["Asa"])
This one finds the first "Asa":

Code: Select all

Text:= ""
Text.="|<A>*162$7.472VF4WzkMA"
;Text.="|<A>*165$8.61UI92NWTYD1U"
Text.="|<s>**50$5.Tb3klz"
;Text.="|<s>**50$5.Tb3klx"
Text.="|<a>**50$5.xA/snz"
;Text.="|<a>**50$6.SH3TnnTU"

ok:=FindText(X, Y, 258-150000, 336-150000, 258+150000, 336+150000, 0.1, 0.1, Text,, 1,["Asa"])
Finds the second one:

Code: Select all

Text:= ""
Text.="|<A>*162$7.472VF4WzkMA"
;Text.="|<A>*165$8.61UI92NWTYD1U"
;Text.="|<s>**50$5.Tb3klz"
Text.="|<s>**50$5.Tb3klx"
;Text.="|<a>**50$5.xA/snz"
Text.="|<a>**50$6.SH3TnnTU"

ok:=FindText(X, Y, 258-150000, 336-150000, 258+150000, 336+150000, 0.1, 0.1, Text,, 1,["Asa"])
This one finds only the third one:

Code: Select all

Text:= ""
Text.="|<A>*162$7.472VF4WzkMA"
Text.="|<A>*165$8.61UI92NWTYD1U"
Text.="|<s>**50$5.Tb3klz"
Text.="|<s>**50$5.Tb3klx"
Text.="|<a>**50$5.xA/snz"
Text.="|<a>**50$6.SH3TnnTU"
FindText().PicLib(Text, 1)

ok:=FindText(X, Y, 258-150000, 336-150000, 258+150000, 336+150000, 0.1, 0.1, FindText().PicN("Asa"),, 1,1)

feiyue
Posts: 349
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

Post by feiyue » 28 Mar 2022, 04:57

@Descolada I updated it. Now combined search is case sensitive. :dance:

Descolada
Posts: 1099
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 28 Mar 2022, 12:12

feiyue wrote:
28 Mar 2022, 04:57
@Descolada I updated it. Now combined search is case sensitive. :dance:
Thanks, works perfectly now! :superhappy: :beer:

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 29 Mar 2022, 02:23

thaihoa3189 wrote:
23 Mar 2022, 22:24
Can someone make a full function youtube tutorial video for this FindText class please? :thumbup:
I only have different ways to incorporate different mouse clicks on this thread. viewtopic.php?p=441482#p441482
I want to do a video tutorial but sorry paid work comes 1st. One day maybe. Took me months to learn FindText Function Properly

thaihoa3189
Posts: 35
Joined: 23 Mar 2022, 22:10

Re: FindText - Capture screen image into text and then find it

Post by thaihoa3189 » 29 Mar 2022, 20:36

SteveMylo wrote:
29 Mar 2022, 02:23
thaihoa3189 wrote:
23 Mar 2022, 22:24
Can someone make a full function youtube tutorial video for this FindText class please? :thumbup:
I only have different ways to incorporate different mouse clicks on this thread. viewtopic.php?p=441482#p441482
I want to do a video tutorial but sorry paid work comes 1st. One day maybe. Took me months to learn FindText Function Properly
Many thanks for your guide. It is very useful to me. While waiting for your next instruction I have 1 question. That is whether the FindText function can find multiple text in a command or not. Like : FindText(X, Y, 0, 0, 1366, 768, 0, 0, Text1|Text2|Text3). If it is possible what is the correct syntax of it? :)

Descolada
Posts: 1099
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 29 Mar 2022, 23:26

thaihoa3189 wrote:
29 Mar 2022, 20:36
SteveMylo wrote:
29 Mar 2022, 02:23
thaihoa3189 wrote:
23 Mar 2022, 22:24
Can someone make a full function youtube tutorial video for this FindText class please? :thumbup:
I only have different ways to incorporate different mouse clicks on this thread. viewtopic.php?p=441482#p441482
I want to do a video tutorial but sorry paid work comes 1st. One day maybe. Took me months to learn FindText Function Properly
Many thanks for your guide. It is very useful to me. While waiting for your next instruction I have 1 question. That is whether the FindText function can find multiple text in a command or not. Like : FindText(X, Y, 0, 0, 1366, 768, 0, 0, Text1|Text2|Text3). If it is possible what is the correct syntax of it? :)
You got it pretty much right. "|<Text1>**50$5.Tb3klz|<Text2>**50$5.Tb3klz" is the correct syntax and will return a match for either Text1 or Text2 (if FindAll is set to 0) or all matches for either Text1 or Text2 (if FindAll is set to 1).
These will also work:

Code: Select all

Text1 := "|<Text1>**50$5.Tb3klz"
Text2 := "|<Text2>**50$5.Tb3klz"
FindText(X, Y, 0, 0, 1366, 768, 0, 0, Text1 . Text2)

Code: Select all

Text := "|<Text1>**50$5.Tb3klz"
Text .= "|<Text2>**50$5.Tb3klz"
FindText(X, Y, 0, 0, 1366, 768, 0, 0, Text)

badge234
Posts: 3
Joined: 08 Nov 2021, 10:48

Re: FindText - Capture screen image into text and then find it

Post by badge234 » 30 Mar 2022, 20:58

After multiple trials and error, I figured it out I think. It seems that if you have your screens setup with different zoom factors, then FindText won't work. Even trying to click on the "capture" button will give you a result from somewhere else on the screen and not where you actually clicked. I have 1 screen setup at 100% and the other at 125% and there seems to be the problem. If I but both at 100%, then everything works well. I dont know if there's a workaround for this, but I'd really like to be able to keep my secdond screen at 125%.

Thanks.

Paul

badge234 wrote:
08 Nov 2021, 10:56
Hi,

I love your code and I started using it for a few projects. However, for some reason after changing my computer for a new one with new monitors, when I try to capture text or images on one of the external monitors, FindText returns a blank capture image (I'm using a laptop with 2 external monitors). If I capture from the laptop screen, everything works but if I caption on either of the other 2 monitors, I get a blank image.

Do you know why this is appening?

Also, if I capture from the laptop monitor but then search for the image string on one of the 2 other monitors, the string/image is not found.

Thank you so much for your work.

Paul

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 30 Mar 2022, 23:18

@badge234 The rules say on page one that all monitors have to be the same DPI and resolutions.
There is a workaround. The start of the discussions starts here with myself and another user. Please read all of our conversations. :arrow: viewtopic.php?p=435203#p435203

Tedious Method:
  • Basically what you have to do, is run FindText GUI in your normal Main Monitor, capturing what you want.
  • Then close down ALL applications including any Autohotkey scripts running. Then go to your Computer settings and make your second monitor your 'MAIN Monitor'.
  • Then Open the FindText GUI again and capture what you need from the monitor. The 'Capture' method will work properly now. Paste the results in your script.
  • Then reset everything again and start your normal main monitor again and continue as normal.
It's tedious to set up the script but you only need to do it once. So the one script will search for the 100% scale and if that fails it'll search for the 125% scale. Or however you like to run the script.

I can't remember but you may be able to skip that method and just take a window snippet of the image on the other screen, drag the snippet into your main monitor and do a CAPTURE from there. But if you are dealing with trying to find actual 'TEXT' and not normal images, then I would use the tedious method. As antialiasing issues can stuff things up with 'text' searching.

Here is a basic example of a script I use that is in a loop, but you can see how I have different search areas in there.
You can search the entire screen of both monitors but it can be a little slow depending on your speed needs. My Script below is fast cause I designated certain small areas for FindText to search. If you are capturing different ranges like my script on the second monitor, then the tedious method is needed for the Capture Range to work properly.

Code: Select all

#NoEnv
SetWorkingDir %A_ScriptDir%
CoordMode, Mouse, Screen
#SingleInstance, Force
#Persistent 

DllCall("SetThreadDpiAwarenessContext", "ptr", -3, "ptr")    ; add this on top of all your multi-monitor scripts. It fixed issues on mousemoves for me on different screens. 

x::
loop
{
   Text:="|<1>171717-000000$22.zzXzzyDzzkTzw0Twk1zV07w7Xzk7TzU8Xz0QDw0UDU60y0s3k3MQ0U3k01T0M3s3cj0TXw0yDkDzzUzzybzzs"
   Text.="|<2>252525-000000$61.1Uzn1za3sCAqTlYzXDwaCTBkrPVjqv7DauPhokPRvbntBrmTDixbntavnDrrSbnw1Rs3vvj7nvtYrnNgbUMBwkPtUq3o"
   Text.="|<3>171717-000000$22.UzVw1s1k303040A000M001k007U00S001w00Ts00y007k00T000M000U0020008001UK0T0w1s"
	
if !FindText(X, Y, -524, -478, -142, -404, 0, 0, Text)    ; 4k monitor 2
if !FindText(X, Y, -2038, 573, -1488, 805, 0, 0, Text)  ;  4k monitor 2
if !FindText(X, Y, 1673, 15, 1919, 149, 0, 0, Text)        ; now search 1080 Main Monitor
	break
MouseClick, Left, X, Y, 1, 1		
}
return

thaihoa3189
Posts: 35
Joined: 23 Mar 2022, 22:10

Re: FindText - Capture screen image into text and then find it

Post by thaihoa3189 » 31 Mar 2022, 23:33

You got it pretty much right. "|<Text1>**50$5.Tb3klz|<Text2>**50$5.Tb3klz" is the correct syntax and will return a match for either Text1 or Text2 (if FindAll is set to 0) or all matches for either Text1 or Text2 (if FindAll is set to 1).
These will also work:

Code: Select all

Text1 := "|<Text1>**50$5.Tb3klz"
Text2 := "|<Text2>**50$5.Tb3klz"
FindText(X, Y, 0, 0, 1366, 768, 0, 0, Text1 . Text2)

Code: Select all

Text := "|<Text1>**50$5.Tb3klz"
Text .= "|<Text2>**50$5.Tb3klz"
FindText(X, Y, 0, 0, 1366, 768, 0, 0, Text)
Thank you very much. This is what I've been looking for for so long ! :D
Last edited by gregster on 31 Mar 2022, 23:35, edited 1 time in total.
Reason: Fixed quote.

shnxx
Posts: 6
Joined: 01 Apr 2022, 07:27

Re: FindText - Capture screen image into text and then find it

Post by shnxx » 07 Apr 2022, 04:56

Hi!

I found this amazing tool and based on the description, it is what I needed. I'm currently doing some hella tasks right now and it is taking a lot of time even though it is easy. What I'm currently doing is that I have this notepad and these strings/texts are listed on there, I just copy one of them then go to the browser, ctrl+F then paste to check if this text exists. If not, then I'll copy now the 2nd string/text from the list, and repeat. I need to do it in 372 files and I have 25 texts to search on each webpage.

For example, assume I have 50 tabs and need to find these 25 strings in each one. And if one of these strings is discovered, I'd like to save it on another notepad and include a link to the webpage where it was discovered. Then it will close the tab (perhaps by pressing CTRL+W), which will switch to another tab, and begin searching again.

Here's what I currently have.

Code: Select all


*^F1:: ; CTRL+F1
Loop 
{
    FileReadLine, line, C:\Users\SHNX\Desktop\strings.txt, %A_Index% 
    if ErrorLevel
        break
    
    ControlSend,, ^f, ahk_exe msedge.exe
    ControlSend,, %line%{Enter}, ahk_exe msedge.exe
    
   ; I need something here that will trigger if the string(s) were found in the webpage
   ; if %line% is found
   ;    Save this %line% to a notepad with the webpage where it was discovered. ; Example: 12-34568-901 - https://someWebsite.com/someWebpageItWasFound.txt 
   ;    return

   ; If finished searching ; Example: Already finished searching for the 25 texts.
         ControlSend,, ^w, ahk_exe msedge.exe ; to close the current tab and go to the next tab that is opened.
         return
   ; else return
}
return

I would like to implement this FindText. If anyone is willing to share their talent and assistance, it would be greatly super duper appreciated!!!!

Descolada
Posts: 1099
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 07 Apr 2022, 05:55

shnxx wrote:
07 Apr 2022, 04:56
Hi!

I found this amazing tool and based on the description, it is what I needed. I'm currently doing some hella tasks right now and it is taking a lot of time even though it is easy. What I'm currently doing is that I have this notepad and these strings/texts are listed on there, I just copy one of them then go to the browser, ctrl+F then paste to check if this text exists. If not, then I'll copy now the 2nd string/text from the list, and repeat. I need to do it in 372 files and I have 25 texts to search on each webpage.

For example, assume I have 50 tabs and need to find these 25 strings in each one. And if one of these strings is discovered, I'd like to save it on another notepad and include a link to the webpage where it was discovered. Then it will close the tab (perhaps by pressing CTRL+W), which will switch to another tab, and begin searching again.

Here's what I currently have.

Code: Select all


*^F1:: ; CTRL+F1
Loop 
{
    FileReadLine, line, C:\Users\SHNX\Desktop\strings.txt, %A_Index% 
    if ErrorLevel
        break
    
    ControlSend,, ^f, ahk_exe msedge.exe
    ControlSend,, %line%{Enter}, ahk_exe msedge.exe
    
   ; I need something here that will trigger if the string(s) were found in the webpage
   ; if %line% is found
   ;    Save this %line% to a notepad with the webpage where it was discovered. ; Example: 12-34568-901 - https://someWebsite.com/someWebpageItWasFound.txt 
   ;    return

   ; If finished searching ; Example: Already finished searching for the 25 texts.
         ControlSend,, ^w, ahk_exe msedge.exe ; to close the current tab and go to the next tab that is opened.
         return
   ; else return
}
return

I would like to implement this FindText. If anyone is willing to share their talent and assistance, it would be greatly super duper appreciated!!!!
Hello,
When Ctrl+F does not find your string then it will display "0/0". This means you can do ImageSearch or FindText to look for that image, and if it is not found then it means your string was found.

Code: Select all

WinGetPos, winX, winY, winW, winH, ahk_exe msedge.exe ; Get browser location and size to use FindText only in the browser, not the whole screen. You could further narrow down to only the search box for added speed benefits.
if (!FindText(X, Y, winX, winY, winX+winW, winY+winH, 0, 0, "|<>*190$21.0000000000000071C1A/88WF34G8MaF34W88YF1B3878C02000004"))
{
  ; String was found, do something here.
}
Also if you have all your tabs open and you know exactly how many tabs you have, then you can send Ctrl+1 to go to the first tab, do your search, then send Ctrl+Tab to switch to the next one (without closing the previous one).

shnxx
Posts: 6
Joined: 01 Apr 2022, 07:27

Re: FindText - Capture screen image into text and then find it

Post by shnxx » 07 Apr 2022, 06:16

Descolada wrote:
07 Apr 2022, 05:55
Hello,
When Ctrl+F does not find your string then it will display "0/0". This means you can do ImageSearch or FindText to look for that image, and if it is not found then it means your string was found.

Code: Select all

WinGetPos, winX, winY, winW, winH, ahk_exe msedge.exe ; Get browser location and size to use FindText only in the browser, not the whole screen. You could further narrow down to only the search box for added speed benefits.
if (!FindText(X, Y, winX, winY, winX+winW, winY+winH, 0, 0, "|<>*190$21.0000000000000071C1A/88WF34G8MaF34W88YF1B3878C02000004"))
{
  ; String was found, do something here.
}
Also if you have all your tabs open and you know exactly how many tabs you have, then you can send Ctrl+1 to go to the first tab, do your search, then send Ctrl+Tab to switch to the next one (without closing the previous one).
Hi @Descolada! Thank you for your response! I tried getting the area of the "0/0" in the search box using AHK's Window Spy and I got this. 180, 13; 180, 26; and add 20 on both Xs to form a rectangle that covers the "0/0". Also, where should I put the "0/0"? Lastly, what does this ("|<>*190$21.0000000000000071C1A/88WF34G8MaF34W88YF1B3878C02000004") long string means?

Thank you very much for your response!

Btw, here's my current code:

Code: Select all

*^F1:: ; CTRL+F1
Loop   ;set the loop to however many lines there is. Will read the text line by line until its over.
{
    FileReadLine, line, C:\Users\SHNX\Desktop\strings.txt, %A_Index% 
    if ErrorLevel
        break
    
    ControlSend,, ^f, ahk_exe msedge.exe
    ControlSend,, %line%{Enter}, ahk_exe msedge.exe

   WinGetPos, winX, winY, winW, winH, ahk_exe msedge.exe ; Get browser location and size to use FindText only in the browser, not the whole screen. You could further narrow down to only the search box for added speed benefits.
   if (!FindText(X, Y, winX, winY, winX+winW, winY+winH, 0, 0, "|<>*190$21.0000000000000071C1A/88WF34G8MaF34W88YF1B3878C02000004"))
   {
      ; Save this %line% to a notepad with the webpage where it was discovered. ; Example: "12-34568-901 - https://someWebsite.com/someWebpageItWasFound.txt" Inside a notepad
      return
   }

   ; If finished searching ; Example: Already finished searching for the 25 texts.
         ControlSend,, ^w, ahk_exe msedge.exe ; to close the current tab and go to the next tab that is opened.
         return
   ; else return

    
}
return

Descolada
Posts: 1099
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 07 Apr 2022, 06:36

shnxx wrote:
07 Apr 2022, 06:16
Hi @Descolada! Thank you for your response! I tried getting the area of the "0/0" in the search box using AHK's Window Spy and I got this. 180, 13; 180, 26; and add 20 on both Xs to form a rectangle that covers the "0/0". Also, where should I put the "0/0" and my variable %list%? Lastly, what does this ("|<>*190$21.0000000000000071C1A/88WF34G8MaF34W88YF1B3878C02000004") long string means?

Thank you very much for your response!
Hello,
I recommend you read a bit more about how FindText works, otherwise this will turn into a very long conversation. But to answer shortly:
1) "the long string" is the "0/0" image taken with the FindText GUI in my computer from Edge and turned into a string representation, which FindText uses to look for it on the screen.
2) The location you got from Window Spy is relative to the window, but FindText uses absolute coordinates, so you would also need to take into consideration the position of the window (in my example, using winX and winY variables).
3) I do not understand the "variable %list%" question. If you use ControlSend to send "%line%{Enter}", then look for "0/0" with FindText, then you don't need to use %list% any more. But you do need to fetch the URL somehow. Maybe if FindText did not find "0/0", then you send Esc to close the searchbox, Alt+D to select the URL, then Ctrl+C and get the URL from clipboard.

shnxx
Posts: 6
Joined: 01 Apr 2022, 07:27

Re: FindText - Capture screen image into text and then find it

Post by shnxx » 07 Apr 2022, 06:48

Descolada wrote:
07 Apr 2022, 06:36
shnxx wrote:
07 Apr 2022, 06:16
Hi @Descolada! Thank you for your response! I tried getting the area of the "0/0" in the search box using AHK's Window Spy and I got this. 180, 13; 180, 26; and add 20 on both Xs to form a rectangle that covers the "0/0". Also, where should I put the "0/0" and my variable %list%? Lastly, what does this ("|<>*190$21.0000000000000071C1A/88WF34G8MaF34W88YF1B3878C02000004") long string means?

Thank you very much for your response!
Hello,
I recommend you read a bit more about how FindText works, otherwise this will turn into a very long conversation. But to answer shortly:
1) "the long string" is the "0/0" image taken with the FindText GUI in my computer from Edge and turned into a string representation, which FindText uses to look for it on the screen.
2) The location you got from Window Spy is relative to the window, but FindText uses absolute coordinates, so you would also need to take into consideration the position of the window (in my example, using winX and winY variables).
3) I do not understand the "variable %list%" question. If you use ControlSend to send "%line%{Enter}", then look for "0/0" with FindText, then you don't need to use %list% any more. But you do need to fetch the URL somehow. Maybe if FindText did not find "0/0", then you send Esc to close the searchbox, Alt+D to select the URL, then Ctrl+C and get the URL from clipboard.

Thank you again for your quick response!!!
I'll be reading more about how it works, but your brief explanation already gave me an idea. You really made a big help! Thank you so much!!!

Regarding your second point, I am using my msedge.exe in fullscreen when I run the script so will it still matter?

Descolada
Posts: 1099
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 07 Apr 2022, 09:24

shnxx wrote:
07 Apr 2022, 06:48
Regarding your second point, I am using my msedge.exe in fullscreen when I run the script so will it still matter?
It might matter, because a full-screen window coordinates might be negative. For example in my computer when Edge is maximized, then its coordinates are X -11, Y -11. Thus the search coordinates should be adjusted accordingly.

shnxx
Posts: 6
Joined: 01 Apr 2022, 07:27

Re: FindText - Capture screen image into text and then find it

Post by shnxx » 07 Apr 2022, 09:56

Descolada wrote:
07 Apr 2022, 09:24
shnxx wrote:
07 Apr 2022, 06:48
Regarding your second point, I am using my msedge.exe in fullscreen when I run the script so will it still matter?
It might matter, because a full-screen window coordinates might be negative. For example in my computer when Edge is maximized, then its coordinates are X -11, Y -11. Thus the search coordinates should be adjusted accordingly.
Oh, my apologies, it was not fullscreen, but only maximized. Sorry for any confusion. So does it still matter if msedge.exe is maximized?

Thank you again @Descolada!

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: FindText - Capture screen image into text and then find it

Post by SteveMylo » 08 Apr 2022, 20:40

shnxx wrote:
07 Apr 2022, 09:56
does it still matter if msedge.exe is maximized?
you should be able to easily do that that yourself. Open up window spy & take Look
Last edited by SteveMylo on 21 Apr 2022, 15:59, edited 1 time in total.

Descolada
Posts: 1099
Joined: 23 Dec 2021, 02:30

Re: FindText - Capture screen image into text and then find it

Post by Descolada » 21 Apr 2022, 12:10

Hello,
I noticed that when looping FindText or using FindText with "wait", the cursor will sometimes blink or flicker, and it is worse when the CPU/GPU is under load. It seems that this is caused by the internal GetBitsFromScreen function, specifically the DllCall for BitBlt with the CAPTUREBLT flag enabled. According to this thread CAPTUREBLT isn't necessary in BWM/Aero mode (so almost anything newer than Windows XP). I propose that a check for DWM composition be included and remove CAPTUREBLT accordingly.

This could be achieved by adding a little bit to the first line of Class FindTextClass:

Code: Select all

static bind:=[], bits:=[], Lib:=[], Cursor:=0, DwmIsCompositionEnabled:=DllCall("Dwmapi\DwmIsCompositionEnabled", "Int*", compositionEnabled), BitBltFlags:=compositionEnabled ? 0xCC0020 : 0xCC0020|0x40000000
and then replacing all instances of "0xCC0020|0x40000000" with "this.BitBltFlags".

I tested it on my Windows 10 setup and it eliminated the flicker while still capturing WS_EX_LAYERED windows.

Post Reply

Return to “Scripts and Functions (v1)”