FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
palendin
Posts: 2
Joined: 15 Apr 2020, 21:28

Re: FindText - Capture screen image into text and then find it

13 Jun 2020, 14:12

hi Feiyue,

I am already failing the first step. I saved the file as FindText and ran the script. It gave me a local variable error line text: local, error: line does not contain recognized action.

Could you help?

Thanks!
gregster
Posts: 4646
Joined: 30 Sep 2013, 06:48

Re: FindText - Capture screen image into text and then find it

14 Jun 2020, 05:59

palendin wrote:
13 Jun 2020, 14:12
It gave me a local variable error line text: local, error: line does not contain recognized action.
Outdated AHK version?
andy5566888
Posts: 10
Joined: 07 May 2020, 02:20

Re: FindText - Capture screen image into text and then find it

14 Jun 2020, 20:41

anyone can help me? i use the bindwindow will see the black picture and can not findthetext my method

1. click capture and choose a local area
2.and will get the gray2Two the page
3.click BindWindow button
4.and choose the step 1 local area
5.will see the black picture

and if i use the Manual input code like

Code: Select all

id:=WinExist("新分頁 - Google Chrome ahk_class Chrome_WidgetWin_1")
FindText_BindWindow(id)  ; Unbind Window using FindText_BindWindow(0)

; #Include <FindText>

 t1:=A_TickCount, X:=Y:=""

Text:="|<>*198$71.00000000000000000000000000000000000000000000000000000000000EEA000AzskA07ss0U0F0FUM1l381U0WaX0k02A81U2Jd69U0Tzw10B9GzH0Ae0070STwMbkFxsUC08a8lA0WeJ0K0p4FWM05Ie1Y1TzX4k0DjI2A7R169U3JGcA80e2DH068ZEk8BLoQa0DzuX0MvU/1A0kWFA0NN0ETzU17gk0C27U00000000000000000000000000000000000000000000000000000000000001"

 if (ok:=FindText(385-150000, 496-150000, 385+150000, 496+150000, 0, 0, Text))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   ; Click, %X%, %Y%
 }
and will need the page on the first then can be find not like the bindwindow can Behind find
User avatar
Chunjee
Posts: 605
Joined: 18 Apr 2014, 19:05
GitHub: Chunjee

Re: FindText - Capture screen image into text and then find it

19 Jun 2020, 11:55

I'm confused about the JoinText parameter. The in-code documentation says ; JoinText --> if the value is 1, Join all Text for combination lookup

Ok fine, the default for this parameter is 0 therefore passing two "Text" queries should FAIL because it by default will NOT combine them?


Actual result: Passing two parameters with the default setting finds and returns both.

Code: Select all

; variables
pizzaGraphic := "|<pizza>*150$45.D1zw01z1sTzU0TsD3zw03z1wTz00TsDzk00zs1zy0Dzy0Dzk1zzk1zy0Dzy0zzk3zy07zy0TzU0zzk3zw07zy0TzU000Dzs00001zy000U"
spaggGraphic := "|<spagg>*150$43.znzzzzzzkzzzzzzsTzzzzzyTzzzzvzzzzzzVzzzzzzkzzzzzzsTzzzzzw3zzzzzzszzzzzzwTzzzzzyTzzzzzzU"
twoqueries := pizzaGraphic spaggGraphic
; => "|<pizza>*150$45.D1zw01z1sTzU0TsD3zw03z1wTz00TsDzk00zs1zy0Dzy0Dzk1zzk1zy0Dzy0zzk3zy07zy0TzU0zzk3zw07zy0TzU000Dzs00001zy000U|<spagg>*150$43.znzzzzzzkzzzzzzsTzzzzzyTzzzzvzzzzzzVzzzzzzkzzzzzzsTzzzzzw3zzzzzzszzzzzzwTzzzzzyTzzzzzzU"

resultlegacyfindtext := FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, twoqueries)
; =>[{"1":1219,"2":421,"3":45,"4":14,"id":"pizza","x":1241,"y":428},{"1":1461,"2":829,"3":45,"4":14,"id":"pizza","x":1483,"y":836},{"1":857,"2":905,"3":45,"4":14,"id":"pizza","x":879,"y":912},{"1":1569,"2":963,"3":45,"4":14,"id":"pizza","x":1591,"y":970},{"1":1851,"2":1275,"3":45,"4":14,"id":"pizza","x":1873,"y":1282},{"1":1808,"2":571,"3":43,"4":12,"id":"spagg","x":1829,"y":577},{"1":1368,"2":967,"3":43,"4":12,"id":"spagg","x":1389,"y":973}]

btw putting a 1 in that parameter WILL make that search fail

Code: Select all

FindText(0, 0, A_ScreenWidth, A_ScreenHeight, 0, 0, twoqueries, 1, 1, 1)
; => 0
Can anyone clarify what a "combination lookup" is? Regardless of the parameter FindText will always split the primary input on line 1407 Loop, Parse, text, |
feiyue
Posts: 226
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

19 Jun 2020, 19:52

Combination search is to use some individual numbers to splice into a string of ID numbers,
or some individual letters to splice into a string of words. The offset of the height of each
letter is not big, and the interval between each letter is not big, which is similar to
generating a dynamic image to find. However, in order to improve the search speed,
only the first letter search mode is used as the combination search mode.
Therefore, when generating a single word library for combination search,
a unified mode must be used, such as "**50". :beer:
Last edited by feiyue on 19 Jun 2020, 21:16, edited 1 time in total.
User avatar
Chunjee
Posts: 605
Joined: 18 Apr 2014, 19:05
GitHub: Chunjee

Re: FindText - Capture screen image into text and then find it

19 Jun 2020, 20:53

Can you post an example I tried to use "|<!!!>**150$71.000000000..." but that is not immediately understandable to me.

Sounds like it is used to join two or more graphics into a longer dynamic "text" search. They have to be close to each other or something like searching for consecutive letters in a word.
feiyue
Posts: 226
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

19 Jun 2020, 21:40

This is to find specific text directly:

Code: Select all

; #Include <FindText>

 t1:=A_TickCount, X:=Y:=""

Text:="|<150$71 00>*160$68.00200000000DXVlw80sQC631gU360PBamFkF80WU4G95WjYH08814WlEe968420FcwI+2F1V0U4G/52UYE8k814WFMe9g6826PBYmHQCD20VXVksM000U00000002"

 if (ok:=FindText(501-150000, 726-150000, 501+150000, 726+150000, 0, 0, Text))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   ; Click, %X%, %Y%
 }

 MsgBox, 4096, Tip, % "Found :`t" Round(ok.MaxIndex())
   . "`n`nTime  :`t" (A_TickCount-t1) " ms"
   . "`n`nPos   :`t" X ", " Y
   . "`n`nResult:`t" (ok ? "Success !" : "Failed !")

 for i,v in ok
   if (i<=2)
     FindText_MouseTip(ok[i].x, ok[i].y)

This is a combination of find specific text:

Code: Select all

; #Include <FindText>

 t1:=A_TickCount, X:=Y:=""

Text:="|<1>*159$3./d99A"

Text.="|<5>*159$5.zX7sV35o"

Text.="|<0>*159$6.QqWWXWWqQU"

Text.="|<$>*159$5.8u4A464TWE"

Text.="|<7>*159$5.yAEW4MV4"

Text.="|<1>*159$3./d99A"

Text.="|<.>*159$6.00000000AA0U"

Text.="|<0>*159$6.QqWWXWWqQU"

Text.="|<0>*159$6.QqWWXWWqQU"

Text.="|<0>*159$6.CPFlllFHCU"

Text.="|<0>*159$6.AGlVVVlGAU"


 if (ok:=FindText(501-150000, 726-150000, 501+150000, 726+150000, 0, 0, Text,1,1,1))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   ; Click, %X%, %Y%
 }

 MsgBox, 4096, Tip, % "Found :`t" Round(ok.MaxIndex())
   . "`n`nTime  :`t" (A_TickCount-t1) " ms"
   . "`n`nPos   :`t" X ", " Y
   . "`n`nResult:`t" (ok ? "Success !" : "Failed !")

 for i,v in ok
   if (i<=2)
     FindText_MouseTip(ok[i].x, ok[i].y)

This is to use the word library to combine and find specific text:

Code: Select all

; #Include <FindText>

 t1:=A_TickCount, X:=Y:=""

Text:="|<1>*159$3./d99A"

Text.="|<5>*159$5.zX7sV35o"

Text.="|<0>*159$5.RiAMlXho"

Text.="|<$>*159$5.8u4A464TWE"

Text.="|<7>*159$5.yAEW4MV4"

Text.="|<.>*159$6.00000000AA0U"

FindText_PicLib(Text, add_to_Lib:=1)   ; add Text to word library

find:=FindText_PicLib("1|5|0|$|7|1|.|0|0")  ; get Text from word library

find:=FindText_PicN("150$71.00")  ; get Text from word library

 if (ok:=FindText(501-150000, 726-150000, 501+150000, 726+150000, 0, 0, find,1,1,1))
 {
   CoordMode, Mouse
   X:=ok.1.x, Y:=ok.1.y, Comment:=ok.1.id
   ; Click, %X%, %Y%
 }

 MsgBox, 4096, Tip, % "Found :`t" Round(ok.MaxIndex())
   . "`n`nTime  :`t" (A_TickCount-t1) " ms"
   . "`n`nPos   :`t" X ", " Y
   . "`n`nResult:`t" (ok ? "Success !" : "Failed !")

 for i,v in ok
   if (i<=2)
     FindText_MouseTip(ok[i].x, ok[i].y)
Last edited by feiyue on 19 Jun 2020, 22:37, edited 2 times in total.
andy5566888
Posts: 10
Joined: 07 May 2020, 02:20

Re: FindText - Capture screen image into text and then find it

19 Jun 2020, 22:14

hi feiyue can you help me the bindwindow question
feiyue
Posts: 226
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

19 Jun 2020, 22:57

It's not very reliable to look for pictures in the background.
Many windows can't get images in the background,
which is why I haven't added this function for a long time.
(in WinXP era, "PrintWindow()" can find pictures in the background,
But it's not good to use now)

Because many people ask, and I find that the success rate of using
"GetDcEx()" to find the background window image is not bad,
so I added this function. After binding, the background window
image will be projected to the foreground.

But it's still up to luck whether we can look for pictures in the background.
Good luck! :xmas: :beer:
andy5566888
Posts: 10
Joined: 07 May 2020, 02:20

Re: FindText - Capture screen image into text and then find it

19 Jun 2020, 23:07

i use gdipbitfromhwnd and can use imagesearch but some text i want to use your findtext because is look like good , and high Accuracy but your findtext is use bitmapfromscreen i don't know how to change this part
feiyue
Posts: 226
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

19 Jun 2020, 23:54

If the background image cannot be found using "GetDCEx",
But use "PrintWindow" to find the background image,
You can modify it as follows:

Code: Select all

    if (id) and !(w<1 or h<1)
    {
/*
      hDC2:=DllCall("GetDCEx", Ptr,id, Ptr,0, "int",3, Ptr)
      DllCall("BitBlt",Ptr,mDC,"int",x-zx,"int",y-zy,"int",w,"int",h
        , Ptr,hDC2, "int",x-wx, "int",y-wy, "uint",0x00CC0020|0x40000000)
      DllCall("ReleaseDC", Ptr,id, Ptr,hDC2)
*/
      VarSetCapacity(bi, 40, 0), NumPut(40, bi, 0, "int")
      NumPut(ww, bi, 4, "int"), NumPut(-wh, bi, 8, "int")
      NumPut(1, bi, 12, "short"), NumPut(32, bi, 14, "short")
      hBM2:=DllCall("CreateDIBSection", Ptr,0, Ptr,&bi
      , "int",0, Ptr "*",0, Ptr,0, "int",0, Ptr)
      hDC2:=DllCall("CreateCompatibleDC", Ptr,0, Ptr)
      oBM2:=DllCall("SelectObject", Ptr,hDC2, Ptr,hBM2, Ptr)
      ;---------------------
      DllCall("PrintWindow", Ptr,id, Ptr,hDC2, "uint",0)
      ;---------------------
      DllCall("BitBlt",Ptr,mDC,"int",x-zx,"int",y-zy,"int",w,"int",h
      , Ptr,hDC2, "int",x-wx, "int",y-wy, "uint",0x00CC0020|0x40000000)
      DllCall("SelectObject", Ptr,hDC2, Ptr,oBM2)
      DllCall("DeleteObject", Ptr,hBM2)
      DllCall("DeleteDC", Ptr,hDC2)
    }
andy5566888
Posts: 10
Joined: 07 May 2020, 02:20

Re: FindText - Capture screen image into text and then find it

20 Jun 2020, 01:08

oh thx feiyue the solve my question very thanks you
songdg
Posts: 114
Joined: 04 Oct 2017, 20:04

Re: FindText - Capture screen image into text and then find it

27 Jun 2020, 21:51

I run the latest version of the script and an error occur saying "Error at line 71.This line does not contain a recognized action."
gregster
Posts: 4646
Joined: 30 Sep 2013, 06:48

Re: FindText - Capture screen image into text and then find it

27 Jun 2020, 23:44

songdg wrote:
27 Jun 2020, 21:51
I run the latest version of the script and an error occur saying "Error at line 71.This line does not contain a recognized action."
What is the line's content? The error message should say so... Edit: It's looks like local here.
Then you are probably running an AHK version older than 1.1.27 which is not compatible (see force-local mode).
songdg
Posts: 114
Joined: 04 Oct 2017, 20:04

Re: FindText - Capture screen image into text and then find it

28 Jun 2020, 04:40

gregster wrote:
27 Jun 2020, 23:44
songdg wrote:
27 Jun 2020, 21:51
I run the latest version of the script and an error occur saying "Error at line 71.This line does not contain a recognized action."
What is the line's content? The error message should say so... Edit: It's looks like local here.
Then you are probably running an AHK version older than 1.1.27 which is not compatible (see force-local mode).
Thank you very much!
c7aesa7r
Posts: 43
Joined: 02 Jun 2016, 21:09

Re: FindText - Capture screen image into text and then find it

05 Jul 2020, 14:50

hi @feiyue theres any option to convert a image to text directly?
i mean instead clicking on "capture" you do something like open>myimage.png and it return the image text code

i have tried convert a image to binarie then convert to b64 but i got different result than your code
quietdolphin900
Posts: 2
Joined: 08 Jul 2020, 04:43

Re: FindText - Capture screen image into text and then find it

11 Jul 2020, 19:04

Hi I am a mega newbie at this stuff.

Is there any way to get the text of the comment to show what it found an match of?

Sorry if this is a dumb question.
c7aesa7r
Posts: 43
Joined: 02 Jun 2016, 21:09

Re: FindText - Capture screen image into text and then find it

16 Jul 2020, 05:54

do someone know if is possible to search inside a pbitmap instead searching for the text on screen?

i tried edit here, but did not work :cry:

Code: Select all

    if (id) and !(w<1 or h<1)
    {
      ;hDC2:=DllCall("GetDCEx", Ptr,id, Ptr,0, "int",3, Ptr)
      ;hDC2	    := Gdip_BitmapFromHWND(id)
      hDC2 := Gdip_CreateBitmapFromFile("C:\Users\User\Documents\x.png") ;
      
      DllCall("BitBlt",Ptr,mDC,"int",x-zx,"int",y-zy,"int",w,"int",h
        , Ptr,hDC2, "int",x-wx, "int",y-wy, "uint",0x00CC0020|0x40000000)
      DllCall("ReleaseDC", Ptr,id, Ptr,hDC2)
    }
at the end of this code it do a

Code: Select all

return bits
I read the values inside bits I got it:

Code: Select all

i1 bits 91881472 
i2 bits 10240 
i3 bits 0 
i4 bits 0 
i5 bits 2560 
i6 bits 1080 
i3 i4 i5 i6 seems to be my screen resolution, starting x0 y0 w2560 h 1080, but I have no idea about values from i1 and i2

If I do

Code: Select all

pBitmap := GdiBitmapFromScreen(0)
And print pBitmap, I have the pbitmap value of that screenshot
Where in findtext the code store the value from the screen being printed atm? If I could get this value I can do something similar, capture the screen using findtext function and store the value of the image.

I thought this was stored in bits (i1) but seems not :eh:
User avatar
Chunjee
Posts: 605
Joined: 18 Apr 2014, 19:05
GitHub: Chunjee

Re: FindText - Capture screen image into text and then find it

16 Jul 2020, 12:38

c7aesa7r wrote:
16 Jul 2020, 05:54
Where in findtext the code store the value from the screen being printed atm? If I could get this value I can do something similar, capture the screen using findtext function and store the value of the image.
I am also interested in this if anyone knows the answer. I'd like to save it for later use in certain circumstances.

quietdolphin900 wrote:
11 Jul 2020, 19:04
Hi I am a mega newbie at this stuff.

Is there any way to get the text of the comment to show what it found an match of?
The default return object is often a var named ok so you would msgbox ok[index].id where index is the number of the "text" found. If you only think one was found, this would be msgbox, % ok[1].id
This of course assumes that you have a comment/id attached to the search string.

Return to “Scripts and Functions”

Who is online

Users browsing this forum: No registered users and 20 guests