FindText - Capture screen image into text and then find it
Re: FindText - Capture screen image into text and then find it
@c7aesa7r You can do this:
1. Click the button: CaptureS, OpenDir, Cancel.
2. Copy the image to the open directory ( %A_Temp%\Ahk_ScreenShot ).
3. Click the button: CaptureS, (select your picture), OK.
4. Your image is displayed in the upper left corner of the screen, you can capture it.
1. Click the button: CaptureS, OpenDir, Cancel.
2. Copy the image to the open directory ( %A_Temp%\Ahk_ScreenShot ).
3. Click the button: CaptureS, (select your picture), OK.
4. Your image is displayed in the upper left corner of the screen, you can capture it.
Re: FindText - Capture screen image into text and then find it
Since saved image is displayed exactly in the upper left corner, it has one additional positive side effect i.e. capturing position is correct even after screen is being magnified. Thank you feiyue!
Re: FindText - Capture screen image into text and then find it
How does one tell the function to click a certain occurrence of the found text?
I wrote this function and here's the use case
The use of function here
This works
This doesn't work
I wrote this function and here's the use case
The use of function here
Code: Select all
ftc(Text:="|<>*159$23.0zw0TzzUzzz1zzy7y7yDs7wTU7sz0Dly0Tbw0zjw3zTwDyzzzxzzzvzzzs0000000E",,,,"146, 79, 922, 569","Y",1)
Code: Select all
ftc(text:="",offX:=0,offY:=0,LR:="Left",range:="780, 272, 1870, 1008",show:="N",occurrence:=1) {
rangeA:=[]
rangeA:=StrSplit(range, ", ")
While !ok:=FindText(rangeA[1],rangeA[2],rangeA[3],range[A], 0, 0, Text)
sleep, 100
If occurrence=1
MouseClick, Left, % ok.1.x+offX, % ok.1.y+offY
If occurrence=2
MouseClick, Left, % ok.2.x+offX, % ok.2.y+offY
If (show="Y") {
for i,v in ok
if (i<=2)
FindText.MouseTip(ok[i].x+offX, ok[i].y+offY)
}
}
Code: Select all
ftc(text:="",offX:=0,offY:=0,LR:="Left",range:="780, 272, 1870, 1008",show:="N",occurrence:=1) {
rangeA:=[]
rangeA:=StrSplit(range, ", ")
While !ok:=FindText(rangeA[1],rangeA[2],rangeA[3],range[A], 0, 0, Text)
sleep, 100
MouseClick, Left, % ok.occurrence.x+offX, % ok.occurrence.y+offY
If (show="Y") {
for i,v in ok
if (i<=2)
FindText.MouseTip(ok[i].x+offX, ok[i].y+offY)
}
}
Re: FindText - Capture screen image into text and then find it
I solved it lel
Code: Select all
ftc(text:="",offX:=0,offY:=0,LR:="Left",range:="780, 272, 1870, 1008",show:="N",occurrence:=1) {
rangeA:=[]
rangeA:=StrSplit(range, ", ")
While !ok:=FindText(rangeA[1],rangeA[2],rangeA[3],range[A], 0, 0, Text)
sleep, 100
MouseClick, Left, % ok[occurrence].x+offX, % ok[occurrence].y+offY
If (show="Y") {
for i,v in ok
if (i<=10)
FindText.MouseTip(ok[i].x+offX, ok[i].y+offY)
}
}
Re: FindText - Capture screen image into text and then find it
@feiyue
If i have various text with the same comment for example:
And I have set:
As the last parameter is 0, it means it will return at the first match.
I know I can see the comment matched inside:
But as all comment have the same name it would not help to know which one was caught, is possible somehow to differentiate it? Maybe the array index of the text caught or ?
If i have various text with the same comment for example:
Code: Select all
text := "|<1>*.......
text .="|<1>*.......
text .="|<1>*.......
text .="|<1>*.......
text .="|<1>*.......
text .="|<1>*.......
Code: Select all
if (ok:=findtext(x, y, x2 , y2,0.1,0.1, text,1,0)) {
}
I know I can see the comment matched inside:
Code: Select all
comment :=ok.1.id
Re: FindText - Capture screen image into text and then find it
Sorry I dont really understand about what you're saying. Hi Feiyue, i try to OCR this bmp but not work (link: https drive.google.com /file/d/1tbtUisFVqi7h-__brb1qJz1BYmrIH0Pm/view?usp=sharing). Broken Link for safetyfeiyue wrote: ↑01 Apr 2021, 16:18Updated to 8.4 version - 2021/04/02
.... 1. Improvement: in multicolor mode,You can find multiple colors and adapt to different resolutions.
....... Now it also supports searching images, for example: Text:="|<>##10$ c:\a.bmp"
....... It's better to limit the search range to a small one.
.... 2. Modify: the return value of OCR identification function is changed.
....... eg: MsgBox, % FindText.Ocr(ok).ocr --> Change to: MsgBox, % FindText.Ocr(ok).text
.... 3. Add: FindText.QPC() and FindText.SetColor() is added.
.... 4. Modify: The search direction (dir = 1-9), dir = 9: From the Center to all around.
The aim is want to OCR multi line and get the value 1 as as variable1 secondline as a variable 2, etc
with this code but not work: is that something wrong?
Code: Select all
#NoEnv
CoordMode, Mouse ; Required: change coord mode to screen vs relative.
F2::
; using findtext8.1 status not working
; This use the monitor size 1280x1028 (show only on monitor 1).
;universal (grey or white background)
;ini minusnya sama area putih supaya nggak cari ke atas menu
;~ Text:="|<->*150$13.0000000000000000000000000C000000000000000E"
;~ Text:="|<1>*132$4./W8W8WU"
;~ Text:="|<2>*143$6.S2226AEEzU"
;~ Text:="|<3>*121$5.s8Ea24/Y"
;~ Text:="|<4>*134$6.66+GGz22U"
;~ Text:="|<5>*131$5.R27V34/Y"
;~ Text:="|<6>*136$6.CMESHFFGCU"
;~ Text:="|<7>*132$5.y4EW40V4"
;~ Text:="|<8>*131$6.CGGGAGFHCU"
;~ ;GrayDiff 50 baru bisa universal
;~ Text:="|<9>**50$6.SHnnnT36QU"
;~ Text:="|<0>**50$6.SHnnnnHHSU"
;~ Text:="|<.>**50$3.000003M04"
;===use this code also not works then trying on monitor 2 only.
;using findtext 8.5 also not working
; This use the monitor size 1280x1028 (show only on monitor 2) not using the left monitor.
Text:="|<->*121$10.00000000000000000000007U000000000000000002"
Text:="|<1>*171$6.6SK66666666U"
Text:="|<2>*134$7.D8k846266621z"
Text:="|<3>*137$8.D4E61UEs1U821bm"
Text:="|<4>*127$8.10kA52FYFDw410G"
Text:="|<5>*132$7.0000000T8421s610UIPk000000000E"
Text:="|<7>*135$7.z0UUEM844211V"
Text:="|<6>*128$8.7X0UE5lWEoB28Vm"
Text:="|<8>*135$8.0000000003kaEY8a7693Eq8w0000000008"
Text:="|<9>*128$8.D6N2EY9aCU8217W"
Text:="|<0>*133$8.D2N2EY93EY92NXm"
;using GrayDiff 50, and then only after that i can get the dot (.) of full stop, in the white and grey area.
Text:="|<.>**50$3.s0000000P0007U"
;MouseGetPos, x, y
t1:=A_TickCount
;------------------------------
;====1128,662, 1253,761
;ok:=FindText(x-150, y-20, x+150, y+20, 0.2, 0.1, Text)
ok:=FindText(1128, 662, 1253, 761, 0.2, 0.1, Text)
arr:=FindText.OCR(ok), OCR:=arr.text
;------------------------------
t1:=A_TickCount-t1
MsgBox, 4096, OCR, OCR Result: [%OCR%] in %t1% ms.
Return
below is findtext copied code.
OCR Result: [...] in 47 ms.
How to fix this, is that something wrong?
and the second question. How to make it showing in red box before searching, just to make sure the program not seeking at the wrong area?
(sorry for my bad english)
Re: FindText - Capture screen image into text and then find it
@feiyue i tried to do this modification but it does not work, maybe because you updated the function a lot? do you know how to edit to work with the newer version?feiyue wrote: ↑16 Jul 2020, 22:16I think you can replace screen data with image files like this, but I haven't tested:c7aesa7r wrote: ↑16 Jul 2020, 05:54do someone know if is possible to search inside a pbitmap instead searching for the text on screen?
i tried edit here, but did not workCode: Select all
if (id) and !(w<1 or h<1) { ;hDC2:=DllCall("GetDCEx", Ptr,id, Ptr,0, "int",3, Ptr) ;hDC2 := Gdip_BitmapFromHWND(id) hDC2 := Gdip_CreateBitmapFromFile("C:\Users\User\Documents\x.png") ; DllCall("BitBlt",Ptr,mDC,"int",x-zx,"int",y-zy,"int",w,"int",h , Ptr,hDC2, "int",x-wx, "int",y-wy, "uint",0x00CC0020|0x40000000) DllCall("ReleaseDC", Ptr,id, Ptr,hDC2) }
Code: Select all
/* if (hBM) and !(w<1 or h<1) { win:=DllCall("GetDesktopWindow", Ptr) hDC:=DllCall("GetWindowDC", Ptr,win, Ptr) mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr) oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,hBM, Ptr) DllCall("BitBlt",Ptr,mDC,"int",x-zx,"int",y-zy,"int",w,"int",h , Ptr,hDC, "int",x, "int",y, "uint",0x00CC0020|0x40000000) DllCall("ReleaseDC", Ptr,win, Ptr,hDC) if (id:=this.BindWindow(0,0,1)) WinGet, id, ID, ahk_id %id% if (id) { WinGetPos, wx, wy, ww, wh, ahk_id %id% left:=x, right:=x+w-1, up:=y, down:=y+h-1 left:=left<wx ? wx:left, right:=right>wx+ww-1 ? wx+ww-1:right up:=up<wy ? wy:up, down:=down>wy+wh-1 ? wy+wh-1:down x:=left, y:=up, w:=right-left+1, h:=down-up+1 } if (id) and !(w<1 or h<1) { hDC2:=DllCall("GetDCEx", Ptr,id, Ptr,0, "int",3, Ptr) DllCall("BitBlt",Ptr,mDC,"int",x-zx,"int",y-zy,"int",w,"int",h , Ptr,hDC2, "int",x-wx, "int",y-wy, "uint",0x00CC0020|0x40000000) DllCall("ReleaseDC", Ptr,id, Ptr,hDC2) } DllCall("SelectObject", Ptr,mDC, Ptr,oBM) DllCall("DeleteDC", Ptr,mDC) } */ ;----------------- if FileExist(file:="C:\Users\User\Documents\x.png") { pBitmap := Gdip_CreateBitmapFromFile(file) w:=Gdip_GetImageWidth(pBitmap), h:=Gdip_GetImageHeight(pBitmap) w:=w>zw ? zw:w, h:=h>zh ? zh:h hBM2:=Gdip_CreateHBITMAPFromBitmap(pBitmap), Gdip_DisposeImage(pBitmap) mDC:=DllCall("CreateCompatibleDC", Ptr,0, Ptr) oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,hBM, Ptr) mDC2:=DllCall("CreateCompatibleDC", Ptr,0, Ptr) oBM2:=DllCall("SelectObject", Ptr,mDC2, Ptr,hBM2, Ptr) DllCall("BitBlt",Ptr,mDC,"int",0,"int",0,"int",w,"int",h , Ptr,mDC2, "int",0, "int",0, "uint",0x00CC0020|0x40000000) DllCall("SelectObject", Ptr,mDC2, Ptr,oBM2) DllCall("DeleteDC", Ptr,mDC2) DllCall("SelectObject", Ptr,mDC, Ptr,oBM) DllCall("DeleteDC", Ptr,mDC) DllCall("DeleteObject", Ptr,hBM2) } ;-----------------
Re: FindText - Capture screen image into text and then find it
Comment is used to distinguish between the results.c7aesa7r wrote: ↑04 May 2021, 11:30But as all comment have the same name it would not help to know which one was caught, is possible somehow to differentiate it? Maybe the array index of the text caught or ?Code: Select all
text := "|<1>*....... text .="|<1>*....... text .="|<1>*....... text .="|<1>*....... text .="|<1>*....... text .="|<1>*.......
Code: Select all
text := "|<1.1>*.......
text .="|<1.2>*.......
text .="|<1.3>*.......
Re: FindText - Capture screen image into text and then find it
@arimania
1. It's a connection, not a replacement.
2. The decimal point should include the surrounding white space.
Or it can be judged by setting a smaller interval to make the result generate a * symbol.
3. The range of OCR can be obtained by clicking the "GetRange" button.
1. It's a connection, not a replacement.
Code: Select all
Text := "|<->*121$10.00000000000000000000007U000000000000000002"
Text .= "|<1>*171$6.6SK66666666U"
Text .= "|<2>*134$7.D8k846266621z"
Or it can be judged by setting a smaller interval to make the result generate a * symbol.
Code: Select all
Text:="|<.>**50$3.s0000000P0007U"
Re: FindText - Capture screen image into text and then find it
@kauan014 You can do it like this:
Code: Select all
#Include <FindText>
f1 := "d:\a.bmp"
f2 := "d:\b.bmp"
FindText.SavePic(f1, 0, 0, 0, 0, ScreenShot:=1) ; This line is not needed if the image already exists
FindText.ShowPic(f1, show:=0)
FindText.SavePic(f2, 0, 0, 100, 100, ScreenShot:=0) ; This line is not needed if the image already exists
Text:="|<>##0$" . f2
if (ok:=FindText(0, 0, 0, 0, 0, 0, Text, ScreenShot:=0))
{
CoordMode, Mouse
X:=ok.1.1, Y:=ok.1.2, Comment:=ok.1.id
MsgBox, Found Pos: %X%`, %Y%
}
else
MsgBox, Not Found !
return
Last edited by feiyue on 06 May 2021, 04:52, edited 1 time in total.
Re: FindText - Capture screen image into text and then find it
feiyue wrote: ↑05 May 2021, 07:19@kauan014 You can do it like this:Code: Select all
#Include <FindText> f1 := "d:\a.bmp" f2 := "d:\b.bmp" FindText.SavePic(f1, 0, 0, 0, 0, ScreenShot:=1) ; This line is not needed if the image already exists FindText.ShowPic(f1, show:=0) FindText.SavePic(f2, 0, 0, 100, 100, ScreenShot:=0) ; This line is not needed if the image already exists Text:="|<>##0$" . f2 if (ok:=FindText(0, 0, 0, 0, 0, 0, Text, ScreenShot:=0)) { CoordMode, Mouse X:=ok.1.1, Y:=ok.1.1, Comment:=ok.1.id MsgBox, Found Pos: %X%`, %Y% } else MsgBox, Not Found ! return
The text inside the variable text is the icon of VSCode
But the code does not find it, am I doing something wrong?
Last edited by kauan014 on 05 May 2021, 14:06, edited 2 times in total.
Re: FindText - Capture screen image into text and then find it
@kauan014 You don't understand the meaning of each line of code, you just need to:
Code: Select all
#Include <FindText>
Text:="|<>*77$......"
; Add this line to load the image into memory. Note that show:=0 and ScreenShot:=0
FindText.ShowPic("c:\b.bmp", show:=0)
if (ok:=FindText(0, 0, 0, 0, 0, 0, Text, ScreenShot:=0))
{
X:=ok.1.1, Y:=ok.1.2, Comment:=ok.1.id
MsgBox, Found Pos: %X%`, %Y%
}
else
MsgBox, Not Found !
return
Last edited by feiyue on 06 May 2021, 04:52, edited 1 time in total.
Re: FindText - Capture screen image into text and then find it
@feiyue thanks! now it finds the image.
In this case, the x1 y1 x2 y2 parameters are not relative to the image loaded or they are ignored and is searched inside the entire image?
In this case, the x1 y1 x2 y2 parameters are not relative to the image loaded or they are ignored and is searched inside the entire image?
Re: FindText - Capture screen image into text and then find it
@kauan014
Because all the coordinates of FindText are relative to the screen coordinates,
it is a bit troublesome to search for images instead of the screen.
The images are loaded into the upper left corner of the screen,
and the upper left corner of the screen is not necessarily located in (0,0).
You can refer to the following code: (Use the files that FindText updated today)
Because all the coordinates of FindText are relative to the screen coordinates,
it is a bit troublesome to search for images instead of the screen.
The images are loaded into the upper left corner of the screen,
and the upper left corner of the screen is not necessarily located in (0,0).
You can refer to the following code: (Use the files that FindText updated today)
Code: Select all
#Include <FindText>
Text:="|<>*77$......"
; Add this line to load the image into memory. Note that show:=0 and ScreenShot:=0
FindText.ShowPic("c:\b.bmp", show:=0, zx, zy, w, h)
; Search range of relative pictures: (x1,y1,x2,y2)
; if (ok:=FindText(x1+zx, y1+zy, x2+zx, y2+zy, 0, 0, Text, ScreenShot:=0))
; Search range of relative pictures: (0,0,w,h)
if (ok:=FindText(zx+0, zy+0, zx+w, zy+h, 0, 0, Text, ScreenShot:=0))
{
; Results of relative pictures:
X:=ok.1.1-zx, Y:=ok.1.2-zy, Comment:=ok.1.id
MsgBox, Found Pos: %X%`, %Y%
}
else
MsgBox, Not Found !
return
Re: FindText - Capture screen image into text and then find it
@feiyue Thanks amazing! I have tested it, working
I got some questions if you don't mind.
Does searching inside images compared to searching on the screen have less performance?
I mean:
1- does it use more CPU power compared to searching on screen in a loop with a very low interval?
2- does the time to find a match is higher compared to when searching on-screen.
I got some questions if you don't mind.
Does searching inside images compared to searching on the screen have less performance?
I mean:
1- does it use more CPU power compared to searching on screen in a loop with a very low interval?
2- does the time to find a match is higher compared to when searching on-screen.
Re: FindText - Capture screen image into text and then find it
A screen size memory image is reserved inside the FindText function, All searches are performed in this memory image.
When searching the screen, use BitBlt() to copy the screen content to this memory image,
When searching the disk image, use LoadPicture() to copy the disk image content to this memory image,
I don't know which copy process is faster, but the search speed of memory image is the same.
When searching the screen, use BitBlt() to copy the screen content to this memory image,
When searching the disk image, use LoadPicture() to copy the disk image content to this memory image,
I don't know which copy process is faster, but the search speed of memory image is the same.
Re: FindText - Capture screen image into text and then find it
@feiyue
Is possible to search for a combination of a specific text?
Example:
I captured just one pink pixel from the image above:
Then I would like to search inside a region if it has 10? of these pixels, combining 10x the text it would look like:
Theres such option?
Is possible to search for a combination of a specific text?
Example:
I captured just one pink pixel from the image above:
Code: Select all
text:="|<1>0xEA00FF@0.00$1.k" ;
Last edited by kauan014 on 06 May 2021, 23:14, edited 3 times in total.
Re: FindText - Capture screen image into text and then find it
@kauan014 Please look at the function description carefully and try it yourself.
Re: FindText - Capture screen image into text and then find it
@feiyue Trust me i have look inside the function the entire day
I discovered this thing of Lib, I thought it was what I'm looking for, but looks like it just combines a pre-saved text and search for each of it and not a combination of all of they.
Code: Select all
text:="|<1>0xEA00FF@0.00$1.k"
FindText.PicLib(text, add_to_Lib:=1)
text:= FindText.PicN("11111111111")
fileappend %text% `n,*
if (ok:=FindText(0, 0, 0, 0, 0,0, text,1,1))
{
fileappend found! `n,*
for i,v in ok
FindText.MouseTip(ok[i].x, ok[i].y)
}
Re: FindText - Capture screen image into text and then find it
look at the function description :JoinText