Currently captured screen image text, to be used on other computers, you must use the same:
1. screen resolution
2. browser magnification
3. Text font
4. smooth screen font edge
So basically, the image is required to be consistent (Allows for a certain tolerance).
Because the text lines are very thin, if you zoom in and out (even if it's very slight),
Text lines will stagger, causing great error,
Even if the use of tolerance is not easy to use.
so the current does not apply to zoom in or out.
The advantage of it over the built-in command ImageSearch is that :
it does not have to use an external image, just use the script, and find faster.
FindText - Capture screen image into text and then find it
Re: FindText - Catch screen image into text and then find it
00000000000000000000000000000000000000000000000000000000000000000000000
00000000______0__0000000000000000__0________00000000000000000_000000000
00000000__00000000000000000000000__0000__0000000000000000000__000000000
00000000__00000__00_0____0000____0_0000__00000____00__000_______0000000
00000000__00000__00__000__00__000__0000__0000__00__00__0__00__000000000
00000000__00000__00__0000_00_0000__0000__0000_0000_000_0_000__000000000
00000000______0__00_00000_00_0000__0000__0000______0000_0000__000000000
00000000__00000__00_00000_00_0000__0000__0000_00000000___000__000000000
00000000__00000__00_00000_00_0000__0000__0000_00000000_0_000__000000000
00000000__00000__00_00000_00__000__0000__0000__000000_00__00__000000000
00000000__00000__00_00000_000____0_0000__00000_____0__000__00___0000000
00000000000000000000000000000000000000000000000000000000000000000000000
hmm... interesting... *thinks of use-cases*
00000000______0__0000000000000000__0________00000000000000000_000000000
00000000__00000000000000000000000__0000__0000000000000000000__000000000
00000000__00000__00_0____0000____0_0000__00000____00__000_______0000000
00000000__00000__00__000__00__000__0000__0000__00__00__0__00__000000000
00000000__00000__00__0000_00_0000__0000__0000_0000_000_0_000__000000000
00000000______0__00_00000_00_0000__0000__0000______0000_0000__000000000
00000000__00000__00_00000_00_0000__0000__0000_00000000___000__000000000
00000000__00000__00_00000_00_0000__0000__0000_00000000_0_000__000000000
00000000__00000__00_00000_00__000__0000__0000__000000_00__00__000000000
00000000__00000__00_00000_000____0_0000__00000_____0__000__00___0000000
00000000000000000000000000000000000000000000000000000000000000000000000
hmm... interesting... *thinks of use-cases*
Re: FindText - Catch screen image into text and then find it
This is an amazing piece of code. I don't know how it works, but it is very fast and accurate.
I would recommend the opening post include this link to ed1chandlers explaination on how to use this code. https://autohotkey.com/boards/viewtopic ... ext#p87947
I don't think we can rely on everyone to read thru all of the replies to this thread to see e1chandles explaination.
I would never have been able to figure out how to use this code without a detailed explanation like ed1chandlers.
thank you feiyue for this great and useful code.
I would recommend the opening post include this link to ed1chandlers explaination on how to use this code. https://autohotkey.com/boards/viewtopic ... ext#p87947
I don't think we can rely on everyone to read thru all of the replies to this thread to see e1chandles explaination.
I would never have been able to figure out how to use this code without a detailed explanation like ed1chandlers.
thank you feiyue for this great and useful code.
Check out my scripts. (MyIpChanger) (ClipBoard Manager) (SavePictureAs)
All my scripts are tested on Windows 10, AutoHotkey 32 bit Ansi unless otherwise stated.
All my scripts are tested on Windows 10, AutoHotkey 32 bit Ansi unless otherwise stated.
- SpeedMaster
- Posts: 494
- Joined: 12 Nov 2016, 16:09
Re: FindText - Catch screen image into text and then find it
Hello,
You can also use it as a Hexadecimal Color Picker
You can also use it as a Hexadecimal Color Picker
Re: FindText - Catch screen image into text and then find it
I added an option for similarity to the color mode,
which can be used in the game for those background transparent text.
Note: when the similarity is less than 1, the tolerance
of the background character (_) should be a little larger,
in some cases it can even be set to 1.
which can be used in the game for those background transparent text.
Note: when the similarity is less than 1, the tolerance
of the background character (_) should be a little larger,
in some cases it can even be set to 1.
Re: FindText - Catch screen image into text and then find it
I used your function in script for searching ~8 (common color) patterns in common area at one moment.
So, most time get GetBitsFromScreen(). Are you considering divide function in parts?For auto release memory on exit from function and for ease of use reasons it can look like:Also I thinking about usage base64 for writing pattern.
Current form spectacularly and wow-effect , but script became too big .
So, most time get GetBitsFromScreen(). Are you considering divide function in parts?
Code: Select all
FindText(pattern) { ; simple one pattern usage
GetBitsFromScreen()
Find(pattern)
}
; multiple pattern usage
GetBitsFromScreen()
Find(pattern1)
Find(pattern2)
...
Find(patternN)
Code: Select all
FindText(pattern) ; simple
FindText([pattern1, pattern2, ..., patternN]) ; multiple, return array of matches
FindText(p) {
GetBitsFromScreen()
if IsObject(p) {
returnValue := []
for index, pattern in p
returnValue.Push(Find(pattern))
Return returnValue
} else
Return Find(p)
}
Current form spectacularly and wow-effect , but script became too big .
Re: FindText - Catch screen image into text and then find it
Your suggestion is very good, I updated script, realized your two needs.
Re: FindText - Catch screen image into text and then find it
Updated to accommodate version 5.1 (Code Cleanup by feiyue). Awesome script. Perfect for my needs of interacting with Citrix Windows. Here's my humble attempt to make a functional wrapper. The utility of this is for linking multiple FindText together. For example, You use Findtext to click a webpage link, then try to click a second link after page reload. If the code fails it is hard to determine if you tested too early or if there was a recognition problem. This function bakes in wait time, post find wait times, and verbose tooltip options.
Code: Select all
WaitFindText(x, y, w, h, err1, err0, Text, verbose := false, wait := 5, action := 1, postwait := 0, xoffset :=0, yoffset := 0,exitonfail := false)
{
; Ver 1.0 - Inital
; Ver 1.1 - update for FindText ver 5.0
; Ver 1.2 - updated 6/9/2017 - less pausing between loops
; Ver 1.3 updated 6/27/2017 - change click type (action=1) to be more robust
; Ver 1.4 updated 7/15/17 - added exitonfail to exit thread if fails
; Arguments 1-7 identical to output from FindText()
; Argument #8 verbose tooltip during search (true/false) default false
; Argument #9 wait - seconds to wait looking for searchtext, default 5 sec
; Argument #10 action : 0 - no action (useful for does it exist?), 1 click via mouse (default), 2 move to found, 3 Click silently (only works on active window)
; Argument #11 postwait - seconds to pause after found, default 0
; Argument #12 xoffest - from found center move x by ... default 0, useful for find label, and click field, etc
; Argument #13 yoffset - from found center move y by ... default 0, useful for find label, and click field, etc
; Argument #14 exitonfail - if function fails to find, then exits thread.
RegExMatch(Text, "<\K[^>]*(?=>)", comment)
start := A_TickCount
Loop {
elapsed := Floor((A_TickCount - start)/1000)
if wft := FindText(x,y,w,h,err1,err0,Text)
{
rx := wft.1 + (wft.3//2) + xoffset, ry := wft.2 + (wft.4//2) + yoffset
if action ; note 0 used to determining exists only
{
CoordMode, Mouse
if action = 1
{ ; note previous Click %rx% %ry% fails in some instances
MouseMove, %rx%, %ry%
Sleep, 20 ; helps to avoid missed clicks somehow
Send, {Click} ; could also use Send {vk01sc000}
}
if action = 2
MouseMove, %rx%, %ry%
if action = 3
{
if WinActive("ahk_class IEFrame") { ; ctrl-l focuses at URLbar - needed
Send, ^l
Sleep, 50
}
WinGetPos, x, y,,, A
x:=rx-x, y:=ry-y
ControlClick, x%x% y%y%, A
}
}
Sleep % (postwait * 1000)
break
}
If Verbose
Tooltip % comment . " Not Found in " . elapsed . " / " . wait . " secs"
If wait
Sleep, 100
} Until (elapsed >= wait)
If verbose
{
Tooltip % (wft) ? (wft.5 . " Found at X: " . rx . " Y: " . ry . " in " . elapsed . " / " . wait . " secs")
: (comment . " Not Found Final @ " . wait . " secs")
SetTimer, WFTKillTooltip, -5000
}
if not wft and exitonfail and action
Exit
return wft
WFTKillTooltip:
Tooltip
return
}
Last edited by c4p on 28 Jul 2017, 09:08, edited 22 times in total.
-
- Posts: 20
- Joined: 05 Jul 2015, 17:51
Re: FindText - Catch screen image into text and then find it
A program I am manipulating via AHK recently updated and controls are no longer used. Before I delve into trying to utilize this, can someone tell me if I can adapt it to click on icons on a toolbar based on text underneath the icon?
-
- Posts: 16
- Joined: 31 Mar 2017, 04:11
Re: FindText - Catch screen image into text and then find it
Hi, I'm a newbie who just built his first GUI. I'm really interested in this script and I'm sure once I attach it properly to what I've done I shouldn't have much problem adding the movement and clicking to the whole process. But I have a more basic problem of actually finding my text outside of the test function in the app.
This is what I have. It's purpose is to initiate mouseover when pressing ctrl+q:
Could you please steer me in the right direction with this, such as a working example? Thank you in advance.
This is what I have. It's purpose is to initiate mouseover when pressing ctrl+q:
Code: Select all
FindText(x,y,w,h,err1,err0,text)
{
xywh2xywh(x-w,y-h,2*w+1,2*h+1,x,y,w,h)
if (w<1 or h<1)
Return, 0
bch:=A_BatchLines
SetBatchLines, -1
;--------------------------------------
GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits)
;--------------------------------------
sx:=0, sy:=0, sw:=w, sh:=h
if (err1=0 and err0=0)
err1:=0.05, err0:=0.05
arr:=[]
Loop, Parse, text, |
{
v:=A_LoopField
IfNotInString, v, $, Continue
OCR:="", e1:=err1, e0:=err0
; You Can Add OCR Text In The <>
if RegExMatch(v,"<([^>]*)>",r)
v:=StrReplace(v,r), OCR:=Trim(r1)
; You can Add two fault-tolerant in the [], separated by commas
if RegExMatch(v,"\[([^\]]*)]",r)
{
v:=StrReplace(v,r), r2:=""
StringSplit, r, r1, `,
e1:=r1, e0:=r2
}
StringSplit, r, v, $
color:=r1, v:=r2
StringSplit, r, v, .
w1:=r1, v:=base64tobit(r2), h1:=StrLen(v)//w1
if (r0<2 or w1>sw or h1>sh or h1<1 or StrLen(v)!=w1*h1)
Continue
if PicFind(Scan0,Stride,sx,sy,sw,sh
,v,color,w1,h1,rx,ry,e1,e0)
{
rx+=x, ry+=y
arr.Push(rx,ry,w1,h1,OCR)
}
}
SetBatchLines, %bch%
Return, arr.MaxIndex() ? arr:0
}
xywh2xywh(x1,y1,w1,h1,ByRef x,ByRef y,ByRef w,ByRef h)
{
SysGet, zx, 76
SysGet, zy, 77
SysGet, zw, 78
SysGet, zh, 79
left:=x1, right:=x1+w1-1, up:=y1, down:=y1+h1-1
left:=left<zx ? zx:left, right:=right>zx+zw-1 ? zx+zw-1:right
up:=up<zy ? zy:up, down:=down>zy+zh-1 ? zy+zh-1:down
x:=left, y:=up, w:=right-left+1, h:=down-up+1
}
GetBitsFromScreen(x,y,w,h,ByRef Scan0,ByRef Stride,ByRef bits)
{
VarSetCapacity(bits, w*h*4, 0)
Ptr:=A_PtrSize ? "Ptr" : "UInt"
win:=DllCall("GetDesktopWindow", Ptr)
hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
hBM:=DllCall("CreateCompatibleBitmap", Ptr,hDC
, "int",w, "int",h, Ptr)
oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,hBM, Ptr)
DllCall("BitBlt", Ptr,mDC, "int",0, "int",0, "int",w, "int",h
, Ptr,hDC, "int",x, "int",y, "uint",0x00CC0020|0x40000000)
;--------------------------
VarSetCapacity(bi, 40, 0), NumPut(40, bi, 0, "int")
NumPut(w, bi, 4, "int"), NumPut(-h, bi, 8, "int")
NumPut(1, bi, 12, "short"), NumPut(bpp:=32, bi, 14, "short")
;--------------------------
DllCall("GetDIBits", Ptr,mDC, Ptr,hBM
, "int",0, "int",h, Ptr,&bits, Ptr,&bi, "int",0)
DllCall("SelectObject", Ptr,mDC, Ptr,oBM)
DllCall("DeleteObject", Ptr,hBM)
DllCall("DeleteDC", Ptr,mDC)
DllCall("ReleaseDC", Ptr,win, Ptr,hDC)
Scan0:=&bits, Stride:=((w*bpp+31)//32)*4
}
PicFind(Scan0,Stride,sx,sy,sw,sh,text,color
, w, h, ByRef rx, ByRef ry, err1, err0)
{
static MyFunc
if !MyFunc
{
x32:="5589E583EC408B45200FAF45188B551CC1E20201D08945F"
. "48B5524B80000000029D0C1E00289C28B451801D08945DCC74"
. "5F000000000837D08000F85210100008B450CC1E81025FF000"
. "0008945D88B450CC1E80825FF0000008945D48B450C25FF000"
. "0008945D0C745F800000000E9DD000000C745FC00000000E9B"
. "B0000008B45F483C00289C28B451401D00FB6000FB6C08945C"
. "C8B45F483C00189C28B451401D00FB6000FB6C08945C88B55F"
. "48B451401D00FB6000FB6C08945C48B45CC2B45D889C28B45C"
. "C3B45D87E07B801000000EB05B8FFFFFFFF0FAFD08B45C82B4"
. "5D489C18B45C83B45D47E07B801000000EB05B8FFFFFFFF0FA"
. "FC18D0C028B45C42B45D089C28B45C43B45D07E07B80100000"
. "0EB05B8FFFFFFFF0FAFC201C83B45107F0B8B55F08B452C01D"
. "0C600318345FC018345F4048345F0018B45FC3B45240F8C39F"
. "FFFFF8345F8018B45DC0145F48B45F83B45280F8C17FFFFFFE"
. "9A30000008B450C83C00169C0E803000089450CC745F800000"
. "000EB7FC745FC00000000EB648B45F483C00289C28B451401D"
. "00FB6000FB6C069D02B0100008B45F483C00189C18B451401C"
. "80FB6000FB6C069C04B0200008D0C028B55F48B451401D00FB"
. "6000FB6C06BC07201C83B450C730B8B55F08B452C01D0C6003"
. "18345FC018345F4048345F0018B45FC3B45247C948345F8018"
. "B45DC0145F48B45F83B45280F8C75FFFFFFC745E8000000008"
. "B45E88945EC8B45EC8945F0C745F800000000EB7CC745FC000"
. "00000EB678B45F08D50018955F089C28B453001D00FB6003C3"
. "175278B45EC8D50018955EC8D1485000000008B453401C28B4"
. "5F80FAF452489C18B45FC01C88902EB258B45E88D50018955E"
. "88D1485000000008B453801C28B45F80FAF452489C18B45FC0"
. "1C889028345FC018B45FC3B453C7C918345F8018B45F83B454"
. "00F8C78FFFFFF8B45242B453C83C00189453C8B45282B45408"
. "3C0018945408B45EC3945E80F4D45E88945DCC745F80000000"
. "0E9E3000000C745FC00000000E9C70000008B45F80FAF45248"
. "9C28B45FC01D08945F48B45448945E48B45488945E0C745F00"
. "0000000EB708B45F03B45EC7D2E8B45F08D1485000000008B4"
. "53401D08B108B45F401D089C28B452C01D00FB6003C31740A8"
. "36DE401837DE40078638B45F03B45E87D2E8B45F08D1485000"
. "000008B453801D08B108B45F401D089C28B452C01D00FB6003"
. "C30740A836DE001837DE00078308345F0018B45F03B45DC7C8"
. "88B551C8B45FC01C28B454C89108B55208B45F801C28B45508"
. "910B801000000EB3B90EB01908345FC018B45FC3B453C0F8C2"
. "DFFFFFF8345F8018B45F83B45400F8C11FFFFFF8B454CC700F"
. "FFFFFFF8B4550C700FFFFFFFFB800000000C9C24C0090"
x64:="554889E54883EC40894D10895518448945204C894D288B4"
. "5400FAF45308B5538C1E20201D08945F48B5548B8000000002"
. "9D0C1E00289C28B453001D08945DCC745F000000000837D100"
. "00F85310100008B4518C1E81025FF0000008945D88B4518C1E"
. "80825FF0000008945D48B451825FF0000008945D0C745F8000"
. "00000E9ED000000C745FC00000000E9CB0000008B45F483C00"
. "24863D0488B45284801D00FB6000FB6C08945CC8B45F483C00"
. "14863D0488B45284801D00FB6000FB6C08945C88B45F44863D"
. "0488B45284801D00FB6000FB6C08945C48B45CC2B45D889C28"
. "B45CC3B45D87E07B801000000EB05B8FFFFFFFF0FAFD08B45C"
. "82B45D489C18B45C83B45D47E07B801000000EB05B8FFFFFFF"
. "F0FAFC18D0C028B45C42B45D089C28B45C43B45D07E07B8010"
. "00000EB05B8FFFFFFFF0FAFC201C83B45207F108B45F04863D"
. "0488B45584801D0C600318345FC018345F4048345F0018B45F"
. "C3B45480F8C29FFFFFF8345F8018B45DC0145F48B45F83B455"
. "00F8C07FFFFFFE9B60000008B451883C00169C0E8030000894"
. "518C745F800000000E98F000000C745FC00000000EB748B45F"
. "483C0024863D0488B45284801D00FB6000FB6C069D02B01000"
. "08B45F483C0014863C8488B45284801C80FB6000FB6C069C04"
. "B0200008D0C028B45F44863D0488B45284801D00FB6000FB6C"
. "06BC07201C83B451873108B45F04863D0488B45584801D0C60"
. "0318345FC018345F4048345F0018B45FC3B45487C848345F80"
. "18B45DC0145F48B45F83B45500F8C65FFFFFFC745E80000000"
. "08B45E88945EC8B45EC8945F0C745F800000000E989000000C"
. "745FC00000000EB748B45F08D50018955F04863D0488B45604"
. "801D00FB6003C31752C8B45EC8D50018955EC4898488D14850"
. "0000000488B45684801C28B45F80FAF454889C18B45FC01C88"
. "902EB2A8B45E88D50018955E84898488D148500000000488B4"
. "5704801C28B45F80FAF454889C18B45FC01C889028345FC018"
. "B45FC3B45787C848345F8018B45F83B85800000000F8C68FFF"
. "FFF8B45482B457883C0018945788B45502B858000000083C00"
. "18985800000008B45EC3945E80F4D45E88945DCC745F800000"
. "000E908010000C745FC00000000E9EC0000008B45F80FAF454"
. "889C28B45FC01D08945F48B85880000008945E48B859000000"
. "08945E0C745F000000000E9800000008B45F03B45EC7D368B4"
. "5F04898488D148500000000488B45684801D08B108B45F401D"
. "04863D0488B45584801D00FB6003C31740A836DE401837DE40"
. "078778B45F03B45E87D368B45F04898488D148500000000488"
. "B45704801D08B108B45F401D04863D0488B45584801D00FB60"
. "03C30740A836DE001837DE000783C8345F0018B45F03B45DC0"
. "F8C74FFFFFF8B55388B45FC01C2488B859800000089108B554"
. "08B45F801C2488B85A00000008910B801000000EB4690EB019"
. "08345FC018B45FC3B45780F8C08FFFFFF8345F8018B45F83B8"
. "5800000000F8CE9FEFFFF488B8598000000C700FFFFFFFF488"
. "B85A0000000C700FFFFFFFFB8000000004883C4405DC39090"
MCode(MyFunc, A_PtrSize=8 ? x64:x32)
}
v:=text
if InStr(color,"-")
{
r:=err1, err1:=err0, err0:=r, v:=StrReplace(v,"1","_")
v:=StrReplace(StrReplace(v,"0","1"),"_","0")
}
if err1!=0
err1:=Round(StrLen(StrReplace(v,"0"))*err1)
if err0!=0
err0:=Round(StrLen(StrReplace(v,"1"))*err0)
mode:=InStr(color,"*") ? 1:0
color:=RegExReplace(color,"[*\-]") . "@"
StringSplit, r, color, @
color:=Round(r1), n:=Round(r2,2)+(!r2)
text:=v, k:=StrLen(text)*4, n:=Floor(255*3*(1-n))
VarSetCapacity(ss, sw*sh, Asc("0"))
VarSetCapacity(s1, k, 0), VarSetCapacity(s0, k, 0)
VarSetCapacity(rx, 8, 0), VarSetCapacity(ry, 8, 0)
Return, DllCall(&MyFunc, "int",mode
, "uint",color, "int",n, "ptr",Scan0, "int",Stride
, "int",sx, "int",sy, "int",sw, "int",sh
, "ptr",&ss, "Astr",text, "ptr",&s1, "ptr",&s0
, "int",w, "int",h, "int",err1, "int",err0
, "int*",rx, "int*",ry)
}
MCode(ByRef code, hex)
{
ListLines, Off
bch:=A_BatchLines
SetBatchLines, -1
VarSetCapacity(code, StrLen(hex)//2)
Loop, % StrLen(hex)//2
NumPut("0x" . SubStr(hex,2*A_Index-1,2)
, code, A_Index-1, "char")
Ptr:=A_PtrSize ? "Ptr" : "UInt"
DllCall("VirtualProtect", Ptr,&code, Ptr
,VarSetCapacity(code), "uint",0x40, Ptr . "*",0)
SetBatchLines, %bch%
ListLines, On
}
base64tobit(s) {
ListLines, Off
s:=RegExReplace(s,"\s+")
Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
. "abcdefghijklmnopqrstuvwxyz"
SetFormat, IntegerFast, d
StringCaseSense, On
Loop, Parse, Chars
{
i:=A_Index-1, v:=(i>>5&1) . (i>>4&1)
. (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
s:=StrReplace(s,A_LoopField,v)
}
StringCaseSense, Off
s:=SubStr(s,1,InStr(s,"1",0,0)-1)
ListLines, On
Return, s
}
bit2base64(s) {
ListLines, Off
s:=RegExReplace(s,"\s+")
s.=SubStr("100000",1,6-Mod(StrLen(s),6))
s:=RegExReplace(s,".{6}","|$0")
Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
. "abcdefghijklmnopqrstuvwxyz"
SetFormat, IntegerFast, d
Loop, Parse, Chars
{
i:=A_Index-1, v:="|" . (i>>5&1) . (i>>4&1)
. (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
s:=StrReplace(s,v,A_LoopField)
}
ListLines, On
Return, s
}
text:=
(
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
_________________________________________
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
00000000___0000___000000000000000000____0
00000000___00000__000000000000000__0__000
00000000___00000__000000000000000000__000
__0___________00___00000000000000_______0
_____000___0___0___00000000000000__0__000
_____000___0___0___00________0000__0__000
_____000___0___0___00000000000000__0__000
_____000___0___0___00000000000000__0__000
___________0___0__000000000000000__0__000
0000000000000000__00000000000000000000000
000000000000000___00000000000000000000000
00000000000000000000000000000000000000000
)
^q::
if FindText(-1238,210,150000,150000,0,0,Text)
{
CoordMode, Mouse
MouseMove, X, Y
}
Could you please steer me in the right direction with this, such as a working example? Thank you in advance.
Re: FindText - Catch screen image into text and then find it
Because the script is updated and is no longer compatible with the previous code,
Please re - capture the screen image, and generate new code.
You can modify the "FindText" function to fit your own style.
Please re - capture the screen image, and generate new code.
You can modify the "FindText" function to fit your own style.
Re: FindText - Catch screen image into text and then find it
Version 5.1 Help - credit to echandler's oppst
The author is does not speak English and his/her posts can be hard to follow. As such, I am trying to fully document this amazing function.
FindText Function - Although named FindText, it also works equally well at finding graphics. Other have suggest it should be called "FindMonochromePattern", and just to be cleart this does NOT have anything to do with OCR (text can be found but not interpreted). The function can be thought of as an alternative to native AHK Imagesearch Function. The native function requires saved graphic files, identical image matching, is hard to troubleshoot and performs in a relatively slow manner. AHK Imagesearch can commonly fail for reasons I will not get into. FindText() approaches the search differently. Think 1980's ASCII art. This function abstracts the screen's image into representative 0's and _'s. Because this is an abstraction, not and bit for bit comparison, it allows fast matching and easy adjustments of fault tolerance. The FindText function was rewritten in version 5, so the ASCII art mostly stays behind the scenes now.
General Usage:
1) Target Selection: Click the "Capture" button and the cursor is accompanied by a transparent red box. Move the red box over the text you want to search for and click. Then move the cursor away from the target to capture a clean target image.
2) Search Image Refinement: "Catch Image To Text" window shows a magnified view of the captured text. The buttons to the lower-left allows trimming of the images edges by 1 or 3 pixels in each direction. Auto does automatic trimming of useless whitespace around the edges, but only after Image has been abstracted. "Reset" resets the original Image, "Invert" inverts black and white (only after Color2Two or Grey2Two). The Modify Checkbox allow you to cleanup extraneous pixel from an image by clicking the pixels.
There two comparison modes: color and greyscale. You can choose either. These are slightly different ways to determine if each pixel color/greyscale in the target image becomes simplified to black or white and eventually a "0" or "_". The threshold is the divide between black and white determination. Threshold too low -> all white, Threshold to high -> all black and at these extremes matching accuracy is low.
3) Testing: You have been taken back to the original window. For fun you can click the long Text string in the lower pane, and the upper window will show the corresponding ASCIIart searchtext. You can now can click the "Test" button to test that the search will actually find what you're looking for. Click the "Copy" button to copy the text block and the related function call to the clipboard for you to paste into your own AHK code. The checkbox Insert FindText() in Copy allows you to include the needed function in your code (if its the first time) or exclude it if it's already there.
If you just want it to work, you done and you can stop reading now.....
More Advanced Understanding
FindText(X,Y,W,H,err0,err1,Text)
the parameters of X, Y, W, H to used determine a search area
X,Y are the center point about which search was conducted (where your red box was when clicked)
W,H are the offset distance from the center defining the width and height of search area (not the red box's width and height),
W and H are hardcoded to 150000 indicating a search of the entire screen.
So, effectively the search range is (X-W, Y-H)-->(X+W, Y+H).
X, Y, W, and H are calculated for you, and generally you're going to leave them alone.
If we are searching the whole screen, why even have the options? Consider 2 appropriate matches on the screen, Findtext will return the first
one found, but you want the second. Having X,Y allows you to get close to one desired, and constraining the W or H arguments can allow for
avoiding the other match and getting the desired match.
err1 is the character "0" fault-tolerant in percentage (0-1) Default 0.
err0 is the character "_" fault-tolerant in percentage.(0-1) Default 0.
Text is the String of data that represents the match. Text may include multiple of findtexts each separated by "|".
Here is an example of outputted code for a single find...
Although we are using the same naming convention, these object properties are entirely different than the X,Y,H,W argument submitted to FindText()
X, Y represents top/left coordinates of the found FindText Area. (The above calculation gets you to the center (instead of X and Y being left and top).
H, W represents width and height of find text area (how big was the red search box),
Comment is self explanatory and comes from code between <>'s in Text.
If you sent more than one find strings (separated by |) in Findtext, you can identify the correct array element by thinking:
ArgsPerMatch := 5
Matches := ok.MaxIndex()//ArgsPerMatch
DesiredIndex := (ArgsPerMatch * (DesiredMatch - 1) + DesiredArgNum
or if we wanted 4th Match's X (1st Arg) --- > (5 * (4-1)) + 1 = 16th array index position
FindText failures are unlikely to occur at the time of creation are unusual. It is worth knowing that any match failures with default of 0 error tolerance are automatically retried with 5% error tolerance (0.05). Future failures may happen on a different computer, OS, resolution, text aliasing, etc. You can manually adjust the error tolerance in the Function by adjusting 5th ("o" fault tolerance) and/or 6th ("_" fault tolerance) argument to hopefully gain a match.
The size of the red capture rectangle can be adjusted in the code.
You must ask yourself why you are doing this? Keeping it small works just fine, and keeps it speedy.
Still want it bigger, change the code below in the function
;----------------------------
; The capture range can be changed by adjusting the numbers
ww:=35, hh:=12
nW:=2*ww+1, nH:=2*hh+1
;---------------------------[/color]
The author is does not speak English and his/her posts can be hard to follow. As such, I am trying to fully document this amazing function.
FindText Function - Although named FindText, it also works equally well at finding graphics. Other have suggest it should be called "FindMonochromePattern", and just to be cleart this does NOT have anything to do with OCR (text can be found but not interpreted). The function can be thought of as an alternative to native AHK Imagesearch Function. The native function requires saved graphic files, identical image matching, is hard to troubleshoot and performs in a relatively slow manner. AHK Imagesearch can commonly fail for reasons I will not get into. FindText() approaches the search differently. Think 1980's ASCII art. This function abstracts the screen's image into representative 0's and _'s. Because this is an abstraction, not and bit for bit comparison, it allows fast matching and easy adjustments of fault tolerance. The FindText function was rewritten in version 5, so the ASCII art mostly stays behind the scenes now.
General Usage:
1) Target Selection: Click the "Capture" button and the cursor is accompanied by a transparent red box. Move the red box over the text you want to search for and click. Then move the cursor away from the target to capture a clean target image.
2) Search Image Refinement: "Catch Image To Text" window shows a magnified view of the captured text. The buttons to the lower-left allows trimming of the images edges by 1 or 3 pixels in each direction. Auto does automatic trimming of useless whitespace around the edges, but only after Image has been abstracted. "Reset" resets the original Image, "Invert" inverts black and white (only after Color2Two or Grey2Two). The Modify Checkbox allow you to cleanup extraneous pixel from an image by clicking the pixels.
There two comparison modes: color and greyscale. You can choose either. These are slightly different ways to determine if each pixel color/greyscale in the target image becomes simplified to black or white and eventually a "0" or "_". The threshold is the divide between black and white determination. Threshold too low -> all white, Threshold to high -> all black and at these extremes matching accuracy is low.
- Grey Mode: This one is the simpler of the two. Click the "Gray2Two" button to convert the magnified image into a black/white version. "Grey Threshold" has been automatically calculated and populated. Although not commonly needed, you can manually set the threshold and reclick Grey2Two button again to allow for more or less detail.
- Color Mode: No automatic Threshold determination on this one. First you must identify a color to separate black and white. For text just click the a black pixel of the word. If it is a non-text graphic select a color that appears to be present commonly in the image by clicking on a pixel in the image. The corresponding hex value of the color selected populates into "Selected Color". When you click the "Color2Two" button, it converts the magnified image into a black and version. The Color Sensitivity slider allows manual adjustment of whether colors slightly darker or lighter than the target color become black or white. For example you do a Color2Two, and don't see much detail then you move the slider to the left, and redo Color2Two. It now shows more detail and is more likely for an accurate match. Slide too far to the left, and again your into less detail and less accuracy.
3) Testing: You have been taken back to the original window. For fun you can click the long Text string in the lower pane, and the upper window will show the corresponding ASCIIart searchtext. You can now can click the "Test" button to test that the search will actually find what you're looking for. Click the "Copy" button to copy the text block and the related function call to the clipboard for you to paste into your own AHK code. The checkbox Insert FindText() in Copy allows you to include the needed function in your code (if its the first time) or exclude it if it's already there.
If you just want it to work, you done and you can stop reading now.....
More Advanced Understanding
FindText(X,Y,W,H,err0,err1,Text)
the parameters of X, Y, W, H to used determine a search area
X,Y are the center point about which search was conducted (where your red box was when clicked)
W,H are the offset distance from the center defining the width and height of search area (not the red box's width and height),
W and H are hardcoded to 150000 indicating a search of the entire screen.
So, effectively the search range is (X-W, Y-H)-->(X+W, Y+H).
X, Y, W, and H are calculated for you, and generally you're going to leave them alone.
If we are searching the whole screen, why even have the options? Consider 2 appropriate matches on the screen, Findtext will return the first
one found, but you want the second. Having X,Y allows you to get close to one desired, and constraining the W or H arguments can allow for
avoiding the other match and getting the desired match.
err1 is the character "0" fault-tolerant in percentage (0-1) Default 0.
err0 is the character "_" fault-tolerant in percentage.(0-1) Default 0.
Text is the String of data that represents the match. Text may include multiple of findtexts each separated by "|".
Here is an example of outputted code for a single find...
Code: Select all
Text.="|<>*139$71.4n3800+95AU0t44E00IHue02G88U00cY5I"
if ok:=FindText(121,44,150000,150000,0,0,Text)
{
CoordMode, Mouse
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
MouseMove, X+W//2, Y+H//2
}
X, Y represents top/left coordinates of the found FindText Area. (The above calculation gets you to the center (instead of X and Y being left and top).
H, W represents width and height of find text area (how big was the red search box),
Comment is self explanatory and comes from code between <>'s in Text.
If you sent more than one find strings (separated by |) in Findtext, you can identify the correct array element by thinking:
ArgsPerMatch := 5
Matches := ok.MaxIndex()//ArgsPerMatch
DesiredIndex := (ArgsPerMatch * (DesiredMatch - 1) + DesiredArgNum
or if we wanted 4th Match's X (1st Arg) --- > (5 * (4-1)) + 1 = 16th array index position
FindText failures are unlikely to occur at the time of creation are unusual. It is worth knowing that any match failures with default of 0 error tolerance are automatically retried with 5% error tolerance (0.05). Future failures may happen on a different computer, OS, resolution, text aliasing, etc. You can manually adjust the error tolerance in the Function by adjusting 5th ("o" fault tolerance) and/or 6th ("_" fault tolerance) argument to hopefully gain a match.
The size of the red capture rectangle can be adjusted in the code.
You must ask yourself why you are doing this? Keeping it small works just fine, and keeps it speedy.
Still want it bigger, change the code below in the function
;----------------------------
; The capture range can be changed by adjusting the numbers
ww:=35, hh:=12
nW:=2*ww+1, nH:=2*hh+1
;---------------------------[/color]
Last edited by c4p on 13 Jul 2017, 11:30, edited 25 times in total.
Re: FindText - Catch screen image into text and then find it
You write a very detailed description of the use. Thank you very much!
Because my English is not very good, Can only rely on software translation,
there should be a lot of grammatical errors, so I wrote very little instructions.
The official forum has a lot of enthusiastic net friend,
Make this tool continuously improved, so that it can be better used.
Because my English is not very good, Can only rely on software translation,
there should be a lot of grammatical errors, so I wrote very little instructions.
The official forum has a lot of enthusiastic net friend,
Make this tool continuously improved, so that it can be better used.
Last edited by feiyue on 07 May 2017, 18:21, edited 5 times in total.
Re: FindText - Catch screen image into text and then find it
feiyue - check your Personal Messages, I sent you an update for consideration.
Re: FindText - Catch screen image into text and then find it
@c4p, FindText() Updated to v5.1 version, Thank you for your help!c4p wrote:feiyue - check your Personal Messages, I sent you an update for consideration.
You've done a lot of work to make the software easier to use.
Re: FindText - Capture screen image into text and then find it
feiyue, I want to say Thank you for your awesome genius work
It helped me a lot ,and as an ambitious human being I wanted to take it to the next level , To make it work in background.
how can I make it work in back ground with ControlClick/Move modes ?
I have tried it but I guess the problem is that If statement can't work in background.
how can I make it detect hidden windows ?
I also tried DetectHiddenWindows, On
I renamed Title to "This is a new title" to avoid conflict.
changed Click , MouseMove to ControlMove,ControlClick
It doesn't Click , only selects the browser. like so https://prntscr.com/fa28cy (Selected) , https://prntscr.com/fa28ky (not selected)
sometimes it brings the desired window "This is a new title" window to the top , but doesn't do anything else.
P.S I have no experience with coding so these are just mere newbie assumptions
a fraction of what I want to be done:
Thank you
It helped me a lot ,and as an ambitious human being I wanted to take it to the next level , To make it work in background.
how can I make it work in back ground with ControlClick/Move modes ?
I have tried it but I guess the problem is that If statement can't work in background.
how can I make it detect hidden windows ?
I also tried DetectHiddenWindows, On
I renamed Title to "This is a new title" to avoid conflict.
changed Click , MouseMove to ControlMove,ControlClick
It doesn't Click , only selects the browser. like so https://prntscr.com/fa28cy (Selected) , https://prntscr.com/fa28ky (not selected)
sometimes it brings the desired window "This is a new title" window to the top , but doesn't do anything else.
P.S I have no experience with coding so these are just mere newbie assumptions
a fraction of what I want to be done:
Code: Select all
^i::
;Click Continue
DetectHiddenWindows, On
Text:="|<Continue>[email protected]$68.00000000000000000000000000000000000D3kUbt891T0A9WA8EH2EI020UHW64sY501U84gVV+91M0M21d8MGGELk60UG+64aY5U0U84XVV8t3M0ANa8MMG6Ma01wD2264UXlw0000000000000000000000000000000000000000000008"
if ok:=FindText(743,113,150000,150000,0,0,Text)
{
CoordMode, Mouse
X:=ok.1, Y:=ok.2, W:=ok.3, H:=ok.4, Comment:=ok.5
ControlMove, , X+W//2, Y+H//2,,, This is a new title, Google Chrome,,
ControlClick, X+W//2 Y+H//2, This is a new title,,,,, Pos
/*
;MouseMove, X+W//2, Y+H//2 ;Original code
;Click, X+W//2, Y+H//2 ;add to Original code to Click
;ControlClick, x255 y152, WinTitle,,,, Pos ;sample code I found
Also tried ControlClick with small x,y letters with changing the original code (x:=ok.1 , and so on)
*/
}
Sleep 300
!r::Reload
;===== Copy The Following Functions To Your Own Code Just once =====
; Note: parameters of the X,Y is the center of the coordinates,
; and the W,H is the offset distance to the center,
; So the search range is (X-W, Y-H)-->(X+W, Y+H).
; err1 is the character "0" fault-tolerant in percentage.
; err0 is the character "_" fault-tolerant in percentage.
; Text can be a lot of text to find, separated by "|".
; ruturn is a array, contains the X,Y,W,H,Comment results of Each Find.
FindText(x,y,w,h,err1,err0,text)
{
xywh2xywh(x-w,y-h,2*w+1,2*h+1,x,y,w,h)
if (w<1 or h<1)
Return, 0
bch:=A_BatchLines
SetBatchLines, -1
;--------------------------------------
GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits)
;--------------------------------------
sx:=0, sy:=0, sw:=w, sh:=h
Loop, 2 {
arr:=[]
Loop, Parse, text, |
{
v:=A_LoopField
IfNotInString, v, $, Continue
Comment:="", e1:=err1, e0:=err0
; You Can Add Comment Text within The <>
if RegExMatch(v,"<([^>]*)>",r)
v:=StrReplace(v,r), Comment:=Trim(r1)
; You can Add two fault-tolerant in the [], separated by commas
if RegExMatch(v,"\[([^\]]*)]",r)
{
v:=StrReplace(v,r), r2:=""
StringSplit, r, r1, `,
e1:=r1, e0:=r2
}
StringSplit, r, v, $
color:=r1, v:=r2
StringSplit, r, v, .
w1:=r1, v:=base64tobit(r2), h1:=StrLen(v)//w1
if (r0<2 or w1>sw or h1>sh or h1<1 or StrLen(v)!=w1*h1)
Continue
if PicFind(Scan0,Stride,sx,sy,sw,sh
,v,color,w1,h1,rx,ry,e1,e0)
{
rx+=x, ry+=y
arr.Push(rx,ry,w1,h1,Comment)
}
}
if (arr.MaxIndex())
Break
if (A_Index=1 and err1=0 and err0=0)
err1:=0.05, err0:=0.05
else Break
}
SetBatchLines, %bch%
Return, arr.MaxIndex() ? arr:0
}
xywh2xywh(x1,y1,w1,h1,ByRef x,ByRef y,ByRef w,ByRef h)
{
SysGet, zx, 76
SysGet, zy, 77
SysGet, zw, 78
SysGet, zh, 79
left:=x1, right:=x1+w1-1, up:=y1, down:=y1+h1-1
left:=left<zx ? zx:left, right:=right>zx+zw-1 ? zx+zw-1:right
up:=up<zy ? zy:up, down:=down>zy+zh-1 ? zy+zh-1:down
x:=left, y:=up, w:=right-left+1, h:=down-up+1
}
GetBitsFromScreen(x,y,w,h,ByRef Scan0,ByRef Stride,ByRef bits)
{
VarSetCapacity(bits, w*h*4, 0)
Ptr:=A_PtrSize ? "Ptr" : "UInt"
win:=DllCall("GetDesktopWindow", Ptr)
hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
hBM:=DllCall("CreateCompatibleBitmap", Ptr,hDC
, "int",w, "int",h, Ptr)
oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,hBM, Ptr)
DllCall("BitBlt", Ptr,mDC, "int",0, "int",0, "int",w, "int",h
, Ptr,hDC, "int",x, "int",y, "uint",0x00CC0020|0x40000000)
;--------------------------
VarSetCapacity(bi, 40, 0), NumPut(40, bi, 0, "int")
NumPut(w, bi, 4, "int"), NumPut(-h, bi, 8, "int")
NumPut(1, bi, 12, "short"), NumPut(bpp:=32, bi, 14, "short")
;--------------------------
DllCall("GetDIBits", Ptr,mDC, Ptr,hBM
, "int",0, "int",h, Ptr,&bits, Ptr,&bi, "int",0)
DllCall("SelectObject", Ptr,mDC, Ptr,oBM)
DllCall("DeleteObject", Ptr,hBM)
DllCall("DeleteDC", Ptr,mDC)
DllCall("ReleaseDC", Ptr,win, Ptr,hDC)
Scan0:=&bits, Stride:=((w*bpp+31)//32)*4
}
PicFind(Scan0,Stride,sx,sy,sw,sh,text,color
, w, h, ByRef rx, ByRef ry, err1, err0)
{
static MyFunc
if !MyFunc
{
x32:="5589E583EC408B45200FAF45188B551CC1E20201D08945F"
. "48B5524B80000000029D0C1E00289C28B451801D08945D8C74"
. "5F000000000837D08000F85F00000008B450CC1E81025FF000"
. "0008945D48B450CC1E80825FF0000008945D08B450C25FF000"
. "0008945CCC745F800000000E9AC000000C745FC00000000E98"
. "A0000008B45F483C00289C28B451401D00FB6000FB6C02B45D"
. "48945EC8B45F483C00189C28B451401D00FB6000FB6C02B45D"
. "08945E88B55F48B451401D00FB6000FB6C02B45CC8945E4837"
. "DEC007903F75DEC837DE8007903F75DE8837DE4007903F75DE"
. "48B55EC8B45E801C28B45E401D03B45107F0B8B55F08B452C0"
. "1D0C600318345FC018345F4048345F0018B45FC3B45240F8C6"
. "AFFFFFF8345F8018B45D80145F48B45F83B45280F8C48FFFFF"
. "FE9A30000008B450C83C00169C0E803000089450CC745F8000"
. "00000EB7FC745FC00000000EB648B45F483C00289C28B45140"
. "1D00FB6000FB6C069D02B0100008B45F483C00189C18B45140"
. "1C80FB6000FB6C069C04B0200008D0C028B55F48B451401D00"
. "FB6000FB6C06BC07201C83B450C730B8B55F08B452C01D0C60"
. "0318345FC018345F4048345F0018B45FC3B45247C948345F80"
. "18B45D80145F48B45F83B45280F8C75FFFFFF8B45242B45488"
. "3C0018945488B45282B454C83C00189454C8B453839453C0F4"
. "D453C8945D8C745F800000000E9E3000000C745FC00000000E"
. "9C70000008B45F80FAF452489C28B45FC01D08945F48B45408"
. "945E08B45448945DCC745F000000000EB708B45F03B45387D2"
. "E8B45F08D1485000000008B453001D08B108B45F401D089C28"
. "B452C01D00FB6003C31740A836DE001837DE00078638B45F03"
. "B453C7D2E8B45F08D1485000000008B453401D08B108B45F40"
. "1D089C28B452C01D00FB6003C30740A836DDC01837DDC00783"
. "08345F0018B45F03B45D87C888B551C8B45FC01C28B4550891"
. "08B55208B45F801C28B45548910B801000000EB2990EB01908"
. "345FC018B45FC3B45480F8C2DFFFFFF8345F8018B45F83B454"
. "C0F8C11FFFFFFB800000000C9C25000"
x64:="554889E54883EC40894D10895518448945204C894D288B4"
. "5400FAF45308B5538C1E20201D08945F48B5548B8000000002"
. "9D0C1E00289C28B453001D08945D8C745F000000000837D100"
. "00F85000100008B4518C1E81025FF0000008945D48B4518C1E"
. "80825FF0000008945D08B451825FF0000008945CCC745F8000"
. "00000E9BC000000C745FC00000000E99A0000008B45F483C00"
. "24863D0488B45284801D00FB6000FB6C02B45D48945EC8B45F"
. "483C0014863D0488B45284801D00FB6000FB6C02B45D08945E"
. "88B45F44863D0488B45284801D00FB6000FB6C02B45CC8945E"
. "4837DEC007903F75DEC837DE8007903F75DE8837DE4007903F"
. "75DE48B55EC8B45E801C28B45E401D03B45207F108B45F0486"
. "3D0488B45584801D0C600318345FC018345F4048345F0018B4"
. "5FC3B45480F8C5AFFFFFF8345F8018B45D80145F48B45F83B4"
. "5500F8C38FFFFFFE9B60000008B451883C00169C0E80300008"
. "94518C745F800000000E98F000000C745FC00000000EB748B4"
. "5F483C0024863D0488B45284801D00FB6000FB6C069D02B010"
. "0008B45F483C0014863C8488B45284801C80FB6000FB6C069C"
. "04B0200008D0C028B45F44863D0488B45284801D00FB6000FB"
. "6C06BC07201C83B451873108B45F04863D0488B45584801D0C"
. "600318345FC018345F4048345F0018B45FC3B45487C848345F"
. "8018B45D80145F48B45F83B45500F8C65FFFFFF8B45482B859"
. "000000083C0018985900000008B45502B859800000083C0018"
. "985980000008B45703945780F4D45788945D8C745F80000000"
. "0E90B010000C745FC00000000E9EC0000008B45F80FAF45488"
. "9C28B45FC01D08945F48B85800000008945E08B85880000008"
. "945DCC745F000000000E9800000008B45F03B45707D368B45F"
. "04898488D148500000000488B45604801D08B108B45F401D04"
. "863D0488B45584801D00FB6003C31740A836DE001837DE0007"
. "8778B45F03B45787D368B45F04898488D148500000000488B4"
. "5684801D08B108B45F401D04863D0488B45584801D00FB6003"
. "C30740A836DDC01837DDC00783C8345F0018B45F03B45D80F8"
. "C74FFFFFF8B55388B45FC01C2488B85A000000089108B55408"
. "B45F801C2488B85A80000008910B801000000EB2F90EB01908"
. "345FC018B45FC3B85900000000F8C05FFFFFF8345F8018B45F"
. "83B85980000000F8CE6FEFFFFB8000000004883C4405DC390"
MCode(MyFunc, A_PtrSize=8 ? x64:x32)
}
if InStr(color,"-")
{
r:=err1, err1:=err0, err0:=r
text:=StrReplace(text,"1","_")
text:=StrReplace(StrReplace(text,"0","1"),"_","0")
}
mode:=InStr(color,"*") ? 1:0
color:=RegExReplace(color,"[*\-]") . "@"
StringSplit, r, color, @
color:=Round(r1), n:=Round(r2,2)+(!r2)
n:=Floor(255*3*(1-n)), k:=StrLen(text)*4
VarSetCapacity(ss, sw*sh, Asc("0"))
VarSetCapacity(s1, k, 0), VarSetCapacity(s0, k, 0)
VarSetCapacity(rx, 8, 0), VarSetCapacity(ry, 8, 0)
len1:=len0:=0
ListLines, Off
Loop, Parse, text
{
i:=((A_Index-1)//w)*sw+Mod(A_Index-1,w)
if A_LoopField
NumPut(i, s1, 4*len1++, "int")
else
NumPut(i, s0, 4*len0++, "int")
}
ListLines, On
err1:=Round(len1*err1), err0:=Round(len0*err0)
Return, DllCall(&MyFunc, "int",mode
, "uint",color, "int",n, "ptr",Scan0, "int",Stride
, "int",sx, "int",sy, "int",sw, "int",sh
, "ptr",&ss, "ptr",&s1, "ptr",&s0
, "int",len1, "int",len0, "int",err1, "int",err0
, "int",w, "int",h, "int*",rx, "int*",ry)
}
MCode(ByRef code, hex)
{
ListLines, Off
bch:=A_BatchLines
SetBatchLines, -1
VarSetCapacity(code, StrLen(hex)//2)
Loop, % StrLen(hex)//2
NumPut("0x" . SubStr(hex,2*A_Index-1,2)
, code, A_Index-1, "char")
Ptr:=A_PtrSize ? "Ptr" : "UInt"
DllCall("VirtualProtect", Ptr,&code, Ptr
,VarSetCapacity(code), "uint",0x40, Ptr . "*",0)
SetBatchLines, %bch%
ListLines, On
}
base64tobit(s) {
ListLines, Off
s:=RegExReplace(s,"\s+")
Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
. "abcdefghijklmnopqrstuvwxyz"
SetFormat, IntegerFast, d
StringCaseSense, On
Loop, Parse, Chars
{
i:=A_Index-1, v:=(i>>5&1) . (i>>4&1)
. (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
s:=StrReplace(s,A_LoopField,v)
}
StringCaseSense, Off
s:=SubStr(s,1,InStr(s,"1",0,0)-1)
ListLines, On
Return, s
}
bit2base64(s) {
ListLines, Off
s:=RegExReplace(s,"\s+")
s.=SubStr("100000",1,6-Mod(StrLen(s),6))
s:=RegExReplace(s,".{6}","|$0")
Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
. "abcdefghijklmnopqrstuvwxyz"
SetFormat, IntegerFast, d
Loop, Parse, Chars
{
i:=A_Index-1, v:="|" . (i>>5&1) . (i>>4&1)
. (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
s:=StrReplace(s,v,A_LoopField)
}
ListLines, On
return, s
}
ASCII(s)
{
if RegExMatch(s,"(\d+)\.([\w+/]{3,})",r)
{
s:=RegExReplace(base64tobit(r2),".{" r1 "}","$0`n")
s:=StrReplace(StrReplace(s,"0","_"),"1","0")
}
else s=
return, s
}
Re: FindText - Capture screen image into text and then find it
See the wrapper I wrote above. It has controlclick functionality. Hidden Windows will not recognize because findtext is searching the screen's graphics. It has been awhile since I treed it, but as I remember, Controlclick works when IE browser is active, but failsed if page controls within browser has focus. Try clicking on IE URL bar, or titlebar then running your script. If that fixes it, it should be a matter coding AHK putting focus on correct part.
Re: FindText - Capture screen image into text and then find it
When I was in WinXP, I realized the function of searching the background window in FindText,
Because there is a good WinApi (PrintWindow), It can capture image of the hidden window.
But in Win7 and above, PrintWindow is not working well, so I remove this function again.
Because there is a good WinApi (PrintWindow), It can capture image of the hidden window.
But in Win7 and above, PrintWindow is not working well, so I remove this function again.
Last edited by feiyue on 21 May 2017, 09:30, edited 1 time in total.
Re: FindText - Capture screen image into text and then find it
I use GetDCEx method instead PrintWindow, btw it faster.
Example screenshoter, using Gdip library (uncomment 6 line for deactivate window before screenshot):
I not found reliable method for take border and titlebar, so I take only client area of window:
Example screenshoter, using Gdip library (uncomment 6 line for deactivate window before screenshot):
Code: Select all
SetBatchLines, -1
pToken := GdipStartup()
Run notepad,,, pid
WinWaitActive ahk_pid %pid%
WinGet hwnd, ID, ahk_pid %pid%
; Send !{Esc} ; WinDeActivate
sleep 1000
pBitmap := Gdip_BitmapFromScreen(hBitmap, "hwnd:" hwnd)
GdipGetImageDimensions(pBitmap, Width, Height)
SS_BITMAP := 0xE, STM_SETIMAGE := 0x172, IMAGE_BITMAP := 0
Gui, -DPIScale ToolWindow
Gui, Margin, 0, 0
Gui, Add, Text, hWndPic w%Width% h%Height% +%SS_BITMAP%
PostMessage, STM_SETIMAGE, IMAGE_BITMAP, hBitmap,, ahk_id %Pic%
Gui, Show
DllCall("gdiplus\GdipDisposeImage", "ptr", pBitmap)
DllCall("DeleteObject", "ptr", hBitmap)
GdipShutdown(pToken)
Return
GuiClose:
GuiEscape:
ExitApp
GdipStartup() {
if !DllCall("GetModuleHandle", "str", "gdiplus", "ptr")
DllCall("LoadLibrary", "str", "gdiplus")
VarSetCapacity(si, A_PtrSize = 8 ? 24 : 16, 0), si := Chr(1)
DllCall("gdiplus\GdiplusStartup", "uptr*", pToken, "ptr", &si, "ptr", 0)
return pToken
}
GdipShutdown(pToken) {
DllCall("gdiplus\GdiplusShutdown", "uptr", pToken)
if hModule := DllCall("GetModuleHandle", "str", "gdiplus", "ptr")
DllCall("FreeLibrary", "ptr", hModule)
return 0
}
GdipGetImageDimensions(pBitmap, ByRef Width, ByRef Height) {
DllCall("gdiplus\GdipGetImageWidth", "ptr", pBitmap, "uint*", Width)
DllCall("gdiplus\GdipGetImageHeight", "ptr", pBitmap, "uint*", Height)
}
Gdip_BitmapFromScreen(ByRef hBitmap, Screen=0, Raster="") {
if (Screen = 0) {
Sysget, x, 76
Sysget, y, 77
Sysget, w, 78
Sysget, h, 79
}
else if (SubStr(Screen, 1, 5) = "hwnd:") {
Screen := SubStr(Screen, 6)
if !WinExist( "ahk_id " Screen)
return -2
WinGetPos,,, w, h, ahk_id %Screen%
x := y := 0
hhdc := GetDCEx(Screen, 3)
}
else if (Screen&1 != "") {
Sysget, M, Monitor, %Screen%
x := MLeft, y := MTop, w := MRight-MLeft, h := MBottom-MTop
}
else {
StringSplit, S, Screen, |
x := S1, y := S2, w := S3, h := S4
}
if (x = "") || (y = "") || (w = "") || (h = "")
return -1
chdc := CreateCompatibleDC(), hbm := CreateDIBSection(w, h, chdc), obm := SelectObject(chdc, hbm), hhdc := hhdc ? hhdc : GetDC()
BitBlt(chdc, 0, 0, w, h, hhdc, x, y, Raster)
ReleaseDC(hhdc)
pBitmap := Gdip_CreateBitmapFromHBITMAP(hbm)
DllCall("gdiplus\GdipCreateHBITMAPFromBitmap", "ptr", pBitmap, "ptr*", hBitmap, "uint", 0xffffffff)
SelectObject(chdc, obm), DeleteObject(hbm), DeleteDC(hhdc), DeleteDC(chdc)
return pBitmap
}
GetDCEx(hwnd, flags=0, hrgnClip=0) {
return DllCall("GetDCEx", "uint", hwnd, "uint", hrgnClip, "int", flags)
}
CreateCompatibleDC(hdc=0) {
return DllCall("CreateCompatibleDC", "uint", hdc)
}
CreateDIBSection(w, h, hdc="", bpp=32, ByRef ppvBits=0) {
hdc2 := hdc ? hdc : GetDC()
VarSetCapacity(bi, 40, 0)
NumPut(w, bi, 4), NumPut(h, bi, 8), NumPut(40, bi, 0), NumPut(1, bi, 12, "ushort"), NumPut(0, bi, 16), NumPut(bpp, bi, 14, "ushort")
hbm := DllCall("CreateDIBSection", "uint" , hdc2, "uint" , &bi, "uint" , 0, "uint*", ppvBits, "uint" , 0, "uint" , 0)
if !hdc
ReleaseDC(hdc2)
return hbm
}
GetDC(hwnd=0){
return DllCall("GetDC", "uint", hwnd)
}
ReleaseDC(hdc, hwnd=0) {
return DllCall("ReleaseDC", "uint", hwnd, "uint", hdc)
}
SelectObject(hdc, hgdiobj) {
return DllCall("SelectObject", "uint", hdc, "uint", hgdiobj)
}
BitBlt(ddc, dx, dy, dw, dh, sdc, sx, sy, Raster="") {
return DllCall("gdi32\BitBlt", "uint", dDC, "int", dx, "int", dy, "int", dw, "int", dh
, "uint", sDC, "int", sx, "int", sy, "uint", Raster ? Raster : 0x00CC0020)
}
Gdip_CreateBitmapFromHBITMAP(hBitmap, Palette=0) {
DllCall("gdiplus\GdipCreateBitmapFromHBITMAP", "uint", hBitmap, "uint", Palette, "uint*", pBitmap)
return pBitmap
}
DeleteObject(hObject) {
return DllCall("DeleteObject", "uint", hObject)
}
DeleteDC(hdc) {
return DllCall("DeleteDC", "uint", hdc)
}
Code: Select all
SetBatchLines, -1
pToken := GdipStartup()
Run notepad,,, pid
WinWaitActive ahk_pid %pid%
WinGet hwnd, ID, ahk_pid %pid%
Send !{Esc} ; WinDeActivate
sleep 1000
pBitmap := Gdip_BitmapFromScreen(hBitmap, "client_hwnd:" hwnd)
GdipGetImageDimensions(pBitmap, Width, Height)
SS_BITMAP := 0xE, STM_SETIMAGE := 0x172, IMAGE_BITMAP := 0
Gui, -DPIScale ToolWindow
Gui, Margin, 0, 0
Gui, Add, Text, hWndPic w%Width% h%Height% +%SS_BITMAP%
PostMessage, STM_SETIMAGE, IMAGE_BITMAP, hBitmap,, ahk_id %Pic%
Gui, Show
DllCall("gdiplus\GdipDisposeImage", "ptr", pBitmap)
DllCall("DeleteObject", "ptr", hBitmap)
GdipShutdown(pToken)
Return
GuiClose:
GuiEscape:
ExitApp
GdipStartup() {
if !DllCall("GetModuleHandle", "str", "gdiplus", "ptr")
DllCall("LoadLibrary", "str", "gdiplus")
VarSetCapacity(si, A_PtrSize = 8 ? 24 : 16, 0), si := Chr(1)
DllCall("gdiplus\GdiplusStartup", "uptr*", pToken, "ptr", &si, "ptr", 0)
return pToken
}
GdipShutdown(pToken) {
DllCall("gdiplus\GdiplusShutdown", "uptr", pToken)
if hModule := DllCall("GetModuleHandle", "str", "gdiplus", "ptr")
DllCall("FreeLibrary", "ptr", hModule)
return 0
}
GdipGetImageDimensions(pBitmap, ByRef Width, ByRef Height) {
DllCall("gdiplus\GdipGetImageWidth", "ptr", pBitmap, "uint*", Width)
DllCall("gdiplus\GdipGetImageHeight", "ptr", pBitmap, "uint*", Height)
}
Gdip_BitmapFromScreen(ByRef hBitmap, Screen=0, Raster="") {
if (Screen = 0) {
Sysget, x, 76
Sysget, y, 77
Sysget, w, 78
Sysget, h, 79
}
else if (SubStr(Screen, 1, 5) = "hwnd:") {
Screen := SubStr(Screen, 6)
if !WinExist( "ahk_id " Screen)
return -2
WinGetPos,,, w, h, ahk_id %Screen%
x := y := 0
hhdc := GetDCEx(Screen, 3)
}
else if (SubStr(Screen, 1, 12) = "client_hwnd:") {
Screen := SubStr(Screen, 13)
if !WinExist( "ahk_id " Screen)
return -2
WinP := WinGetP(Screen)
x := WinP.Client2Win.x, y := WinP.Client2Win.y, w := WinP.Client2Win.w, h := WinP.Client2Win.h
hhdc := GetDCEx(Screen, 3)
}
else if (Screen&1 != "") {
Sysget, M, Monitor, %Screen%
x := MLeft, y := MTop, w := MRight-MLeft, h := MBottom-MTop
}
else {
StringSplit, S, Screen, |
x := S1, y := S2, w := S3, h := S4
}
if (x = "") || (y = "") || (w = "") || (h = "")
return -1
chdc := CreateCompatibleDC(), hbm := CreateDIBSection(w, h, chdc), obm := SelectObject(chdc, hbm), hhdc := hhdc ? hhdc : GetDC()
BitBlt(chdc, 0, 0, w, h, hhdc, x, y, Raster)
ReleaseDC(hhdc)
pBitmap := Gdip_CreateBitmapFromHBITMAP(hbm)
DllCall("gdiplus\GdipCreateHBITMAPFromBitmap", "ptr", pBitmap, "ptr*", hBitmap, "uint", 0xffffffff)
SelectObject(chdc, obm), DeleteObject(hbm), DeleteDC(hhdc), DeleteDC(chdc)
return pBitmap
}
GetDCEx(hwnd, flags=0, hrgnClip=0) {
return DllCall("GetDCEx", "uint", hwnd, "uint", hrgnClip, "int", flags)
}
CreateCompatibleDC(hdc=0) {
return DllCall("CreateCompatibleDC", "uint", hdc)
}
CreateDIBSection(w, h, hdc="", bpp=32, ByRef ppvBits=0) {
hdc2 := hdc ? hdc : GetDC()
VarSetCapacity(bi, 40, 0)
NumPut(w, bi, 4), NumPut(h, bi, 8), NumPut(40, bi, 0), NumPut(1, bi, 12, "ushort"), NumPut(0, bi, 16), NumPut(bpp, bi, 14, "ushort")
hbm := DllCall("CreateDIBSection", "uint" , hdc2, "uint" , &bi, "uint" , 0, "uint*", ppvBits, "uint" , 0, "uint" , 0)
if !hdc
ReleaseDC(hdc2)
return hbm
}
GetDC(hwnd=0){
return DllCall("GetDC", "uint", hwnd)
}
ReleaseDC(hdc, hwnd=0) {
return DllCall("ReleaseDC", "uint", hwnd, "uint", hdc)
}
SelectObject(hdc, hgdiobj) {
return DllCall("SelectObject", "uint", hdc, "uint", hgdiobj)
}
BitBlt(ddc, dx, dy, dw, dh, sdc, sx, sy, Raster="") {
return DllCall("gdi32\BitBlt", "uint", dDC, "int", dx, "int", dy, "int", dw, "int", dh
, "uint", sDC, "int", sx, "int", sy, "uint", Raster ? Raster : 0x00CC0020)
}
Gdip_CreateBitmapFromHBITMAP(hBitmap, Palette=0) {
DllCall("gdiplus\GdipCreateBitmapFromHBITMAP", "uint", hBitmap, "uint", Palette, "uint*", pBitmap)
return pBitmap
}
DeleteObject(hObject) {
return DllCall("DeleteObject", "uint", hObject)
}
DeleteDC(hdc) {
return DllCall("DeleteDC", "uint", hdc)
}
WinGetP(hwnd) {
WinGetPos, x, y, w, h, ahk_id %hWnd%
WinP := {x:x, y:y, w:w, h:h}
VarSetCapacity(pt, 16)
NumPut(x, pt, 0) || NumPut(y, pt, 4) || NumPut(w, pt, 8) || NumPut(h, pt, 12)
if (!DllCall("GetClientRect", "uint", hwnd, "uint", &pt))
Return
if (!DllCall("ClientToScreen", "uint", hwnd, "uint", &pt))
Return
x := NumGet(pt, 0, "int"), y := NumGet(pt, 4, "int")
w := NumGet(pt, 8, "int"), h := NumGet(pt, 12, "int")
Client := {x:x, y:y, w:w, h:h}
Client2Win := {x:x-WinP.x, y:y-WinP.y, w:w, h:h}
Return WinP := {x:WinP.x, y:WinP.y, w:WinP.w, h:WinP.h, Client2Win:Client2Win, Client:Client}
}
Last edited by stealzy on 24 May 2017, 06:29, edited 1 time in total.
Return to “Scripts and Functions (v1)”
Who is online
Users browsing this forum: No registered users and 195 guests