FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools
paulpma
Posts: 19
Joined: 08 Sep 2018, 22:05

Re: FindText - Capture screen image into text and then find it

30 Oct 2018, 22:59

@leosouza85
I have been thinking about your idea (wild character) and agree it's much better then auto error calculation. I use the function for OCR purposes and it works great and it's fast, but at times I have to play with thresholds because pixels show or don't show in certain areas depending on position. The wild character will definitelyly solve this issue. I wonder what feiyue has to say about your idea.
feiyue
Posts: 162
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

11 Nov 2018, 07:01

Updated to the v6.2 version. :beer:
A parameter is added at the end of the function to allow the last screenshot to be used.
Of course, saving the last screenshot will take up a certain amount of memory.
In addition, ScreenShot() function is added, which can be used before the loop.
pk23
Posts: 67
Joined: 24 Apr 2015, 00:49

Re: FindText - Capture screen image into text and then find it

11 Nov 2018, 11:39

feiyue wrote:
11 Nov 2018, 07:01
Updated to the v6.2 version. :beer:
A parameter is added at the end of the function to allow the last screenshot to be used.
Of course, saving the last screenshot will take up a certain amount of memory.
In addition, ScreenShot() function is added, which can be used before the loop.
Awesome! In fact I've also tried the famous 大漠插件 and feel that the performance of this tool is better than that. thank you for maintaining and improving this tool constantly. :thumbup:
iseahound
Posts: 446
Joined: 13 Aug 2016, 21:04
GitHub: iseahound

Re: FindText - Capture screen image into text and then find it

11 Nov 2018, 22:31

Regarding false positives/negatives, I think there are some improvements that can be made. I might add some stuff from FindText into my project Vis2, if that is okay with you feiyue.
CyL0N
Posts: 210
Joined: 27 Sep 2018, 09:58

Re: FindText - Capture screen image into text and then find it

12 Nov 2018, 02:42

iseahound wrote:
11 Nov 2018, 22:31
Regarding false positives/negatives, I think there are some improvements that can be made. I might add some stuff from FindText into my project Vis2, if that is okay with you feiyue.
Color me intrigued,but how would that work given you use tesseract for OCR, and given i'm on the subject, might i suggest refining tesseract output with a WordList, https://github.com/Paris/AutoHotkey-Scr ... stance.ahk & https://rosettacode.org/wiki/Longest_Common_Substring ,i gave it a shot a while back but was too slow,you might have better luck...(FYI, i merely refer to correcting an OCR output of 'euc11de' to 'ecuclide' for example,which happens a lot with tesseract).

Cheers & Sorrry if i detracted from this topic,i'm just intrigued by how you can use this to improve Vis2 cos it's my go tofor OCR & i didn't know it did much else, just as FindText is my goto for fast image search.
live ? long & prosper : regards
pk23
Posts: 67
Joined: 24 Apr 2015, 00:49

Re: FindText - Capture screen image into text and then find it

24 Nov 2018, 01:59

Any ways to generate patterns at runtime? I mean, the parameter Text of FindText() hasn't been determined before running, so cannot be written fixed in the code. Any ways to capture the image and create the parameter Text during running?
iseahound
Posts: 446
Joined: 13 Aug 2016, 21:04
GitHub: iseahound

Re: FindText - Capture screen image into text and then find it

24 Nov 2018, 18:28

It should be possible. I'd do it, but I would take my time.
ahkcrazy
Posts: 4
Joined: 26 Feb 2018, 02:19

Re: FindText - Capture screen image into text and then find it

25 Nov 2018, 04:01

Im trying to figure out what the FindText variable is and make an if/else statement and loop it.
If FindText finds a match I want it to hit Numpad2, else hit numpad1, wait for a second and then loop. Not sure if Im doing this right but this is what I got:

Code: Select all

;;pause script
Pause::Pause

Loop
{
t1:=A_TickCount

Text:="|<Auto>-*59$62.000000000003k000000001y0000A0000TU000700007w0003k0003z0000w0000zk3UQTsDk0SS0w7by7z07bUD1tzXzs1tw3kSDVwT0wD0w7XkS3kD3kD1swD0y7ky3kSD3k7VzzUw7Xkw1sTzwD1swD0SDzz3kyD3sDXk3kwDXkS3lw0yDbszbvwS07Xzy7szy7U1wTrVy7z00001k070C000000000008"

if (ok:=FindText(211, 414, 150000, 150000, 0, 0, Text))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
}

;;MsgBox, 4096,, % "Time:`t" (A_TickCount-t1) " ms`n`n"
;;  . "Pos:`t" X ", " Y "`n`n"
;;  . "Result:`t" (ok ? "Success !":"Failed !"), 3
;;MouseMove, X, Y
if (Text:=<Auto>) {
Send {Numpad2}
}else{
Send {Numpad1}
}
sleep, 1000
}


;===== Copy The Following Functions To Your Own Code Just once =====


; Note: parameters of the X,Y is the center of the coordinates,
; and the W,H is the offset distance to the center,
; So the search range is (X-W, Y-H)-->(X+W, Y+H).
; err1 is the character "0" fault-tolerant in percentage.
; err0 is the character "_" fault-tolerant in percentage.
; Text can be a lot of text to find, separated by "|".
; ruturn is a array, contains the [X,Y,W,H,Comment] results of Each Find.

FindText(x,y,w,h,err1,err0,text)
{
  xywh2xywh(x-w,y-h,2*w+1,2*h+1,x,y,w,h)
  if (w<1 or h<1)
    return, 0
  bch:=A_BatchLines
  SetBatchLines, -1
  ;--------------------------------------
  GetBitsFromScreen(x,y,w,h,Scan0,Stride,bits)
  ;--------------------------------------
  sx:=0, sy:=0, sw:=w, sh:=h, arr:=[]
  Loop, Parse, text, |
  {
    v:=A_LoopField
    IfNotInString, v, $, Continue
    Comment:="", e1:=err1, e0:=err0
    ; You Can Add Comment Text within The <>
    if RegExMatch(v,"<([^>]*)>",r)
      v:=StrReplace(v,r), Comment:=Trim(r1)
    ; You can Add two fault-tolerant in the [], separated by commas
    if RegExMatch(v,"\[([^\]]*)]",r)
    {
      v:=StrReplace(v,r), r1.=","
      StringSplit, r, r1, `,
      e1:=r1, e0:=r2
    }
    StringSplit, r, v, $
    color:=r1, v:=r2
    StringSplit, r, v, .
    w1:=r1, v:=base64tobit(r2), h1:=StrLen(v)//w1
    if (r0<2 or h1<1 or w1>sw or h1>sh or StrLen(v)!=w1*h1)
      Continue
    ;--------------------------------------------
    if InStr(color,"-")
    {
      r:=e1, e1:=e0, e0:=r, v:=StrReplace(v,"1","_")
      v:=StrReplace(StrReplace(v,"0","1"),"_","0")
      color:=StrReplace(color,"-")
    }
    mode:=InStr(color,"*") ? 1:0
    color:=StrReplace(color,"*") . "@"
    StringSplit, r, color, @
    color:=mode=1 ? r1 : ((r1-1)//w1)*Stride+Mod(r1-1,w1)*4
    n:=Round(r2,2)+(!r2), n:=Floor(255*3*(1-n))
    VarSetCapacity(allpos, 1024*4, 0), k:=StrLen(v)*4
    VarSetCapacity(s1, k, 0), VarSetCapacity(s0, k, 0)
    len1:=len0:=0, j:=Stride-w1*4+4, i:=-j
    ListLines, Off
    Loop, Parse, v
    {
      i:=Mod(A_Index,w1)=1 ? i+j : i+4
      if A_LoopField
        NumPut(i, s1, 4*len1++, "int")
      else
        NumPut(i, s0, 4*len0++, "int")
    }
    ListLines, On
    ;--------------------------------------------
    if (num:=PicFind(mode,color,n,Scan0,Stride,sx,sy,sw,sh
      ,s1,s0,len1,len0,e1,e0,w1,h1,allpos))
      or (err1=0 and err0=0
      and (num:=PicFind(mode,color,n,Scan0,Stride,sx,sy,sw,sh
      ,s1,s0,len1,len0,0.05,0.05,w1,h1,allpos)))
    {
      Loop, % num
        pos:=NumGet(allpos, 4*(A_Index-1), "uint")
        , rx:=(pos&0xFFFF)+x, ry:=(pos>>16)+y
        , arr.Push( [rx,ry,w1,h1,Comment] )
    }
  }
  SetBatchLines, %bch%
  return, arr.MaxIndex() ? arr:0
}

PicFind(mode, color, n, Scan0, Stride
  , sx, sy, sw, sh, ByRef s1, ByRef s0
  , len1, len0, err1, err0, w1, h1, ByRef allpos)
{
  static MyFunc
  if !MyFunc
  {
    x32:="5589E55383EC50C745EC000000008B45242B454483C0018"
    . "945D48B45282B454883C0018945D08B45200FAF45188B551CC"
    . "1E20201D08945CC8B45343945380F4D45388945C8837D08000"
    . "F854D020000C745F000000000E930020000C745F400000000E"
    . "9140200008B453C8945DC8B45408945D88B45F00FAF45188B5"
    . "5F4C1E20201C28B45CC01D08945C48B55C48B450C01D08945C"
    . "08B45C083C00289C28B451401D00FB6000FB6C08945BC8B45C"
    . "083C00189C28B451401D00FB6000FB6C08945B88B55C08B451"
    . "401D00FB6000FB6C08945B4C745F800000000E94C0100008B4"
    . "5F83B45340F8D9A0000008B45F88D1485000000008B452C01D"
    . "08B108B45C401D08945C08B45C083C00289C28B451401D00FB"
    . "6000FB6C02B45BC8945E88B45C083C00189C28B451401D00FB"
    . "6000FB6C02B45B88945E48B55C08B451401D00FB6000FB6C02"
    . "B45B48945E0837DE8007903F75DE8837DE4007903F75DE4837"
    . "DE0007903F75DE08B55E88B45E401C28B45E001D03B45107E0"
    . "E836DDC01837DDC000F88EF0000008B45F83B45380F8D96000"
    . "0008B45F88D1485000000008B453001D08B108B45C401D0894"
    . "5C08B45C083C00289C28B451401D00FB6000FB6C02B45BC894"
    . "5E88B45C083C00189C28B451401D00FB6000FB6C02B45B8894"
    . "5E48B55C08B451401D00FB6000FB6C02B45B48945E0837DE80"
    . "07903F75DE8837DE4007903F75DE4837DE0007903F75DE08B5"
    . "5E88B45E401C28B45E001D03B45107F0A836DD801837DD8007"
    . "8508345F8018B45F83B45C80F8CA8FEFFFF8B45EC8D5001895"
    . "5EC8D1485000000008B454C01D08B4D208B55F001CA89D3C1E"
    . "3108B4D1C8B55F401CA09DA8910817DECFF0300000F8FFE010"
    . "000EB0490EB01908345F4018B45F43B45D40F8CE0FDFFFF834"
    . "5F0018B45F03B45D00F8CC4FDFFFFE9D70100008B450C83C00"
    . "169C0E803000089450CC745F000000000E98D000000C745F40"
    . "0000000EB788B45F00FAF45188B55F4C1E20201C28B45CC01D"
    . "08945C48B45C483C00389C28B451401D08B55C483C20289D18"
    . "B551401CA0FB6120FB6D269CA2B0100008B55C483C20189D38"
    . "B551401DA0FB6120FB6D269D24B0200008D1C118B4DC48B551"
    . "401CA0FB6120FB6D26BD27201DA3B550C0F9CC288108345F40"
    . "18B45F43B45247C808345F0018B45F03B45280F8C67FFFFFF8"
    . "345CC03C745F000000000E901010000C745F400000000E9E50"
    . "000008B453C8945DC8B45408945D88B45F00FAF45188B55F4C"
    . "1E20201C28B45CC01D08945C4C745F800000000EB708B45F83"
    . "B45347D2E8B45F88D1485000000008B452C01D08B108B45C40"
    . "1D089C28B451401D00FB6003C01740A836DDC01837DDC00787"
    . "B8B45F83B45387D2E8B45F88D1485000000008B453001D08B1"
    . "08B45C401D089C28B451401D00FB60084C0740A836DD801837"
    . "DD80078488345F8018B45F83B45C87C888B45EC8D50018955E"
    . "C8D1485000000008B454C01D08B4D208B55F001CA89D3C1E31"
    . "08B4D1C8B55F401CA09DA8910817DECFF0300007F2BEB0490E"
    . "B01908345F4018B45F43B45D40F8C0FFFFFFF8345F0018B45F"
    . "03B45D00F8CF3FEFFFFEB0490EB01908B45EC83C4505B5DC24"
    . "8009090"
    x64:="554889E54883EC50894D10895518448945204C894D28C74"
    . "5F0000000008B45482B858800000083C0018945D88B45502B8"
    . "59000000083C0018945D48B45400FAF45308B5538C1E20201D"
    . "08945D08B45683945700F4D45708945CC837D10000F8585020"
    . "000C745F400000000E968020000C745F800000000E94C02000"
    . "08B45788945E08B85800000008945DC8B45F40FAF45308B55F"
    . "8C1E20201C28B45D001D08945C88B55C88B451801D08945C48"
    . "B45C483C0024863D0488B45284801D00FB6000FB6C08945C08"
    . "B45C483C0014863D0488B45284801D00FB6000FB6C08945BC8"
    . "B45C44863D0488B45284801D00FB6000FB6C08945B8C745FC0"
    . "0000000E96C0100008B45FC3B45680F8DAA0000008B45FC489"
    . "8488D148500000000488B45584801D08B108B45C801D08945C"
    . "48B45C483C0024863D0488B45284801D00FB6000FB6C02B45C"
    . "08945EC8B45C483C0014863D0488B45284801D00FB6000FB6C"
    . "02B45BC8945E88B45C44863D0488B45284801D00FB6000FB6C"
    . "02B45B88945E4837DEC007903F75DEC837DE8007903F75DE88"
    . "37DE4007903F75DE48B55EC8B45E801C28B45E401D03B45207"
    . "E0E836DE001837DE0000F88090100008B45FC3B45700F8DA60"
    . "000008B45FC4898488D148500000000488B45604801D08B108"
    . "B45C801D08945C48B45C483C0024863D0488B45284801D00FB"
    . "6000FB6C02B45C08945EC8B45C483C0014863D0488B4528480"
    . "1D00FB6000FB6C02B45BC8945E88B45C44863D0488B4528480"
    . "1D00FB6000FB6C02B45B88945E4837DEC007903F75DEC837DE"
    . "8007903F75DE8837DE4007903F75DE48B55EC8B45E801C28B4"
    . "5E401D03B45207F0A836DDC01837DDC00785A8345FC018B45F"
    . "C3B45CC0F8C88FEFFFF8B45F08D50018955F04898488D14850"
    . "0000000488B85980000004801D08B4D408B55F401CAC1E2104"
    . "189D08B4D388B55F801CA4409C28910817DF0FF0300000F8F3"
    . "D020000EB0490EB01908345F8018B45F83B45D80F8CA8FDFFF"
    . "F8345F4018B45F43B45D40F8C8CFDFFFFE9160200008B45188"
    . "3C00169C0E8030000894518C745F400000000E9A4000000C74"
    . "5F800000000E9880000008B45F40FAF45308B55F8C1E20201C"
    . "28B45D001D08945C88B45C883C0034863D0488B45284801D08"
    . "B55C883C2024863CA488B55284801CA0FB6120FB6D269CA2B0"
    . "100008B55C883C2014C63C2488B55284C01C20FB6120FB6D26"
    . "9D24B020000448D04118B55C84863CA488B55284801CA0FB61"
    . "20FB6D26BD2724401C23B55180F9CC288108345F8018B45F83"
    . "B45480F8C6CFFFFFF8345F4018B45F43B45500F8C50FFFFFF8"
    . "345D003C745F400000000E929010000C745F800000000E90D0"
    . "100008B45788945E08B85800000008945DC8B45F40FAF45308"
    . "B55F8C1E20201C28B45D001D08945C8C745FC00000000E9840"
    . "000008B45FC3B45687D3A8B45FC4898488D148500000000488"
    . "B45584801D08B108B45C801D04863D0488B45284801D00FB60"
    . "03C01740E836DE001837DE0000F88910000008B45FC3B45707"
    . "D368B45FC4898488D148500000000488B45604801D08B108B4"
    . "5C801D04863D0488B45284801D00FB60084C0740A836DDC018"
    . "37DDC0078568345FC018B45FC3B45CC0F8C70FFFFFF8B45F08"
    . "D50018955F04898488D148500000000488B85980000004801D"
    . "08B4D408B55F401CAC1E2104189D08B4D388B55F801CA4409C"
    . "28910817DF0FF0300007F2BEB0490EB01908345F8018B45F83"
    . "B45D80F8CE7FEFFFF8345F4018B45F43B45D40F8CCBFEFFFFE"
    . "B0490EB01908B45F04883C4505DC39090909090909090"
    MCode(MyFunc, A_PtrSize=8 ? x64:x32)
  }
  return, DllCall(&MyFunc, "int",mode
    , "uint",color, "int",n, "ptr",Scan0, "int",Stride
    , "int",sx, "int",sy, "int",sw, "int",sh
    , "ptr",&s1, "ptr",&s0, "int",len1, "int",len0
    , "int",Round(len1*err1), "int",Round(len0*err0)
    , "int",w1, "int",h1, "ptr",&allpos)
}

xywh2xywh(x1,y1,w1,h1,ByRef x,ByRef y,ByRef w,ByRef h)
{
  SysGet, zx, 76
  SysGet, zy, 77
  SysGet, zw, 78
  SysGet, zh, 79
  left:=x1, right:=x1+w1-1, up:=y1, down:=y1+h1-1
  left:=left<zx ? zx:left, right:=right>zx+zw-1 ? zx+zw-1:right
  up:=up<zy ? zy:up, down:=down>zy+zh-1 ? zy+zh-1:down
  x:=left, y:=up, w:=right-left+1, h:=down-up+1
}

GetBitsFromScreen(x,y,w,h,ByRef Scan0,ByRef Stride,ByRef bits)
{
  VarSetCapacity(bits,w*h*4,0), bpp:=32
  Scan0:=&bits, Stride:=((w*bpp+31)//32)*4
  Ptr:=A_PtrSize ? "UPtr" : "UInt", PtrP:=Ptr . "*"
  win:=DllCall("GetDesktopWindow", Ptr)
  hDC:=DllCall("GetWindowDC", Ptr,win, Ptr)
  mDC:=DllCall("CreateCompatibleDC", Ptr,hDC, Ptr)
  ;-------------------------
  VarSetCapacity(bi, 40, 0), NumPut(40, bi, 0, "int")
  NumPut(w, bi, 4, "int"), NumPut(-h, bi, 8, "int")
  NumPut(1, bi, 12, "short"), NumPut(bpp, bi, 14, "short")
  ;-------------------------
  if hBM:=DllCall("CreateDIBSection", Ptr,mDC, Ptr,&bi
    , "int",0, PtrP,ppvBits, Ptr,0, "int",0, Ptr)
  {
    oBM:=DllCall("SelectObject", Ptr,mDC, Ptr,hBM, Ptr)
    DllCall("BitBlt", Ptr,mDC, "int",0, "int",0, "int",w, "int",h
      , Ptr,hDC, "int",x, "int",y, "uint",0x00CC0020|0x40000000)
    DllCall("RtlMoveMemory", Ptr,Scan0, Ptr,ppvBits, Ptr,Stride*h)
    DllCall("SelectObject", Ptr,mDC, Ptr,oBM)
    DllCall("DeleteObject", Ptr,hBM)
  }
  DllCall("DeleteDC", Ptr,mDC)
  DllCall("ReleaseDC", Ptr,win, Ptr,hDC)
}

MCode(ByRef code, hex)
{
  ListLines, Off
  bch:=A_BatchLines
  SetBatchLines, -1
  VarSetCapacity(code, StrLen(hex)//2)
  Loop, % StrLen(hex)//2
    NumPut("0x" . SubStr(hex,2*A_Index-1,2), code, A_Index-1, "char")
  Ptr:=A_PtrSize ? "UPtr" : "UInt"
  DllCall("VirtualProtect", Ptr,&code, Ptr
    ,VarSetCapacity(code), "uint",0x40, Ptr . "*",0)
  SetBatchLines, %bch%
  ListLines, On
}

base64tobit(s)
{
  ListLines, Off
  Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    . "abcdefghijklmnopqrstuvwxyz"
  SetFormat, IntegerFast, d
  StringCaseSense, On
  Loop, Parse, Chars
  {
    i:=A_Index-1, v:=(i>>5&1) . (i>>4&1)
      . (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
    s:=StrReplace(s,A_LoopField,v)
  }
  StringCaseSense, Off
  s:=SubStr(s,1,InStr(s,"1",0,0)-1)
  s:=RegExReplace(s,"[^01]+")
  ListLines, On
  return, s
}

bit2base64(s)
{
  ListLines, Off
  s:=RegExReplace(s,"[^01]+")
  s.=SubStr("100000",1,6-Mod(StrLen(s),6))
  s:=RegExReplace(s,".{6}","|$0")
  Chars:="0123456789+/ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    . "abcdefghijklmnopqrstuvwxyz"
  SetFormat, IntegerFast, d
  Loop, Parse, Chars
  {
    i:=A_Index-1, v:="|" . (i>>5&1) . (i>>4&1)
      . (i>>3&1) . (i>>2&1) . (i>>1&1) . (i&1)
    s:=StrReplace(s,v,A_LoopField)
  }
  ListLines, On
  return, s
}

ASCII(s)
{
  if RegExMatch(s,"(\d+)\.([\w+/]{3,})",r)
  {
    s:=RegExReplace(base64tobit(r2),".{" r1 "}","$0`n")
    s:=StrReplace(StrReplace(s,"0","_"),"1","0")
  }
  else s=
  return, s
}

; You can put the text library at the beginning of the script,
; and Use Pic(Text,1) to add the text library to Pic()'s Lib,
; Use Pic("comment1|comment2|...") to get text images from Lib

Pic(comments, add_to_Lib=0) {
  static Lib:=[]
  if (add_to_Lib)
  {
    re:="<([^>]*)>[^$]+\$\d+\.[\w+/]{3,}"
    Loop, Parse, comments, |
      if RegExMatch(A_LoopField,re,r)
        Lib[Trim(r1)]:=r
  }
  else
  {
    text:=""
    Loop, Parse, comments, |
      text.="|" . Lib[Trim(A_LoopField)]
    return, text
  }
}

FindTextOCR(nX, nY, nW, nH, err1, err0, Text, Interval=8) {
  OCR:="", Right_X:=nX+nW
  While (ok:=FindText(nX, nY, nW, nH, err1, err0, Text))
  {
    ; For multi text search, This is the number of text images found
    Loop, % ok.MaxIndex()
    {
      ; X is the X coordinates of the upper left corner
      ; and W is the width of the image have been found
      i:=A_Index, x:=ok[i].1, y:=ok[i].2
        , w:=ok[i].3, h:=ok[i].4, comment:=ok[i].5
      ; We need the leftmost X coordinates
      if (A_Index=1 or x<Left_X)
        Left_X:=x, Left_W:=w, Left_OCR:=comment
    }
    ; If the interval exceeds the set value, add "*" to the result
    OCR.=(A_Index>1 and Left_X-Last_X>Interval ? "*":"") . Left_OCR
    ; Update nX and nW for next search
    x:=Left_X+Left_W-2, nW:=(Right_X-x)//2, nX:=x+nW, Last_X:=x
  }
  Return, OCR
}


/***** C source code of machine code *****

int __attribute__((__stdcall__)) PicFind(int mode
  , int c, int n, unsigned char * Bmp
  , int Stride, int sx, int sy, int sw, int sh
  , int * s1, int * s0, int len1, int len0
  , int err1, int err0, int w1, int h1, int * allpos)
{
  int o, i, j, k, x, y, w, h, num=0;
  int r, g, b, rr, gg, bb, e1, e0, max;
  w=sw-w1+1; h=sh-h1+1; k=sy*Stride+sx*4;
  max=len1>len0 ? len1 : len0;
  if (mode==0)  // Color Mode
  {
    for (y=0; y<h; y++)
    {
      for (x=0; x<w; x++)
      {
        e1=err1; e0=err0; o=y*Stride+x*4+k;
        j=o+c; rr=Bmp[2+j]; gg=Bmp[1+j]; bb=Bmp[j];
        for (i=0; i<max; i++)
        {
          if (i<len1)
          {
            j=o+s1[i]; r=Bmp[2+j]-rr; g=Bmp[1+j]-gg; b=Bmp[j]-bb;
            if (r<0) r=-r; if (g<0) g=-g; if (b<0) b=-b;
            if (r+g+b>n && (--e1)<0) goto NoMatch1;
          }
          if (i<len0)
          {
            j=o+s0[i]; r=Bmp[2+j]-rr; g=Bmp[1+j]-gg; b=Bmp[j]-bb;
            if (r<0) r=-r; if (g<0) g=-g; if (b<0) b=-b;
            if (r+g+b<=n && (--e0)<0) goto NoMatch1;
          }
        }
        allpos[num++]=(sy+y)<<16|(sx+x);
        if (num>=1024) goto Return1;
        NoMatch1:
        continue;
      }
    }
    goto Return1;
  }
  // Gray Threshold Mode
  c=(c+1)*1000;
  for (y=0; y<sh; y++)
  {
    for (x=0; x<sw; x++)
    {
      o=y*Stride+x*4+k;
      Bmp[3+o]=Bmp[2+o]*299+Bmp[1+o]*587+Bmp[o]*114<c ? 1:0;
    }
  }
  k=k+3;
  for (y=0; y<h; y++)
  {
    for (x=0; x<w; x++)
    {
      e1=err1; e0=err0; o=y*Stride+x*4+k;
      for (i=0; i<max; i++)
      {
        if (i<len1 && Bmp[o+s1[i]]!=1 && (--e1)<0) goto NoMatch2;
        if (i<len0 && Bmp[o+s0[i]]!=0 && (--e0)<0) goto NoMatch2;
      }
      allpos[num++]=(sy+y)<<16|(sx+x);
      if (num>=1024) goto Return1;
      NoMatch2:
      continue;
    }
  }
  Return1:
  return num;
}

*/

;================= The End =================

;
How can I tell if it found it? (If its true/false)
leosouza85
Posts: 34
Joined: 22 Jul 2016, 16:28

Re: FindText - Capture screen image into text and then find it

25 Nov 2018, 06:36

You are using it wrong.

The text variable is set by you, the script will not automatically set the text variable.

If you know how to interpret ahk code you can use the functions of the script to get an automatic snap and then search for it, it is all what I can say, it will take so much effort to explain further, and I think your conception of what the script is meant for is wrong since the beginning.

Also, you don't use ":=" in a if statement to compare values, you use "=="

The image that you are trying to search, you dont know in advance how it will look like?
ahkcrazy
Posts: 4
Joined: 26 Feb 2018, 02:19

Re: FindText - Capture screen image into text and then find it

25 Nov 2018, 14:42

leosouza85 wrote:
25 Nov 2018, 06:36
You are using it wrong.

Clearly, otherwise I wouldn't be here asking questions.

The text variable is set by you, the script will not automatically set the text variable.

So your saying <auto>? This was generated by the script.

If you know how to interpret ahk code you can use the functions of the script to get an automatic snap and then search for it, it is all what I can say, it will take so much effort to explain further, and I think your conception of what the script is meant for is wrong since the beginning.

Also, you don't use ":=" in a if statement to compare values, you use "=="

I'm new to ahk so if I added something in the wrong spot (that's why I added my code)but the Text:=<auto> part was copied from the code of this script. I tried many. An example would be helpful.

The image that you are trying to search, you dont know in advance how it will look like?

Of course I do. How do you think <auto> was generated? From the FindText script that generated the code from the image.

d582693456
Posts: 4
Joined: 25 Nov 2018, 20:32
Facebook: d582693456
Google: [email protected]
GitHub: d582693456

Re: FindText - Capture screen image into text and then find it

25 Nov 2018, 22:50

PrintScreen
Last edited by d582693456 on 30 Nov 2018, 03:32, edited 1 time in total.
feiyue
Posts: 162
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

26 Nov 2018, 03:25

@ahkcrazy, You can do it like this:

Code: Select all


Loop
{

Text:="|<Auto>-*59$62.000000000003k000000001y0000A0000TU000700007w0003k0003z0000w0000zk3UQTsDk0SS0w7by7z07bUD1tzXzs1tw3kSDVwT0wD0w7XkS3kD3kD1swD0y7ky3kSD3k7VzzUw7Xkw1sTzwD1swD0SDzz3kyD3sDXk3kwDXkS3lw0yDbszbvwS07Xzy7szy7U1wTrVy7z00001k070C000000000008"

if (ok:=FindText(211, 414, 150000, 150000, 0, 0, Text))
{
Send {Numpad2}
}else{
Send {Numpad1}
}

sleep, 1000
}

;;pause script
Pause::Pause

leosouza85
Posts: 34
Joined: 22 Jul 2016, 16:28

Re: FindText - Capture screen image into text and then find it

26 Nov 2018, 04:15

ahkcrazy wrote:
25 Nov 2018, 14:42
leosouza85 wrote:
25 Nov 2018, 06:36
You are using it wrong.

Clearly, otherwise I wouldn't be here asking questions.

The text variable is set by you, the script will not automatically set the text variable.

So your saying <auto>? This was generated by the script.

If you know how to interpret ahk code you can use the functions of the script to get an automatic snap and then search for it, it is all what I can say, it will take so much effort to explain further, and I think your conception of what the script is meant for is wrong since the beginning.

Also, you don't use ":=" in a if statement to compare values, you use "=="

I'm new to ahk so if I added something in the wrong spot (that's why I added my code)but the Text:=<auto> part was copied from the code of this script. I tried many. An example would be helpful.

The image that you are trying to search, you dont know in advance how it will look like?

Of course I do. How do you think <auto> was generated? From the FindText script that generated the code from the image.

Text:="|<Auto>-*59$62.000000000003k000000001y0000A0000TU000700007w0003k0003z0000w0000zk3UQTsDk0SS0w7by7z07bUD1tzXzs1tw3kSDVwT0wD0w7XkS3kD3kD1swD0y7ky3kSD3k7VzzUw7Xkw1sTzwD1swD0SDzz3kyD3sDXk3kwDXkS3lw0yDbszbvwS07Xzy7szy7U1wTrVy7z00001k070C000000000008"
Loop
{
if (ok:=FindText(211, 414, 150000, 150000, 0, 0, Text))
{
Send {Numpad2}
}
else
{
Send {Numpad1}
}
sleep, 1000
}
EvetsNalon
Posts: 11
Joined: 25 Aug 2018, 17:55

Re: FindText - Capture screen image into text and then find it

13 Dec 2018, 19:52

Hello, and Thank you so much for creating and sharing this absolutely awesome script/function!!! I love it and am using already....

I do have a couple of questions regarding clarification of some of this.. Any assistance would be greatly appreciated....

I am trying to get the response or "find" time as quick as possible. With that in mind, I must ask the following..

1- I am confused a bit regarding the comments and post regarding MCode .. Would Compiling the script make it much faster, and if so, how can I do that with this MCode or "mingw64-gcc"... ??

2-Also, after playing with this for a good bit and even reading through all of the comments and explanations, I unfortunately am still not grasping how to adjust the "Area" in which the Search for the Text will be conducted.. I assume that of course if I only need to search in a portion of the window, that it would be far faster at finding the text than it is scanning the whole window or screen.

That makes sense of course, but I still cannot figure out how to adjust that Search Area with the sample code given..

My output from Capture and Trim section:

Code: Select all


	if (ok:=FindText(797-150000//2, 433-150000//2, 150000, 150000, 0, 0, Text))
		
	{
		CoordMode, Mouse
		X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
		Click, %X%, %Y%
		Send {Enter}
		Pause
		
	}
Any assistance with this would be greatly appreciated...

My particular need is for the function is to Only search the Left approximately 1/3rd of the screen or window...

Example:
SearchArea.png
SearchArea.png (12.1 KiB) Viewed 2128 times
Thanks in advance for any assistance and help. It would greatly assist in my endeavors and work progress... :)

Peace!
feiyue
Posts: 162
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

14 Dec 2018, 10:48

Hello, EvetsNalon, here is my reply: :D

1. MCode is a function written in C language code,
then compiled by Gcc installed in mingw64-gcc,
and then extracted the machine code.
Friends discuss this issue mainly not to modify C language code themselves,
but to manually try to compile MCode, so as to compare the MCode I provide,
make sure my code doesn't add code that could damage the computer.
Of course, if they are not sure they can use their own compiled Code.

The following is my compiler switch:
"C:\TDM-GCC-64\bin\gcc.exe -m64(or -m32) -O2"

2. The first four parameters of FinText(x, y, w, h, ...)
should be explained clearly in the introduction:

;===========================================
; Introduction of function parameters:
;
; returnArray := FindText(
; X --> the search scope's upper left corner X coordinates
; , Y --> the search scope's upper left corner Y coordinates
; , W --> the search scope's Width
; , H --> the search scope's Height
; , Character --> "0" fault-tolerant in percentage --> 0.1=10%
; , Character --> "_" fault-tolerant in percentage --> 0.1=10%
; , text --> The Base64 encoding string for the text to find
; , ScreenShot --> if the value is 0, the last screenshot will be used
; )


If you want to search the full screen, you can use:
FindText(0, 0, A_ScreenWidth, A_ScreenHeight, ...)

If you want to search for a window, you can use:
WinGetPos, x, y, w, H
FindText(x, y, w, h, ...)

Maybe the automatically generated code is a little strange,
because I like to locate the center of the search scope first,
then calculate a width and height to determine the range.

Others should first locate in the upper left corner of the search scope,
then calculate a width and height to determine the range.

I'm sorry that my personal hobbies have confused users. :headwall: :xmas:
EvetsNalon
Posts: 11
Joined: 25 Aug 2018, 17:55

Re: FindText - Capture screen image into text and then find it

15 Dec 2018, 15:33

feiyue wrote:
14 Dec 2018, 10:48
Hello, EvetsNalon, here is my reply: :D

1. MCode is a function written in C language code,
then compiled by Gcc installed in mingw64-gcc,
and then extracted the machine code.
Friends discuss this issue mainly not to modify C language code themselves,
but to manually try to compile MCode, so as to compare the MCode I provide,
make sure my code doesn't add code that could damage the computer.
Of course, if they are not sure they can use their own compiled Code.

The following is my compiler switch:
"C:\TDM-GCC-64\bin\gcc.exe -m64(or -m32) -O2"

2. The first four parameters of FinText(x, y, w, h, ...)
should be explained clearly in the introduction:

;===========================================
; Introduction of function parameters:
;
; returnArray := FindText(
; X --> the search scope's upper left corner X coordinates
; , Y --> the search scope's upper left corner Y coordinates
; , W --> the search scope's Width
; , H --> the search scope's Height
; , Character --> "0" fault-tolerant in percentage --> 0.1=10%
; , Character --> "_" fault-tolerant in percentage --> 0.1=10%
; , text --> The Base64 encoding string for the text to find
; , ScreenShot --> if the value is 0, the last screenshot will be used
; )


If you want to search the full screen, you can use:
FindText(0, 0, A_ScreenWidth, A_ScreenHeight, ...)

If you want to search for a window, you can use:
WinGetPos, x, y, w, H
FindText(x, y, w, h, ...)

Maybe the automatically generated code is a little strange,
because I like to locate the center of the search scope first,
then calculate a width and height to determine the range.

Others should first locate in the upper left corner of the search scope,
then calculate a width and height to determine the range.

I'm sorry that my personal hobbies have confused users. :headwall: :xmas:




Let me first start out by saying THANK YOU "feiyue" again for this fantastic and well written function.... I cannot say enough about that.. Also... Do Not Apologize about anything.. I think more than anything that it is a language barrier issue and others who have done their level best to interpret(Very Well I Would Say), your Code and thoughts on the function being discussed.. But IN NO WAY, should you be sorry about anything at all. Your efforts are much appreciated by myself and many I am sure..

Your way of thinking is a bit different than most of us, but it does not make it wrong or in need of any apology.. Heck, We may be the ones not thinking the way we should.. LOL.. Either way it is all good, and your recent reply to my questions gave me the understanding I was looking for.. I guess all of our own personalities and brains work in different ways and some of us.. (me< included) just do not grasp things as quick when they do not match what we have previously done.. It's all good.. Thanks again, and I cannot say that enough..

I still have not figured out the Compiling in C with the (compiler switch:
"C:\TDM-GCC-64\bin\gcc.exe -m64(or -m32) -O2"
))...... But eventually I will.... LOL.....

I will continue to investigate and attempt to consume the knowledge you have posted here:
Note to English Users: Use "Google Translate"

https://autohotkey.com/boards/viewtopic ... 28&t=18405


Anyhow,

I am working on a couple of refinements to the great explanations of your functions that both "ed1chandler", and "c4p" have already done so well. Maybe It will help those like me..


Thanks again for your great work, and please let us know as you come up with other great ideas and scripts...!!

Talk Soon..

Thanks again!


With all that said in English.. Here is the best I can do in your native language... Thanks again..

让我首先开始说再次谢谢你“飞跃”这个奇妙而精心编写的功能......我不能说清楚......还有......不要为任何事情道歉......我想的不仅仅是它是一个语言障碍问题,其他人已经完成了最好的解释(很好,我会说),你的代码和正在讨论的功能的想法..但不,你不应该对任何事情感到抱歉。你的努力非常受我自己的赞赏,我相信很多人。

你的思维方式与我们大多数人有点不同,但它并没有使它错误或需要任何道歉......哎呀,我们可能是那些没有想到我们应该这样做的人..大声笑..无论哪种方式,它都是一切都很好,你最近对我的问题的回答给了我理解我正在寻找的东西..我猜我们所有的个性和大脑都以不同的方式工作,我们中的一些人......(我<包括在内)只是不把握事情当他们不符合我们之前所做的那样时很快..这一切都很好......再次感谢,我不能说够了..

我仍然没有想到用C编译器(编译器开关:
“C:\ TDM-GCC-64 \ bin \ gcc.exe -m64(或-m32)-O2”
))......但最终我会.... LOL .....

 我将继续调查并尝试使用您在此处发布的知识:
英文用户须知:使用“谷歌翻译”

https://autohotkey.com/boards/viewtopic ... 28&t=18405


无论如何,

我正在对你的函数的重要解释进行一些改进,“ed1chandler”和“c4p”都已经做得很好。也许它会帮助像我这样的人..


再次感谢您的出色工作,请在我们提出其他伟大的想法和脚本时告诉我们...... !!

一会再聊..

再次感谢!

TalkSoon...
:D
itz4sking
Posts: 8
Joined: 17 Dec 2018, 12:33

Re: FindText - Capture screen image into text and then find it

17 Dec 2018, 12:39

For some reason pressing the RButton doesn't work when attempting to capture an image. The red box just stays there waiting for me to start the image creation.
itz4sking
Posts: 8
Joined: 17 Dec 2018, 12:33

Re: FindText - Capture screen image into text and then find it

17 Dec 2018, 13:10

Does anyone have an older version of the script? I'd like to see if I can get one of those to work. Thanks
itz4sking
Posts: 8
Joined: 17 Dec 2018, 12:33

Re: FindText - Capture screen image into text and then find it

18 Dec 2018, 07:58

I've actually figured out my problem. For some reason GetKeyState() doesn't work through teamviewer.... anyone have a work around?
feiyue
Posts: 162
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

18 Dec 2018, 12:52

itz4sking wrote: I've actually figured out my problem. For some reason GetKeyState() doesn't work through teamviewer.... anyone have a work around?
@itz4sking, I also encountered this problem when I was remotely controlling the computer.
Now I have upgraded the script to solve this problem. :dance:

Updated to 6.3 version - 2018/12/18
.... 1. Changed the way to capture, Now first click the right mouse button,
.... you can move the mouse, and then click the right mouse button again.
.... 2. Add MouseTip(). It is used to prompt mouse position in remote assistance.

Return to “Scripts and Functions”

Who is online

Users browsing this forum: robodesign and 31 guests