FindText - Capture screen image into text and then find it

Post your working scripts, libraries and tools for AHK v1.1 and older
sigartrader
Posts: 9
Joined: 01 Feb 2018, 20:34

Re: FindText - Capture screen image into text and then find it  Topic is solved

09 Feb 2018, 14:27

Code: Select all

t1:=A_TickCount

Text:="|<Up Arrow>*70$11.yTsTUS0M0004MMllrXz7yDwT"

if (ok:=FindText(1099, 559, 150000, 150000, 0, 0, Text))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
}

; MsgBox, 4096,, % "Time:`t" (A_TickCount-t1) " ms`n`n"
;  . "Pos:`t" X ", " Y "`n`n"
;  . "Result:`t" (ok ? "Up Arrow Found !":"Up Arrow Not Found !"), 3
MouseMove, X, Y
Click, 822, 400
sleep 3000
Click, 1135, 339
return 

on this part
; MsgBox, 4096,, % "Time:`t" (A_TickCount-t1) " ms`n`n"
; . "Pos:`t" X ", " Y "`n`n"
; . "Result:`t" (ok ? "Up Arrow Found !":"Up Arrow Not Found !"), 3
MouseMove, X, Y
Click, 822, 400
sleep 3000
Click, 1135, 339
return

i try to eliminate message box, then after text/image is found, script will right click mouse at Click, 822, 400 then sleep 3 seconds then right cick again at Click, 1135, 339

is there something wrong, need help please, thanks
pedroabs
Posts: 9
Joined: 04 Oct 2017, 23:09

Re: FindText - Capture screen image into text and then find it

14 Feb 2018, 03:28

I am new to coding but I think this should be the code that you need.




t1:=A_TickCount

Text:="|<Up Arrow>*70$11.yTsTUS0M0004MMllrXz7yDwT"

if (ok:=FindText(1099, 559, 150000, 150000, 0, 0, Text))
{
CoordMode, Mouse
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
; Click, %X%, %Y%
Click, right, 822, 400
sleep 3000
Click, right, 1135, 339
return
}

; MsgBox, 4096,, % "Time:`t" (A_TickCount-t1) " ms`n`n"
; . "Pos:`t" X ", " Y "`n`n"
; . "Result:`t" (ok ? "Up Arrow Found !":"Up Arrow Not Found !"), 3
MouseMove, X, Y
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

16 Feb 2018, 16:36

Updated to the v5.7 version, I changed the way to capture,
Before, click the left button of the mouse, and then move the mouse out of 100 pixels.
Now, press down the right button of the mouse, and then move a certain distance and release the right button.
In addition, I updated the "Capture Image To Text" window,
allowing the use of tags to split the text at one time and generate multiple text libraries.
garry
Posts: 3720
Joined: 22 Dec 2013, 12:50

Re: FindText - Capture screen image into text and then find it

28 Feb 2018, 14:52

@feiyue, thank you for your script

also this , your ahk-script :
AHK-Quellcode Encryptor v1.5 , url =
https://autohotkey.com/boards/viewtopic ... 28&t=42494

this is an encoded example ( contains a gui with a text and button to start url )
save this script as xy.ahk and run

Code: Select all

#NoEnv
#NoTrayIcon
#SingleInstance force
s=
s.="u4105957551u1664783769u302272433u3617706743u2723639953u42613"
s.="82992u236786073u11533420u2257073979u3688155385u2515987797u33"
s.="20702031u3200729149u3033717368u673603469u2160101244u18611295"
s.="43u3086375289u3381628729u800863623u1193713769u3556266240u157"
s.="1479361u3074389548u2178282451u2299034137u2474702685u39635559"
s.="99u3951100437u2570528232u2463202485u2941174652u2872522463u18"
s.="33421529u1459084225u776461975u2705591617u1203191856u35549119"
s.="77u29985388u2531261803u2646340537u2022654309u609936751u24898"
s.="45485u2634088664u4032058589u3462492924u1903244407u429737657u"
s.="3943192393u2578624551u2549548313u1180245984u2922638225u14180"
s.="75948u2188170499u1220665305u2910794605u1738894015u1940085701"
s.="u2099056712u1678595589u2154929404u4202758159u2149057049u3333"
s.="533137u3964547639u3460124913u660157456u3824568121u118734188u"
s.="73880731u1472565113u435784565u1996641679u2529076637u10109681"
s.="20u3300970541u1748587900u3044938663u1660411129u2023411289u28"
s.="58892999u4004158921u1892110272u2133149665u1707396396u2725935"
s.="667u1861517209u1527072381u1437480927u1871525237u669500584u35"
s.="19734869u1278630012u2974846783u243236441u4138586081u21452379"
s.="75u1833555105u3573551088u2886324361u963021676u431221195u1847"
s.="644217u638985861u2123617455u2802340685u3367904920u961407101u"
s.="397824252u1297296599u2304636217u3853237353u1966964839u215541"
s.="3881u2179421344u1944896305u931738924u4163100771u3631255129u2"
s.="384232077u4280663295u1227800357u3702071048u1801681573u895285"
s.="500u4126249327u2392301977u1304000241u3277254007u3419416401u3"
s.="181196240u717548249u205884268u3380255995u897650681u145261915"
s.="7u1018587855u3588557053u4080320504u2571923149u2804650108u273"
s.="9669511u602795641u284171897u424888327u2197402665u1438118528u"
s.="1005030017u126436396u833472659u2786130201u3030109853u5755384"
s.="63u1591145941u2173583464u27268085u135011964u785524895u346904"
s.="3417u2599389953u2320606231u1368897793u1912214448u1622915369u"
s.="3654995308u277967147u1613048761u2063026085u1611091695u326028"
s.="2541u3892117848u3346912541u3067453692u2095482935u2995774137u"
s.="3798749321u2726048423u3810674905u3700300896u1315202513u38191"
s.="49356u3598522563u1622505945u2485922989u3856807743u1043573381"
s.="u937049032u884678725u1624649980u3833523919u78859033u42067563"
s.="69u4137689783u1841766833u1001990288u4067714425u939024748u196"
s.="1601627u349242489u3431772085u3172894991u3920852573u318816914"
s.="4u68426605u299441276u3653270375u1173548025u1953823641u893811"
s.="271u3326745737u3004719680u1000256545u1121883180u387462899u33"
s.="09848729u3628558269u4111763551u3428131125u1961434152u1482553"
s.="749u2142394748u2325991167u2773783385u2598499361u340411991u11"
s.="22363489u961780592u2590284745u3961054828u346010507u341958072"
s.="9u3799421125u3716892719"

Ahk=
Exec(s, Ahk)
ExitApp

Exec(str, Ahk="", arg="") {
  static WriteFile, f1, f2, f3, f4, f5
  s:=RegExReplace(str,"\s"), StrReplace(s,"u","",size)
  VarSetCapacity(str,(size+1)*4,0), s:=Trim(s,"u")
  Loop, Parse, s, u
    NumPut(A_LoopField,str,(A_Index-1)*4,"uint")
  ;-----------------------------
  Ptr:=A_PtrSize ? "UPtr":"UInt"
  Ahk:=Ahk ? Ahk : A_IsCompiled  ? A_ScriptDir "\AutoHotkey.exe" : A_AhkPath
  IfNotExist, %Ahk%
  {
    MsgBox, 4096, Error!, `n`nCan't Find: %Ahk% !`n`n
    return, 0
  }
  name:="\\.\pipe\AHK" . A_TickCount
  Loop, 2
    if (p%A_Index% := DllCall("CreateNamedPipe","str",name,"uint",2
    ,"uint",0,"uint",255,"uint",0,"uint",0,Ptr,0,Ptr,0))=-1
      return, 0
  EnvSet, AhkPath, %A_ScriptFullPath%
  Run, %Ahk% "%name%" %arg%
  DllCall("ConnectNamedPipe",Ptr,p1,Ptr,0)
  DllCall("CloseHandle",Ptr,p1)
  DllCall("ConnectNamedPipe",Ptr,p2,Ptr,0)
  if !WriteFile
  {
    x32:="5589E583EC58C744240800000000C744240400100000C70"
    . "424000000008B450CFFD08945F08B452483C001C1E00289442"
    . "408C7442404000000008B45F08904248B4510FFD08945ECC74"
    . "5D407000000C745D80D000000C745DC11000000C745E013000"
    . "000C745F400000000EB268B45F483E0038945E88B45E88B448"
    . "5D469D0830000008B45F401C28B45E8895485D48345F401837"
    . "DF46376D4C745F400000000EB5A8B45F483E0038945E88B45E"
    . "88B4485D469D0830000008B45F401C28B45E8895485D48B45F"
    . "48D1485000000008B452001D08B108B45E88B4485D431D0894"
    . "5E48B45F48D1485000000008B45EC01D08B55E4C1CA0889108"
    . "345F4018B45F43B4524729E8B45F48D1485000000008B45EC0"
    . "1D0C700000000008B452483C0018D148500000000C74424100"
    . "00000008B45288944240C895424088B45EC894424048B451C8"
    . "904248B4508FFD08B45EC89442408C7442404000000008B45F"
    . "08904248B4514FFD08B45F08904248B4518FFD0B800000000C"
    . "9C22400909090"
    x64:="554889E54883EC6048894D10488955184C8945204C894D2"
    . "8488B451841B800000000BA00100000B900000000FFD048894"
    . "5F08B454883C0018D148500000000488B4DF0488B45204189D"
    . "0BA00000000FFD0488945E8C745D007000000C745D40D00000"
    . "0C745D811000000C745DC13000000C745FC00000000EB268B4"
    . "5FC83E0038945E48B45E48B4485D069D0830000008B45FC01C"
    . "28B45E4895485D08345FC01837DFC6376D4C745FC00000000E"
    . "B608B45FC83E0038945E48B45E48B4485D069D0830000008B4"
    . "5FC01C28B45E4895485D08B45FC488D148500000000488B454"
    . "04801D08B108B45E48B4485D031D08945E08B45FC488D14850"
    . "0000000488B45E84801D08B55E0C1CA0889108345FC018B45F"
    . "C3B454872988B45FC488D148500000000488B45E84801D0C70"
    . "0000000008B454883C001448D1485000000004C8B4550488B5"
    . "5E8488B4D3848C744242000000000488B45104D89C14589D0F"
    . "FD0488B55E8488B4DF0488B45284989D0BA00000000FFD0488"
    . "B55F0488B45304889D1FFD0B8000000004883C4605DC39090"
    hex:=A_PtrSize=8 ? x64:x32
    VarSetCapacity(WriteFile, len:=StrLen(hex)//2)
    Loop, % len
      NumPut("0x" SubStr(hex,2*A_Index-1,2),WriteFile,A_Index-1,"char")
    DllCall("VirtualProtect",Ptr,&WriteFile,Ptr,len,"uint",0x40,Ptr "*",0)
    kernel32:=DllCall("GetModuleHandle", "Str","kernel32", Ptr)
    f=WriteFile,HeapCreate,HeapAlloc,HeapFree,HeapDestroy
    For i,v in StrSplit(f, ",")
      f%i%:=DllCall("GetProcAddress", Ptr,kernel32, "AStr",v, Ptr)
  }
  DllCall(&WriteFile, Ptr,f1, Ptr,f2, Ptr,f3, Ptr,f4, Ptr,f5
  , Ptr,p2, Ptr,&str, "uint",size, "uint*",0)
  DllCall("CloseHandle",Ptr,p2)
  return, 1
}
guest2

Re: FindText - Capture screen image into text and then find it

25 Mar 2018, 08:38

Great script. It takes about 60ms to find an image.

Now suppose I want to write a script to attach a note a specific image out of anyone of 200 images , to be run every half second.

Any comments from readers about best way? example run 200 scripts simultaneous etc
User avatar
ivill
Posts: 124
Joined: 13 May 2016, 02:23

Re: FindText - Capture screen image into text and then find it

11 Apr 2018, 00:07

Hi, FeiYue, i have a question, your script works great, except the different displayer issue, can you maybe make a fuzzy match setting at the next update?
here is my issue: the same image/text to find on different displayer will not work as the same, let's say i captured this on PC1:
__00000__________00_________________0________00000___________00_____
_0____00_________00_________________0_______00___00__________00_____
00____00___0000__00___0000___0000__000_____00_____00__00000_0000__00
_00_______00__00_00__00__00__0___0__0______00________00___00_00__00_
_00000___00____0_00_00____0_00______0______00_____________00_00__0__
____0000_0000000_00_0000000_00______0______00__________00000_00_0000
_______0_00______00_00______00______0______00________000__00_00_00__
00_____0_00______00_00______00______0______00_____00_00___00_00__0__
_00___00__00__00_00__00__00__0___0__0_______00___00__00__000_00__00_
__00000____0000__00___0000___00000__00_______00000____000000__00__00

it will not work on the PC2(with different displayer but the same resolution 1440x900), if i capture the image on PC2, it looks like this:
_00000___________0_________________0_________0000____________0______
0_____0__________0_________________0________0____0___________0______
0_____0___0000___0___0000____000__0000_____0______0___0000__0000__00
0________0____0__0__0____0__0___0__0_______0_________0____0__0___0__
_000_____0____0__0__0____0__0______0_______0______________0__0___0__
____000__000000__0__000000__0______0_______0___________0000__0___000
______0__0_______0__0_______0______0_______0__________0___0__0___0__
0_____0__0_______0__0_______0______0_______0______0__0____0__0___0__
0_____0__0____0__0__0____0__0___0__0________0____0___0___00__0___0__
_00000____0000___0___0000____000___000_______0000_____000_0__000__00

as you can see, the second one got a smaller size, i think it's the problem? or maybe i was misunderstanding about using the script?
thanks! FeiYue, and everybody in the forum.
guest3456
Posts: 3453
Joined: 09 Oct 2013, 10:31

Re: FindText - Capture screen image into text and then find it

11 Apr 2018, 07:24

ivill wrote:your script works great, except the different displayer issue
what does different "displayer" mean? obviously you would need to make sure everything is exactly the same on the other PC, otherwise you cant guarnatee that the graphics will be the same and therefore the imagesearch will not match. that text looks different because maybe the scaling % is different, or maybe you have ClearType fonts or something to make the fonts more smooth. either way i would definitely not expect this to work across different computers with different display settings.

Twisted_J
Posts: 1
Joined: 19 Apr 2018, 00:39

Re: FindText - Capture screen image into text and then find it

19 Apr 2018, 01:00

Greetings everyone,

I am very new to AHK, but have dabbled with coding from time to time. My question I have is how can I get this to pass back TRUE or FALSE?

I have managed to get the script to find the small area I am looking for, but now I would just like to pass a TRUE statement that it found the text.

Code: Select all

if ((ok:=FindText(448, 605, 100, 100, 0, 0, "|Text"))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
}
Bascially I am looking to have it confirm that text is populating as the program scrolls text down the window (DISCORD Server Chat window). What I am trying to do is when it matches the text and I get a valid hit, I would do something like:

Code: Select all

If (FindText:= VarName)
	{
	send, Hello %VarName% {enter}
	}
Thank you for any assistance.
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

20 Apr 2018, 10:26

I think you want to cycle check the text in the chat window to automatically reply.
This can be achieved in this way:

Code: Select all


pic1:="|<send1>..."
pic2:="|<send2>..."
pic3:="|<send3>..."
send1:="Hello Everyone!"
send2:="Go Go Go!"
send3:="You're Too Bad!"
AllText:=pic1 . pic2 . pic3
SetTimer, CheckChatWindow, 1000
return

CheckChatWindow:
if (ok:=FindText(448, 605, 100, 100, 0, 0, AllText))
{
  CoordMode, Mouse
  X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
  ; Click, %X%, %Y%
  VarName:=%Comment%
  Send, %VarName%{Enter}
}
return

Last edited by feiyue on 22 Apr 2018, 23:55, edited 1 time in total.
Cantus

Re: FindText - Capture screen image into text and then find it

22 Apr 2018, 22:13

I've spent hours looking at examples, and i'm totally lost.

I can run the original script that will detect and move mouse to the chosen image/text. I recognize how to remove the popup just before the mouse is automatically moved to the position of the found text/image.

My problem is creating a script that will run continuously, clicking on a default image every ~5 seconds. But, with a higher priority, the script should check for another image/text that is rare (not always visible), but should be clicked first, if it exists on screen.

Can anyone give guidance or help me in creating such a script?
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

22 Apr 2018, 23:47

I think you can do it like this :

Code: Select all


#NoEnv
#Persistent
CoordMode, Mouse

pic1:="|<default>..."
pic2:="|<higher1>..."
pic3:="|<higher2>..."
pic4:="|<higher3>..."
AllText:=pic1 . pic2 . pic3 . pic4
SetTimer, Check, 5000
return

Check:
if (ok:=FindText(448, 605, 100, 100, 0, 0, AllText))
{
  ;-- higher images
  For i,v in ok
  if InStr(ok[i].5, "higher")
  {
    X:=ok[i].1, Y:=ok[i].2, W:=ok[i].3, H:=ok[i].4, X+=W//2, Y+=H//2
    Click, %X%, %Y%
    Sleep, 1000
    Goto, %A_ThisLabel%
  }
  ;-- default image
  X:=ok[1].1, Y:=ok[1].2, W:=ok[1].3, H:=ok[1].4, X+=W//2, Y+=H//2
  Click, %X%, %Y%
}
return

brutus_skywalker
Posts: 175
Joined: 24 Dec 2016, 13:16
Location: Antarctica

Re: FindText - Capture screen image into text and then find it

24 Apr 2018, 23:48

Sup feiyue, could you possibly post the MCode generator you used, I suspect it was McodeGenHelper.ahk, but I can't find any links to that script as it was on the old forums... Thx.

Amazin'Script BTW,my goTo for image/text search, as is your awesome macro recorder as well...
Outsourcing Clicks & Presses Since 2004.
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

25 Apr 2018, 05:07

@brutus_skywalker, The machine code generator I used was written by myself.

It's very simple. I've already posted it in Chinese forum. You can go and have a look. :wave:

https://autohotkey.com/boards/viewtopic ... 28&t=18405
brutus_skywalker
Posts: 175
Joined: 24 Dec 2016, 13:16
Location: Antarctica

Re: FindText - Capture screen image into text and then find it

26 Apr 2018, 06:17

feiyue wrote:@brutus_skywalker, The machine code generator I used was written by myself.

It's very simple. I've already posted it in Chinese forum. You can go and have a look. :wave:

https://autohotkey.com/boards/viewtopic ... 28&t=18405

Thanks, sweet code BTW. I just learned a few new tricks...
Outsourcing Clicks & Presses Since 2004.
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

26 Apr 2018, 17:42

I recompiled the machine code, using the optimization option: " -O2". :xmas:

So the operation efficiency is improved. The previous was not optimized. :headwall:
Ajjervon
Posts: 3
Joined: 02 May 2018, 18:54

Re: FindText - Capture screen image into text and then find it

02 May 2018, 20:22

Here is what I have. It only runs once i open it one time. Of course the rest of the code is there after the" ;===== Copy The Following Functions To Your Own Code Just once =====" part
I cant figure out where to put the other function codes ex: the left click and or send text code.(ex: FindText_Send_Text(Ok,"text sent",EnterKey:=1,X_Adj:=0,Y_Adj:=0,MoveBack:=0))

also once it finds the text and send my text how do i get it to do it again?
below is a copy of what i have. ant help would be great.
**********************************************************************************************************************************************
t1:=A_TickCount

Text:="|<gmail>*192$35.3k001gMk003MUk000n0DiDBa0NanPASnA6qMBaNxgEPAqPMkqNgqkSAnDhU00000E"

if (ok:=FindText(2655, 134, 150000, 150000, 0, 0, Text))
{
CoordMode, Mouse
X:=ok.1.1, Y:=ok.1.2, W:=ok.1.3, H:=ok.1.4, Comment:=ok.1.5, X+=W//2, Y+=H//2
; Click, %X%, %Y%
}

MsgBox, 4096,, % "Time:`t" (A_TickCount-t1) " ms`n`n"
. "Pos:`t" X ", " Y "`n`n"
. "Result:`t" (ok ? "Success !":"Failed !"), 3
MouseMove, X, Y


;===== Copy The Following Functions To Your Own Code Just once =====
Ajjervon
Posts: 3
Joined: 02 May 2018, 18:54

Re: FindText - Capture screen image into text and then find it

02 May 2018, 20:45

So I got it to reload i just cant figure out how to make it click the text then write a message then hit enter. I'm still working on it any help would be loved!
Also is there a way to make it reload on its own like every 5-10seconds?
Ajjervon
Posts: 3
Joined: 02 May 2018, 18:54

Re: FindText - Capture screen image into text and then find it

02 May 2018, 21:47

I got it yay!!!!!!!
this has been edited.
Now it only works with the MsgBox uuugggghhh
Also is there a way to make it keep searching until the "text" comes up?
jt_kirk

Re: FindText - Capture screen image into text and then find it

16 May 2018, 10:24

Hi, feiyue!

Thank you for your code, it is amazing!

Is it possible to make a function (based on your code) that take as input image file, converts it to the string and then returns the string? i need to convert several small images (.PNG) files and then search for their images.

as conversion type it can use Gray2Two as it can set gray threshold automatically
as parameters it can accept offsets from image borders

Thanks in advance!
feiyue
Posts: 348
Joined: 08 Aug 2014, 04:08

Re: FindText - Capture screen image into text and then find it

18 May 2018, 05:49

@jt_kirk, I think this is a relatively easy solution: :)

Code: Select all

;----------------------------
;  Load Pictures to GUI
;----------------------------

Gui, +AlwaysOnTop +ToolWindow
Gui, Color, DDEEFF
Gui, Font, cRed s28
Gui, Add, Text,, Drag Pictures here to Load
  . `n`nPress Enter or Space to Clear.
Gui, Show,, Load Pictures
OnMessage(0x201, "LButton_Down")
return

LButton_Down() {
  ListLines, Off
  if (A_Gui=1 or A_Gui=2) and (A_GuiControl="")
    SendMessage, 0xA1, 2
}

GuiClose:
ExitApp

GuiDropFiles:
Gui, +OwnDialogs
Loop, parse, A_GuiEvent, `n
{
  file:=A_LoopField
  Gosub, LoadPic
}
return

LoadPic:
Gui, 2:Default
Gui, +LastFound -Caption
Gui, Margin, 0, 0
Gui, Color, White
Gui, Add, Button, w50 Default, OK
Gui, Add, Picture, vPic, %file%
GuiControlGet, p, Pos, Pic
if pW>50
  GuiControl, Move, Button1, w%pW%
Gui, Show
WinWaitClose
return

2ButtonOK:
2GuiEscape:
Gui, Destroy
return

;----------- Script End -----------

;
Last edited by feiyue on 17 Jun 2018, 15:36, edited 2 times in total.

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: furqan, kashmirLZ and 79 guests