Is there a way (maybe even built-in within the function?) to set a search range related to the mouse cursor position at any given time?
Meaning, search for and only execute the part of the code within the “if (ok:=” brackets if the image is under the cursor or within a predetermined range related to it i.e X-5 until X+5 and Y-5 t until Y+5? In other words, pretty much replace the predetermined search range with one related to the cursor? That would be an excellent and instant (since the search area would be so small) way to make context sensitive hotkeys, so I am sure I can’t be the only one who thought about it?
FindText - Capture screen image into text and then find it
Re: FindText - Capture screen image into text and then find it
@Epoch I think what you are after is just getting the mouse position at the beginning yeah?
Code: Select all
MouseGetPos, aX, aY
if (ok:=FindText(X, Y, aX-50, aY-50, aX+50, aY+50, 0, 0, Text))
{
FindText().Click(ok[1].x, ok[1].y, "L", 1)
}
Re: FindText - Capture screen image into text and then find it
Always fast and to the point, thanks mate. I figured it'd most likely have to do with MouseGetPos, but I didn't expect you could just put the X and Y from that into the Findtext function. Dunno, I think this could be really useful with audio or video editing programs, you can have the same mouse button doing different things depending where your cursor is (toolbars, video/audio timeline, piano rolls etc.). Cheers!SteveMylo wrote: ↑26 Jan 2023, 20:02@Epoch I think what you are after is just getting the mouse position at the beginning yeah?Code: Select all
MouseGetPos, aX, aY if (ok:=FindText(X, Y, aX-50, aY-50, aX+50, aY+50, 0, 0, Text)) { FindText().Click(ok[1].x, ok[1].y, "L", 1) }
Re: FindText - Capture screen image into text and then find it
@Epoch Yeah It is, I do it, It just isn't reliable on other peoples computers cause of different screen resolutions and scales. But yeah sure make your own it's great
Re: FindText - Capture screen image into text and then find it
is this encodes image to base64?
Re: FindText - Capture screen image into text and then find it
@pirik33, it encodes a pixelated ASCII style representation of the image to base64, yes.
-
- Posts: 60
- Joined: 12 Sep 2019, 21:15
Re: FindText - Capture screen image into text and then find it
Hi guys, I use an app called Authy Desktop and it got updated automatically today (it's possible to downgrade, but the upgrade happens automatically). After the upgrade, it broke my scripts. Turns out that this app uses Electron and the app can now hide itself from screenshot apps, including FindText. I've tried Window's native screenshot and also a 3rd party tool called PicPick and they all cannot take screenshots of Authy Desktop, it'll just hide itself. For apps that have a similar issue, how can we use FindText to scan the images?
Re: FindText - Capture screen image into text and then find it
Updated to 9.0 version - 2023/04/15
1. Modify: Fixed the bug that the returned width and height were incorrect if the image scaling parameter (zoomW, zoomH) was used. In version 8.9.
2. Modify: Fixed a bug in the compiled program where the generated code does not contain library function content.
3. Modify: When searching for image files, a transparent color can be specified without the need to create transparent images yourself.
You can use Text:="##10-RRGGBB $ d:\a.bmp", then the 0xRRGGBB(+/-10) as transparent color.
You can use FindText().ImageSearch(x,y,0,0,0,0, "*10 *Trans112233 d:\a.bmp")
4. Add: Added WaitNotChange() function.
1. Modify: Fixed the bug that the returned width and height were incorrect if the image scaling parameter (zoomW, zoomH) was used. In version 8.9.
2. Modify: Fixed a bug in the compiled program where the generated code does not contain library function content.
3. Modify: When searching for image files, a transparent color can be specified without the need to create transparent images yourself.
You can use Text:="##10-RRGGBB $ d:\a.bmp", then the 0xRRGGBB(+/-10) as transparent color.
You can use FindText().ImageSearch(x,y,0,0,0,0, "*10 *Trans112233 d:\a.bmp")
4. Add: Added WaitNotChange() function.
Last edited by feiyue on 12 Aug 2023, 11:20, edited 1 time in total.
Re: FindText - Capture screen image into text and then find it
@chaoscreater, setting display affinity to 0 for Authy Desktop seems to fix that problem. I'm not sure whether it's necessary to call SetWindowDisplayAffinity before every FindText call or just periodically, but the following code seems to fix it. Unfortunately it's an incomplete solution, because I only got it working with Authy Desktop, and calling it on other programs will most likely crash the program. Perhaps somebody else can figure out the cause (I suspect it's because LoadLibrary isn't called from the external thread?).
Code: Select all
hWnd := WinExist("Twilio")
OutputDebug, % GetWindowDisplayAffinity(hWnd) "`n"
SetWindowDisplayAffinity(hWnd, 0)
OutputDebug, % GetWindowDisplayAffinity(hWnd) "`n"
GetWindowDisplayAffinity(hWnd) {
DllCall("GetWindowDisplayAffinity", "int", hWnd, "int*", out)
return out
}
SetWindowDisplayAffinity(hWnd, dwAffinity:=0) {
static pGlobal, mcode:="2,x86:g+wci0QkIItQCIlUJATB+h+JVCQIi1AEiRQk/xCDxBzD,x64:TItBCEiLURBIiwFMicFI/+A=", e := {1:4, 2:1}, c := (A_PtrSize=8) ? "x64" : "x86"
, hFunc := DllCall("GetProcAddress", "ptr", DllCall("GetModuleHandle", "str", "user32.dll", "ptr"), "astr", "SetWindowDisplayAffinity", "ptr")
/*
#include <windows.h>
BOOL SetAffinity(void* args[]) {
BOOL(*func)(HWND, long long) = (BOOL(*)(HWND, long long))args[0];
return func((HWND)args[1], (long long)args[2]);
}
*/
; Load MCode into local memory
if (!pGlobal) {
if (!regexmatch(mcode, "^([0-9]+),(" c ":|.*?," c ":)([^,]+)", m))
return
if (!DllCall("crypt32\CryptStringToBinary", "str", m3, "uint", 0, "uint", e[m1], "ptr", 0, "uint*", s, "ptr", 0, "ptr", 0))
return
pGlobal := DllCall("GlobalAlloc", "uint", 0, "ptr", s, "ptr")
if (c="x64")
DllCall("VirtualProtect", "ptr", pGlobal, "ptr", s, "uint", 0x40, "uint*", op)
if (!DllCall("crypt32\CryptStringToBinary", "str", m3, "uint", 0, "uint", e[m1], "ptr", pGlobal, "uint*", s, "ptr", 0, "ptr", 0))
DllCall("GlobalFree", "ptr", pGlobal)
}
; Write MCode function pointer, window hWnd, and dwAffinity parameters into a buffer to be copied over
WinGet, pid, PID, ahk_id %hWnd%
VarSetCapacity(buffer, A_PtrSize*3)
NumPut(hFunc, buffer, 0, "ptr")
NumPut(hWnd, buffer, A_PtrSize, "ptr")
NumPut(dwAffinity, buffer, A_PtrSize*2, "ptr")
; Get process handle for process
hProc := DllCall("OpenProcess", "int", 0x000F0000 | 0x00100000 | 0xFFFF, "int", 0, "int", pid) ; PROCESS_ALL_ACCESS
; Allocate memory for the process, and then copy the parameters from buffer
remotebuffer := DllCall("VirtualAllocEx", "ptr", hProc, "ptr", 0, "int", A_PtrSize*3, "int", 0x00001000 | 0x00002000, "int", 0x04, "ptr")
DllCall("WriteProcessMemory", "ptr", hProc, "ptr", remotebuffer, "ptr", &buffer, "int", A_PtrSize*3, "int*", written)
; Write the MCode function in the process memory
pRemoteFunc := DllCall("VirtualAllocEx", "ptr", hProc, "ptr", 0, "int", s, "int", 0x00001000 | 0x00002000, "int", 0x40, "ptr")
DllCall("WriteProcessMemory", "ptr", hProc, "ptr", pRemoteFunc, "ptr", pGlobal, "int", s, "int*", written)
; Execute new thread using MCode function and allocated parameters
hThread := DllCall("CreateRemoteThread", "ptr", hProc, "uint", 0, "uint", 0, "ptr", pRemoteFunc, "ptr", remotebuffer, "uint", 0, "uint", 0, "ptr")
DllCall("WaitForSingleObject", "ptr", hThread, "UInt", 0xFFFFFFFF)
DllCall("VirtualFree", "ptr", pRemoteFunc, "int", 0, "int", 0x00008000) ; MEM_DECOMMIT
DllCall("VirtualFree", "ptr", remotebuffer, "int", 0, "int", 0x00008000)
DllCall("CloseHandle", "ptr", hThread)
DllCall("CloseHandle", "ptr", hProc)
}
Re: FindText - Capture screen image into text and then find it
Hi guys, is it possible to take a new function for find text that allows searching within an image file instead of the screen?
Re: FindText - Capture screen image into text and then find it
@Leox23 You can try this:
Code: Select all
pic1:="d:\big.bmp"
pic2:="d:\small.jpg"
if FindTextInPic(rx, ry, pic1, 0, 0, Text:="##10$" . pic2)
MsgBox % "Found small pic in big pic : " rx ", " ry
else
MsgBox % "Can't Found small pic in big pic !"
return
FindTextInPic(ByRef rx, ByRef ry, pic, err1, err0, Text, args*)
{
FindText().ShowPic( pic, show:=0, x, y, w, h ), args[1]:=0
if (ok:=FindText( rx, ry, x, y, x+w-1, y+h-1, err1, err0, Text, args*))
{
For i,v in ok
v[1]-=x, v[2]-=y, v.x-=x, v.y-=y
rx:=ok[1][1], ry:=ok[1][2]
}
return ok
}
Re: FindText - Capture screen image into text and then find it
Will this work on v2?
Re: FindText - Capture screen image into text and then find it
Will it continue to be updated to v1?
Re: FindText - Capture screen image into text and then find it
I have a few questions for its use,
I might have misunderstood. If so, I'm sorry.
For example, I will search for an image or pixel at the coordinates 200,200,200,200, and when it is found, I want the 1 key to be pressed.
I have 10 different images that I want to be searched at coordinates 200,200,200,200. Can I specify these images in the script and press the keys I want quickly when the pictures appear at the specified coordinate?
Thanks for any help fei
I might have misunderstood. If so, I'm sorry.
For example, I will search for an image or pixel at the coordinates 200,200,200,200, and when it is found, I want the 1 key to be pressed.
I have 10 different images that I want to be searched at coordinates 200,200,200,200. Can I specify these images in the script and press the keys I want quickly when the pictures appear at the specified coordinate?
Thanks for any help fei
Re: FindText - Capture screen image into text and then find it
@Galakrond I think you can do it like this:
Code: Select all
Text:=""
Text.="|<pic1>..." ; Text.="|<color1>##10 $ 0/0/RRGGBB" ; Search for the first color
Text.="|<pic2>..." ; Text.="|<color2>##10 $ 0/0/RRGGBB" ; Search for the second color
Text.="|<pic3>..." ; Text.="|<color3>##10 $ 0/0/RRGGBB" ; Search for the third color
Loop {
if (ok:=FindText(x,y, 200,200,300,300, 0.1,0.1,Text,, FindAll:=0))
{
id:=ok[1].id
if (id="pic1")
Send a
else if (id="pic2")
Send b
else if (id="pic3")
Send c
Sleep 200
}
}
; Or use arrays to simplify code
Key:={ pic1:"a", pic2:"b", pic3:"c" }
Loop {
if (ok:=FindText(x,y, 200,200,300,300, 0.1,0.1,Text,, FindAll:=0))
{
Send % Key[ ok[1].id ]
Sleep 200
}
}
Re: FindText - Capture screen image into text and then find it
Ive been struggling with tryng to get variable inputs to work for the range that would be loaded in through iniread. If anyone would be able to give ANY direction or hint or any help I'd ,much appreciate it
-
- Posts: 60
- Joined: 12 Sep 2019, 21:15
Re: FindText - Capture screen image into text and then find it
thank you, that worked!Descolada wrote: ↑15 Apr 2023, 14:34@chaoscreater, setting display affinity to 0 for Authy Desktop seems to fix that problem. I'm not sure whether it's necessary to call SetWindowDisplayAffinity before every FindText call or just periodically, but the following code seems to fix it. Unfortunately it's an incomplete solution, because I only got it working with Authy Desktop, and calling it on other programs will most likely crash the program. Perhaps somebody else can figure out the cause (I suspect it's because LoadLibrary isn't called from the external thread?).
Code: Select all
hWnd := WinExist("Twilio") OutputDebug, % GetWindowDisplayAffinity(hWnd) "`n" SetWindowDisplayAffinity(hWnd, 0) OutputDebug, % GetWindowDisplayAffinity(hWnd) "`n" GetWindowDisplayAffinity(hWnd) { DllCall("GetWindowDisplayAffinity", "int", hWnd, "int*", out) return out } SetWindowDisplayAffinity(hWnd, dwAffinity:=0) { static pGlobal, mcode:="2,x86:g+wci0QkIItQCIlUJATB+h+JVCQIi1AEiRQk/xCDxBzD,x64:TItBCEiLURBIiwFMicFI/+A=", e := {1:4, 2:1}, c := (A_PtrSize=8) ? "x64" : "x86" , hFunc := DllCall("GetProcAddress", "ptr", DllCall("GetModuleHandle", "str", "user32.dll", "ptr"), "astr", "SetWindowDisplayAffinity", "ptr") /* #include <windows.h> BOOL SetAffinity(void* args[]) { BOOL(*func)(HWND, long long) = (BOOL(*)(HWND, long long))args[0]; return func((HWND)args[1], (long long)args[2]); } */ ; Load MCode into local memory if (!pGlobal) { if (!regexmatch(mcode, "^([0-9]+),(" c ":|.*?," c ":)([^,]+)", m)) return if (!DllCall("crypt32\CryptStringToBinary", "str", m3, "uint", 0, "uint", e[m1], "ptr", 0, "uint*", s, "ptr", 0, "ptr", 0)) return pGlobal := DllCall("GlobalAlloc", "uint", 0, "ptr", s, "ptr") if (c="x64") DllCall("VirtualProtect", "ptr", pGlobal, "ptr", s, "uint", 0x40, "uint*", op) if (!DllCall("crypt32\CryptStringToBinary", "str", m3, "uint", 0, "uint", e[m1], "ptr", pGlobal, "uint*", s, "ptr", 0, "ptr", 0)) DllCall("GlobalFree", "ptr", pGlobal) } ; Write MCode function pointer, window hWnd, and dwAffinity parameters into a buffer to be copied over WinGet, pid, PID, ahk_id %hWnd% VarSetCapacity(buffer, A_PtrSize*3) NumPut(hFunc, buffer, 0, "ptr") NumPut(hWnd, buffer, A_PtrSize, "ptr") NumPut(dwAffinity, buffer, A_PtrSize*2, "ptr") ; Get process handle for process hProc := DllCall("OpenProcess", "int", 0x000F0000 | 0x00100000 | 0xFFFF, "int", 0, "int", pid) ; PROCESS_ALL_ACCESS ; Allocate memory for the process, and then copy the parameters from buffer remotebuffer := DllCall("VirtualAllocEx", "ptr", hProc, "ptr", 0, "int", A_PtrSize*3, "int", 0x00001000 | 0x00002000, "int", 0x04, "ptr") DllCall("WriteProcessMemory", "ptr", hProc, "ptr", remotebuffer, "ptr", &buffer, "int", A_PtrSize*3, "int*", written) ; Write the MCode function in the process memory pRemoteFunc := DllCall("VirtualAllocEx", "ptr", hProc, "ptr", 0, "int", s, "int", 0x00001000 | 0x00002000, "int", 0x40, "ptr") DllCall("WriteProcessMemory", "ptr", hProc, "ptr", pRemoteFunc, "ptr", pGlobal, "int", s, "int*", written) ; Execute new thread using MCode function and allocated parameters hThread := DllCall("CreateRemoteThread", "ptr", hProc, "uint", 0, "uint", 0, "ptr", pRemoteFunc, "ptr", remotebuffer, "uint", 0, "uint", 0, "ptr") DllCall("WaitForSingleObject", "ptr", hThread, "UInt", 0xFFFFFFFF) DllCall("VirtualFree", "ptr", pRemoteFunc, "int", 0, "int", 0x00008000) ; MEM_DECOMMIT DllCall("VirtualFree", "ptr", remotebuffer, "int", 0, "int", 0x00008000) DllCall("CloseHandle", "ptr", hThread) DllCall("CloseHandle", "ptr", hProc) }
Re: FindText - Capture screen image into text and then find it
Is there a way to search an image file or a pointer to bitmap instead of the screen?
Also, how can I convert many image files to text form without using the gui?
Also, how can I convert many image files to text form without using the gui?
- KennethQuicy
- Posts: 1
- Joined: 15 Jul 2023, 17:34
- Contact:
-
DEear T.Magenheimer
I have also faced similar problem 3rd time or 4th.
I waited for caluclation finished for about 10-20 minutes, however the calculation was not stopping or seems not to process, or stuck.
Finally I restarted PC.
I have also faced similar problem 3rd time or 4th.
I waited for caluclation finished for about 10-20 minutes, however the calculation was not stopping or seems not to process, or stuck.
Finally I restarted PC.
Re: FindText - Capture screen image into text and then find it
Updated to 9.1 version - 2023/07/30
1. Modify: Omitting the first two parameters is no longer allowed (improvements made in v8.9 have been removed).
Because considering compatibility with code before v8.5 would add a lot of complexity.
The first and second parameters can be default values, as long as the subsequent parameters are not all empty.
eg: ok:=FindText(,,,,,,,, Text)
2. Modify: Improved the speed of ImageSearch() in finding large image files.
ImageSearch()、ShowPic() can now use image handles. eg: HBITMAP:*%handle%
3. Add: Added PixelSearch(), PixelCount(), ControlClick(), GetRange() function.
1. Modify: Omitting the first two parameters is no longer allowed (improvements made in v8.9 have been removed).
Because considering compatibility with code before v8.5 would add a lot of complexity.
The first and second parameters can be default values, as long as the subsequent parameters are not all empty.
eg: ok:=FindText(,,,,,,,, Text)
2. Modify: Improved the speed of ImageSearch() in finding large image files.
ImageSearch()、ShowPic() can now use image handles. eg: HBITMAP:*%handle%
3. Add: Added PixelSearch(), PixelCount(), ControlClick(), GetRange() function.
Return to “Scripts and Functions (v1)”
Who is online
Users browsing this forum: rakizi and 115 guests