Help speeding up image snippet utility I made

Get help with using AutoHotkey and its commands and hotkeys
SteaIthe
Posts: 10
Joined: 03 Jan 2019, 18:55

Help speeding up image snippet utility I made

03 Jan 2019, 21:11

In my never ending battle with my biggest nemesis of scripting with image recognition on images that change slightly and having to battle between too high and too low variation and which pixels to make transparent/execution time I came up with a script to do it way faster and way more precise.

-The script allows you to create a rectangle around the spot to be searched on screen
-From the gui you choose the directory/snippet image
-You then Start the search
-It searches the rectangle for the image starting at variation of 1 and going up til it finds it
-It then creates a GDIP Pbitmap image of it
-Then it uses GDIP Lockbits to get the RGB values 1 pixel at a time and separates them into each color individually
-It then puts each color value for every pixel into a variable
-it then repeats this process starting from creating the Pbitmap of the same coordinates as before around 30 ms per iteration
- Every iteration it checks if the new pixel color values are higher or lower then the highest and lowest intensity for that color.If it is then it overwrites the variable with the new highest or lowest
-So you basically have 6 variables for every pixel. R high, R low, G high, G low, B high, B low
-When you stop the search it takes the high and low and gets the middle intensity(high + low/2)
-It also takes (Middle - low) to give the variation needed for that color
-It then creates a new bitmap using Gdip setpixel with all the middle values and will make pixels requiring too high variation a transparent color predefined in the gui
-The new snippet you now have is the very middle ground for every pixel that the varying image changes to making image search with variation as fast,efficient and reliable as possible


Sorry for the wall of text but understanding what the script is doing will help understanding my issue and with extra ideas.

I'm looking for ways to speed up the process ultimately and that the idea at least for this gets in the hands of someone way more knowledgeable than myself.

Here is my version of the utility minus the gui as it would be too big.

Code: Select all

buttonstop:
GuiControl,, callog,Generating Image
stopit=1
return

buttoncontinue:
continued=1

buttonstart:
if isitfocus:= LV_GetNext()
{
infocus=1
}
else
infocus=0
Thread, NoTimers
GuiControl, hide, continue
GuiControl, hide, calibrate
GuiControl, show, nocalibrate
GuiControl,, callog,Finding Image
ifequal,infocus,1
{
LV_GetText(FileName, isitfocus)
gosub presearch
variation=0

loop
{      
GuiControl,, callog,%variation%
ImageSearch, OutputVarX, OutputVarY, X1, Y1, X2, Y2, *%variation% *trans0x%Mytransrgb% %image%
if not errorlevel
{
break
}
variation:=variation + 5
ifgreater,variation,255
{
GuiControl,, callog,Could Not Find Image
continued=0
GuiControl, show, continue
GuiControl, show, calibrate
GuiControl, hide, nocalibrate
Thread, NoTimers , false
return
}
}
GuiControl,, callog,Image Found
pToken := Gdip_StartUp()

   pBitmap := Gdip_CreateBitmapFromFile(image)
   Gdip_GetImageDimensions(pBitmap, imagewidth, imageheight)
   Gdip_DisposeImage(pBitmap)
 screen51 := OutputVarX . "|" . OutputVarY . "|" . imagewidth . "|" imageheight
}
else
{
gosub presearch
pToken := Gdip_StartUp()
imagewidth:=Capture_x2 - Capture_x
imageheight:=Capture_y2 - Capture_y
screen51 := Capture_x . "|" . Capture_y . "|" . imagewidth . "|" imageheight
}
raster := 0x40000000 + 0x00CC0020  
ifequal,continued,1
firstrun:=0
else
firstrun:=1
cuns=
rows:=imagewidth, columbs:=imageheight
stopit=0
stoping=0
indexes=2
loop 
{
end:=a_tickcount - start
ifnotequal,stopit,1
ifnotequal,a_index,1
{
cuns:=cuns end "`n"
GuiControl,, callog,Iterations: %a_index%`n%end2%`n%cuns%
}
else
GuiControl,, callog,Iterations: %a_index%
Gdip_DisposeImage(pBitmap)
start:=a_tickcount
pBitmap := Gdip_BitmapFromScreen(screen51,raster)
Gdip_LockBits(pBitmap, 0, 0, Gdip_GetImageWidth(pBitmap), Gdip_GetImageHeight(pBitmap), Stride, Scan0, BitmapData)
gosub hexfileconvert2
Ptr := A_PtrSize ? "UPtr" : "UInt"
DllCall("Gdiplus\GdipBitmapUnlockBits", Ptr, pBitmap, Ptr, &BitmapData)
if firstrun
firstrun=0
if stoping
break
if stopit
{
stoping=1
}
}
continued=0
x:=0, y:=0

loop %columbs%
{
loop %rows%
{
range:="M", toler:="T", assemble:="0xff"
loop 3
{
lastcolor:=x%x%y%y%%range%%a_index%
lasttolerance:=x%x%y%y%%toler%%a_index%
ifgreater,lasttolerance,%myslider%
{
assemble=0xff%Mytransrgb%
Gdip_SetPixel(pBitmap, x, y, assemble)
break
}

SetFormat, IntegerFast, hex
lastcolor += 0
lastcolor .= "" 
SetFormat, IntegerFast, d
ifless,lastcolor,16
{
stringtrimleft,lastcolor,lastcolor,2
lastcolor=0%lastcolor%
}
else
stringtrimleft,lastcolor,lastcolor,2

assemble:= assemble lastcolor
ifequal,a_index,3
{
Gdip_SetPixel(pBitmap, x, y, assemble)
}
}
x++
}
y++, x:=0
}
ifequal,infocus,1
file8=%temp%\%subo%
else
file8=%temp%\Nonfocused.bmp
filedelete,%file8%
Gdip_SaveBitmapToFile(pBitmap,file8)
run, %file8%
Gdip_DisposeImage(pBitmap)
Gdip_Shutdown(pToken)
GuiControl, show, continue
GuiControl, show, calibrate
GuiControl, hide, nocalibrate
GuiControl,, callog,Image Created
continued=0
Thread, NoTimers , false
return

presearch:
subo:=FileName
   image:=myfold "\" subo
  ifequal,searchareaChoice,region
   {
      x1:= Capture_x
      y1:= Capture_y
      x2:= Capture_x2
      y2:= Capture_y2
   }else  {
      x1:= 0
      y1:= 0
      x2:= VirtualWidth
      y2:= VirtualHeight
   }
return


hexfileconvert2:
x:=0, y:=0
loop %columbs%
{
loop %rows%
{
start2=1
SetFormat, IntegerFast, hex
pixelColor :=NumGet(Scan0+0, (x*4)+(y*Stride), "UInt")
StringTrimLeft,pixelColor,pixelColor,4
loop 3
{
StringMid, acolor, pixelColor,%start2% , 2
SetFormat, integer, d
acolor=0x%acolor%
if firstrun
{
x%x%y%y%L%a_index%:=acolor, x%x%y%y%H%a_index%:=acolor
}
else
{
if acolor NOT between % x%x%y%y%L%a_index% and % x%x%y%y%H%a_index%
{
ifless,acolor,% x%x%y%y%L%a_index%
{
x%x%y%y%L%a_index%:=acolor
}
else
{
ifgreater,acolor,% x%x%y%y%H%a_index%
{
x%x%y%y%H%a_index%:=acolor
}
}
}
if stoping
{
mid:=(x%x%y%y%H%a_index% + x%x%y%y%L%a_index%) / 2
if mid is float
mid:=round(mid), tol:=mid - x%x%y%y%L%a_index% + 1
else
tol:=mid - x%x%y%y%L%a_index%
x%x%y%y%M%a_index%:=mid, x%x%y%y%T%a_index%:=tol
}
}
start2+=2
}
x++
}
x:=0, y++
}
return
swagfag
Posts: 3972
Joined: 11 Jan 2017, 17:59

Re: Help speeding up image snippet utility I made

04 Jan 2019, 07:46

The script allows you to create a rectangle around the spot to be searched on screen
From the gui you choose the directory/snippet image
You then Start the search
It searches the rectangle for the image starting at variation of 1 and going up til it finds it
It then creates a GDIP Pbitmap image of it
Then it uses GDIP Lockbits to get the RGB values 1 pixel at a time and separates them into each color individually
It then puts each color value for every pixel into a variable
ok
it then repeats this process starting from creating the Pbitmap of the same coordinates as before around 30 ms per iteration
repeats what process? and an iteration of what? accessing an R, G, B component? accessing a pixel? processing the entire bitmap?
Every iteration it checks if the new pixel color values are higher or lower then the highest and lowest intensity for that color.If it is then it overwrites the variable with the new highest or lowest
what new pixel color? what is the lowest/highest intensity?
SteaIthe
Posts: 10
Joined: 03 Jan 2019, 18:55

Re: Help speeding up image snippet utility I made

04 Jan 2019, 10:52

repeats what process? and an iteration of what? accessing an R, G, B component? accessing a pixel? processing the entire bitmap?
The process it repeats is
-creating a new pbitmap of the selected rectangled area
-getting and separating the rgb components of each pixel
-checking whether the new r color intensity is higher or lower than whats currently in the highest and lowest variable.It does the same thing for g and b as well
-If it is lower or higher then it overwrites the variable making it the new highest or lowest depending on which one it was.if it isn't higher or lower it does nothing and continues

As far as an iteration of what?
-iteration is a full loop of the process just described

what new pixel color? what is the lowest/highest intensity?
The new pixel color is the color of the pixel from the new pbitmap it created at the beginning of the iteration

-The lowest/highest intensity is the only variables kept of each pixels separated colors
-All it is doing throughout the entire process is recording what the highest and lowest color intensity for that pixel has ever been throughout all the iterations.

For example: Pixel X20, Y25, After 5000 iterations the highest it's red has ever been is 255 and the lowest it was is 170.It keeps the same information of the green and blue as well

Then when your done it takes the highest and lowest of each and finds the middle which in the above example would be 212 for red.When it makes the new bitmap it will make the red value of pixel X20,Y25 212
and it does the same for every color of every pixel of the bitmap.
swagfag
Posts: 3972
Joined: 11 Jan 2017, 17:59

Re: Help speeding up image snippet utility I made

05 Jan 2019, 13:34

ah i see, i didnt realize this was supposed to happen over time. so how is this supposed to work anyway, u let it run, and after some arbitrary time u cancel it and wait for the bitmap to be produced, or what?
whats the use case? like, say, u had this here scene:
scene
and u wanted to make bitmap for the shrine:
shrine
u select a rectangle around it and let it run. suppose ure using *TransFF00FF magenta as the passthrough color, after some time it spits out an image that has a select few yellowish pixels in specific places and tells u to use *26 for the variation. something like this:
Spoiler
is that the idea?
SteaIthe
Posts: 10
Joined: 03 Jan 2019, 18:55

Re: Help speeding up image snippet utility I made

06 Jan 2019, 18:59

Exactly!...Only it does it perfect.
For example I made snippets of all the guns in Fortnite to allow my script to detect which guns I have and which one is active that way to set the proper anti recoil.

Problem I ran into is the background of the gold items is animated and also the gun itself has a glimmer effect

Here was my original snippet for finding the image
Assault Rifle (Gold).png
Assault Rifle (Gold).png (6.83 KiB) Viewed 1043 times
Issue with it was with the animated background the image required a 90 variance to detect causing image search to take 3x longer and getting false positives sometimes compared to the other non gold guns.

Other option I tried was using a transparent color:
Assault Rifle (Gold) - Copy.png
Assault Rifle (Gold) - Copy.png (3.14 KiB) Viewed 1043 times
Issue with this is adding transparent color to image search cause it to take 10x longer and gave great chance of false positives and because the gun itself also glimmers slightly still needed a 40 variation.

So I used my tool to make the image..I ran it for about 20 seconds to let it get through its full animation cycle:
Assault Rifle (Gold2).png
Assault Rifle (Gold2).png (6.7 KiB) Viewed 1043 times
This image now requires no transparent coloring and only requires a 15 variation for image search to always see it.
To the eye it looks very similar to the first image however the tool has made every single pixel's color in the very middle of the intensity range of the entire animation making image search as efficient as possible...15 variation their is no chance of a false positive,image search is also way faster.

The tool can also if set cause any pixel that will be over a set variation to be a transparent color making the image similar to the 2nd image only still alot better because it also fixes the slight gun glimmer and also knows the exact pixels to make transparent vs a human just kind of trimming it themselves.
However the image would still require a lot more search time do to adding transparency to it.

So after testing came up with the result it is way better and faster to have the image with the background at a slightly higher variation than to have a lower variation with transparent coloring.

The tool can also be pause to allow you to move the image to a different area and continue the process.

But i'm not a great script writer and their are people who know way more that if knew the idea of it could make something way better possibly even more an image that constantly is moving around the screen
guest3456
Posts: 3134
Joined: 09 Oct 2013, 10:31

Re: Help speeding up image snippet utility I made

24 Nov 2019, 00:06

@SteaIthe do you care to publish your final code for this? the C mcode would be nice too

also i want to note, that the problem this approach solves, is the fact that you've chosen too large of a needle image. your square is capturing the surrounding part of the gun for example. if you chose a smaller rectangle INSIDE the gun, then perhaps that part wouldn't be changing as much


Return to “Ask For Help”

Who is online

Users browsing this forum: Albireo, Google [Bot], mikeyww, PlumpSamurai, RubbeH, Stewn and 44 guests