Jump to content


Overcoming the limitations of PixelSearch/PixelGetColor


  • Please log in to reply
12 replies to this topic

#1 ahakoyofne

ahakoyofne
  • Guests

Posted 08 August 2012 - 09:18 PM

I'm trying to improve the usability of the AHK functions which are responsible for capturing screen color data for manipulation.
The inbuilt methods, while easy to use, have limitations.

The limitations which I'm motivated to work around are:

- Both PixelSearch and PixelGetColor fail to return valid results for any Fullscreen OpenGL or D3D process in Windows 7.
They worked fine in XP but are broken now, I'm unsure exactly why or if its fixable with some OS setting. It isn't Aero related afaik.

- Pixelsearch only searches one direction in fast mode, leaving 3/4ths of any searched-for object invisible to the script.
Say I wanted to get the bottom or right limit of an object by scanning pixels, I couldn't without some slow iterative loop.

- Prior attempts to work around these have either been FAR too slow (say, ten million times slower than PixelSearch, Fast was in WinXP) or too resource-intensive.

With these in mind, my goal is to create a replacement function which is comparable in speed and resource use to the optimized default fastmode,
while removing the directional constraint and the fullscreen broken-ness.

My plan is to first identify which single-pixel read method is most efficient, provided it works for fullscreen apps.
After that, I'd write some search functions using it, either by capturing a chunk of the screen at once and parisng it, or by iterating one pixel at a time.
My guess is that the parse method is the better idea, but I'm not at that stage yet.
Finally, I'd replicate the search function in a language which can be compiled to MCode (duno if ahk can) and use the result of that in my scripts.

I'll likely need some help with each stage, but for now I'll just ask:
What is the most efficient (read: fast and not resource-hungry) way of getting pixel color data from a fullscreen OpenGL or D3D application in Windows 7?


I've included an attempt using GDI+ bitmap from screen, which is hugely resource hungry and eats up an entire processor core while causing the target applications smooth operation to suffer. Clearly, this is not the method I should be using.

;// TRYING to build a method of reading pixels quickly from a fullscreen D3D or OpenGL window without causing lag
;// the default "pixelsearch, fast" mode is quick enough, unfortunately it is directionally limited and incapable of seeing pixels in any fullscreen application due to some windows 7 malarkey.
;// this method is the start of my attempt to rectify these weaknesses.
;// it is really inefficient and causes a huge performance hit :(
#Persistent 
#KeyHistory, 0
#NoEnv
#HotKeyInterval 1
#MaxHotkeysPerInterval 127
#SingleInstance, Force 
SetKeyDelay,-1, 8
SetControlDelay, -1
SetMouseDelay, -1
SetWinDelay,-1
SendMode, Event
SetBatchLines,-1
ListLines, Off
CoordMode, Pixel, Relative, RGB
CoordMode, Mouse, Relative
PID := DllCall("GetCurrentProcessId")
Process, Priority, %PID%, Low

#Include Gdip.ahk
pToken := GDIP_StartUp()

Loop, {
pBitMap := GDIP_BitmapFromScreen()
MouseGetPos, x_in, y_in
PixelGetColor, PRGB, %x_in%, %y_in%, RGB
ARGB := GDIP_GetPixel( pbitmap, x_in, y_in )
GDIP_DisposeImage( pBitMap )
MRGB := ARGB2RGB( ARGB )

ToolTip, %x_in% | %y_in% | %ARGB% | %MRGB%
GoSub, SleepF
}

pixelsearchup() {
	;to be done once i have a useable pixel reading method that doesnt lag me
}
pixelsearchdown() {
	;to be done once i have a useable pixel reading method that doesnt lag me
}
pixelsearchleft() {
	;to be done once i have a useable pixel reading method that doesnt lag me
}
pixelsearchright() {
	;to be done once i have a useable pixel reading method that doesnt lag me
}
	

argb2rgb(argb){
    argb := argb & 0x00ffffff
    return "0x" . argb
}

SleepF:
SleepDuration = 1
TimePeriod = 1
DllCall("Winmm\timeBeginPeriod", uint, TimePeriod)
Iterations = 1
StartTime := A_TickCount
	Loop, %Iterations% {
		DllCall("Sleep", UInt, TimePeriod)
	}
DllCall("Winmm\timeEndPeriod", UInt, TimePeriod)
Return


#2 Wingfat

Wingfat
  • Members
  • 932 posts

Posted 08 August 2012 - 09:26 PM

interesting.. my PixelSearch can be pointed into one area of the screen i thought..

not my script by a very useful one from GamerGirl:
!g::
ScreenGrabber:
If (Grabflag)
{   Gosub EndGrab
   Return
}
GrabFlag = 1
;this sets the mouse return position
MouseGetPos, oposx, oposy
Sleep 100
; we'll use a 1shot timer since the duration of the code
; in the timer is actually longer than the duration of the 
; timer. this will prevent timer hit conflicts.
SetTimer, WatchMouse, -100
Return

;------------------------------------------------
; 100 ms timer watches the mouse position
;------------------------------------------------
;............................
; entry point for mbutton grab section. mbutton
; calls this when it has a location and color to
; be saved
;............................
WatchMouse:
MouseGetPos, mx, my
Sleep 100

; Get the color at the cursor position It's possible
; this color could be different than the color at the
; same position when the mouse is not over the item.
; If it is different color it is likely to be highlighted
; in some way. We will actually try to grab both
; highlighted and non highlighted colors
PixelGetColor, mcHilight, %mx%, %my%, RGB
Sleep 100
ToolTip, Line%A_LineNumber% %AppName% Activate window.`nCtrl-MButton to grab.`nx=%mx% y=%my% c=%mcHilight%
;reset the timer for the next go round
SetTimer, WatchMouse, -100
return

;------------------------------------------------
; end grabbing a screen location/color here is 
; where we'll grab the unhiglighted color for 
; the selected location. We'll put all this info
; on the clipboard so it can be pasted in where
; needed.
;------------------------------------------------
^MButton::
EndGrab:
If (GrabFlag) ; If grabbing loc and color
{   GrabFlag = 0
   Clipboard =
   
   ; turn off the cursor watch timer
   SetTimer, WatchMouse, Off
   Sleep 1000
   Tooltip,
   
   ;grab the unhighlighted color
   MouseMove, %oposx%, %oposy%
   Sleep 100
   PixelGetColor, mcLolight, %mx%, %my%, RGB
   Sleep 300
   
   ; Save the data on the clipboard for use with ctrl-v or "paste"
   Clipboard = %mx%,%my%,%mcLolight%,%mcHilight%
   ClipWait
   
   ; put the data into notepad so it can
   ; be seen
   Run, Notepad
   WinActivate, Untitled
   WinWaitActive, Untitled
   Send, ^v
}
Return


#3 hakfoyofne

hakfoyofne
  • Members
  • 3 posts

Posted 08 August 2012 - 09:44 PM

Sorry, but it is evident that you did not read, or read and did not understand, the original post.
Delete your post (or an admin can do it) and try again?

#4 Wingfat

Wingfat
  • Members
  • 932 posts

Posted 08 August 2012 - 09:46 PM

My plan is to first identify which single-pixel read method is most efficient, provided it works for fullscreen apps.
After that, I'd write some search functions using it, either by capturing a chunk of the screen at once and parisng it, or by iterating one pixel at a time.

mmm so i gave you a screen grabber and you werent happy? oh well.

#5 hakfoyofne

hakfoyofne
  • Members
  • 3 posts

Posted 09 August 2012 - 12:59 AM

Stop trolling :roll:
There is no chance you read the original post and came to the conclusion that your "screen grabber" script is in any way helpful.
You expect me to be grateful for something I didn't ask for, and in fact EXPLAINED THE USELESSNESS OF before you posted it?

I'm tempted to call you extremely stupid, but you're not even there yet: you're worse than wrong because you didn't TRY to read before posting.

Again, if you can't/wont read, don't write. If you can't understand what I'm attempting, how could you expect your input to be valuable?

#6 girlgamer

girlgamer
  • Moderators
  • 2039 posts

Posted 09 August 2012 - 01:58 AM

Please do not engage in name calling or disrespectful postings. It is easily possible for people to mis-understand another personn's post. And abusive posts do nothing to encourage others to help you or even provide suggestions.

#7 VxE

VxE
  • Fellows
  • 3504 posts

Posted 09 August 2012 - 02:49 AM

hakfoyofne's reaction was not entirely unprovoked. Try this analogy: you go to a doctor and tell him you need an apendectomy, then he gives you an enema and says "wut, you're not happy?" (this isn't a comment on commercial helthcare in the USA).


@hakfoyofne, it isn't windows being funky that prevents pixelsearch from 'seeing' fullscreen games, it's because the game is using hardware acceleration to send graphics info to your graphics card (without the "windows" middle-man).

In order to perform pixel operations in such cases, you have two options: disable graphics hardware acceleration (not always an option but is usually achieved by running a game in 'windowed' mode) or write a "graphics driver hook" (not sure what to call it) that dumps one frame's worth of image data from graphics memory to normal memory.

IMHO, writing a 'god-mode' pixelsearch alternative is a very ambitious project.

#8 girlgamer

girlgamer
  • Moderators
  • 2039 posts

Posted 09 August 2012 - 05:41 AM

Everyone has limits vxe and, yes, I'm well aware of that and Obviously he felt provoked. But, it would have been less confrontational, however, to have said, "Thank you but this is not what I need at the moment. I'm looking for a way overcome the limitations that pixel routines now have." and then let it go at that. Or simply saying nothing at all. We all try to be respectful toward each other when asking for and offering help. That's what makes these forums work as well as they do. Cooperation brings out the best in everyone. But like anything else, that's my two cents and I would rather err on the side of caution than allow a situation to spiral out of control.

#9 Wingfat

Wingfat
  • Members
  • 932 posts

Posted 09 August 2012 - 02:37 PM

there is no need to defend him or I VeX,
the screen grabber and pixel searcher i had added works quick with out taking up a lot of processing power.. part of what he is asking for help with.
"What is the most efficient (read: fast and not resource-hungry) way of getting pixel color data from a fullscreen OpenGL or D3D application in Windows 7?"

In the fourms we like to help people and point them in the right directions to make the scripts the way they want. Without us just giving everyone a full script every time they ask for it. He is new on the fourms here so i will give him a chance.. but yeah name calling like Stupid really shows how young this kid is and I see how fast he is getting a response now. He stopped by for an apendectomy, i gave him a Scalpel and a book and showed where to cut.

#10 Guests

  • Guests

Posted 09 August 2012 - 07:32 PM

In the fourms we like to help people and point them in the right directions...

Except that you have done neither and the OP called you out on it. Girlgamer posted a courtesy check and VXe actually posted something relevant to the topic.

Any questions?

#11 Wingfat

Wingfat
  • Members
  • 932 posts

Posted 09 August 2012 - 07:45 PM

In the fourms we like to help people and point them in the right directions...

Except that you have done neither and the OP called you out on it. Girlgamer posted a courtesy check and VXe actually posted something relevant to the topic.

Any questions?

yes anyone going to help this dude?? or is everyone besides GamerGirl just type at me about how I tried to help with pointing him to a good script that did part of what he asked for. He could have said "sorry that didnt help any other examples of lowering the processing time of a screen search?"

With these in mind, my goal is to create a replacement function which is comparable in speed and resource use to the optimized default fastmode,
while removing the directional constraint and the fullscreen broken-ness

Fullscreen is one thing.. but the directional part was solved useing a pixel get color type thing where the mouse is.. which could have been modded to have the mouse move to where he wanted instead of pixel searching a FULLscreen. - bam solved the processing time.
again oh well. I tried to help, others tried trolling it with out helping.

#12 girlgamer

girlgamer
  • Moderators
  • 2039 posts

Posted 09 August 2012 - 08:01 PM

@wingfat
Please, just let it go. You did what you thought was right just like the other posters here have tried to do. We don't always have the answers that others want or even need. Sometimes they have to be allowed to find them for themselves too. Occasionally our errors most often teach us more than our successes. All I'm trying to do is ensure that we all try to treat each other with respect for our differences while we fumble through our own errors.

#13 hakfoyofne

hakfoyofne
  • Members
  • 3 posts

Posted 13 August 2012 - 01:51 AM

Thanks for those who actually read my query :D

it would have been less confrontational, however, to have said, "Thank you but this is not what I need at the moment. I'm looking for a way overcome the limitations that pixel routines now have."

Why should I have to repeat myself?
If that confused person couldn't get from the original post exactly the info you suggest I reiterate, what good would repeating myself do?
Confrontation is no sin when the subject of confrontation is belligerent ignorance.

Being confused about the specifics of a problem is one thing, disregarding a careful explanation of a problem and taking offense when your total lack of process is pointed out is another.
Wasting someones time and insisting that you aren't is a higher form of disrespect than pointing out that someone is wasting their time and yours, agree?


And if this is such an ambitious idea, it's only because the AHK pixelsearch algorithm is incompatible with the majority of potential use cases, and because MS for whatever reason decided to segregate portions of its OS which were previously connected.
I'm sure MS had a reason to break this feature, I just expect that it was a stupid one.

Anyway,
Indeed it seems the only viable solution is to write a generic driver wrapper of some kind.
Dumping entire frames seems inefficient and performance costly but I see no other way.
FAR from ideal, when it shouldn't be required at all.
I doubt any competent OS developer wants its users forced into driver hacking just to survive a version migration.

I'm not yet able to understand all the nuances of that level of manipulation yet, but I'll do some reading and head+wall bashing, if I find something valuable I will post it here.
If such a thing can be created in AHK at all I'll be surprised...

[EDIT]
Apparently I've received a warning as a result of this thread.
Good to know that anti-contributions by the willing illiterate are passively welcomed while the disappreciation of such is actively punished.
Let me know how that philosophy works out.
[/EDIT]