Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

Using Post/SendMessage To Send Clicks To Game Window


  • Please log in to reply
11 replies to this topic
Ripture
  • Members
  • 17 posts
  • Last active: Jun 07 2014 07:23 AM
  • Joined: 19 Aug 2013

Hello all,

 

I've been trying to figure out how to send mouse clicks to a game window (specifically, Eve Online) without much luck.  ControlSend seems to function fine, though I don't have it implemented wholly yet, but ControlClick does not and I've been trying to find out why.  I've read that some windows simply do not play well with ControlClick but I haven't given up.  I managed to find a 64bit version of Spyxx to monitor exactly what messages were being sent to and from the window to find out why ControlClick (which apparently uses PostMessage) does not work.

 

Below is an excerpt of the message log surrounding a legitimate mouse click:

eMeFO5x.png

 

 

 

 

I tried to examine this as much as possible to see what else might be going on such that a simple PostMessage for WM_LBUTTONDOWN wouldn't work.  I went so far as to completely mimic the messages sent from a legitimate click.  Unfortunately, this behaved identically to how ControlClick did.  That excerpt is below as well:

BuxIWi5.png

 

 

 

As far as I can see, the only discernible difference is on line 4 of the simulated click (second image).  The message WM_SETCURSOR returns a false for HaltProcessing.  I'm not sure what exactly this return value affects or how to force it to return True like all of the other WM_SETCURSORs.

 

 

My question is are there any layers deeper than sending messages directly to windows?  Phrased differently, to a window, is a message sent from a script *identical* to a physical user input?



guest3456
  • Members
  • 1704 posts
  • Last active: Nov 19 2015 11:58 AM
  • Joined: 10 Mar 2011
hrmmmm from
http://msdn.microsof...2(v=vs.85).aspx
 

WM_SETCURSOR message

If an application processes this message, it should return TRUE to halt further processing or FALSE to continue.


since your simulated ControlClick is returning FALSE, then it seems that the game isn't processing the message. but i don't know why

have you tried a basic Click ?

Ripture
  • Members
  • 17 posts
  • Last active: Jun 07 2014 07:23 AM
  • Joined: 19 Aug 2013

hrmmmm from
http://msdn.microsof...2(v=vs.85).aspx
 

since your simulated ControlClick is returning FALSE, then it seems that the game isn't processing the message. but i don't know why

have you tried a basic Click ?

 

Click works fine, but this carries the requirement that the window either have focus or the clicking position have line-of-sight, so to speak, of the window.

 

I was hoping to find a way to send input directly to the window, bypassing the need for focus.  ControlSend performs this task but it works with keyboard input.  All I need to figure out is how to send mouse clicks to the window.

 

 

I may experiment more with examining messages in an attempt to discover why the application fails to process the message, as you pointed out from MSDN.  However, my initial tests  showed no variation in outcome; the application simply did not want to process the simulated click.



guest3456
  • Members
  • 1704 posts
  • Last active: Nov 19 2015 11:58 AM
  • Joined: 10 Mar 2011

Click works fine, but this carries the requirement that the window either have focus or the clicking position have line-of-sight, so to speak, of the window.


yes, you are correct. ControlClick or PostMessage normally allows you to send clicks when the window is behind others. not really sure why this is being blocked. perhaps someone smarter will chime in

Ripture
  • Members
  • 17 posts
  • Last active: Jun 07 2014 07:23 AM
  • Joined: 19 Aug 2013

This may be premature, but I appear to have it working.  Moments ago, it appeared to work only if my mouse was on my primary monitor.  The game window did not need focus and clicks were being sent to the correct location.  However, if the mouse was unfocused and on the second monitor, it would not function.  Now, for some reason, it appears to work regardless of where the mouse is.  Below is the script I used:

x := 1062
y := 938

lParam := x | (y << 16)



;SendMessage, 0x086 , 0x00000000 , 0x00000000, , ahk_id 0x00040708	;NCACTIVATE SEND
;SendMessage, 0x006 , 0x00000000 , 0x00000000, , ahk_id 0x00040708	;ACTIVATE SEND
;SendMessage, 0x01C , 0x00000000 , 0x00001774, , ahk_id 0x00040708	;ACTIVATEAPP SEND
;SendMessage, 0x281 , 0x00000000 , 0xC000000F, , ahk_id 0x00040708	;IME_SETCONTEXT SEND

SendMessage, 0x084 , 0x00000000 , %lParam%, , 	ahk_id 0x00040708	;NCHITTEST SEND
SendMessage, 0x020 , 0x00040708 , 0x02040001, , ahk_id 0x00040708	;SETCURSOR SEND
PostMessage, 0x200 , 0x00000000 , %lParam%, , 	ahk_id 0x00040708	;MOUSEMOVE POST
Sleep,100
PostMessage, 0x204 , 0x00000002 , %lParam%, , 	ahk_id 0x00040708	;RBUTTONDOWN POST
SendMessage, 0x084 , 0x00000000 , %lParam%, , 	ahk_id 0x00040708	;NCHITTEST SEND
SendMessage, 0x020 , 0x00040708 , 0x02050001, , ahk_id 0x00040708	;SETCURSOR SEND
PostMessage, 0x205 , 0x00000000 , %lParam%, , 	ahk_id 0x00040708	;RBUTTONUP POST

I was experimenting with the activate messages because as I said, when the mouse was off-screen, it wasn't working but it now appears to be, so I commented them out.  Posting the click with the coordinates in the lparam did not seem to matter to the window, if I sent a MOUSEMOVE prior, it now believed the mouse was at those coordinates.  The NCHITTEST and SETCURSOR may also be unnecessary but I'm just putting this up as it seems to function.  I also added the Sleep because it seemed like the window was processing the RBUTTONDOWN before the MOUSEMOVE, causing it to appear to hold the right mouse button and then drag the mouse from where it was to the coordinates specified.  Sleep made that issue go away.

 

I will continue to see what is unnecessary and post if anything changes.

 

 

 

 

UPDATE:

As I suspected, the culprit was MOUSEMOVE.  I commented out everything except the MOUSEMOVE and the CLICK posts and it seems to work regardless of focus, mouse position or line-of-sight.  One small thing now is that if the mouse is moving in the window when I activate the script, it uses the current mouse position.  This might be because, similarly to the window processing RBUTTONDOWN before the MOUSEMOVE, the mouse position is being updated mid-script and that's where the click is being sent.  I could probably BlockInput but that seems sloppy, however, it might be the only way since I don't really have control over how the window processes the messages or when it pulls certain data like mouse position.  

 

Since it only occurs when the mouse is moving in the window and the window has focus, I could just do an IfWinActive and a BlockInput for that case, that would be fine.  Otherwise, the mouse position is not updated for the window and the click is sent fine.  I even have it working to where if the window has focus and a mouse button is already being held (BlockInput does not undo this, it only stops new input), the click will still go through without conflict by sending an RBUTTONUP and LBUTTONUP before anything else.



guest3456
  • Members
  • 1704 posts
  • Last active: Nov 19 2015 11:58 AM
  • Joined: 10 Mar 2011

Posting the click with the coordinates in the lparam did not seem to matter to the window, if I sent a MOUSEMOVE prior, it now believed the mouse was at those coordinates.


very interesting discovery, thanks a lot.

One small thing now is that if the mouse is moving in the window when I activate the script, it uses the current mouse position. This might be because, similarly to the window processing RBUTTONDOWN before the MOUSEMOVE, the mouse position is being updated mid-script and that's where the click is being sent.


the click ends up using the current mouse position instead of the lparam value that was passed? thats kinda crappy, but yeah maybe blockinput is the best way to do it.

Ripture
  • Members
  • 17 posts
  • Last active: Jun 07 2014 07:23 AM
  • Joined: 19 Aug 2013

the click ends up using the current mouse position instead of the lparam value that was passed? thats kinda crappy, but yeah maybe blockinput is the best way to do it.

 

I will test later to see if the lparam sent with LBUTTONDOWN even matters at all, it may not.  The application may only consider the last position MOUSEMOVE says it was which would be why my script wouldn't work until I updated MOUSEMOVE myself with given coordinates.  Because of this, while the mouse is moving inside of the window either in focus or out of focus and with line-of-sight, MOUSEMOVE is being constantly updated every frame or however often it's set to update.  This is fast enough to get in-between the script and change MOUSEMOVE after I have set it in the script but before it has a chance to process the LBUTTONDOWN message.



guest3456
  • Members
  • 1704 posts
  • Last active: Nov 19 2015 11:58 AM
  • Joined: 10 Mar 2011
well, the reason i ask is because i have used only the LBUTTONDOWN in the past without MOUSEMOVE. and so the only way that the window would know what coordinate to click at is through the lparam. no other information is sent.
 
PostLeftClick(x, y, hwnd) {

   PostMessage, 0x201, 0x0001, ((y<<16)^x), , ahk_id%hwnd%         ;WM_LBUTTONDOWN
   PostMessage, 0x202 , 0, ((y<<16)^x), , ahk_id%hwnd%               ;WM_LBUTTONUP

}
so the lparam must matter because i am not providing any other information to background windows, and the clicks are being processed correctly

and i'm pretty sure that ControlClick (which i now use instead,) only passes the lparam as well, using the same method in that function

Ripture
  • Members
  • 17 posts
  • Last active: Jun 07 2014 07:23 AM
  • Joined: 19 Aug 2013

well, the reason i ask is because i have used only the LBUTTONDOWN in the past without MOUSEMOVE. and so the only way that the window would know what coordinate to click at is through the lparam. no other information is sent.
 

PostLeftClick(x, y, hwnd) {

   PostMessage, 0x201, 0x0001, ((y<<16)^x), , ahk_id%hwnd%         ;WM_LBUTTONDOWN
   PostMessage, 0x202 , 0, ((y<<16)^x), , ahk_id%hwnd%               ;WM_LBUTTONUP

}
so the lparam must matter because i am not providing any other information to background windows, and the clicks are being processed correctly

and i'm pretty sure that ControlClick (which i now use instead,) only passes the lparam as well, using the same method in that function

 

 

Oh yes, sorry, I didn't mean to make it sound like I was saying the lparam is irrelevant for all applications.  In my particular case, what I should have specified, is where I require the MOUSEMOVE in addition to the clicking messages.  I just tested it and my BUTTONDOWN and UP messages do not require an lparam.  It just so happens with my particular application (a game, drawn in a single window), I need to "move" the mouse to the position and click.  I suppose this is because of how the designers decided to interpret mouse messages.

 

Using MOUSEMOVE in place of an lparam might work for other applications as well.  Of course, you wouldn't do that if you didn't need to but it seems to accomplish the same task in a different way.



guest3456
  • Members
  • 1704 posts
  • Last active: Nov 19 2015 11:58 AM
  • Joined: 10 Mar 2011
ah right ok. and yes its a good discovery to keep in mind. if i run into troubles in the future i will try a mousemove first

Ripture
  • Members
  • 17 posts
  • Last active: Jun 07 2014 07:23 AM
  • Joined: 19 Aug 2013

Indeed.  My final hurdle seems to be with left-clicking.  I'm not sure if this happens in general with left-clicking (even with ControlClick) because when I send a right-click, nothing happens to my cursor.  However, when I send a left-click, my cursor is dragged directly to the edge of the application window if it was outside of it prior.  If my mouse is within the confines of the application window (on top, focused, anything) nothing happens to the cursor, all is well, but if the cursor is on my second monitor, for example (but anywhere outside of the window has this happen), my cursor will be taken to the window.

 

Is this normal behavior for sending left-clicks to windows?  I can't see anything on MSDN that would differentiate between a left and right-click.  They seem to do identical things.



guest3456
  • Members
  • 1704 posts
  • Last active: Nov 19 2015 11:58 AM
  • Joined: 10 Mar 2011
no, thats not normal for me. my cursor doesn't move