Mouse movement not acting like I'd like it to Topic is solved

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 10 Dec 2021, 22:12

Hey AHK dudes! I'm very close to finishing up my script, we're at 742 lines and it's been a long journey. Some of those lines record key positions and write it to a file, so you can read it back later and replicate those clicks, so if you need to fill out a form on one screen, you can "train" it to do it on another screen that might be another size, or different zoom, or what have you. It all works like it should, but I've run into a strange issue I've never had a problem with until now.

MouseClick and MouseMove do not move to the correct coordinates if you are on a monitor that is not the main monitor.

Just to confirm I wasn't loading bad numbers, I made this short script

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.


CoordMode, Mouse, Screen ; Regardless of being used or not, still doesn't work correctly, using this or not, it will work on the laptop screen but not the other two connected monitors
global x := 0
global y := 0


^+4::
	MouseGetPos, x, y
	;ToolTip Coords are %x% and %y%
return

^+5::
	MouseMove, x, y, 5
return
As you can see, if you hit Ctrl Shift 4, it'll record the mouses position, and if you hit Ctrl Shift 5, it will move the mouse to that position. I also can't get it to move slow, it always moves instantly, don't know why.
I've confirmed that the position it's recording is also correct and makes sense.

This script works just fine on the main monitor. However, the moment you try to save a coordinate on another monitor and move to it, it's anyones guess on where it's going. I almost feel like it's adding an extra monitors worth of horizontal distance.

I'm using a laptop hooked up by USB C to a dock that uses two HMDI's to two monitors. It looks like
Monitor Monitor
Laptop

Please send help, thanks in advance,
Bloo

User avatar
boiler
Posts: 16768
Joined: 21 Dec 2014, 02:44

Re: Mouse movement not acting like I'd like it to

Post by boiler » 11 Dec 2021, 02:20

See this excerpt from the Remarks section of the MouseMove documentation:
The following is an alternate way to move the mouse cursor that may work better in certain multi-monitor configurations:

Code: Select all

DllCall("SetCursorPos", "int", 100, "int", 400)  ; The first number is the X-coordinate and the second is the Y (relative to the screen).
Once you move the mouse pointer to a location using that method, you could then use Click or MouseClick with no coordinates specified to perform a mouse click at that location.

BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Re: Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 11 Dec 2021, 03:20

Absolute legend, thank you. Kinda peeved about it, I'd love to know why it's not working like that but I'm cool with it.

BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Re: Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 11 Dec 2021, 03:43

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.


CoordMode, Mouse, Screen ; Regardless of being used or not, still doesn't work correctly
global x := 0

global y := 0


^+4::
	MouseGetPos, x, y
	PixelGetColor, color, x, y ; , "Alt" or "Slow" do not seem to work either
	ToolTip %color%
	Sleep 500
	ToolTip
return

^+5::
	DllCall("SetCursorPos", "int", x, "int", y)

return
Now I can move the mouse correctly, I'm not getting accurate color values. Again, it works on the main screen, but any other screen it throws 0xFFFFFFF or 0x000000, it's important for me to get accurate colors because part of my program requires waiting until a background has changed from one color to another, as sometimes webpages take varying amounts of time to load.

User avatar
boiler
Posts: 16768
Joined: 21 Dec 2014, 02:44

Re: Mouse movement not acting like I'd like it to

Post by boiler » 11 Dec 2021, 07:03

It would seem that you need this line at the top of your code like you have for Mouse:

Code: Select all

CoordMode, Pixel, Screen

BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Re: Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 11 Dec 2021, 20:47

You're absolutely right, I didn't read over the CoordMode page very carefully and just assumed Mouse would have affected PixelGetColor as well, not sure why.

I'm so glad this forum is so active, it's wonderful to see people getting help as fast as they do. Thanks.

BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Re: Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 11 Dec 2021, 20:59

I hate to make a post right after claiming a working solution, but it still doesn't seem to fix PixelGetColor.

I have this forum post on my very right most monitor, and even with setting CoordMode, Pixel, Screen at the top to auto execute, it is still claiming the whole page is 0xFFFFFF
My second monitor in the middle now shows different colors depending on where I put my mouse, but they are not accurate, as it seems to be pulling pixels from somewhere else as I multiple points on a white background will give me very different colors.
Just to be absolutely positive, this is the code that I'm using now

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.


CoordMode, Mouse, Screen
CoordMode, Pixel, Screen
global x := 0

global y := 0


^+4::
	MouseGetPos, x, y
	PixelGetColor, color, x, y ; Also attemtped with Alt and Slow, neither fixed the issue
	ToolTip %color%
	Sleep 500
	ToolTip
return

^+5::
	DllCall("SetCursorPos", "int", x, "int", y)
return
I'm wondering if I'll also need a DllCall for pixel color grabbing, but reading up on the DllCall page, I have no idea what I can and cannot call, and where I even find what I could call.

User avatar
boiler
Posts: 16768
Joined: 21 Dec 2014, 02:44

Re: Mouse movement not acting like I'd like it to

Post by boiler » 11 Dec 2021, 22:00

Try running the following script and compare what it shows in the ToolTip to what the Window Spy tool shows for the color under the mouse at the same time. Do they always agree, or is this script showing different values?

Code: Select all

CoordMode, Mouse, Screen
CoordMode, Pixel, Screen

loop {
	MouseGetPos, x, y
	PixelGetColor, color, x, y, RGB
	ToolTip %color%
	Sleep 50
}
return

Esc::ExitApp

BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Re: Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 12 Dec 2021, 00:05

Window spy was reading the same thing PixelGetColor was reading. I forgot to run your script, so I took pictures using the code I provided, but just to absolutely verify I did do the exact same test but using your snippet, which was much easier to verify then me hitting buttons over and over, and yeah, same results. Monitor 1 reads normal, monitor 2 gets wonky, and monitor 3 always reads FFFFFFFF.

Not sure how to proceed from here.
Monitor1.png
Monitor1.png (38.53 KiB) Viewed 4103 times
Monitor2.png
Monitor2.png (64.19 KiB) Viewed 4103 times
Monitor3.png
Monitor3.png (68.39 KiB) Viewed 4103 times

User avatar
boiler
Posts: 16768
Joined: 21 Dec 2014, 02:44

Re: Mouse movement not acting like I'd like it to  Topic is solved

Post by boiler » 12 Dec 2021, 00:45

It does look like it has to do with multiple-monitor issues, which I can’t do much to try to recreate the problem right now. Maybe someone else can.

If it’s feasible, you might want to try having each monitor run with 100% scaling factor in Windows settings if they aren’t already. Laptops and very high-res monitors often are running at something like 150% scaling factor, and maybe a combination of different scaling factors messes up one or more of these commands.

BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Re: Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 12 Dec 2021, 01:46

Damn, right on the money. Scaling on the main laptop screen was at 150%, dropped down to 100% and now everything is reading correctly no matter which monitor. I never took into account how scaling might affect something like this, throws a huge wrench in my plans, and the people I plan on giving my program to most likely use screens that have been scaled differently. I don't mind doing math to account for scaling, because I run all mouse moves and clicks and coordinate grabbing from several functions, modifying it should be fairly simple, but I wouldn't even know where to begin. If I can figure out how to grab the info for monitor resolutions and positioning of windows, I could make make believe boxes to correspond mouse movements to, and grab pixels from an offset which should theoretically grab the correct one on screen. And, I can't bare to look at my laptop in 100% scaling, it's tiny.

Thanks Boiler. You're awesome.

User avatar
boiler
Posts: 16768
Joined: 21 Dec 2014, 02:44

Re: Mouse movement not acting like I'd like it to

Post by boiler » 12 Dec 2021, 05:23

Glad I could help identify the issue.
BlooDoesntPlay wrote: If I can figure out how to grab the info for monitor resolutions and positioning of windows
Use the SysGet sub-commands MonitorCount to find the number of monitors and Monitor to find the bounding coordinates of each monitor (which then gives you resolution with some math). Use WinGetPos to get the window positions.

It would seem to be quite a task to make what you describe work for all varieties of multiple-monitor setups. Good luck.

BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Re: Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 13 Dec 2021, 22:54

I know, hate to bring up a resolved ask for help, but just maybe to let others know, I found a post that described a similar issue to mine, and their solution was to change the properties of the autohotkey.exe file, Compatibility, Change high DPI settings, and High DPI scaling override on Application, and it solved the whole issue.

User avatar
SteveMylo
Posts: 233
Joined: 22 Jun 2021, 00:50
Location: Australia
Contact:

Re: Mouse movement not acting like I'd like it to

Post by SteveMylo » 16 Dec 2021, 21:59

BlooDoesntPlay wrote:
13 Dec 2021, 22:54
change the properties of the autohotkey.exe file, Compatibility, Change high DPI settings, and High DPI scaling override on Application, and it solved the whole issue.
I would looooove a link to where you found that info. I need that myself. The whole DPI thing.
I would appreciate it so much.
Or maybe share that part of the script??
Many Thanks

BlooDoesntPlay
Posts: 17
Joined: 26 Nov 2021, 07:39

Re: Mouse movement not acting like I'd like it to

Post by BlooDoesntPlay » 16 Jan 2022, 05:08

SteveMylo wrote:
16 Dec 2021, 21:59
BlooDoesntPlay wrote:
13 Dec 2021, 22:54
change the properties of the autohotkey.exe file, Compatibility, Change high DPI settings, and High DPI scaling override on Application, and it solved the whole issue.
I would looooove a link to where you found that info. I need that myself. The whole DPI thing.
I would appreciate it so much.
Or maybe share that part of the script??
Many Thanks
DllCall("SetThreadDpiAwarenessContext", "ptr", -3, "ptr")
I've been using this line since this seems to force auto hotkey to run in the dpi compatibility thing regardless of settings, which is useful for when I give me program to co workers and they aren't savvy enough to change the options themselves. By dropping this at the top of the script, it'll fix all the positioning if you are using monitors with different scaling, but it'll make tool tips and what not look strange on other screens.
viewtopic.php?t=84980

As for the info I read on changing the options in the autohotkey exe, I can't seem to find the exact post, but what I described is exactly what you would do. I'd go with the single line though, since it seems to guarantee compatibility regardless of options chosen on the exe.

vad
Posts: 1
Joined: 18 May 2022, 16:56

Re: Mouse movement not acting like I'd like it to

Post by vad » 18 May 2022, 17:59

BlooDoesntPlay wrote:
16 Jan 2022, 05:08
DllCall("SetThreadDpiAwarenessContext", "ptr", -3, "ptr")
You were on the right path. The missing explanation there is: A long long time ago, Windows only worked for a default DPI of 96 (DPI UNAWARE). A long time ago, Windows added a DPI setting that barely anyone changed because it made so many programs look horrible with automatic scaling. There was only one setting for all monitors, thus it is called SYSTEM-WIDE AWARE. However 1440p and 4K displays became more popular and people began using mixed setups, one 1080p monitor where 96 DPI looks normal and 4K monitors where you'd want 150% scaling. As you noticed, in Windows 10 you can set PER-MONITOR DPI, hence it's PER-MONITOR AWARE.

Windows always tried to be backwards compatible. I mentioned auto-scaling above that works without any changes to the code of the program, Windows does it itself. To remain backwards compatible, it often changes how internals work for old programs. Like "Program Files" vs "Program Files (x86)". Different programs will get different folders. This is further done with so called shims etc.
AutoITv3 has the same issue, they've not touched DPI awareness yet as is evident by my bug report: https://www.autoitscript.com/trac/autoit/ticket/3884 - you're not alone, but you're in the minority who needs it. To start supporting DPI awareness properly, you must target DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2, yes you must not only be DPI-aware in your code, but also be multi-monitor-aware. Fun, a lot of work isn't it?

To explain your inexplicable mouse movement positions: when you have multiple monitors in EXTENDED mode (essentially: not mirrored) then you must imagine a big rectangular canvas whose dimensions fit all of your monitors. Like if you have a 4K+1080p display (side by side), then your virtual screen space will be width=3840+1920 and height=2160. It must fit both, remember? Any space that doesn't project onto a real monitor is practically inaccessible to a cursor but you still have to deal with the coordinates. So you really must know the monitor positions (or instead deal with relative window positions) when moving the cursor. Whatever is easier. See in the attachments, a diagonal 4K+1080p setup.

To implement this, you'll need to read a lot of MSDN. The DPI awareness setting is just the start. The reason your cursor showed white, because it moved to a wrong spot... which is because Windows API lied to Autohotkey. AHK asked the cursor to move to X,Y but Windows DPI-scaled the coordinates of your UNAWARE application to X*dpiScale,Y*dpiScale. And if AHK asked what coords the mouse is at, Windows will lie again and say: X,Y.

Further reading:
viewtopic.php?p=205386
https://stackoverflow.com/questions/50239138/dpi-awareness-unaware-in-one-release-system-aware-in-the-other
https://docs.microsoft.com/en-us/windows/win32/api/windef/ne-windef-dpi_awareness
https://docs.microsoft.com/en-us/windows/win32/hidpi/dpi-awareness-context
Attachments
display-setup-diagonal.png
display-setup-diagonal.png (11.74 KiB) Viewed 3573 times

Post Reply

Return to “Ask for Help (v1)”