Screen scaling, DPI, and making scripts work in different computers

[Descolada]

Introduction

This tutorial is about DPI, screen scaling, multi-monitor setups, and tries to answer the question "why does my script work in one computer, but stops working in another". It's a fairly complicated topic and some details about it are unclear to me as well, but we shall try anyway.

For easy reference I'll attach here also the tools used in this tutorial: the Dpi.ahk library and the accompanying WindowSpyDpi tool.

This tutorial will go over what is DPI and display resolution, discuss DPI awareness of programs, how to adjust scripts for DPI, and finally we'll take a look at how to make scripts that don't care about DPI (mostly how to save coordinates in such a way, and how to perform ImageSearch regardless of DPI). We won't be discussing making AHK GUIs more DPI-aware.

What is DPI and resolution?

First lets define DPI, short for dots per inch: DPI measures how many "dots" (or pixels) fit in a one-inch line. Importantly, DPI is NOT screen resolution, nor does it represent the amount of physical pixels in your monitor.
Mostly for historic reasons, the default DPI for monitors has been defined as 96DPI, which in Windows' settings corresponds to a screen scaling of 100%. This means that a monitor scaled to 100% will display one logical inch of content on 96 pixels.

You might ask why I used the term "logical inch" not just "inch". This is because Windows usually doesn't account for the real physical screen size. Some monitors don't report their physical size, and in some cases talking about physical size doesn't even make sense: for example, if using a projector then its distance from the wall determines the physical size of the screen, and there is no good way for Windows to get that info. This is why Windows deals in logical inches, and the DPI determines the number of pixels for one such inch.

Correspondingly, for font sizes 1pt (point) is defined to be 1/72 logical inches so a 72pt font will be displayed as 96 pixels tall on a 96DPI screen. If displayed on a monitor scaled to 150% then a 72pt font would be 96*1.5=144 pixels tall.

So what is the difference between display resolution and DPI? The display resolution defines how many of the physical pixels Windows uses to display output. If physically using a 1920x1080 screen but setting the display resolution to 960x540 (twice as small), then the quality of the image decreases by two-fold (there will be twice as few pixels to display content). Display resolution can't really be set higher than the physical resolution of the screen, it wouldn't make sense.
DPI on the other hand, as we already know, means how many pixels are used for one inch of content. The higher the DPI, the more pixels are used to display the same amount of content, and thus the higher the quality of the displayed content.
If we had two screens, both physically same size, but one with 1200x800 physical resolution and the other with 600x400 resolution, then the 1200x800 running @ 200% scaling would look much crisper: it would have twice the pixel density for the same images.

An easy way to understand resolution vs DPI is playing around with your monitor settings, which in Windows 10 are under "Change the resolution of the display" in Start menu. I will use as an example my 14' laptop which has a 16:9 aspect ratio and 1920x1080 pixel resolution (or pixels per inch (PPI) of ~157). The default display settings are DPI of 144 (scale 150%) and display resolution of 1920x1080. However, if I change resolution to 1600x900 and scale to 125%, everything on the screen looks exactly the same size as before (1920/150 = 1600/125), but everything is more blurry. This is because now we have less pixels to display the content than before.

Hopefully this makes it clear why DPI is necessary: higher DPI can allow for crispier images on high-resolution displays. Also, since modern high-resolution displays have much more pixels than 10-20 years ago, then the historical DPI of 96 can get uncomfortable because everything is displayed too small. In addition, some users (including sight-impaired people) might prefer bigger content and a higher DPI allows that without lowering the quality (resolution).

DPI awareness
Suppose we don't have just one monitor, but two. And suppose that these monitors have different resolutions and DPIs. In this case things get tricky when moving windows between monitors and trying to figure out coordinates to use MouseMove/Click with.

My following examples use my current setup: the primary monitor is a 14' laptop screen with 16:9 aspect ratio, 1920x1080 pixel resolution, and DPI 144 (150%); the secondary monitor is a 34' screen with 21:9 aspect ratio, 3440x1440 resolution, and DPI 96 (100%); running the latest Windows 10 (as of 01.09.2023).
In this setup AHKs built-in variables A_ScreenWidth and A_ScreenHeight contain 1920 and 1080 respectively, because this is the resolution of the primary monitor.

Our first task will be to move the Calculator app to the second monitors' x100 y100 coordinates. On the primary monitor this is easy: WinMove(100, 100) works perfectly fine. However, AHK scripts and other programs are by default DPI system-aware, which means they expect the primary monitors' DPI everywhere and Windows adjusts for that. So if we try to use WinMove(A_ScreenWidth+100, 100), we get an unexpected result:

MouseMove00WithoutDpi_resized.png (47.38 KiB) Viewed 6131 times

Obviously this is wrong and the most probable culprit is the differing DPIs. This is confirmed by using WinMove(A_ScreenWidth*144/96+100, 100) instead, which moves the window to the correct location. Why that calculation fixes the current issue, I'll leave for the reader to ponder. However, it's easy to see that this will get more and more messy if dealing with three monitors or getting and calculating the DPIs and resulting coordinates for both primary and secondary monitor windows. Fortunately, Windows has provided a solution for this: per-monitor DPI awareness. This causes Windows to adjust the coordinates taking also account other monitors' DPIs.

Setting our script to per-monitor aware is simple: just add the following line to the top of your script: DllCall("SetThreadDpiAwarenessContext", "ptr", -3, "ptr")
Recommended extra reading: AHK docs DPI Scaling section.

Yay, this fixes WinMove, MouseMove, and other related functions! Case solved? No!

Adjusting for DPI
While DPI per-monitor awareness fixes a bunch of things, it doesn't solve the issue of scaled coordinates. Namely, WindowSpy reports Client and Window coordinates adjusted to DPI. This means that if we record a coordinate in one DPI, then it won't work in another DPI.

For example, let's try writing a script to click the "7" button that will need to work on both my monitors.

CalculatorSevenButtonLocation.png (82.66 KiB) Viewed 6131 times

(Note that this image is taken on the primary monitor, so DPI doesn't need to be per-monitor.)

Let's use those Client coordinates to click: Click(67, 494). Easy enough and works on the primary monitor, but let's try it on the second monitor:

CalculatorSevenButtonNoDpiAdjust.png (20.46 KiB) Viewed 6131 times

Naturally, the cursor ends up in the wrong location, because coordinates recorded in DPI 144 will represent a different point in DPI 96. Adjusting for the DPI by dividing by 144 and multiplying by 96 fixes the problem: Click(67*96/144, 494*96/144).
However, now the code doesn't work on the primary monitor!

It seems that to completely fix this issue we would need to get the DPI of the monitor the window is located in, and then adjust for that accordingly. This can be achieved by the following function:

Code: Select all

; Gets the DPI for the specified window
WinGetDpi(WinTitle?, WinText?, ExcludeTitle?, ExcludeText?) {
    return (hMonitor := DllCall("MonitorFromWindow", "ptr", WinExist(WinTitle?, WinText?, ExcludeTitle?, ExcludeText?), "int", 2, "ptr") ; MONITOR_DEFAULTTONEAREST
    , DllCall("Shcore.dll\GetDpiForMonitor", "ptr", hMonitor, "int", 0, "uint*", &dpiX:=0, "uint*", &dpiY:=0), dpiX)
}

Using the WinGetDpi function we can try again with Click(67*WinGetDpi("A")/144, 494*WinGetDpi("A")/144). And success, it works in both monitors properly! This stands true even if we start the application in one monitor and drag it over to the other, or change DPI, or use it in a computer with a different resolution. Resizing Calculator to a much bigger size will naturally break it, because the buttons move to a different location then.

Simplifying DPI adjustment
To simplify this process of adjusting for DPI, I have written the Dpi.ahk library and the accompanying WindowSpyDpi tool.
WindowSpyDpi is adjusted to be DPI per-monitor aware and also automatically scales all coordinates and sizes to DPI 96. Then, the Dpi library can be used with the coordinates to perform the desired actions, and the code should perform the same with any DPI.

For example, WindowSpyDpi reports Calculator (resized to be as small as possible) number 7 button client coordinates to be x47 y329. This means that DPI.Click(47, 329) should automatically work across all monitors.

Some notes:
1) Since scaling down from a very high DPI (eg 192) to 96 will lose some precision, then the A_StandardDpi variable can be adjusted in WindowSpyDpi and also in the script that includes Dpi.ahk, and an arbitrarily large number (eg DPI.StandardDpi := 960) can be used instead.
2) The Dpi functions are necessarily slower than the native functions because of all the extra calculations. This shouldn't be noticable though, unless calling the functions thousands of times per second.
3) The Dpi library might break existing scripts using hard-coded coordinates because it changes DPI awareness which the existing script does not account for. It might also change the appearence of GUIs and GUI menus, so it might be desirable to use SetThreadDpiAwarenessContext(-2) before creating those.

DPI-agnostic ImageSearch & FindText
In some cases it's also possible to perform ImageSearch across multiple DPIs. This involves using ImageSearch's *w and *h options to scale the image to the correct size. This will not always work and pretty much always requires adjusting the variation (*n) to a high level, because the scaled image will never be exactly the same as the one captured at the target DPI. In addition, antialiasing and differing font spacing/size might change the image so much that it can't be found at any other DPI than the native one: this is especially true in browsers and browser-based applications (eg Spotify, Discord etc). I've found that FindText works more reliably across DPIs, even when using image files.

However, sometimes things work out. The following code works for screen scales 100% to 225% to find the Calculator icon:

Code: Select all

#include DPI.ahk
wTitle := "Calculator"
WinActivate wTitle
WinWaitActive wTitle
WinGetPos(&wX, &wY, &wW, &wH, wTitle)
if DPI.ImageSearch(&outX, &outY, 0, 0, wW, wH, "*150 Calculator_icon_225%.png",,216)
    MouseMove outX, outY
else
    MsgBox("Could not find image")

Note: the 216 in DPI.ImageSearch is the DPI of the file Calculator_icon_225%.png, because it was taken on the secondary monitor and the image metadata contains the DPI of the primary monitor (144) which would not work. If the image were captured on the primary monitor then that argument could be omitted.


I've found FindText to be a bit easier and more forgiving method of image-searching. The following code finds the reload button in Chrome:

Code: Select all

#include DPI.ahk
#include FindText.ahk
wTitle := "ahk_exe chrome.exe"
WinActivate wTitle
WinWaitActive wTitle
WinGetPos(&wX, &wY, &wW, &wH, wTitle)
Text:="|<ChromeReloadDPI216>**50$41.0000000000000000000000000000000000000y00000DzU2001s3kA00Dzyss00zUDvE03w07wU07U03100S006201g00A403k00k80B0030E0S00A0U0w00zz01M000002U000005000000/000000S000000w000801c000s01s003k03M007U03k00S007k01s007s0Dk007w1z0007TzQ0007U3U0003zw00000T0000000000000000000000000000004"

if (ok:=FindText(&X, &Y, wX, wY, wX+wW, wY+wH, 0.3, 0.2, Text, , 0, , , , , zoomW:=DPI.GetForWindow(wTitle)/216, zoomW)) ; The Text was taken at DPI 216 (225%)
    FindText().MouseTip(X, Y)
else
    MsgBox("Could not find image")

FindText also appears to have a bit more success with using image files for searches, eg for Chrome reload button:

Code: Select all

#include DPI.ahk
#include FindText.ahk

wTitle := "ahk_exe chrome.exe"
WinActivate wTitle
WinWaitActive wTitle
WinGetPos(&wX, &wY, &wW, &wH, wTitle)
if (ok:=FindText(&X, &Y, wX, wY, wX+wW, wY+wH, 0.2, 0.1, "##10$Chrome_Reload_100%.png", , 0, , , , , zoomW:=DPI.GetForWindow(wTitle)/96, zoomW))
    FindText().MouseTip(X, Y)
else
    MsgBox("Could not find image")

Extras
The following is for easy reference of converting between DPI and scale:

Code: Select all

DPI     Scale   Percentage
96      1       100%
120     1.25    125%
144     1.5     150%
168     1.75    175%
192     2       200%
216     2.25    225%
240     2.5     250%
288     3       300%
336     3.5     350%

Conclusion
Hopefully this tutorial has been instructional, and while writing it I've definitely learned a lot. If anyone has ideas on how to make adjusting for DPI even easier or how to ImageSearch more reliably, be sure to leave a comment!

Change log

Code: Select all

07.09.23: breaking change: all DPI functions have been encapsulated in the DPI class (eg instead of ClickDpi it's now DPI.Click). Added multiple monitor-related methods (GetForMonitor, GetMonitorHandles, MonitorFromPoint, MonitorFromWindow).

[Descolada]

Reserved for future use.

[OvercastBTC]

Hey Descolada,

Here is the code we worked on, converting this to v2.

Feel free to delete my post and make it yours

Code: Select all

;@include-winapi
; --------------------------------------------------------------------------------
#MaxThreads 255 ; Allows a maximum of 255 instead of default threads.
#Warn All, OutputDebug
#SingleInstance Force
SendMode("Input")
SetWorkingDir(A_ScriptDir)
SetTitleMatchMode(2)
; --------------------------------------------------------------------------------
DetectHiddenText(true)
DetectHiddenWindows(true)
; --------------------------------------------------------------------------------
#Requires AutoHotkey v2+
; --------------------------------------------------------------------------------
SetControlDelay(-1)
SetMouseDelay(-1)
SetWinDelay(-1)
; --------------------------------------------------------------------------------
DllCall("SetThreadDpiAwarenessContext", "ptr", -4, "ptr")
; DllCall("SetThreadDpiAwarenessContext", "ptr", -3, "ptr")
; --------------------------------------------------------------------------------
/**
 * @author Descolada (main v2 author)
 * @author OvercastBTC (mod)
 * @author justme (original v1 author)
 * @author iPhilip (v1 mod)
 * @param EnumMonitors 
 */
#HotIf WinActive(A_ScriptName)
#^e::
{
hwnd := WinExist("A"), List := ""
for Each, hMonitor in EnumMonitors() {
	dpi := GetDpiForMonitor(hMonitor)
	List .= 'Index: [' Each ']`n' 'mhWnd: ' hMonitor '`n' 'dpi.x: ' dpi.x  '`n' 'dpi.y: ' dpi.y '`n' 'dpi.Window: ' GetDpiForWindow(hwnd) "`n"
}

MsgBox List
}
#HotIf

EnumMonitors() {
	static EnumProc := CallbackCreate(MonitorEnumProc)
	Monitors := []
	return DllCall("User32\EnumDisplayMonitors", "Ptr", 0, "Ptr", 0, "Ptr", EnumProc, "Ptr", ObjPtr(Monitors), "Int") ? Monitors : false
}

MonitorEnumProc(hMonitor, hDC, pRECT, ObjectAddr) {
	Monitors := ObjFromPtrAddRef(ObjectAddr)
	Monitors.Push(hMonitor)
	return true
}

GetDpiForMonitor(hMonitor, Monitor_Dpi_Type := 0) {  ; MDT_EFFECTIVE_DPI = 0 (shellscalingapi.h)
	if !DllCall("Shcore\GetDpiForMonitor", "Ptr", hMonitor, "UInt", Monitor_Dpi_Type, "UInt*", &dpiX:=0, "UInt*", &dpiY:=0, "UInt")
		return {x:dpiX, y:dpiY}
}

GetDpiForWindow(hwnd) {
	return DllCall("User32\GetDpiForWindow", "Ptr", hwnd, "UInt")
}

[Descolada]

@OvercastBTC, thank you for your suggestions, I have added the methods to the library. I have also encapsulated everything in the DPI class which is a breaking change, alas it is likely necessary for the future since global function definitions are prone for namespace conflicts.

[wpb]

@Descolada, just wanted to say thank you for this. It's a tricky topic, and this is a great summary of the problem and potential solutions. Much appreciated.

[iseahound]

@Descolada Did you manage to figure out which function windows uses to scale with DPI? That would fix most of the DPI related imagesearch issues.

[Descolada]

@iseahound I have not managed to figure out the algorithm which Windows uses, but I think any algorithm we could stumble upon would still be unreliable as Windows might change it at any time (it is undocumented afterall). However, I did find a work-around (at least for myself...).

So, the three types of DPI awareness are DPI unaware, DPI system-aware, and DPI per-monitor aware. There is no algorithm and no known way to generate the images for DPI per-monitor applications nor DPI system-aware (supposing you start the app in another DPI). The reason is that applications draw their windows according to the DPI (and per-monitor aware applications update dynamically to DPI changes), so unless you know how the app decides to draw its window you are out of luck. The app might use different icons for differing DPIs, change the position of UI elements, the text is rendered differently etc.

However, for DPI unaware windows or DPI system-aware ones we can use GetDCEx or PrintWindow to get the window image in it's native DPI!

How it works:
1) DPI unaware. These windows think that screen scaling is 100% all the time, but Windows will automatically stretch the window to target size depending on monitor PDI. This means that if screen scaling isn't 100% then the window will look blurry as Windows stretches it. DPI unaware windows are captured by GetDCEx and PrintWindow in their native DPI (100% scaling), so you could for example use the FindText library in conjunction with BindWindow (depending on the mode it uses GetDCEx or PrintWindow) to perform the ImageSearch, and if the searched image was also captured at 100% scaling then this method should work in any scaling the screen is set to. You will also need to adjust the returned coordinates, because they will be at screen scaling up to the target window top left corner, and from that point they will be 100% scaling.
2) DPI system-aware. These windows adjust their content depending on what the DPI was when the program was first started. For example, if you start Notepad++ in 100% and then set screen scaling to 150% then it'll look blurry (Windows stretching algorithm at play), but if you restart Notepad++ in 150% scaling then it won't be blurry any more as the window uses different image resources to draw the window. GetDCEx/PrintWindow will capture the window in the scaling that the window was first started in, not the scaling that Windows stretches it to. So in this case you could capture search images in all relevant screen scalings, figure out the DPI that the window was started in, and then use GetDCEx/PrintWindow to search the image. Again you will also need to adjust the returned coordinates, because they will be at screen scaling up to the target window top left corner, and from that point they will be at whatever scaling the window was first started in.

You can get the DPI awareness for a given window with DpiAwareness := DllCall("GetAwarenessFromDpiAwarenessContext", "ptr", DllCall("GetWindowDpiAwarenessContext", "ptr", WinExist("Target window"), "ptr"), "int") where 0 = unaware, 1 = system-aware, 2 = per-monitor aware.