HotGestures - Customize and recognize mouse gestures

[Tebayaki]

HotGestures

---

This is a light Autohotkey library for creating and recognizing mouse gestures. It helps you to customize any one-stroke pattern and bind it to any function.
It can accurately recognize complex patterns, including straight lines, arcs, and polylines. So you can draw L, O, M, U, Z, and even a star shape on the screen to do whatever you want.

The guide and source code are available at https://github.com/Tebayaki/HotGestures.

Demo

---

This example creates 3 gestures for notepad. Left-slide to backspace, right-slide to wrap, and circle to select all.

Code: Select all

; A left slide gesture
leftSlide := HotGestures.Gesture("←:-1,0")
; Create A right slide gesture
rightSlide := HotGestures.Gesture("→:1,0")
; A O-Shape gesture
circle := HotGestures.Gesture("O:-20,0|-20,3|-19,5|-19,7|-18,10|-16,12|-15,14|-13,15|-11,17|-9,18|-6,19|-4,20|-1,20|1,20|4,20|6,19|9,18|11,17|13,15|15,14|16,12|18,10|19,7|19,5|20,3|20,0|20,-3|19,-5|19,-7|18,-10|16,-12|15,-14|13,-15|11,-17|9,-18|6,-19|4,-20|1,-20|-1,-20|-4,-20|-6,-19|-9,-18|-11,-17|-13,-15|-15,-14|-16,-12|-18,-10|-19,-7|-19,-5|-20,-3")

hgs := HotGestures()
hgs.Register(leftSlide, "Backspace", _ => Send("{BackSpace}"))
hgs.Register(rightSlide, "Wrap", _ => Send("{Enter}"))
hgs.Register(circle, "Select All", _ => Send("^a"))

HotIfWinactive("ahk_class Notepad")
hgs.Hotkey("RButton")
HotIfWinactive()

demo.gif (168.6 KiB) Viewed 1794 times

[Feather]

Hi, could you release a separate DTW() library? I tried to test this algorithm in my brush script, but I found that I had a hard time extracting it.
Also, do you consider the performance of the algorithm, for example if it is still as fast as 3 gestures when 100 gestures exist.

[viv]

Thank you very much for posting

The gestures added using GestureRecorder are a bit slow
Like below, comment out the longer items below,check is almost imperceptible
Adding longer items a check took 60+ms in my environment

It would be nice if there was a way to streamline longer entries(extract key nodes?)

Code: Select all

    _L := gs.AddGesture("←:-1,0")
    _R := gs.AddGesture("→:1,0")
    _U := gs.AddGesture("↑:0,-1")
    _D := gs.AddGesture("↓:0,1")
    _UL := gs.AddGesture("↖:-1,-1")
    _UR := gs.AddGesture("↗:1,-1")

    _DR := gs.AddGesture("↘:1,1")
    gs.AddGesture("↘:1,0|2,2|5,2|7,3|7,3|8,3|8,4|9,4|8,3|8,3|7,3|6,3|5,2|5,2|4,2|3,1|2,1|2,1|1,1|1,1|1,0", , _DR)

    _DL := gs.AddGesture("↙:-1,1")

    _R_L := gs.AddGesture("→←:1,0|-1,0")
    gs.AddGesture("↘↙:1,1|-1,1", , _R_L)
    gs.AddGesture("→←2:1,0|1,0|1,0|2,0|2,0|2,0|3,1|4,1|4,0|4,1|5,0|5,0|6,0|6,1|6,0|7,0|7,0|6,0|7,0|6,0|6,0|5,0|4,0|4,0|4,1|3,0|2,0|1,0|2,0|1,0|1,0|0,1|1,0|0,1|0,1|0,1|0,1|-1,0|-1,1|-2,1|-2,1|-2,2|-3,1|-4,2|-4,2|-5,2|-5,2|-6,2|-6,2|-7,2|-7,2|-7,2|-7,2|-8,3|-7,3|-6,3|-5,2|-5,2|-4,2|-3,2|-3,1|-2,1|-1,1|-1,0|-1,0", , _R_L)

    _L_R := gs.AddGesture("←→:-1,0|1,0")
    gs.AddGesture("↙↘::-1,1|1,1", , _L_R)
    gs.AddGesture("←→:-1,0|-1,0|-1,1|-1,0|-1,1|-1,1|-1,0|-1,1|-1,1|-1,1|-2,1|-2,1|-2,1|-2,2|-3,2|-3,2|-3,2|-3,2|-4,2|-4,3|-5,3|-5,3|-5,4|-5,3|-5,3|-5,3|-5,3|-5,3|-5,3|-5,2|-4,3|-5,2|-4,2|-4,2|-4,2|-4,2|-3,2|-2,1|-1,0|-1,1|0,1|1,0|1,1|2,0|2,0|3,0|3,1|3,1|4,1|5,2|5,2|6,2|6,2|6,2|6,2|6,2|6,2|6,2|6,3|6,2|6,3|5,2|5,2|4,2|4,2|3,2|3,1|2,1|2,1|1,0|1,1|1,0", , _L_R)

    _D_R := gs.AddGesture("↓→:0,1|1,0")
    gs.AddGesture("↘↗::1,1|1,-1", , _D_R)
    gs.AddGesture("↓←2:0,-1|0,1|0,3|1,4|1,5|2,6|2,7|2,7|2,6|3,7|2,7|3,6|3,6|3,6|3,5|3,5|3,5|3,5|3,5|3,4|3,4|2,3|2,3|2,3|3,2|2,2|3,2|3,2|3,2|2,2|3,1|3,2|3,2|3,1|3,2|3,1|3,1|4,1|4,1|4,1|5,1|4,1|5,0|5,0|5,0|5,0|5,0|6,-1|6,-1|5,-1|5,-2|5,-2|5,-1|4,-1|5,-1|4,-1|3,-1|3,-1|3,0|2,0|2,0|2,-1|2,-1|1,0|2,0|1,-1|1,0|1,0|1,-1|1,0|1,0|0,-1|1,0", , _D_R)

    _D_L := gs.AddGesture("↓←:0,1|-1,0")
    gs.AddGesture("↘↙::-1,1|1,1", , _D_L)

    _U_R := gs.AddGesture("↑→:0,-1|1,0")
    gs.AddGesture("↑→2:-1,0|0,-1|0,-1|-1,-1|-1,-2|-1,-2|0,-2|-1,-4|-2,-4|-2,-5|-2,-5|-2,-6|-2,-7|-2,-6|-1,-6|-2,-8|-1,-6|-1,-7|-1,-7|0,-7|0,-7|1,-7|1,-6|1,-6|2,-6|2,-6|3,-6|3,-6|3,-6|3,-5|4,-5|4,-5|3,-4|4,-5|5,-4|5,-4|5,-5|5,-4|6,-4|6,-4|6,-4|6,-4|6,-4|6,-4|6,-4|6,-3|6,-3|5,-2|4,-2|4,-2|3,-1|2,-1|2,-1|1,0|1,0|1,0", , _U_R)

    _U_L := gs.AddGesture("↑←:0,-1|-1,0")
    gs.AddGesture("↑←2:0,-1|-1,-2|0,-2|0,-3|-1,-3|-1,-4|-1,-4|-1,-5|-1,-5|-2,-6|-1,-5|-1,-6|-2,-6|-2,-5|-2,-5|-2,-6|-2,-6|-2,-5|-3,-5|-2,-5|-2,-4|-3,-4|-2,-3|-2,-3|-2,-4|-3,-3|-2,-2|-3,-3|-3,-2|-3,-2|-4,-2|-4,-1|-4,-1|-5,-1|-6,-1|-5,-2|-5,-1|-6,0|-6,0|-6,0|-6,0|-6,0|-7,1|-6,1|-7,1|-6,2|-7,3|-6,3|-6,3|-6,3|-6,4|-6,4|-5,3|-5,3|-5,4|-4,3|-4,3|-3,2|-3,1|-2,1|-1,1|-1,1|0,1", , _U_L)

[Tebayaki]

Hi, could you release a separate DTW() library? I tried to test this algorithm in my brush script, but I found that I had a hard time extracting it.
Also, do you consider the performance of the algorithm, for example if it is still as fast as 3 gestures when 100 gestures exist.

Hi.
Here is a dtw library by me: https://github.com/Tebayaki/AutoHotkeyScripts/blob/main/lib/DTW.ahk
About the performance, I added a threshold value to the algorithm. Once the distance of a certain gesture exceeds this value the calculation is not continued. I haven't tested more than 100 gestures because I don't think there are any scenarios that require that many gestures. I've tested about 50 gestures and it lags slightly but holds up well enough.

[Tebayaki]

@viv
Thank you! It's a great idea. I will optimize it later.

[Feather]

Thank you.
The "[6, 3, 5, 7, 5]" in the example, is that the distance travelled from one point to the next? If two lines have the same shape but huge difference in size, do they need to be scaled to the same size?
In some mouse gesture software, 100 gestures are possible. Even this script, if you let it work globally, could have a lot of gestures.

[Tebayaki]

Thank you.
The "[6, 3, 5, 7, 5]" in the example, is that the distance travelled from one point to the next? If two lines have the same shape but huge difference in size, do they need to be scaled to the same size?
In some mouse gesture software, 100 gestures are possible. Even this script, if you let it work globally, could have a lot of gestures.

No. DTW calculates the distance between 2 sequences.
In the example, I defined a distance function "(a, b) => Abs(a - b)", so the distance between two elements from sequence x and sequence y is their absolute value.
If you want to calculate the distance between 2 sequences of vectors or coordinates, you also need to provide a function to calculate the distance between two vectors/coordinates.
In HotGestures of the current version, I only calculate the angle between two vectors and ignore the lengths, so they don't need to be scaled to the same size.

Code: Select all

#Include <DTW>
vectors1 := [[1, 0], [1, 0], [3, 0], [0, 1]]
vectors2 := [[1, 0], [0, 1]]
Distance(a, b) => 1 - (a[1] * b[1] + a[2] * b[2]) / (Sqrt(a[1] ** 2 + a[2] ** 2) * Sqrt(b[1] ** 2 + b[2] ** 2))
MsgBox SimpleDTW(vectors1, vectors2, Distance) ; distance is 0.

I plan to optimize the algorithm for distance calculation and filter out the redundant data to reduce computation volume, but not sure if it will meet the performance requirements of 100 gestures. After all, ahk's basic math computation performance is probably 100x slower than C. And I don't want to use machine code...

[Pingouin]

Thanks!
The example shows how to have different (or no) trigger button in a HotIf statement.
Is it possible to have a different gesture result depending on the active window?

I tried something like this (register different action for the same gesture using hotif) but it didn't work:

Code: Select all

HotIfWinactive("ahk_class Notepad")
hgs.Register(MyGesture, "A in Notepad", _ => Send("A"))
HotIfWinactive()
hgs.Register(MyGesture, "B in anything else", _ => Send("B"))

[GollyJer]

Thanks @Pingouin!

Some feedback.
I'm using it with triple monitors.
The gesture trail only shows when drawing on the primary monitor.
The discovered gesture message always shows on the primary monitor, not the monitor with the mouse.

Thanks again. This is a really cool piece of code.

[GollyJer]

Is there an easy way to set a combo HotKey?
I tried `hgs.Hotkey("Control+RButton")` without success.

[gregster]

@GollyJer, I didn't test it, but I would have tried AHK hotkey syntax: "^Rbutton"; it looks like this built-in function is also used in the source code.

[GollyJer]

Thanks @gregster. I should have just looked at the code.

Code: Select all

if GetKeyName(keyName) == ""
            throw Error("invalid key name", , keyName)

Line 49 defines a valid hotkey as anything that can be called with GetKeyName, which is any single key, no combinations.