How to automate arrow buttons overlaid on different pictures?
How to automate arrow buttons overlaid on different pictures?
Trying to assign keyboard shortcuts on instagram, so that I can scroll through in-post photos (i.e. 1 & 2), instead of through posts only (3 & 4). How do I automate 1 & 2 since the arrow buttons are overlaid on different images AND also have different sizes depending on how big the original image is?
5 is the indicator which shows whether there are multiple images to scroll through within 1 instagram post.S
5 is the indicator which shows whether there are multiple images to scroll through within 1 instagram post.S
Re: How to automate arrow buttons overlaid on different pictures?
You might be able to look for the edge of the circle, trying a few different colors, or possibly just look for the colors in the circles.
Re: How to automate arrow buttons overlaid on different pictures?
My attempt at this using UIAutomation library. Requires Instagram to be in English (or get the name of the buttons with UIAViewer or Accessibility Insights, and change variables nextButtonName and prevButtonName accordingly), and seems to work smoother in Edge.
Code: Select all
#SingleInstance, force
#NoEnv ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn
SetWorkingDir %A_ScriptDir% ; Ensures a consistent starting directory.
SetTitleMatchMode, 2
#include <UIA_Interface>
global UIA := UIA_Interface(), prevHwnd, TW_Next, TW_Prev, nextButtonName := "Next", prevButtonName := "Go Back"
TW_Next := UIA.CreateTreeWalker(UIA.CreateAndCondition(UIA.CreateCondition("ControlType", "Button"), UIA.CreateCondition("Name", nextButtonName)))
TW_Prev := UIA.CreateTreeWalker(UIA.CreateAndCondition(UIA.CreateCondition("ControlType", "Button"), UIA.CreateCondition("Name", prevButtonName)))
#IfWinActive Instagram
$Right::
$Left::
if (prevHwnd != (newHwnd := WinActive("Instagram")))
window := UIA.ElementFromHandle(prevHwnd := newHwnd, True)
if ((UIA.GetFocusedElement().CurrentControlType != 50004) && (but := (InStr(A_ThisHotkey, "Right") ? TW_Next : TW_Prev).GetLastChildElement(window)))
but.Click()
else
Send, % "{" (InStr(A_ThisHotkey, "Right") ? "Right" : "Left") "}"
return
#If
~F1::ExitApp
Re: How to automate arrow buttons overlaid on different pictures?
Thanks! I just checked using UIViewer and the button names are correct. However nothing seems to happen after using the hotkeys, no errors as well. tested on a fresh script as is (with the UIAInterface.ahk directory updated)Descolada wrote: ↑27 Jun 2022, 02:32My attempt at this using UIAutomation library. Requires Instagram to be in English (or get the name of the buttons with UIAViewer or Accessibility Insights, and change variables nextButtonName and prevButtonName accordingly), and seems to work smoother in Edge.
Code: Select all
#SingleInstance, force #NoEnv ; Recommended for performance and compatibility with future AutoHotkey releases. ; #Warn SetWorkingDir %A_ScriptDir% ; Ensures a consistent starting directory. SetTitleMatchMode, 2 #include <UIA_Interface> global UIA := UIA_Interface(), prevHwnd, TW_Next, TW_Prev, nextButtonName := "Next", prevButtonName := "Go Back" TW_Next := UIA.CreateTreeWalker(UIA.CreateAndCondition(UIA.CreateCondition("ControlType", "Button"), UIA.CreateCondition("Name", nextButtonName))) TW_Prev := UIA.CreateTreeWalker(UIA.CreateAndCondition(UIA.CreateCondition("ControlType", "Button"), UIA.CreateCondition("Name", prevButtonName))) #IfWinActive Instagram $Right:: $Left:: if (prevHwnd != (newHwnd := WinActive("Instagram"))) window := UIA.ElementFromHandle(prevHwnd := newHwnd, True) if ((UIA.GetFocusedElement().CurrentControlType != 50004) && (but := (InStr(A_ThisHotkey, "Right") ? TW_Next : TW_Prev).GetLastChildElement(window))) but.Click() else Send, % "{" (InStr(A_ThisHotkey, "Right") ? "Right" : "Left") "}" return #If ~F1::ExitApp
Re: How to automate arrow buttons overlaid on different pictures?
Interesting... I tried it on three different computers and it worked everywhere. So lets try some debugging: first, the hotkey only works if the window is active, so are you sure Instagram is active?
If yes, try running this with a Instagram window open where there are some Next and Go Back buttons available:
It should go through every step one-by-one and might give a hint as to where the problem lies.
If yes, try running this with a Instagram window open where there are some Next and Go Back buttons available:
Code: Select all
#SingleInstance, force
#NoEnv ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn
SetWorkingDir %A_ScriptDir% ; Ensures a consistent starting directory.
SetTitleMatchMode, 2
#include <UIA_Interface>
global UIA := UIA_Interface(), prevHwnd, TW_Next, TW_Prev, nextButtonName := "Next", prevButtonName := "Go Back"
instagramHwnd := WinExist("Instagram")
TW_Next := UIA.CreateTreeWalker(UIA.CreateAndCondition(UIA.CreateCondition("ControlType", "Button"), UIA.CreateCondition("Name", nextButtonName)))
TW_Prev := UIA.CreateTreeWalker(UIA.CreateAndCondition(UIA.CreateCondition("ControlType", "Button"), UIA.CreateCondition("Name", prevButtonName)))
WinActivate, ahk_id %instagramHwnd%
WinWaitActive, ahk_id %instagramHwnd%
window := UIA.ElementFromHandle(instagramHwnd, True)
nextEl := TW_Next.GetLastChildElement(window)
MsgBox, % "Found: " nextEl.Dump() "`n`nPress OK to display found ""Next"" button"
WinActivate, ahk_id %instagramHwnd%
br := nextEl.CurrentBoundingRectangle
RangeTip(br.l, br.t, br.r-br.l, br.b-br.t)
MsgBox, % "Press OK to try clicking ""Next"""
nextEl.Click()
Sleep, 1000
RangeTip()
prevEl := TW_Prev.GetLastChildElement(window)
MsgBox, % "Found: " prevEl.Dump() "`n`nPress OK to display found ""Go Back"" button"
WinActivate, ahk_id %instagramHwnd%
br := prevEl.CurrentBoundingRectangle
RangeTip(br.l, br.t, br.r-br.l, br.b-br.t)
MsgBox, % "Press OK to try clicking ""Go Back"""
prevEl.Click()
Sleep, 1000
RangeTip()
MsgBox, Press OK to Exit
ExitApp
RangeTip(x:="", y:="", w:="", h:="", color:="Red", d:=2) ; from the FindText library, credit goes to feiyue
{
local
static id:=0
if (x="")
{
id:=0
Loop 4
Gui, Range_%A_Index%: Destroy
return
}
if (!id)
{
Loop 4
Gui, Range_%A_Index%: +Hwndid +AlwaysOnTop -Caption +ToolWindow
-DPIScale +E0x08000000
}
x:=Floor(x), y:=Floor(y), w:=Floor(w), h:=Floor(h), d:=Floor(d)
Loop 4
{
i:=A_Index
, x1:=(i=2 ? x+w : x-d)
, y1:=(i=3 ? y+h : y-d)
, w1:=(i=1 or i=3 ? w+2*d : d)
, h1:=(i=2 or i=4 ? h+2*d : d)
Gui, Range_%i%: Color, %color%
Gui, Range_%i%: Show, NA x%x1% y%y1% w%w1% h%h1%
}
}
Re: How to automate arrow buttons overlaid on different pictures?
I tried again on this link and msgbox says Found, yet it did not move through the images...
https://www.instagram.com/p/CfW4IKgpk-Z/?hl=en
https://www.instagram.com/p/CfW4IKgpk-Z/?hl=en
Re: How to automate arrow buttons overlaid on different pictures?
the colors are different, but the shape is very similar to the bigger navigation buttons on the extreme corners.. hence the issue
Re: How to automate arrow buttons overlaid on different pictures?
The search can exclude areas by coordinates.
Re: How to automate arrow buttons overlaid on different pictures?
@newcod3r, please post an image of the first MsgBox shown. The posted Instagram page doesn't contain a Go Back button, so "Found:" was empty. The first MsgBox should contain this:
If it doesn't (displays just "Found:" without the Type and Name after it), then the element wasn't found. Since UIAViewer found the correct ControlType and Name, then check again that your UIA_Interface.ahk is uptodate (is an older version getting loaded from another Lib folder perhaps?).
Code: Select all
Found: Type: 50000 Name: "Next" LocalizedControlType: "button"
Press OK to display found "Next" button