Help on using a context menu selection to click on specific coordinates

[thomas messi]

Hello good day. Grateful for the answer and helped a lot.
I have another question:
By pressing a key combination I open a context menu and can navigate through the items with up and down arrows.

Now, I would like to make this work.
By clicking on the item and pressing enter, he would click on the coordinate that I have mapped.

for example.:

when clicking by pressing the ENTER key on item 1 of the menu, it went to the position MouseClick left, 130, 140
As there are many items to be mapped, I found the solution to make a context menu and place the items and when clicking go to the position and select the item.

Thank you for your help.
note: this application that i am trying to map and make accessible, for us blind the screen reader cannot read anything. So the only resource for us to use the functions of this application and mapping and clicking with the mouse via shortcuts.

[Pepineros]

Hi Thomas!

If I understand correctly, you would have a context menu with for example:
- Start Chrome
- New email
- Start MS Word

Now if you hit Enter on the first option, you want the mouse to click the coordinates of the shortcut to Chrome on your desktop. If you hit Enter on the second option, you want the mouse to click the coordinates of the "New email" button in Gmail (for example). Am I understanding you correctly?

- P

[Pepineros]

Oh wait, I think I understand now. You're trying to automate a specific application that the screen reader doesn't work on.

This should not be difficult, provided that you know the size of the application (so every clickable thing on your screen is always in the same place).

No time to post full code now but I'll come back to this later if nobody beats me to it.

[thomasmessi]

It is an application that, when loading on the screen, has nothing accessible for screen readers for blind people.
And as there are many options that can be clicked, and each one does something different. I think the solution is to create a context menu with several items, and each item would trigger an option on the screen of this application.
Because, when creating a menu, the screen reader can already interpret and we can move the arrow up and down between the options of the items. Now just missing, when clicking with ENTER on the option the mouse moves and go to click on the mapped position.
I already managed to make the context menu
I do not know if it's right.

! ^ + c ::; alt-control-shift-c
MouseClick right, 350, 350
Gosub, ChannelMenu
Return
; open the menu selection

ChannelMenu:
Menu, MyChannelMenu, Add
Menu, MyChannelMenu, DeleteAll
Menu, MyChannelMenu, Add, item 1, MyChannelMenuHandler
Menu, MyChannelMenu, Add, item 2, MyChannelMenuHandler
Menu, MyChannelMenu, Add, item 3, MyChannelMenuHandler
Menu,
MyChannelMenu, Show
Return

MyChannelMenuHandler:
Send {Appskey}
Sleep 100; give the context menu some time to appear
Send {down% A_ThisMenuItemPos%}
Return




thank you for your help

[Pepineros]

As a general rule, build your menu once (in the autoexec part of the script), then use a hotkey only to show it.

Code: Select all

#noenv
#singleinstance,force

menu,channelmenu,add,Save,click1
menu,channelmenu,add,Open new,click2
menu,channelmenu,add,Select,click3

appskey::
  winactivate,the application you want to click on
  winmaximize   ; make sure the app is fixed size so the things you want to click are always in the same place
  menu,channelmenu,show
return

click1:
  click,120 60   ; clicks once at x120 y60
return
click2:
  click,400 20 2   ; double-clicks at x400 y20
return
click3:
  click,200 225 D   ; hold the left button down at x200 y200
  click,100 0 0 Rel   ; move the mouse 100px to the right while holding down mouse button (to select text for example)
  click,u   ; release mouse button
return

Now when you hit the appskey a menu should appear at cursor position with the three items defined above. Just change the names of the items and the coordinates / other parameters of the click command to suit your needs.

If you start adding lots of options in your menu including submenus you might want to look at using loops to build the menu but that's up to you.

You can use functions instead of subroutines for the if you prefer. Subroutines will be marginally quicker (although not so as you would notice while using the script) but if you want to use the menu for more than just clicking the mouse a function may be more appropriate.

Let me know how you get on!

[thomasmessi]

hello noble friend! Good Morning!
I sent you a private message explaining in more detail what I'm trying to do
I want to record my screen on video and send it to you, so you can know how I'm doing and how you could try to help me.
look at your message box here on the forum.

Hugs