Page 1 of 2

Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 01:49
by apoklyps3
Is there any way to automate an app if the only thing winspy detects is the apps ui is its main window with class "Qt5QWindowIcon" ?
no controls of the window get detected

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 04:10
by John Mile
apoklyps3 wrote:Is there any way to automate an app if the only thing winspy detects is the apps ui is its main window with class "Qt5QWindowIcon" ?
no controls of the window get detected
jesswg might know a way via acc

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 04:35
by Guest
Two options probably:

* you can send mouse clicks to specific coordinates to activate controls (relative to the Gui so check the CoordMode command). If the controls move around when resized you could try ImageSearch (which should be pretty fast if you can narrow it down to your Window position not your entire desktop)

* you may be able to send shortcuts, for example a "new" button may have the n underlined meaning sending Alt-n will press the new button

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 08:40
by apoklyps3
John Mile wrote:
apoklyps3 wrote:Is there any way to automate an app if the only thing winspy detects is the apps ui is its main window with class "Qt5QWindowIcon" ?
no controls of the window get detected
jesswg might know a way via acc
escuse my stupidity but what does "Acc" stand for?

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 09:32
by jeeswg
I would recommend that you start by trying out the AccViewer GUI window spy, to see the information that it can retrieve. See also the comments at the start of tutorial below:

jeeswg's Acc tutorial (Microsoft Active Accessibility) (MSAA) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=7&t=40590

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 10:55
by apoklyps3
are these still compatible with the latest version of autohotkey?
i have cread lib folder in as it wasn't event there.
placed acc.ahk and anchor.ahk
when i run AccViewer I get error at line 133

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 12:13
by jeeswg
- I just tested the scripts on the latest version of AutoHotkey (v1.1.28.02), and they appeared to work fine.
- Can you state the text of the error message, thanks.

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 14:47
by apoklyps3
err.png
err.png (4.49 KiB) Viewed 4223 times
err.png
err.png (4.49 KiB) Viewed 4223 times

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 15:49
by swagfag
??? did u just copy paste the page source into an ahk file

https://github.com/Drugoy/Autohotkey-sc ... Viewer.ahk

Re: Automate app with no (or hidden) controls?

Posted: 14 Apr 2018, 17:42
by jeeswg
I've provided a zip called AccPack.zip which you can download here:
Acc library (MSAA) and AccViewer download links - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=26201

Re: Automate app with no (or hidden) controls?

Posted: 15 Apr 2018, 05:46
by apoklyps3
getting somewhere :)
could any of the objects be hidden using this?

Re: Automate app with no (or hidden) controls?

Posted: 15 Apr 2018, 15:58
by apoklyps3
is it possible to hide objects within an application's gui using acc and autohotkey?
or only automate the app?
been reading the tutorials, but still can make sense of them so far. it's chinese to me lol
all I got so far is that Acc is winspy on steroids.

Re: Automate app with no (or hidden) controls?

Posted: 17 Apr 2018, 05:11
by apoklyps3
@jeeswg ??

Re: Automate app with no (or hidden) controls?

Posted: 17 Apr 2018, 12:29
by jeeswg
I don't know of Acc being able to hide controls.

Re: Automate app with no (or hidden) controls?

Posted: 17 Apr 2018, 16:43
by apoklyps3
Ok.
Thanks.
Much apreciated

Re: Automate app with no (or hidden) controls?

Posted: 21 Apr 2018, 04:06
by apoklyps3
going forward does Acc work with hidden windows and if so what would be the code to wait for a button (let's say it's object 1.3 in acc basic) to show up and then press it?

Re: Automate app with no (or hidden) controls?

Posted: 21 Apr 2018, 05:00
by jeeswg
AFAIK Acc works on hidden windows. I'd use WinExist/WinWait with DetectHiddenWindows set to on to check/wait for a window's existence (I'd expect that any controls/GUI elements would also exist).

Re: Automate app with no (or hidden) controls?

Posted: 21 Apr 2018, 06:04
by apoklyps3
no controls except for the main window class Qt5QWindowIcon.
other than that just objects detected with Acc basic.
I have no ideea at all how to start with Acc ...

Re: Automate app with no (or hidden) controls?

Posted: 21 Apr 2018, 09:52
by Frosti
The manufacturer says about QT(http://doc.qt.io): "Qt is designed for developing applications and user interfaces once and deploying them across several desktop and mobile operating systems.".

At least I tried to read information from a QT based desktop app(an interface for a drug-database). The best usable data I got with the UISpy.exe from Microsoft.
More beautiful is Inspect.ahk. Both (the exe and the script) based on Microsofts UIAutomation framework. QT is a different framework. To develope a QT-Application for windows desktop,
they wrote that for some platforms there must be some adjustments. I am not a programmer. I only discovered Autohotkey last year and found that it's easy with it to automate large and complex workflows.
In the meantime, I am happy that the software on my work is not state-of-the-art. It's enough for me that the software consists of 4 different Microsoft frameworks. The first edition was in the year 1995.
One part - the newest additional part and unfortunately too - was devoloped in QT. As I said it's only a user-interface for a drug-database but in QT they need about, I stopped counting at 200+ visible and
hidden windows, to build it. QT is scriptable with Javascript, COM and more, but it must be enabled (it must be a part of your app). Look at the following spoiler.
With UIA, mostly you find unnamed Controls, but at least you always get a BoundingRectangle. You can collect all BoundingRectangles and use this positions for SearchImage. I think its the fastest way.
And moving of elements is no problem anymore. If you can identify controls, they have mostly a fixed axis (x oder y). So this axis I would use to identify groups. I've done a lot of interface automation since Sep.17
and learned that it's more efficient to build a database about all controls of the visible interface I want to control as somehow to go on to program.

But if you have a collection of 200+ windows and thousands of custom controls like me, you must be a masochist to try it further with Qt.
Spoiler
I do not want to deprive you all of the poente. The slogan of QT is: "The future is written with Qt!". No! It would be more correct: "In your future, it will never end to write with QT!

Hope, I can help you a little bit and did not scare you with my bad english.

Re: Automate app with no (or hidden) controls?

Posted: 22 Apr 2018, 16:19
by apoklyps3
Your english is more than ok, but I didn't get anything that could help me start with my script...