UIA v2

[cgx5871]

@Descolada
Notion' SidePeek
I seem to have found a way.
AutomationId of the newly created Column: "\:r1sf\:"
It's different from the old one.
should be able to get the old AutomationId first, and click New Btn. Then get the new Column AutomationId.
If it is different, it is judged that the new AutomationId is created successfully.
But I don’t know how to write the specific code?
For example, how to get AutomationId

This should correspond to the path of the Notion database

[firsthanddead]

Hey there! I've been diving into the world of toast notifications in Skype, and after scouring the forums, it seems like UI Automation (UIA) is the go-to method for interacting with them. I've gone ahead and installed the UIA library, but I'm hitting a bit of a roadblock in understanding how to make it work.

My goal is to intercept these notifications as they pop up and smoothly relocate them to the opposite side of the screen to avoid any interference. Now, here's where I'm a bit stuck. I'm not entirely sure if I should be extracting an ID for use with WinMove(), or if there's a specific UIA method for moving the window that I should be employing.

I've taken a look at the documentation and even used UIAViewer, but translating the information I see there into a practical solution has proven to be a bit tricky. I've experimented with a few approaches, but nothing seems to have clicked yet.

If anyone has experience or insights on how to achieve this with UIA, I'd greatly appreciate a nudge in the right direction. Perhaps some guidance on which piece of information UIAViewer provides should be used where would be immensely helpful. It seems like I'm just one puzzle piece away from solving this, and any assistance would be fantastic!

[Descolada]

@firsthanddead oddly enough this has already been discussed in this thread: viewtopic.php?f=83&t=113065&p=511987#p511987

EDIT: I think I might have an even improved version:

Code: Select all

#Requires AutoHotkey v2
SetWinDelay -1

WinEventProc := CallbackCreate(CallbackFunc, "F", 7)
hHook := DllCall("SetWinEventHook", "Int", 32780, "Int", 32792, "Ptr", 0, "Ptr", WinEventProc, "UInt", 0, "UInt", 0, "UInt", 0x2) ; EVENT_OBJECT_UNCLOAKED
OnExit((*) => DllCall("UnhookWinEvent", "Ptr", hHook))
Sleep 1000
TrayTip "#1", "This is TrayTip #1"
Sleep 5000
ExitApp

CallbackFunc(hWinEventHook, event, hWnd, *) {
    PrevDetectHiddenWindows := A_DetectHiddenWindows
    DetectHiddenWindows 1
    try if WinGetClass(hWnd) = "Windows.UI.Core.CoreWindow" && InStr(WinGetTitle(hWnd), "notification")
        WinMove(0,,,,hWnd)
    DetectHiddenWindows PrevDetectHiddenWindows
}

@Spitzi if there is a bug with substring matching then it's most likely Microsofts fault because I haven't implemented it. I have only implemented StartsWith and Regex (the one that is working for you)... Be sure to notify me if you encounter the same problem somewhere testable, then I could file a bug report for it.

[cgx5871]

Element.FindElement(condition, scope:=4, index:=1, order:=0, startingElement:=0, cacheRequest:=0)
Isn't the Element in this Element.FindElement the starting el?
Why do we need the parameter startingElement:=0?

[Descolada]

@cgx5871 Element in Element.FindElement(condition, scope:=4, index:=1, order:=0, startingElement:=0, cacheRequest:=0) is the starting element if no startingElement is provided. Otherwise it's the root element which the search is performed relative to.

See this image:

image.png (66.14 KiB) Viewed 1049 times

Source

If scope=4 (TreeScope Descendants) and no startingElement is provided then the search goes like this: R->D->C->M->I->H->Q->B->A->L->G->F->E
If the startingElement was for example M then the search would be: I->H->Q->B->A->L->G->F->E
If you wanted to also include the startingElement in the search then the scope needs to be Subtree (scope:=5).

[cgx5871]

WaitElement is very slow when waiting for the right-click menu. At least 1500ms+
Can I add the waiting method as path.
I tested with the following code. It only takes 70ms

Code: Select all

NotionEl := UIA.ElementFromHandle("ahk_exe Notion.exe")
    Send "{RButton}"
    Loop
        try
            el:=NotionEl.ElementFromPath("VQ")
        until IsSet(el)

[Descolada]

@cgx5871 WaitElement is slow because you are using the default TreeScope of Descendants and it searches the whole tree because of it every time. Yet you don't want to search the whole tree, because the element in question is in the second layer. Yes, you can use that method to wait for ElementFromPath, or you can use Element.WaitElementFromPath, or you can limit your search with NotionEl[1].WaitElement({Type:"Custom", scope:2}). In this case both approaches have the same speed.

[cgx5871]

@Descolada
Didn't attention to the built-in WaitElementFromPath
thank you for your help
I'll learn more

[Nixcalo]

@songdg, the WaitElement call doesn't even work on the penultimate page because the button is of Type Link, so no element is found with the condition {Name:"»", Type:"Text"}. Try MsgBox chromeEl.ElementExist({Name:"»", Type:"Text"}).

@Virlix, I currently have no plans on creating examples on Javascript injection, because there are tons of Javascript tutorials out there to learn from, and JSExecute just inserts the Javascript in the address bar. JSGetElementPos uses Javascript injection as well and does some calculations to get the elements position relative to Client/Window. So there isn't much UIA involved in the process (only the injection part uses UIA)...

Hi Descolada,

I am using your UIA library for AHK v1 and I am finding some troubles/errors in the cases where the script doesn't find the particular element it is supposed to find (perhaps because the website decides to take a long time to load, or it changes in that precise moment). Other than that, I am trying to learn JS injection to do some web scraping, combining it with UIA. Since you speak that you know tons of resources for Javascript, could you please point me to some tutorials/video/courses where I could learn the basics for Javascript injection so I can use the RunJsFromChromeAddressBar function?

Thank you!

Nixcalo

[Descolada]

Since you speak that you know tons of resources for Javascript, could you please point me to some tutorials/video/courses where I could learn the basics for Javascript injection so I can use the RunJsFromChromeAddressBar function?

I do not claim to know tons of resources for Javascript, I claim to know that there are tons of resources. Google or search in YouTube for "Javascript tutorial" and you'll find lots of them, eg W3Schools. You don't need to learn specifically about Javascript injection, because it is just Javascript:
1) Write your Javascript code, example:

Code: Select all

function testNum(a) {
  let result;
  if (a > 0) {
    result = 'positive';
  } else {
    result = 'NOT positive';
  }
  return result;
}

console.log(testNum(-5));

(source)
2) Navigate to a webpage that has Javascript enabled (most do)
3) Copy the code, type into the address bar javascript: and then press Ctrl+v to paste
4) Press Enter to "navigate" to the javascript "url" to execute it
5) Since our code used console.log, press F12 to open the Dev tools panel and select the Console tab to see the output. Alternatively replace console.log with alert

[songdg]

@Descolada
Can you explain JSClickElement(selector) Uses Javascript's querySelector to get and click a Javascript element a bit more. Say I want to click the "Full Editor & Preview" button.

[Descolada]

@songdg, search Google for "Javascript selectors" for more information. cUIA.JSClickElement('#qr_full_editor') should click the "Full Editor & Preview" button. I got the selector by pressing F12 in Chrome, inspecting the button, right-clicking the highlighted HTML code and selecting Copy->Copy selector. The equivalent Javascript for clicking it is document.querySelector('#qr_full_editor').click();

[songdg]

@Descolada
Thank you very much, learn something new again But I got a weird problem, it only work the first time, after that the Javascript code keep enter the Quick Reply's inputbox.
Here is the video https://www.bilibili.com/video/BV1SW4y1A7xX/?share_source=copy_web&vd_source=58bdee7036337147fffef79e251aaa18

[songdg]

However I use the console run the Javascript code without any problem at all.

[Descolada]

@songdg I followed the steps in your video and used the same code, everything worked fine. Could you make sure you are running the latest versions of UIA and UIA_Browser? If updating them doesn't fix the problem then please send me in private message the outputs of A_Clipboard := cUIA.URLEditElement.Dump() and A_Clipboard := cUIA.DumpAll():

Code: Select all

#Requires AutoHotkey v2
#include UIA.ahk
#include UIA_Browser.ahk

WinActivate "ahk_exe chrome.exe"
WinWaitActive "ahk_exe chrome.exe"
cUIA := UIA_Browser()
;A_Clipboard := cUIA.URLEditElement.Dump()
A_Clipboard := cUIA.DumpAll()

[songdg]

@songdg I followed the steps in your video and used the same code, everything worked fine. Could you make sure you are running the latest versions of UIA and UIA_Browser? If updating them doesn't fix the problem then please send me in private message the outputs of A_Clipboard := cUIA.URLEditElement.Dump() and A_Clipboard := cUIA.DumpAll():

Code: Select all

#Requires AutoHotkey v2
#include UIA.ahk
#include UIA_Browser.ahk

WinActivate "ahk_exe chrome.exe"
WinWaitActive "ahk_exe chrome.exe"
cUIA := UIA_Browser()
;A_Clipboard := cUIA.URLEditElement.Dump()
A_Clipboard := cUIA.DumpAll()

Of course I use the latest versions of UIA and UIA_Browser.

[songdg]

@songdg
The special version works like a charm, it also solve the SelectTab problem I encounter before, much appreciated for your help, words failed me when I wanted to express my thanks.

[songdg]

What about selector like this body > div.float-foot > div > ul > li:nth-child(1)

[mora145]

Hi, I'm trying obtain the last child element of this father element. The page is

Spoiler

https://therealacademy.org/

Screenshot_72.png (148.36 KiB) Viewed 653 times

Code: Select all

page := cUIA.GetCurrentDocumentElement()
MsgBox page.DumpAll()
finalElement := UIA.TreeWalkerTrue.GetLastChildElement(page)
MsgBox finalElement.DumpAll()

But I continue obtaining this first element.

Screenshot_73.png (9.05 KiB) Viewed 653 times

You can say me what Im making wrong? @Descolada Ty!!!

[Descolada]

@mora145 you are doing everything correct, but Chrome is not (yet for another unknown reason). You can use finalElement := UIA.ContentViewWalker.GetLastChildElement(page) or finalElement := page[-1] as workarounds.