AccViewer and toast notification

[Martin]

Hello friends!
How can I retrive text from a toast notification in windows 10? (title and text)
WindowSpy.ahk doesn't see them, only AccViewer.ahk does the job.
But AccViewer is very complicated for me...

I want to capture notifications from Telegram from certain channels where I subscribed and store them in variables.

Do you have an example that is easier to understand?

Thanks!

[Martin]

I succeeded in something...
I wrote down and use hwnd of Telegram notification toast, that I got with AccViewer.
Then, I ran the script below.
When notification received via Telegram appears, I press "q" and display everything I want on the screen.

But its one more problem: Telegram hwnd window changes when I restart my computer...

I think I could scan all notifications and filtering by app_name = Telegram.

But how to do this?

Code: Select all

#Include Acc.ahk
;Class (NN) = Windows.UI.Core.CoreWindow
;Process = ShellExperienceHost.exe
;hwnd = 0x30084

q::
oAcc := Acc_Get("Object", "4.1.3", 0, "ahk_id " 0x30084)
app_name:= oAcc.accName(0)
oAcc := ""

oAcc := Acc_Get("Object", "4.1.1", 0, "ahk_id " 0x30084)
sender:= oAcc.accName(0)
oAcc := ""

oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_id " 0x30084)
message:= oAcc.accName(0)
oAcc := ""

msgbox %app_name%`n%sender%`n%message%

return

[jeeswg]

Perhaps this. Cheers.

Code: Select all

;before:
"ahk_id " 0x30084

;after:
"ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe"

;after (you need to change 'MyWindowTitle'):
"MyWindowTitle ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe"

[Martin]

I have tested different combinations between :
"MyWindowTitle"
"ahk_id 0x301D0"
"ahk_class Windows.UI.Core.CoreWindow"
"ahk_exe ShellExperienceHost.exe"

Code: Select all

#Include Acc.ahk
DetectHiddenText, On


q::
;=============== TEST ====================
message_test:=[]

;message_test_1		 <=== OK
oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_id 0x301D0")
message_test[1]:= oAcc.accName(0)
oAcc := ""

;message_test_2
oAcc := Acc_Get("Object", "4.1.2", 0, "New notification ahk_id 0x301D0")
message_test[2]:= oAcc.accName(0)
oAcc := ""

;message_test_3
oAcc := Acc_Get("Object", "4.1.2", 0, "New notification ahk_exe ShellExperienceHost.exe")
message_test[3]:= oAcc.accName(0)
oAcc := ""

;message_test_4
oAcc := Acc_Get("Object", "4.1.2", 0, "New notification ahk_class Windows.UI.Core.CoreWindow")
message_test[4]:= oAcc.accName(0)
oAcc := ""

;message_test_5
oAcc := Acc_Get("Object", "4.1.2", 0, "New notification ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe")
message_test[5]:= oAcc.accName(0)
oAcc := ""

;message_test_6
oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe")
message_test[6]:= oAcc.accName(0)
oAcc := ""

;message_test_7		<=== OK
oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_id 0x301D0 ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe")
message_test[7]:= oAcc.accName(0)
oAcc := ""

;message_test_8		<=== OK
oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_id 0x301D0 ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe")
message_test[8]:= oAcc.accName(0)
oAcc := ""

message_test[9]:="End"


;================ TEST RESULT ===============
array := ""
For Each, Element In message_test {
   If (array <> "") 
		array .= "`n"
   array .= A_Index . " -> " . Element
}
MsgBox, %array%


return

Tests 1, 7, 8 works ok.
So, to work properly, "ahk_id" must be specified.
But I can only get it using AccViewer.
I tried to get it using the code below, but unfortunately "0x301D0" did not appear...

Code: Select all

DetectHiddenText, On
WinGet, id, List,,, Program Manager

Loop, %id%
{
    this_id := id%A_Index%
    WinActivate, ahk_id %this_id%
    WinGetClass, this_class, ahk_id %this_id%
    WinGetTitle, this_title, ahk_id %this_id%
    MsgBox, 4, , Visiting All Windows`n%A_Index% of %id%`nahk_id %this_id%`nahk_class %this_class%`n%this_title%`n`nContinue?
    IfMsgBox, NO, break
}

[jeeswg]

Try:
WinGet, vWinList, List, % "ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe"
And perhaps set DetectHiddenWindows on/off as desired.

[Martin]

Thanks jeeswg for suggestions, I modified my script for testing and also tried some combinations.
Unfortunately, my test is a failure :

I re-checked the toast's hwnd, with AccViewer .
It is "0x301D0".
Then I ran the script below.
And ... detects many hwnds, but there is no "0x301D0" in the list.

I can't explain how AccViewer can see toast's hwnd (though hovering over it), and the script below doesn't list it ...

Code: Select all

DetectHiddenText, On
DetectHiddenWindows, On

q::

WinGet, Win, List
;also tested, without success :
;WinGet, Win, List, % "ahk_class Windows.UI.Core.CoreWindow"
;WinGet, Win, List, "ahk_class" Windows.UI.Core.CoreWindow
;WinGet, Win, List, ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe
;WinGet, Win, List,% "ahk_class Windows.UI.Core.CoreWindow ahk_exe ShellExperienceHost.exe"

FileDelete, win_list.txt
Loop %Win% {
	ID := Win%A_Index%
	WinGetTitle, Title, % "ahk_id " ID
	IfEqual, Title,, Continue
	winList .= ( (WinList<>"") ? "|" : "" ) Title "  {" ID "}"
	FileAppend, %Title% -> %ID%`r`n, win_list.txt
 
}

Gui, Destroy
Gui, -Caption +ToolWindow +Border
Gui, Margin, 0, 0
Gui, Add, ComboBox, Simple R60 w580 -E0x200 vID , %winList%
Gui, Add, Button, +Default gWinActivate, WinActivate
Gui, Show
Return

WinActivate:
     GuiControlGet, ID
     RegExMatch(ID,"\{(.*)\}",TID), hWnd:=TID1
     Gui, Destroy
     WinActivate, ahk_id %hWnd%
Return

GuiEscape:
     Gui, Destroy
Return

[Martin]

I've made some progresses, not many, but hey, its better then nothing..

I noticed that all toasts, no matter what application they come from, use the same hWnd.
hWnd which is only seen through AccViewer (basic or v2).

I understand there are some problems with capturing toast's hwnd "ShellExperienceHost.exe" and "Windows.UI.Core.CoreWindow".
From the topics below, plus a few:
https://www.autohotkey.com/boards/viewtopic.php?t=43338
https://www.autohotkey.com/boards/viewtopic.php?f=14&t=42517
https://www.autohotkey.com/boards/viewtopic.php?t=42994

The only ways to read a toast from Telegram (and not only), I think are two: move the cursor over the notification and capture the information, or to pass some coordinates (x / y of the screen), where the notification window is, and capture information without moving cursor over it.

For example, the second variant looks promising, but it's something basic.
With a loop I can periodically check from which application the notification comes, what is the message, sender, and then filtter "Telegram" toasts.
But it works as long as the notification received is first in toast stack (because of the coordinates) ...

Code: Select all

#Include, acc.ahk

f1:: MsgBox % GetText(1445,938).app_name
f2:: MsgBox % GetText(1445,938).sender
f3:: MsgBox % GetText(1445,938).message

GetText(x,y) {
	Acc := Acc_ObjectFromPoint(child, x,y)
	hWnd := Acc_WindowFromObject(Acc)
	hWnd := Format("0x{:X}", hWnd)

	oAcc := Acc_Get("Object", "4.1.3", 0, "ahk_id " hWnd)
	app_name:= oAcc.accName(0)
	oAcc := ""

	oAcc := Acc_Get("Object", "4.1.1", 0, "ahk_id " hWnd)
	sender:= oAcc.accName(0)
	oAcc := ""

	oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_id " hWnd)
	message:= oAcc.accName(0)
	oAcc := ""
	
	return {app_name:app_name, sender: sender, message: message}
}

[rommmcek]

I tried an idea. Despite I ran out of time and knowledge there is some result :

Code: Select all

#NoEnv
#SingleInstance Force
;~ #Persistent
SetBatchLines, -1
global Tt_Update, hFont, UnhkWE, GlobF ;, TTUd, Hk:=""

HookProcAdr := RegisterCallback( "HookProc", "F" )
hWinEventHook := SetWinEventHook( 0x800B, 0x800B, 0, HookProcAdr, 0, 0, 0) ; 0x800B EVENT_OBJECT_LOCATIONCHANGE
OnExit, HandleExit
SoundBeep, 400
SoundBeep, 800
#Include Acc.ahk
Return

;Based on Serenity's hook https://autohotkey.com/board/topic/32662-tool-wineventhook-messages/
HookProc( hWinEventHook, Event, hWnd, idObject, idChild, dwEventThread, dwmsEventTime ) {
        WinGetTitle, Title, % "ahk_id " hwnd
        WinGetPos,,,, h, % "ahk_id " hwnd
        if (h && Title == "New notification") {
            oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_id " hwnd),  message:= oAcc.accName(0), oAcc:= ""
            ToolTip % Format("0x{:x}", hwnd) "`n" Title "`n" message
        }
        else if (!h && Title = "New notification")
            ToolTip
}

SetWinEventHook(eventMin, eventMax, hmodWinEventProc, lpfnWinEventProc, idProcess, idThread, dwFlags) {
	DllCall("CoInitialize", Uint, 0)
	return DllCall("SetWinEventHook"
	, UInt, eventMin
	, UInt, eventMax
	, Ptr, hmodWinEventProc
	, Ptr, lpfnWinEventProc
	, UInt, idProcess
	, UInt, idThread
	, UInt, dwFlags
	, Ptr)
	
}

UnhookWinEvent() {
	Global
	UnhkWE:= DllCall( "UnhookWinEvent", Ptr, hWinEventHook )
    Sleep, -1
}

+Esc::
!F2::
HandleExit:
	UnhookWinEvent()
	if (A_ThisLabel == "!F2")
		Reload
    Else {
        SoundBeep, 800
        SoundBeep, 400
    }
ExitApp

Esc::ExitApp

P.s.: Code is based on my ToolTipAll, which suffers I believe under memory leakage when exiting the script!

[Martin]

I made great progress, thanks for codes, guys !
rommmcek, you illuminated my mind, I forgot about "Events"..

As for the code, I made some minor changes:
- I replaced "0x800B -> EVENT_OBJECT_LOCATIONCHANGE" with 0x8002 -> EVENT_OBJECT_SHOW ";
- "WinGetTitle, Title,%" ahk_id "hwnd" didn't work, even if I formatted it with "Format (" 0x {: x} ", hwnd", I dont know why...
- I put "try" as a safety measure. Because sometimes it fails ("Object not found, Continue script?"), so in this case it can jump over minor errors;
- when receiving a Telegram notification, the script pops up a small window with the message and sender, gives a beep, and saves everything in a file.

The script works very well, I tested it.
But it has some bugs:
- when the toast appears, the script displays the message and beeps several times (it had to be one beep!). I tried to save the "sender" and "message" and compare them with the new ones, and if they are the same, just ignore them. But for some reason it doesn't work.
You can see the same message being recorded in the file several times;
- I put on "exit from script" (Esc key) a triple beep. But if you press Esc, the triple beep is played twice. I do not know why.

It was tested on a machine with Win10/x64, Authotkey Unicode/v1.1.31.01.
For testing I use two different Telegram accounts installed on the same PC and two programs.
One account uses Telegram software, and another uses Unigram software, one sends to the other one.

Here's the new script and bugs:

Code: Select all

;===== HEADER =====
{
	;#NoEnv
	#SingleInstance Force
	#Include Acc.ahk
	Global sender_o, message_o

	HookProcAdr := RegisterCallback( "HookProc", "F" )
	hWinEventHook := SetWinEventHook( 0x8002, 0x8002, 0, HookProcAdr, 0, 0, 0) ; 0x8002 = EVENT_OBJECT_SHOW
	OnExit, HandleExit
	SoundBeep, 800,250
	SoundBeep, 1100,100
	SoundBeep, 1500,100

	Return
}



;Based on Serenity's hook https://autohotkey.com/board/topic/32662-tool-wineventhook-messages/
HookProc( hWinEventHook, Event, hWnd, idObject, idChild, dwEventThread, dwmsEventTime ) {

	app_name:=""
	sender:=""
	message:=""

	try{	

		oAcc := Acc_Get("Object", "4.1.3", 0, "ahk_id " hWnd)
		app_name:= oAcc.accName(0)
		oAcc := ""

		oAcc := Acc_Get("Object", "4.1.1", 0, "ahk_id " hWnd)
		sender:= oAcc.accName(0)
		oAcc := ""

		oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_id " hWnd)
		message:= oAcc.accName(0)
		oAcc := ""
	}	
	
	if (app_name == "Telegram"){
		if ( (sender != sender_o) && (message != message_o) ){
			sender_ := sender
			message_:= message
			
			SplashTextOn, 400, 100, % app_name, %sender% -> %message%
			SoundBeep, 1100,100
			sleep 2000
			SplashTextOff

			FileAppend, %app_name% - > %sender% -> %message%`r`n, messages.txt
		}
	}

			
}




;==== END ====
{

	SetWinEventHook(eventMin, eventMax, hmodWinEventProc, lpfnWinEventProc, idProcess, idThread, dwFlags) {
		DllCall("CoInitialize", Uint, 0)
		return DllCall("SetWinEventHook"
		, UInt, eventMin
		, UInt, eventMax
		, Ptr, hmodWinEventProc
		, Ptr, lpfnWinEventProc
		, UInt, idProcess
		, UInt, idThread
		, UInt, dwFlags
		, Ptr)
		
	}

	UnhookWinEvent() {
		Global
		UnhkWE:= DllCall( "UnhookWinEvent", Ptr, hWinEventHook )
		Sleep, -1
	}

	Esc::
	!F2::
	HandleExit:
		if (A_ThisLabel == "!F2")
			Reload
		Else {
			SoundBeep, 1500,100
			SoundBeep, 1100,100
			SoundBeep, 800,500
		}
ExitApp

}

BTW...
I hope to use this script in a noble way ...
When it will be ready, it will stay on that PC, and will be connected to three Telegram channels, belonging to an earth physics institute, which gives earthquake warnings (20-30 seconds before the earthquake arrives). It brodcast on three channels depending on the magnitude of earthquake (M5-6, M6-7, M7+).
Then, the PC will send a notification to a Wemos D1 mini, which will switch a relay to which a 130dB, 30w siren is connected outdoor, on the street.
I hope everyone hears my warning, so fewer people will be injured.
So I can deserve a beer!

[rommmcek]

I forgot to mention, that I was able to detect Toasts form (un/)plugging audio jack and USB mass storage device on Win10 Ahk 1.1.31.01.
Your script does not detect those Toasts for me, but obviously works for your needs.

In my opinion (but I'm not the competent one to tell) stuff like SoundBeep, Sleep and the like inhibitors for fluency of the hook does not belong there.
Addition: For sound feedback use e.g. Midi and to switch of the SplashText use e.g.: SetTimer TurnSplashTextOff, -2000

bye!

Edit: I forgot, first line of HandleExit: routine should be UnhookWinEvent(). I will edit my post.
Edit2: You have double exit (once by the Esc hotkey and once by the OnExit message). Use Esc:: ExitaApp to avoid it!
(Thanks, now I release object oAcc:= "" too!)
Edit3: It seems to me you have a typo in the script, should be:

Code: Select all

		if ( (sender != sender_o) && (message != message_o) ){
			sender_o := sender ; you have "sender_"
 			message_o:= message ; you have "message_"

[Martin]

Yes, I also noticed your edits earlier
Works very well without UnhookWinEvent(), too .

I returned to the value you said "0x800B -> EVENT_OBJECT_LOCATIONCHANGE".

Because I also tried 0x8002 -> EVENT_OBJECT_SHOW, but it doesn't display the first notification received, I don't know why.
But neither 0x800B, works flawless: because it displays the first notification correctly, but if others notifications appear until the visible one disappears, it does not see them. It's a bug, but it's acceptable.

0x8002 sees all notifications except the first one, but I will stay for now with 0x800B.

But I'm pleased with how the script works, I have to fill in here-there with what I need and we'll see how it will work.

rommmcek, jeeswg, many thanks !

There is the last version, unfinished, if it interests anyone:

Code: Select all

;===== HEADER =====
{
	;#NoEnv
	#SingleInstance Force
	#Persistent
	SetBatchLines,-1
	#Include Acc.ahk
	;#Include TTS.ahk

	HookProcAdr := RegisterCallback( "HookProc", "F" )
	hWinEventHook := SetWinEventHook( 0x800B , 0x800B , 0, HookProcAdr, 0, 0, 0) ;  "0x800B -> EVENT_OBJECT_LOCATIONCHANGE" OR 0x8002 -> EVENT_OBJECT_SHOW ";
-
	SoundBeep, 800,100
	SoundBeep, 1100,100
	SoundBeep, 1500,100

	sender_o	:=	""
	message_o 	:=	""
	Return
}



;Based on Serenity's hook https://autohotkey.com/board/topic/32662-tool-wineventhook-messages/
HookProc( hWinEventHook, Event, hWnd, idObject, idChild, dwEventThread, dwmsEventTime ) {
	Global sender_o, message_o
	
	app_name:=""
	sender:=""
	message:=""

	try{	

		oAcc := Acc_Get("Object", "4.1.3", 0, "ahk_id " hWnd)
		app_name:= oAcc.accName(0)
		oAcc := ""

		oAcc := Acc_Get("Object", "4.1.1", 0, "ahk_id " hWnd)
		sender:= oAcc.accName(0)
		oAcc := ""

		oAcc := Acc_Get("Object", "4.1.2", 0, "ahk_id " hWnd)
		message:= oAcc.accName(0)
		oAcc := ""
	}	
	
	if (app_name == "Telegram"){
		message:= RegExReplace(message,"ims):"," ")
		message:= RegExReplace(message,"ims)\.",",")
		message:= RegExReplace(message,"ims)`r"," ")
		message:= RegExReplace(message,"ims)`n"," ")
		message:= RegExReplace(message,"ims)\s+"," ")
		message:= RegExReplace(message,"ims)\/\/"," ")
		
		if (message != message_o){

			sender_o := sender
			message_o:= message
				
			ACTIONS(sender, message)
		}
		
		return
	}

			
}



;==== ACTIONS ===
{

	ACTIONS(sender, message){
			
		Record(sender,message)
		;Outdoor_Horn_ON()
		;TTS_Alarm(sender,message)
		Display_Message(sender,message)
		;PC_Alarm_ON()
	
	return
	}


	Display_Message(sender,message){
		SplashTextOn, 400, 100, Message, %sender% -> %message%
		SoundBeep, 1100,100
		sleep (2000)
		SplashTextOff
	return
	}
	

	Record(sender,message){
		FileAppend,  %sender% -> %message%`r`n, messages.txt	
		return
	}
	
	/* 	
	TTS_Alarm(sender,message){
		Global
		SoundSet, 100, MASTER
		VoiceName 	:= "IVONA 2 Carmen"
		VoiceRate	:= 3
		VoiceVolume := 100
		VoicePitch	:= 0
		IvonaVoice	 := TTS_CreateVoice(VoiceName, VoiceRate, VoiceVolume, VoicePitch)

		TTS(IvonaVoice, "Speak", message)		
		return
	}
	 */
	
	Outdoor_Horn_ON(){
		;\....\
		return
	}


	Outdoor_Horn_OFF(){
		;\....\
		return
	}
	

	PC_Alarm_ON(){
		SoundSet, 100, MASTER
		;\....\
		return
	}


	PC_Alarm_OFF(){
		Global sound_o
		SoundSet, sound_o, MASTER
		;\....\
		return
	}



	
}


;==== END ====
{
	
	SetWinEventHook(eventMin, eventMax, hmodWinEventProc, lpfnWinEventProc, idProcess, idThread, dwFlags) {
		DllCall("CoInitialize", Uint, 0)
		return DllCall("SetWinEventHook"
		, UInt, eventMin
		, UInt, eventMax
		, Ptr, hmodWinEventProc
		, Ptr, lpfnWinEventProc
		, UInt, idProcess
		, UInt, idThread
		, UInt, dwFlags
		, Ptr)

		;DllCall( "UnhookWinEvent", Ptr,hWinEventHook )
		;DllCall( "GlobalFree", Ptr,&HookProcAdr, Ptr )
	}

	UnhookWinEvent() {
		Global
		UnhkWE:= DllCall( "UnhookWinEvent", Ptr, hWinEventHook )
		Sleep, -1
	}
	

	Esc::
	!F2::
		if (A_ThisLabel == "!F2")
			Reload
		Else {
			UnhookWinEvent()
			SoundBeep, 1500,100
			SoundBeep, 1100,100
			SoundBeep, 800,500
			ExitApp
		}
	

}