Xaml? (v2.0-beta.2)

Discuss Autohotkey related topics here. Not a place to share code.
Forum rules
Discuss Autohotkey related topics here. Not a place to share code.
User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Xaml? (v2.0-beta.2)

Post by kczx3 » 17 Oct 2021, 08:47

[Moderator's note: Topic split from v2.0-beta.2 announcement.]
lexikos wrote:
16 Oct 2021, 22:42
Added maxversiontested to application manifest (allows scripts to create XAML islands).
Don't want to read into this too much but was it done merely because it was simple to do and what it would ultimately allow? (Even if there are very few users that have the knowledge/desire to get Xaml Islands working in-script)

lexikos
Posts: 9494
Joined: 30 Sep 2013, 04:07
Contact:

Re: Xaml? (v2.0-beta.2)

Post by lexikos » 23 Oct 2021, 20:50

It relates to a script I have been working on, in which I have created a simple XAML GUI purely as a test. The purpose of the script is to make Windows Runtime APIs easier to consume with AutoHotkey. It is essentially a "language projection" of the APIs into AutoHotkey, similar to what is available in JavaScript.

User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Re: Xaml? (v2.0-beta.2)

Post by kczx3 » 24 Oct 2021, 16:03

That's exciting to hear. The Runtime opens up a lot of possibilities and functionality that has previously been difficult (or impossible) to achieve.

I tried messing with the Xaml Islands demo on MSDN after reading the particular change but I am very slow and progress at understanding some of the concepts. Really didn't get very far because I couldn't figure out getting an IDesktopWindowXamlSourceNative from the created DesktopWindowXamlSource object. It says to "cast" the object to it which I've never heard of in terms of AHK. Since Windows Runtime is based on COM I guess it would be a call to ComObjQuery with the interfaces IID. Might be way off on that.

Look forward to your progress on this project.

lexikos
Posts: 9494
Joined: 30 Sep 2013, 04:07
Contact:

Re: Xaml? (v2.0-beta.2)

Post by lexikos » 25 Oct 2021, 02:54

Yes, in the C# and C++ language projections, typecasts ultimately boil down to QueryInterface calls. Methods are generally split up into several interfaces, introduced in different Windows versions. The proper interface and IID definitions are hard to find, because WinRT is intended to be used via a language projection, and the projection takes care of those details based on metadata. ILSpy can be used to read the metadata (C:\Windows\System32\WinMetadata\*.winmd). For IDesktopWindowXamlSource, it shows:
[Windows.Foundation.Metadata.Guid(3582312417u, 255, 20926, 186, 29, 161, 50, 153, 86, 234, 10)]

Some of the details have been hard to work out, because the documentation heavily favours the C# and C++ language projections (although there are others), and is sometimes light on details that would only be needed for languages without a projection.

In the current iteration of my script, class activation looks like this:

Code: Select all

dwxs := WinRT("Windows.UI.Xaml.Hosting.DesktopWindowXamlSource")()
... but I also intend to support this:

Code: Select all

dwxs := Windows.UI.Xaml.Hosting.DesktopWindowXamlSource()
The projection already takes care of most of the low-level COM details, so you just get an object of the appropriate class. All properties and methods are defined through the normal AutoHotkey v2 mechanisms and prototype chain.

The XAML test code doesn't serve as a good example of the WinRT projection, since there's only a small percentage of the test code that visibly uses the projection. (At the moment, a third of the code is for making keyboard input work, and another quarter of the code is just to make the titlebar black. ;)) This is a better example (see Example_JsRT_Toast.ahk for comparison):

Code: Select all

    appId := "{6D809377-6AF0-444b-8957-A3773F02200E}\AutoHotkey\AutoHotkey.exe"
    TNM := WinRT("Windows.UI.Notifications.ToastNotificationManager")  ; Get the class itself.
    TNM.History.Clear appId  ; History is a static property.
    toastXml := TNM.getTemplateContent(1)  ; Enum not supported yet, so pass an integer.
    textEls := toastXml.getElementsByTagName("text")
    textEls.GetAt(0).InnerText := "Hello, world!"  ; GetAt is a member of IVectorView<IXmlNode>, implemented by XmlNodeList.
    toastXml.getElementsByTagName("image").Item(0)  ; Item is equivalent to GetAt, for XmlNodeList.
            .setAttribute("src", "C:\Data\Scripts\Drafts\ActiveScript\sample.png")
    toastNotifier := TNM.createToastNotifier(appId)  ; Call a static method.
    notification := WinRT("Windows.UI.Notifications.ToastNotification")(toastXml)  ; Instantiate a class.
    toastNotifier.show(notification)  ; Call an instance method.
This was working test code (until I started refactoring).


At the moment I am cleaning up and restructuring the script, but there are some parameter and return types that aren't implemented yet, making it difficult or unsafe to use some methods/properties. In particular, I haven't yet implemented delegates, structs or array parameters.


Eventually I anticipate built-in support for WinRT will be implemented, but before then I will probably work on more general but related functionality and "architectural" changes. For instance, struct support and something to replace DllCall.Bind (in a way that can handle more complex parameter types). Working on this has given me some ideas and inspiration.


Also of interest is microsoft/win32metadata, which uses the same metadata format (ECMA-335) but with different semantics. Unfortunately the metadata for this is currently about 15 MB and not included with the OS (yet?). Automatic built-in support for the entire Win32 API is probably out of reach, but the metadata can be used to automatically generate wrappers that are more usable and robust than using DllCall directly, and don't require manual calculation of offsets or similar work.

kczx3 wrote:I tried messing with the Xaml Islands demo on MSDN after reading the particular change but I am very slow and progress at understanding some of the concepts.
The documentation and examples are terrible at showing exactly what is actually required to get XAML islands working. The only official samples I could find were over-complicated and some of the details obscured by the C++/WinRT language projection.

I read a lot of documentation, and I don't think the maxversiontested requirement was clearly stated anywhere. It also didn't come up in my searches. I downloaded the official sample and tried to reduce it to the minimal necessities, but that failed spectacularly. It had some kind of dependency on a composable "app" class which, when I attempted to remove it, inexplicably caused the project to crash. I felt that the sample completely missed the point of XAML islands, which was to use them to modernize existing Win32 apps:
XAML Islands is a technology that enables Windows developers to use new pieces of UI from the Universal Windows Platform (UWP) on their existing Win32 Applications [...]. This allows them to gradually modernize their apps at their own pace, making use of their current code as much as they want.
Source: XAML Islands - A deep dive - Part 1 - Windows Developer Blog
The uwplib source code was a bit more helpful, but it still wasn't obvious. (Its readme does say "Create an app with the attached manifest", but doesn't explain that one specific element in the manifest is required for XAML islands to function.) I created a new C++ project from scratch and reproduced the issue I had in script. Eventually while comparing the working projects with the non-working projects, I realized the significance of the maxversiontested attribute. Then looking back at the MS article, it was apparently there all along:
5. Set the maxversiontested value in your application manifest to specify that your application is compatible with Windows 10, version 1903.
Source: Host a standard WinRT XAML control in a C++ desktop (Win32) app using XAML Islands - Windows apps | Microsoft Docs
In my defense, it doesn't say why or that this is a requirement of XAML islands; it's like a bent needle in a haystack of mostly irrelevant information.

So by now, the DesktopWindowXamlSource was being successfully constructed (instead of throwing an incomprehensible error code), but the GUI was blank. Part of the process used in all of the examples I found was like this:

Code: Select all

// Update the XAML Island window size because initially it is 0,0.
SetWindowPos(hWndXamlIsland, 0, 200, 100, 800, 200, SWP_SHOWWINDOW);
... so we should be able to resize it with ControlMove, right? That didn't solve the problem. After some time, I noticed SWP_SHOWWINDOW. The window is not only initially 0,0, but apparently also hidden. This isn't mentioned in the comment. :headwall:

So then ControlShow gave me a visible XAML island - a black box filling the GUI - but it was completely blank. It was supposed to have a TextBox on it. :facepalm:

Eventually, I replaced ControlMove and ControlShow with SetWindowPos, and it all worked.

User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Re: Xaml? (v2.0-beta.2)

Post by kczx3 » 25 Oct 2021, 08:15

Its always a pleasure reading your posts and how fluently you speak of such complicated topics and details :)

I have the same sentiment regarding the documentation and the heavy use of the projections. They make a lot of assumptions that no one will/would ever want to fiddle with it in a direct manner. Even the basic intro app uses some nuget package that ultimately generates something used by the project. I think it used those *.winmd files you mentioned. I hadn't thought to do more research on what those files were and frankly didn't consider it would use them as metadata for code generation.
lexikos wrote:
25 Oct 2021, 02:54
(At the moment, a third of the code is for making keyboard input work, and another quarter of the code is just to make the titlebar black. ;))
:shock: Wow!

The Toast notification sample is great to look at and compare with the ActiveScript version. Where does the GUID used in the appId variable come from? Random or from the AHK source manifest perhaps? Would this new "projection" allow background tasks? I'm thinking something like windows.applicationmodel.background.toastnotificationhistorychangedtrigger or closed app activation. I'm thinking these would require compiling your script and modifying the EXE's manifest or metadata otherwise the AHK executable would be triggered for any app used as a script. Just things coming to mind (don't let me digress here).
lexikos wrote:
25 Oct 2021, 02:54
Eventually I anticipate built-in support for WinRT will be implemented, but before then I will probably work on more general but related functionality and "architectural" changes. For instance, struct support and something to replace DllCall.Bind (in a way that can handle more complex parameter types). Working on this has given me some ideas and inspiration.
Sounds like a hefty goal! Always great (and fun!) when you can find things like this that give satisfaction and "ideas and inspiration". At least, I hope you're having fun with it :)

Re: maxversiontested

Bothers the heck out of me when its not clear why a requirement is exactly that. What problem does it solve, you know? Argh!
lexikos wrote:
25 Oct 2021, 02:54
After some time, I noticed SWP_SHOWWINDOW
I noticed this as well and it seemed odd. I hadn't gotten that far but it was something I had noticed and thought about as to how I would pass that flag along with the moving of the window.

Thanks for sharing your work and thoughts on the subject!

lexikos
Posts: 9494
Joined: 30 Sep 2013, 04:07
Contact:

Re: Xaml? (v2.0-beta.2)

Post by lexikos » 27 Oct 2021, 00:35

kczx3 wrote:
25 Oct 2021, 08:15
Where does the GUID used in the appId variable come from?
It corresponds to FOLDERID_ProgramFilesX64, but I didn't make up the ID; I retrieved it with PowerShell:

Code: Select all

Get-StartApps | where name -contains AutoHotkey
I think on some versions of Windows it is/was required to specify an ID of an app that's pinned to the Start menu. It seems that some versions/systems display the notification regardless.
You must include the AppUserModelID of your app's shortcut on the Start screen each time that you call CreateToastNotifier. If you fail to do this, your toast will not be displayed.
Source: Sending a toast notification from the desktop - Win32 apps | Microsoft Docs

Would this new "projection" allow background tasks?
I do not know what that is.
...
Now that I've read up on it, it looks feasible, but may require a few things that I don't plan to implement initially.

I'll be releasing the script when it's ready for basic use. If there's anything that's still difficult to do, we can sort it out. Anything that's not applicable to the Windows Runtime in general (like the additional code for XAML islands), should go in an optional #include.
To be clear, these are for detecting different events, and only the first link relates to background tasks. The protocol activation type is much easier to use, although if you want it to launch your script, you must register a protocol handler for that in the registry. A protocol handler can just execute a command line, although I tested it with "ms-calculator:", which launches the Calculator app.

The second link is C#-specific; it says unpackaged apps must use the ToastNotificationManagerCompat class to detect notification activation. That class is part of a toolkit written in C#, not a standard OS component. It basically automates some of the details described in the link below.

Refer to Send a local toast notification from other types of unpackaged apps. Basically:
  • Implement the INotificationActivationCallback interface.
  • Implement the IClassFactory interface and register the object with CoRegisterClassObject.
  • Write some values to the registry.
  • Set your AppUserModelID on a shortcut in the Start menu, and use the same ID when displaying the toast.
These are fairly trivial interfaces to implement. They both extend IUnknown, so are plain COM interfaces, not WinRT.

I'm thinking these would require compiling your script and modifying the EXE's manifest or metadata
When the Windows Runtime or UWP documentation refers to a manifest, it's usually the package manifest, i.e. package.appxmanifest. You might need to build a sparse MSIX package (or package your script as an app) to use background tasks. On the other hand, the documentation says that's required for background tasks and notifications (but notifications work without it).

Bothers the heck out of me when its not clear why a requirement is exactly that. What problem does it solve, you know? Argh!
The best parts are:
Note that setting a higher value means older versions of Windows won't run the app properly because every Windows release only knows of versions before it. If you want the app to run on Windows 10, version 1903 (build 10.0.18362), you should either leave the 10.0.18362.0 value as is, or add multiple maxversiontested elements for the different values the app supports.
Source: Host a standard WinRT XAML control in a C++ desktop (Win32) app using XAML Islands - Windows apps | Microsoft Docs
It's just stupid on too many levels.

User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Re: Xaml? (v2.0-beta.2)

Post by kczx3 » 27 Oct 2021, 07:45

Thanks for the information! It seems I've (once again) dove far beneath my level of knowledge in some of these topics. Its a pattern that I can't seem to stop repeating :)

I very much look forward to diving into the script.

lexikos
Posts: 9494
Joined: 30 Sep 2013, 04:07
Contact:

Re: Xaml? (v2.0-beta.2)

Post by lexikos » 04 Jul 2022, 17:16

I haven't done much with it in the last several months, but I've tidied it up a bit and uploaded to GitHub:
https://github.com/Lexikos/winrt.ahk

User avatar
thqby
Posts: 391
Joined: 16 Apr 2021, 11:18
Contact:

Re: Xaml? (v2.0-beta.2)

Post by thqby » 06 Jul 2022, 08:59

@lexikos
The current version cannot run with 32 bits. I've added the missing cdecl.
An error occurred in RoGetParameterizedTypeInstanceIID.

User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Re: Xaml? (v2.0-beta.2)

Post by kczx3 » 06 Jul 2022, 13:51

I also am finding that throwing an error from within an event's callback crashes the script with code 3221226107. Take your xaml.ahk example, modify the callback for the Cancel button to instead call a nested function named "Cancel" and have it simply throw Error("INVALID").

User avatar
thqby
Posts: 391
Joined: 16 Apr 2021, 11:18
Contact:

Re: Xaml? (v2.0-beta.2)

Post by thqby » 06 Jul 2022, 20:21

because only OSError is handled in the delegate.

User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Re: Xaml? (v2.0-beta.2)

Post by kczx3 » 07 Jul 2022, 10:20

For now I'll just not throw from these callbacks and instead do nothing. I didn't see any notes about that in the readme. Thanks for doing the hard work of digging in the code @thqby.

lexikos
Posts: 9494
Joined: 30 Sep 2013, 04:07
Contact:

Re: Xaml? (v2.0-beta.2)

Post by lexikos » 08 Jul 2022, 05:56

thqby wrote:
06 Jul 2022, 08:59
The current version cannot run with 32 bits.
I hardly bother to think about 32-bit nowadays.
I've added the missing cdecl.
An error occurred in RoGetParameterizedTypeInstanceIID.
Did you add any "cdecl" other than for the msvcrt calls?

The whole parameterized IID API is just weird. For instance, the documentation says:

Code: Select all

HRESULT RoGetParameterizedTypeInstanceIID(
                  UINT32                     nameElementCount,
  [in]            PCWSTR                     *nameElements,
  [in]            const IRoMetaDataLocator & metaDataLocator,
  [out]           GUID                       *iid,
  [out, optional] ROPARAMIIDHANDLE           *pExtra
);
No calling convention is declared, so it's cdecl, right?

Spoiler

Then, for IRoMetaDataLocator,

Code: Select all

struct IRoMetaDataLocator {
  HRESULT Locate(
    PCWSTR                     nameElement,
    IRoSimpleMetaDataBuilder & metaDataDestination
  );
};
The struct has no fields or virtual methods, so it makes no sense. The actual definition is:

Code: Select all

struct IRoMetaDataLocator
{
    STDMETHOD(Locate)(
        PCWSTR                      nameElement,
        _In_ IRoSimpleMetaDataBuilder&   metaDataDestination
    ) const = 0;
};
STDMETHOD uses virtual, so really, it is an interface...

but it's not a COM interface. As you can see, it doesn't extend IUnknown, let alone IInspectable.

And notice the parameter is a reference, not a pointer, because again, IRoSimpleMetaDataBuilder isn't a COM interface. It's expected to be constructed on the stack in C++. That's actually not bad for us; it makes things simpler.

The real problem is that the GUID parameters are passed by value, which is very uncommon with COM, so also unexpected. I think I noticed this while I was prototyping the script (last year), and wrote code for x64 (where structs larger than 8 bytes are implicitly passed by address) with the intent to fix it later, once things were actually working.

User avatar
thqby
Posts: 391
Joined: 16 Apr 2021, 11:18
Contact:

Re: Xaml? (v2.0-beta.2)

Post by thqby » 08 Jul 2022, 07:33

lexikos wrote:
08 Jul 2022, 05:56
The real problem is that the GUID parameters are passed by value, which is very uncommon with COM, so also unexpected. I think I noticed this while I was prototyping the script (last year), and wrote code for x64 (where structs larger than 8 bytes are implicitly passed by address) with the intent to fix it later, once things were actually working.
Thank you. I forgot to think about it.
After modification, it can run normally in 32 bits.

Consider adding support for passing structures by value in later versions of AHK? And I can pull request.
Further, add support for custom types, similar to the Python's ctypes from_param.

lexikos
Posts: 9494
Joined: 30 Sep 2013, 04:07
Contact:

Re: Xaml? (v2.0-beta.2)

Post by lexikos » 08 Jul 2022, 18:13

kczx3 wrote:
06 Jul 2022, 13:51
I also am finding that throwing an error from within an event's callback crashes the script with code 3221226107.
In general, a process "crash" is when a native exception is thrown and not handled, and the OS "handles" it by terminating the process and writing to the event log (or displaying a dialog). In this case, the event log says:
Unhandled exception at 0x00007FFDF71F4980 (Windows.UI.Xaml.dll) in AutoHotkey.exe: 0xC000027B: An application-internal exception has occurred
So basically, this is the normal behaviour of unhandled exceptions for XAML...

I didn't really consider how the caller of the callback would handle a failure result. The delegate code is meant to be general, not specifically for XAML. PopupMenu, for instance, just ignores it; there's nothing even shown in the VC++ debugger output. It looks like the ToastNotification events ignore it as well.

There's an event that seems relevant, but doesn't seem to be working. Perhaps it isn't applicable to desktop apps?

Code: Select all

Windows.UI.Xaml.Application.Current.add_UnhandledException(Unhandled)
The UnhandledException event is used to notify the app about exceptions encountered by the XAML framework or by the Windows Runtime in general that have not been handled by app code.

For example, if the Windows Runtime invokes app code like an event handler, and the app code throws an exception and does not catch it, the exception will propagate back to the Windows Runtime. The Windows Runtime will then fire the UnhandledException event to notify the app of this exception.
Source: Application.UnhandledException Event (Windows.UI.Xaml) - Windows UWP applications | Microsoft Docs

thqby wrote:because only OSError is handled in the delegate.
Um, no...
catch Any as e
OSError is a special case because it is assumed to contain a HRESULT code already (although now I realize the callback could call some other function that throws a non-HRESULT Win32 error). If you threw OSError(0x80004005), the callback return value would be the same as for any non-OSError. Throwing OSError(2) gives the same result anyway; process termination with code 0xC000027B.

lexikos
Posts: 9494
Joined: 30 Sep 2013, 04:07
Contact:

Re: Xaml? (v2.0-beta.2)

Post by lexikos » 08 Jul 2022, 18:54

thqby wrote:
08 Jul 2022, 07:33
Consider adding support for passing structures by value in later versions of AHK?
Yes.

While working on this script, I got caught up in trying to prototype a pattern that could be used by DllCall/ComCall/something else to support such things natively. Since I was also trying to come up with a single pattern suitable for actually implementing the script efficiently, I wasn't satisfied and ended up putting it all aside for several months.

What I have currently is ArgPassInfo for passing to/from DllCall and ReadWriteInfo and class-specific static methods for reading/writing struct fields and reading callback parameters. Some ReadWriteInfo are implemented by wrapping ArgPassInfo or static/class methods (string, structs, enums).

In WinRT, there are:
  • Strings as HSTRING, which should always be converted to/from our native format.
  • Enums which return as an object, but can be passed as integer or string and are converted to integer.
  • Structs which are often passed or returned by value, but are sometimes passed by address.
  • Interface pointers that require AddRef/Release in some contexts, and wrapping as objects.
So there are not only objects that decide how they are passed, but classes that decide how a parameter or return type is marshaled, without the class necessarily being instantiated.

I'm looking more to the future, not specifically at how to cram this into DllCall/ComCall. It would be better to have an API for composing function objects that call native functions, or define structs and other native types. However, DllCall/ComCall isn't going away (at least until v3?) so I also consider implementing straight-forward additions that improve their flexibility or convenience.
And I can pull request.
Are you volunteering to implement it? Anyone can submit a pull request. Until I see a pull request, I will not know whether it is something I want to merge.
Further, add support for custom types, similar to the Python's ctypes from_param.
Thanks for the reference. That looks similar to some of my own ideas, although I think the name is too generic.

User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Re: Xaml? (v2.0-beta.2)

Post by kczx3 » 11 Jul 2022, 01:24

Is in-code data binding possible? I tried something super simple like the C# example here. I tried setting the Binding's Source to an AHK string, using StrPtr(), and passing an HSTRING using the class in hstring.ahk but none worked. Passing a plain string gave an error stating "Expected a Number but got a String". Passing an HSTRING resulted in "Type mismatch. Specifically: ptr". Passing the ptr returned from StrPtr resulted in "An exception was thrown (0xc0000005)".

lexikos
Posts: 9494
Joined: 30 Sep 2013, 04:07
Contact:

Re: Xaml? (v2.0-beta.2)

Post by lexikos » 11 Jul 2022, 02:21

When you have a property that accepts any kind of value, in C# it looks like object, but it's actually IInspectable. That's an interface pointer.

To "box" a string, you can use Windows.Foundation.PropertyValue.CreateString("...").

As noted in the readme, the projection should automatically box primitive values this way in future. In C#, you can just assign the value.

One thing I learned only after wasting a lot of time: ILSpy, which is what I use to browse the metadata, can show the raw WinRT types of parameters and return values if you turn off the "Apply Windows Runtime projections on loaded assemblies" setting.

User avatar
jNizM
Posts: 3183
Joined: 30 Sep 2013, 01:33
Contact:

Re: Xaml? (v2.0-beta.2)

Post by jNizM » 23 Aug 2022, 07:46

Lexikos is your winrt github code WinUI2?

Looks a bit more like Win10 than WinUI3 (Win11)
Image

Anyway thank you for this repo to play with. It is really interesting.
Cant wait to see more.

Tested with WinUI 3 Gallery by Microsoft
(https://github.com/microsoft/WinUI-Gallery)
[AHK] v2.0.5 | [WIN] 11 Pro (Version 22H2) | [GitHub] Profile

User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Re: Xaml? (v2.0-beta.2)

Post by kczx3 » 23 Aug 2022, 15:26

XAML Islands only currently supports WinUI2

Post Reply

Return to “General Discussion”