Modules

Discuss the future of the AutoHotkey language
lexikos
Posts: 9721
Joined: 30 Sep 2013, 04:07
Contact:

Modules

Post by lexikos » 24 May 2024, 00:59

Added Modules, #Module, Import and Export.
Source: v2.1-alpha.11
For context, I would recommed reading the documentation first.


#Module

#Module X is analogous to ending the current file, beginning Lib\X.ahk and loading it as a module without importing it. It just defines a new module and adds it to the module list. There is no implied connection between it and any other module.

Helgef's namespace/module pull requests used syntax like Module X {}, which I have not implemented in this release. In contrast with #Module, I would expect a "module block" to:
  • Add "X" to the enclosing namespace, like a class definition.
  • Have access to names defined in the enclosing namespace, consistent with functions and classes. That would be all module-level names, not just exported names.
A block generally encourages a level of indentation, and having an extra level of indentation over perhaps the whole file is probably undesirable. C# always had the syntax namespace X {}, with that enclosing most of the file in 99.7% of files (according to Microsoft). This is cited as a motivation for the addition of file-scoped namespace X; in C# 10.

I also considered module X:, but I think a directive is more familiar.

A directive is simpler to parse without breaking backward-compatibility (than a statement beginning with an identifier, whether terminated by a colon or followed by a block). Hypothetically, a module in "v2.0-compatible mode" must allow "module" as a user-defined variable or function name, but such a module can be safely terminated with #Module since it has no other valid interpretation.

#Module might be changed to allow reopening an existing module.

It might make sense for #Module to automatically end at the end of the current file, when used inside a file being loaded as a module (including the case where #Module is used immediately before #Include). However, I'm not too sure how #Module should relate to nested modules or packages.


Import

Import can import a module, import all of a module's exports, or import specific exports from a module. I implemented syntax mainly influenced by Python and JavaScript.

Python uses import m and from m import a, b as x. Having the module name always at the start has some appeal especially when the list of names becomes long. from m import * would need special-casing to avoid line continuation, but so would import * from . (hypothetical import from current package). The choice of ordering has implications for how backward-compatibility can be handled.

JavaScript has several different forms, but they all start with import and end with "module-name";. The examples above would translate to something like import m from "module-name"; or import * as m from "module-name"; and import {a, b as x} from "module-name";. module-name would typically be a URL rather than a simple name (when an import map is not used).

I started with the intention to implement import a, b as x from m, but ended up adding braces, mainly to make it easier to parse (since the module name is needed before anything else). It also helps to show the grouping of imported names vs. module name. import {} from m is valid; it will load the module but not add any names to the current namespace.

I thought that pre-existing destructuring syntax might have been a reason for JavaScript using braces in import, but they are actually not consistent: import uses "as" where destructuring uses ":".

import m and import real_m as m work like in Python, I think. Equivalent JavaScript would probably be import m from "m"; if there is a default export, otherwise import * as m from "m";.

import {a} from m works like Python's from m import a and JavaScript's import {a} from "m"; (except for how module names work).

Aliasing the module's exports with "as alias" seems to be the same in all.


Export

Function and class definitions can now be prefixed with export or export default. This seemed to be the obvious way to mark something for export from the module. For global variables, global is replaced with export; e.g. export answer := 42.

Requiring exports to be explicit reduces the risk of conflict when a script uses import * from a and import * from b. It also reduces the risk of changes within a module breaking scripts which use the module (e.g. a user can't rely on internal functions that aren't exported).

One drawback is that a v2.0-compatible library can't have exports. One way around this is to use a separate file which #includes the v2.0 library and exports the relevant names. Another way is to use #Module and #Include at the end of the script. However, this needs further consideration with respect to whether a v2.0-compatible module can even contain exports (see Backward-Compatibility below).

Python exports everything which doesn't begin with an underscore by default, but allows __all__ to define the set of names which correspond to *. This relies on the module executing before the importing module - Python apparently doesn't allow cyclic imports.

Declarations allow exports to be found through static analysis, which means the importing module can bind to the imported module's exports at load-time, before either module is executed. This allows functions and classes to be used even if the module hasn't yet executed.

In some cases it might be preferable to export everything by default, such as for a v2.0 module which doesn't or can't use export (see Backward-Compatibility). There is no allowance for this as yet.


Lib

Import m looks for m.ahk in the usual Lib folders. I intend to add a check for an environment variable to override the Lib folder locations with a list of paths. The variable might be based on version, so v2.2 may have different libraries to v2.1, with some in common.


Module extends Any

In this release, Module is implemented as a simple dynamic interface which looks up an exported variable and gets, sets or invokes its value. This is by no means final; the goal for this release was just the core functionality. import m binds a Module to m and as such is not optimized at load time.

Normal properties can be defined in Module.Prototype, but are of course overridden by any export with the relevant name.

At one point I was thinking that a Module could be the entry point for a script to reflect on its own variables, functions and classes, but that mightn't make sense if only explicit exports are accessible.

Instead, I am thinking that Module can be an Object, with a property defined for each export. This would enable the debugger and script to enumerate the exports (properties) of a module. However, exports can't be implemented as a standard value or method property, as the module itself should not be passed as a parameter when calling. If GetOwnPropDesc is used on an export property, it might appear as a property getter with opaque implementation (nested classes are like this already) rather than as a value or constant.


Packages

Currently a module name should be a plain identifier. I may implement packages and module directories, such that import a.b may load Lib\a\b\__init.ahk, Lib\a\b.ahk or Lib\a\__init.ahk (in that order, and maybe Lib\a.ahk); and add a to the current module.

If packages are a thing, it may make sense to borrow Python's package-relative imports, like import .mod_in_same_pkg, import ..mod_in_parent_pkg or import * from . (Python: from . import *).

#Module a.b should (but doesn't yet) also create a top-level module/package a which exports b.

Placing a module in a package is just a matter of organisation and identification of modules. Lib\a\b.ahk would not implicitly have access to names defined within module a; that would be an effect of lexical nesting, as in module a { module b {} }.

I am undecided as to whether import a.b, which adds package a to the current namespace, should imply import a, which would load Lib\a\__init.ahk. I think that import a.b should not load Lib\a\c.ahk, except possibly in the case that it loads Lib\a\__init.ahk and that explicitly imports .c.


Backward-Compatibility

v2.1.0 will be required to have the capability to run v2.0 scripts without changes. The current alpha may not achieve this. My plan is for each module to have a target version for compatibility, with v2.0 being the default target version in an empty script. How exactly the v2.1 mode will be enabled is not yet decided.

Python has from __future__ import feature, which is special-cased by the compiler. I am of two minds about implementing something like this. I think that sharing code will be easier if required to conform to a target version number, but enabling individual changes may ease the process of upgrading existing scripts (or using specific new features in existing scripts without fully upgrading them).

Using v2.0 code within a v2.1 script would involve putting the v2.0 code in its own module. If export must be restricted for backward-compatibility, a v2.0 module might need different export behaviour.

Ideally, making use of new syntax should not require a thorough review of the entire script to make it compatible with the new version. New code can instead be placed within a module which is v2.1-enabled, or code can be gradually migrated between modules. If the syntax to define or load a module is to be used in a v2.0-compatible context, it must be parsed in a way that doesn't change the interpretation of valid code. For example, "Module" and "Import" can't be reserved words.


Debugger

v2.1-alpha.11 has some notable limitations for debugging (with a DBGp client):
  1. Querying a variable which refers to a Module does not list the module's exports, although property_get and property_set should work for directly accessing the export property.
  2. By design, context_get -c 1 returns global variables of the current module and names which are explicitly imported, but not names which are implicitly imported with import * or from the built-in module.
vscode-autohotkey-debug v1.11.0 uses a roundabout method of querying properties (in DBGp, a variable is a property), and will not display the values of properties which are not listed within the parent property or by context_get. This means that it refuses to get the value of an export via a Module reference (due to #1 above) or any implicitly imported variable/function/class (due to #2 above).

niCode
Posts: 321
Joined: 17 Oct 2022, 22:09

Re: Modules

Post by niCode » 24 May 2024, 01:28

lexikos wrote:
24 May 2024, 00:59
v2.1.0 will be required to have the capability to run v2.0 scripts without changes. The current alpha may not achieve this.
Is this why one of my scripts will not work? Should I revert to 2.0.10 2.1-alpha10 or is there something I can add to the script? My issue in particular seems to be that it doesn't recognize some built-in things like WinGetID, Gui(), SetTimer, and OnMessage. AHK thinks they are variables that haven't been assigned a value.

lexikos
Posts: 9721
Joined: 30 Sep 2013, 04:07
Contact:

Re: Modules

Post by lexikos » 24 May 2024, 06:53

@niCode
No, I was mainly referring to Import and Export, which were allowed to be user-defined functions in v2.0. Feel free to bring up any similar issues.

There's a bug which prevents assume-global functions from resolving built-in names or import *. I assume that's where you're seeing the issue. Thanks for bringing my attention to it.

There's no easy workaround as far as I can tell, aside from avoiding assume-global.

eugenesv
Posts: 178
Joined: 21 Dec 2015, 10:11

Re: Modules

Post by eugenesv » 24 May 2024, 08:08

Python's import syntax is nicer since all module names are in the same column at the start, so easier to read, and also allows (in principle) auto-completion since by the type you type specific imports it's already known what module they'll be imported from

niCode
Posts: 321
Joined: 17 Oct 2022, 22:09

Re: Modules

Post by niCode » 24 May 2024, 14:30

lexikos wrote:
24 May 2024, 06:53
There's a bug which prevents assume-global functions from resolving built-in names or import *. I assume that's where you're seeing the issue.
Yes, that would explain it. It was my first script converted to v2 during v2's alpha and haven't gotten around to finishing the version that doesn't use global variables. Thanks for the hard work introducing new features!

niCode
Posts: 321
Joined: 17 Oct 2022, 22:09

Re: Modules

Post by niCode » 24 May 2024, 14:33

I swear I was hitting edit and forum is reposting with more quotes and I can't delete previous posts wtf.
Last edited by gregster on 24 May 2024, 14:49, edited 1 time in total.
Reason: Removed duplicate posts.

iseahound
Posts: 1479
Joined: 13 Aug 2016, 21:04
Contact:

Re: Modules

Post by iseahound » 25 May 2024, 07:18

Python's package syntax requires empty __init__.py files. This is one feature I would delete from python. https://stackoverflow.com/questions/37139786/is-init-py-not-required-for-packages-in-python-3-3

Appreciate the great work!

crocodile
Posts: 100
Joined: 28 Dec 2020, 13:41

Re: Modules

Post by crocodile » 25 May 2024, 21:22

Am I to understand that the current “modules” are mainly functioning to create different scopes for libs, rather than providing higher performance like Python's NumPy?

lexikos
Posts: 9721
Joined: 30 Sep 2013, 04:07
Contact:

Re: Modules

Post by lexikos » 26 May 2024, 04:34

@iseahound Why? Without the file, how does one know that a directory is intended to be module? Anyway; no, it does not require empty files, you can use non-empty files. Also, the Q&A you linked seems to refute what you're saying; a namespace package is created only if there is no __init__.py file.

@crocodile If you read the official Python tutorial for modules, I think you won't find any reference to modules which "provide higher performance". Modules are primarily for use within the language. Once the language has modules, then you can think about using it to package whatever functionality you want.

Currently 60% of the code on the NumPy GitHub repository is Python, and 35.7% is C. I assume the "higher performance" part comes from the latter. That is presumably the nature of what NumPy provides. It has very little to do with modules. A module is just the interface you use to access the functionality from Python, because modules are part of the Python language. It could just as well be through a set of functions or objects.

High-performance libraries can already be interfaced with AutoHotkey through ComObject, DllCall or ComCall. Like I wrote before v2.0.0 was released, improving the methods of interfacing scripts with native functions is on the roadmap. For v2.1.0, I am focusing on structs and modules.

iseahound
Posts: 1479
Joined: 13 Aug 2016, 21:04
Contact:

Re: Modules

Post by iseahound » 26 May 2024, 11:36

I'm just a little skeptical of
import a.b may load Lib\a\b\__init.ahk, Lib\a\b.ahk or Lib\a\__init.ahk
. Is there truly a need to use . as a substitute for \ to parse subdirectories?

Specifically, since AutoHotkey lacks a package manager, and is commonly used by newer users to programming in general, reducing the complexity (with edge cases such as . not being allowed in directory names or script names) would be beneficial.

Also go allows statements like

Code: Select all

import "github.com/google/go-cmp/cmp"
which fetches from a remote repository and creates a local directory structure %USERPROFILE%\go\pkg\mod\github.com\google\go-cmp@v0.6.0\cmp

lexikos
Posts: 9721
Joined: 30 Sep 2013, 04:07
Contact:

Re: Modules

Post by lexikos » 27 May 2024, 03:22

@iseahound Packages are a means of organising modules, with the interpreter deriving paths from the module names. The module names themselves must be valid identifiers, as they may be used directly within the script to access the module. It is not a means of specifying a path. If you want to import from a specific path, there will need to be syntax for that, like Import "Path" as Identifier.

bonobo
Posts: 78
Joined: 03 Sep 2023, 20:13

Re: Modules

Post by bonobo » 27 May 2024, 08:44

This is a fantastic addition!

And if I understand this correctly, it will be possible for two different modules to import each other (something not possible with #include), is that correct?
lexikos wrote:
26 May 2024, 04:34
High-performance libraries can already be interfaced with AutoHotkey through ComObject, DllCall or ComCall. Like I wrote before v2.0.0 was released, improving the methods of interfacing scripts with native functions is on the roadmap. For v2.1.0, I am focusing on structs and modules.
This is great to hear.

lexikos
Posts: 9721
Joined: 30 Sep 2013, 04:07
Contact:

Re: Modules

Post by lexikos » 28 May 2024, 04:12

@bonobo It will be, and is already possible for two modules to import each other. However, in those cases, it may be hard to predict which module's body will execute first (e.g. to initialize global variables).

Two libraries can generally #include each other without problem.

What I meant to say about v2.1.0 is that I am trying to focus on the core syntax and functionality for structs and modules. My main reason for implement modules in this release is that they are part of my strategy for keeping backward-compatibility while enabling progress. The simplest example is that defining new built-in classes will no longer break scripts which define their own by the same names.

User avatar
thqby
Posts: 442
Joined: 16 Apr 2021, 11:18
Contact:

Re: Modules

Post by thqby » 17 Jun 2024, 23:23

Code: Select all

%'v'% := 'set'
MsgBox v ?? 'unset', '__main'
#module abc
; import * from __main
MsgBox v ?? 'unset', 'abc'
@lexikos
Whether there is an import statement or not will produce two results, which is expected?
set, __main or set, abc

lexikos
Posts: 9721
Joined: 30 Sep 2013, 04:07
Contact:

Re: Modules

Post by lexikos » 18 Jun 2024, 06:27

set, __main is expected, because obviously you are assigning v in __main, but what we actually get with import is unset, __main.

set, abc is not expected regardless of import, because v is not exported.

The problem is apparently that the "current module" is set incorrectly during execution of module dependencies. This affects ListVars and dynamic references.

Code: Select all

ListVars  ; shows abc's variables (in v2.1-alpha.13)
current_module_is_main := 1
#module abc
import __main
current_module_is_abc := 1
It does not affect code inside a function, as the global "current module" is not used there.

Code: Select all

f() => ListVars()  ; shows __main's variables
f
current_module_is_main := 1
#module abc
import __main
current_module_is_abc := 1

User avatar
thqby
Posts: 442
Joined: 16 Apr 2021, 11:18
Contact:

Re: Modules

Post by thqby » 19 Jun 2024, 06:32

Code: Select all

import { fn } from abc
import { fn as fn2 } from mod
MsgBox(fn == fn2)  ; 0
#module mod
import { fn } from abc
export fn
abc.ahk

Code: Select all

fn() => 0
@lexikos
Use the same library module in two modules through "import" to get different modules. Is this expected?

User avatar
thqby
Posts: 442
Joined: 16 Apr 2021, 11:18
Contact:

Re: Modules

Post by thqby » 19 Jun 2024, 22:50

Code: Select all

import * from cc
aa()
#module aa
export aa() => 0
#module bb
export bb() => 0
#module cc
import * from aa
import * from bb
In javascript, the export of multiple modules can be imported into a common module, and then imported from the common module.
Will this be implemented in ahk?

lexikos
Posts: 9721
Joined: 30 Sep 2013, 04:07
Contact:

Re: Modules

Post by lexikos » 20 Jun 2024, 04:24

No, both import { fn } from abc are supposed to get the same fn. I'd guess that it is loading the module from file twice, which is not intended.

It is not possible to import something which isn't exported. cc must export aa, bb. If aa is referenced but not declared anywhere in cc, it will be imported from aa due to import *. Currently export aa counts as a declaration for this purpose, so will prevent aa from being imported into cc. Thus, the main module's aa will be the unset aa defined by cc. In order to export it from cc, you must explicitly import it. You can do it like this:

Code: Select all

import * from cc
aa()

#module aa
export aa() => 0

#module cc
import { aa } from aa
export aa

I think JavaScript has a shortcut like export import * from aa. This is not implemented.

Post Reply

Return to “AutoHotkey Development”