Page 1 of 1

use AHK.exe to parse scripts

Posted: 07 Jun 2020, 06:36
by toralf
Several people have coded AHK scripts in the past that parse other AHK scripts to get information from it to support programming.
This can get quite complicated depending on what type of information should be extracted from the script.
I was wondering, if the parser in AHK.exe itself could be used to extract information of a given script?
I would imagine a call of AHK.exe on the command line with a special command line parameter (e.g. \ANALYSE) and a script name as input. Then AHK.exe would not execute the script, but only parse the script and return "some" information on the script.
This information could be
  • List of variables
  • list of functions and parameters
  • list of labels
  • list of objects
  • warnings
  • errors
  • etc
Is this even feasible?

With this information calltips, auto-completion, syntax tidy or similar things would be much easier.

Re: use AHK.exe to parse scripts

Posted: 07 Jun 2020, 23:44
by lexikos
Yes, except for:
  • List of objects. Maybe you meant classes? There is no internal list of classes, just the list of global variables that includes variables containing class objects. I suppose that those could be listed separately, and their contents could be enumerated to list nested classes, methods and whatever else is available without running the script.
  • Errors, plural. AutoHotkey isn't designed to keep parsing after an error, so you'd only get the first error.
  • etc (unspecified things might not be feasible). ;)
What specific format should the information be in?

It should be possible to include local variables by function, hotkey variants by context, and so on. Everything that is used to execute the script is perhaps trivial to enumerate with C++ and dump to stdout or a file. Code size might be an issue, in which case I'd be inclined to offer this capability only as a separate executable, which could exclude a lot of code, like the implementation of each built-in function. The trick would be to do that without littering the code with conditional compilation directives.

Re: use AHK.exe to parse scripts

Posted: 08 Jun 2020, 02:07
by toralf
Thank you for your reply and consideration.

Regarding the output and exchange of information, I have no real concept for it right now. There are some basic requirements and random thoughts:
- it should be easy to access and use by an AHK script
- it will be structured information, thus an object would most likely be the right final entity.
- to end up with an object, I assume that an output on stdout or file should have an xml or JSON format.
- if it is an extra executable, maybe it could be a DLL, then it could be called with DLLCall and return the object directly?

I’ll continue at a later point in time

Re: use AHK.exe to parse scripts

Posted: 08 Jun 2020, 03:35
by lexikos
Which AutoHotkey version(s) are you concerned with?

Re: use AHK.exe to parse scripts

Posted: 08 Jun 2020, 09:43
by toralf
In a Perfect world v1 and v2. But if it has a big impact v2 would I guess be the only one practical.
I still use v1.

Re: use AHK.exe to parse scripts

Posted: 08 Jun 2020, 19:14
by kczx3
I agree this would be awesome to have. JSON or xml are ideal formats even though we must rely on third party libs for JSON. I’d say v2 unless it isn’t much work to also do v1.

Re: use AHK.exe to parse scripts

Posted: 08 Jun 2020, 23:18
by toralf
More random thoughts.
This might not be complete yet, but I saw in the past that you have a good sense of spotting corner cases and unspecified areas. ;)

The information provided should contain
A) a hierarchical structure of all the files involved (include and lib files)
- with absolute or relative path to the main script
B) for a tool that would help to tidy the indentation
- for each line of each file the level of indentation based on code not on real/current indentation
- nice to have
-- the position at which position a comment starts
-- an indication of what type the line is, e.g. pure comment, block comment, continuation line, func def, class def, etc.
C) for a too that would help to eliminate “bad code”
- for each warning and the one error the file and line number and a description on what is wrong.
- nice to have
-- the position in the line
D) for a tool to support auto completion
- a list of all variables Names of all files (built in variables could be maintained within the tool)
-- an indication if they are super global, global or local
-- an indication if they are a class
- a list of functions with their parameters of all files (built in functions could be maintained within the tool)
- a list of classes and their hierarchy of methods (with parameters) and properties
- Nice to have
-- For each the file and line number (and position) they occur
-- For each variable in which labels, functions, classes, methoden they are used
E) for a tool to provide calltips / intellisence
- ...

use cases:
in all cases the word tool, means a tool (e.g. an ahk script) that uses AHK.exe to get the information on the file and that does the job for the user or interacts with the user)
B) Tidy: user selects some lines of code in its editor and asks a tool to do the indentation. Since the tool knows for each line the level of indention it can add indentation according to the indentation type defined by the user. If it knows the type of line, it should be able to also create different styles of indentation.
C) Bad Code:
-- 1) a tool provides a list of warning and the error. User selects one and the cursor jumps in their editor to the right location in their code and if possible selects the portion of the code that creates the issue.
-- 2) an editor shows with custom highlighting the portion of code with issues and provides a tooltip on them on what the warning is.
D) Autocomplete:
-- 1) User types part of a function name, the tool provides a list of functions available with their parameters, and since the tool knows on which line the functions are defined, it could check if the lines above or below the definition are comments (see B) and check them if they contain additional information for the function.
-- 2) User types part of a variable name, the tool provides a list of variables available including where they are defined and if they are super global/global/local. Then the user might spot that he needs to set a variable global to be able to access it.
-- 3) User has selected a variable that contains a class. The tool provides a list of methods and properties of that class.
E) Calltips/Intellisence:
-- ...

Re: use AHK.exe to parse scripts

Posted: 09 Jun 2020, 03:34
by TheArkive
This is an amazing idea, and would cut well over 1000 lines from my CallTipsForAll script. Some more ideas:

Since there is no "list of objects" Is it possible to add checking (maybe reference checking) for members (methods and properties) associated with a particular var, to extend the var list into a possible object list?

If a list of super global vars is possible, and if it is possible to check a var's association with data members, then one could cross reference to find out which elements are classes, and which are objects.

I have ideas for structure and output, but that also depends on the extent of the capabilities.