parsing AutoHotkey scripts

Get help with using AutoHotkey and its commands and hotkeys
User avatar
jeeswg
Posts: 6644
Joined: 19 Dec 2016, 01:58
Location: UK

parsing AutoHotkey scripts

24 Nov 2017, 21:38

I'm working on scripts to parse AutoHotkey scripts.

DllCall converter/cleaner (e.g. x32 to x64/x32 two-way compatible) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=31365
- Clean DllCall lines.
- Clean NumGet/NumPut lines
- Separate parameters in command/function lines:
- Identify continuation section lines.

indent code (automated code indentation) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=6&t=37270

[This may be complete in a few weeks.]
AHK v1 to AHK v2 converter - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 37&t=36754
- Check for lines that don't meet certain criteria.
- Clean AHK v1 scripts.
- Make AHK v1 scripts more AHK v2 ready, e.g. StringReplace -> StrReplace, var = value -> var := value.
- Convert AHK v1 scripts to AHK v2 scripts.

I'm starting this thread, for sharing ideas/problems regarding parsing AutoHotkey scripts. I would be interested to hear from anyone who regularly parses AHK scripts, e.g. people who have worked on AHK editors / the AHK source code.

The first point I thought I'd bring up is, I had a script to add/remove initial commas from commands. But it incorrectly converted a variable with the same name as a command. I've since added a look-ahead to the RegEx line, to check for ' := ', but I thought this was a classic example of a conversion gotcha.
hotkey := NumGet(hotkey)
to
Hotkey, := NumGet(hotkey)

I would be interested in:
- any links to good converter/parser scripts
- any other classic conversion/parsing issues
- any comments on parsing objects, this is something I need to add to my converter scripts
- any general comments on converting from AHK v1 to AHK v2, i.e. tidbits of information v. just giving me a massive script to look through
- complete parsing of a script and listing/(wrapping in text) all the functions/variables
Thanks for reading.

==================================================

[EDIT:] I'm collecting links re. parsing issues.

Winmove error with "[" in window title - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 64#p187564
scrap commands? - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 61#p187561
scrap commands? - AutoHotkey Community
https://autohotkey.com/boards/viewtopic ... 31#p187731
v2-thoughts
https://autohotkey.com/v2/v2-thoughts.htm
•Output variables of built-in functions are currently not allowed to be an expression when parentheses are omitted. For example, MouseGetPos whichvar ? x : y is not valid, but MouseGetPos(whichvar ? x : y) is.
Last edited by jeeswg on 10 Dec 2017, 02:40, edited 1 time in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
Helgef
Posts: 3890
Joined: 17 Jul 2016, 01:02
Contact:

Re: parsing AutoHotkey scripts

25 Nov 2017, 10:57

to check for ' := ',
.= ++ -- ... :think:
User avatar
jeeswg
Posts: 6644
Joined: 19 Dec 2016, 01:58
Location: UK

Re: parsing AutoHotkey scripts

25 Nov 2017, 12:18

- Yes, probably all the = operators. With ++ and -- you also have check if anything is after them.
- I'm not sure if && || & | could be issues.
- Btw I don't personally advocate that variables should be banned if they have the same name as a command, hotkey is quite a useful variable name for example.

Code: Select all

q::
var := 3000
Sleep ++var
MsgBox, % var
return

w::
Sleep := 3000
Sleep ++
MsgBox, % Sleep
return
- With parsing it can be complicated, but there are often little clever tricks you can do.
- Escaped characters, `, `; `" "" `` `% have been giving me some problems.
- If no-one is available for tips or advice, if anyone can point me to the bit of the AutoHotkey source code that handles this, or other code that parses scripts that would be useful. Thanks. Obviously AutoHotkey itself has to make a decision on all of these issues before running the script.

[EDIT:] Any tips for identifying a 'hotkey::code' line, or a valid hotkey, would be great.
[EDIT:] Relevant link:
does string look like a hotkey - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=58131
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
User avatar
jeeswg
Posts: 6644
Joined: 19 Dec 2016, 01:58
Location: UK

Re: parsing AutoHotkey scripts

09 Apr 2019, 12:52

- Bump.
- I achieved all of the parsing objectives that I wanted to achieve.
- However, I thought that I might work towards the goal of completely parsing an AHK script. For potential benefits such as script tidying, variable/function listing, and for parsing other languages.
- I'm just mentioning this in case anyone has any suggestions, or ideas for things to consider, or recommended scripts to look at (e.g. editors), before I embark upon this. Thanks.
Last edited by jeeswg on 10 Apr 2019, 18:45, edited 1 time in total.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
toralf
Posts: 607
Joined: 27 Apr 2014, 21:08
Location: Germany

Re: parsing AutoHotkey scripts

10 Apr 2019, 18:42

ciao
toralf
User avatar
jeeswg
Posts: 6644
Joined: 19 Dec 2016, 01:58
Location: UK

Re: parsing AutoHotkey scripts

10 Apr 2019, 18:48

Thanks for the link. Appreciated. Do you have any general comments/recollections? Often this is as useful as, or more useful than, the code. Thanks.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
toralf
Posts: 607
Joined: 27 Apr 2014, 21:08
Location: Germany

Re: parsing AutoHotkey scripts

11 Apr 2019, 00:28

In which respect do you want to receive comments?
I believe the code is well commented.
The only thing that the code doesn’t do it to identify user variables
ciao
toralf
User avatar
jeeswg
Posts: 6644
Joined: 19 Dec 2016, 01:58
Location: UK

Re: parsing AutoHotkey scripts

11 Apr 2019, 01:57

- Do you remember any interesting/difficult challenges?
- I'd say identifying continuation sections and hotkeys/hotstrings are two of the more fiddly things.
- In this post, it was mentioned that w::Pause is ambiguous, a remapping (what it did) versus the use of the Pause command.
Auto-Clicker Not Pausing (Total Noob) - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=5&t=63512&p=271855#p271855
- And the 'variable with the same name as a command' issue mentioned above, e.g. 'hotkey', is an interesting one.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
toralf
Posts: 607
Joined: 27 Apr 2014, 21:08
Location: Germany

Re: parsing AutoHotkey scripts

11 Apr 2019, 14:04

continuation sections and hotkeys/hotstrings Are not really a problem I believe. But currently commands and variables are not identified. Commands might be easy since they are either followed by a space or comma and something that is not an operator.
To identify variables seems to me more difficult since they can be specified in several locations and nested inside of expressions and other extreme areas. Thus I believe that it can not be ensured to identity all variables. But on the other hand the AHK exe can, so it is possible. But the AHK exe always has an uptodate list of all available build in commands, variables and functions by default. A generic parser should be independent of the list to avoid continuously updating the list.
ciao
toralf
User avatar
jeeswg
Posts: 6644
Joined: 19 Dec 2016, 01:58
Location: UK

Re: parsing AutoHotkey scripts

11 Apr 2019, 14:57

- Yes, in commands, abc could be a literal string or a ByRef variable, depending on the parameter. So you need parameter info for each command, which I have in my converter script. This may have been the original source, or one of them:
GitHub - Lexikos/Scintillua-ahk: AutoHotkey lexers for Scintillua + SciTE4AutoHotkey
https://github.com/Lexikos/Scintillua-ahk
- Fortunately there have been no new commands since AHK v1.1.08 (SetRegView).
list of every command/function/variable from across all versions - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=74&t=27321&p=131642#p131642
- It can be straightforward to identify a line that starts with a command. To fully parse such a line can be harder, because you could have a forced expression, which could contain nested functions. If a line just contains {}, is that an empty array created and discarded, or a NoOp.
- One curious ambiguity could be curly braces (flow control) v. defining an associative array.
- Generally speaking I'm yet to try parsing obj.key and [] and {} assignments.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
toralf
Posts: 607
Joined: 27 Apr 2014, 21:08
Location: Germany

Re: parsing AutoHotkey scripts

11 Apr 2019, 16:45

Flow control with curly braces is recognized by my parser. It is easy to identify since they have to be first in a line or for a few commands the opening brace could be on the last character (OTB).
Why do you want to parse obj.key and {} or [] assignments?
ciao
toralf
User avatar
jeeswg
Posts: 6644
Joined: 19 Dec 2016, 01:58
Location: UK

Re: parsing AutoHotkey scripts

12 Apr 2019, 00:23

To get anything, you must parse everything.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
toralf
Posts: 607
Joined: 27 Apr 2014, 21:08
Location: Germany

Re: parsing AutoHotkey scripts

12 Apr 2019, 05:23

What do you mean with the word “anything”?
My parser already detects a lot.
What do you want to achieve? And why?
Or is this just an accidentally trial to see what is needed to detect everything?
ciao
toralf
User avatar
jeeswg
Posts: 6644
Joined: 19 Dec 2016, 01:58
Location: UK

Re: parsing AutoHotkey scripts

12 Apr 2019, 06:16

- To get each instance of anything, you must parse everything. So, if you want to list all variables, you have to parse things such 'obj.key' as well, where 'obj' is one of the variables.
- As mentioned, I've parsed certain aspects of scripts in the past, but, it might be useful to parse everything, pretty much as the AHK parser does.
- I mentioned above: script tidying, variable/function listing, and for parsing other languages. It could also be useful for doing subtle checks on functions like InStr/SubStr, looking for zero/negative numbers, or, if the AHK v2 functionality changes, you have to update your AHK v2 code.
- Interestingly, if I want to do further work re. parsing, it looks to me that it makes most sense, efficiency/effort/benefits wise, to try and fully parse script.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA

Return to “Ask For Help”

Who is online

Users browsing this forum: aircooled, flyingDman, Kobaltauge, TAC109 and 189 guests