48 replies to this topic

[PhiLho]

This is a script that parses the code of any AutoHotkey script and extract the function definitions, the labels and the hotstuff, ie. hotkeys, hotstrings and key remapping.

It is less trivial that it seems, at least for function definitions, because Chris chose not to prefix them with something, unlike most languages I know (Fun, Sub, function, return type...). So it is quite hard to distinguish a simple function call (ie. with only variable names as parameters in the call) from a function definition. The only hint we have is that a function definition is eventually followed by an opening brace.
The problem is that lot of stuff can go between the function name and this opening brace... So if you want to handle all legal cases, the script has to be a bit complex. I wouldn't have tried to do it without the help of regular expressions, code would have been much longer!

The script being more than 200 lines long, I just uploaded it. Get ListAHKFunctions.ahk and, if you want, SyntaxFunctions.ahk which attempts to make a legal files exploring all the limits of the syntax.

Found in 'SyntaxFunctions.ahk':
# 5 functions:

[25] Function1()
[36] Function2(a, [bb], ?c="", d=8, f=1.6)
[48] Function3( _a, ByRef @b, c?="",__d__=65536, #f=-3.14 )
[65] Function4( )
[70] Function5( x, y )

# 2 labels:

[41] Labels!Can_Be/In\Functions...
[81] X

# 8 hotkeys, hotstrings and key remappings:

[88] Numpad0 & F1
[90] Escape
[91] a
[92] q
[93] z
[94] w
[95] ::rtfm
[96] :*c:kk

What is the use of such script?
You can use it to grab quickly the API of an unknown library, to get the functions of you own library to document it, etc.
Or, primary purpose, if augmented of some code to get the source of the currently edited script, to navigate your source in your favorite editor.
This as been done before: Active GoTo v4
I must admit I didn't tried it. I just took a look at the source now, it seems not to support parameters on several lines nor additional lines between function definition and opening brace.
Of course, this covers most scripts -- not some in the manual...
And Rajat's script can be changed to use my code for better function detection.

Limitation: I was lazy and didn't tried to match only legal hotkeys and such. It will just list all lines with double : char in them, including occurrences in strings (not in comments nor continuation sections, though).

I was planning to make an AHK script to parse the source in the current buffer of SciTE as an example of use, but finally I prefer to rewrite this algorithm in Lua, so a simple Lua script will present a menu to jump to a section.
Work in progress, I will publish the result here.

[EDIT] I rewrote the code to be cleaner, hiding the gory details of state handling. If you want better performance (?) or just want to compare to the previous version, just get it: ListAHKFunctions-.ahk

Some details on implementation: I used an automaton to manage the states of the lexer/parser.
I used, of course, regular expressions to verify what kind of line I have.
If it matches, I set a state indicating the line is in a comment, a continuation section, in a function parameter list or before the opening brace.
Transitions between states are made by regex matching or non-matching: if state is INPARAMS, if the current line doesn't match the syntax of parameter list, we know we don't have a function definition so we return to DEFAULT state.

Project history:

1.03.000 -- 2006/11/30 (toralf & PL) -- Better detection of hotstuff,
             splitted between hotkeys (and key remapping) and hotstrings.
             Use GUI instead of MsgBox to display results.
 1.02.001 -- 2006/11/29 (PL) -- Making labels more tolerant.
 1.02.000 -- 2006/11/21 (PL) -- Encapsulation of state handling.
 1.01.000 -- 2006/11/18 (PL) -- Added listing of labels and hotstuff.
 1.00.000 -- 2006/11/17 (PL) -- Creation.

[PhiLho]

I finished the Lua version of the script: ListAHKFunctions.lua
It was less obvious than I thought, because Lua regular expressions are much more limited than I thought (can't do "(%s; )?"...).
Well, I managed to get it done, but currently it is only a standalone version, to run from the command line.
I have to hack it to use it in SciTE. Stay tuned (is anybody interested, really?).

[PhiLho]

I continue to answer myself... :-)
I made the script for SciTE, it is largely fast enough. Fortunately, as I have to parse the file twice, because SciTE forgets the result between the list display and the management of user choice...
The file is ListAHKFunctions4SciTE.lua.

You have to add stuff to your SciTEStartup.lua, and of course the .properties file where you define your tool commands.
In SciTEStartup.lua:

-- User Defined Lists (generic)
local udl_Separator = '¤'
local udl_SuperAbbr = 10
local udl_ListTools = 15
local udl_ListAHKFunctions = 20


-- Iterator
function AllLines()
	local currentLineNb = -1
	local totalLineNb = editor.LineCount
	return function ()
		currentLineNb = currentLineNb + 1
		if currentLineNb < totalLineNb then
			return editor:GetLine(currentLineNb)
		end
	end
end

function UserListShow(udl, list)
	editorACS = editor.AutoCSeparator
	editor.AutoCSeparator = string.byte(udl_Separator)
	editor:UserListShow(udl, list)
	editor.AutoCSeparator = editorACS
end

ahkFunctions = {}
dofile(props["SciteDefaultHome"] .. [[\ListAHKFunctions4SciTE.lua]])

function CreateAHKFunctionList()
	local functionNb, functionList, labelNb, labelList,
			hotkeyNb, hotkeyList, hotstringNb, hotstringList =
					ListAutoHotkeyFunctions()
	if functionNb == nil then return "" end
	local menuItem
	local rf, rl, rhk, rhs = '', '', '', ''

	for i = 1, functionNb do
		pl = string.gsub(functionList[i].parameters, "%s%s*", " ")
		menuItem = "F [" .. functionList[i].line .. "] " .. functionList[i].name .. "(" .. pl .. ")"
		rf = rf .. menuItem .. udl_Separator
		ahkFunctions[menuItem] = functionList[i].line
	end
	for i = 1, labelNb do
		menuItem = "L [" .. labelList[i].line .. "] " .. labelList[i].name
		rl = rl .. menuItem .. udl_Separator
		ahkFunctions[menuItem] = labelList[i].line
	end
	for i = 1, hotkeyNb do
		menuItem = "K [" .. hotkeyList[i].line .. "] " .. hotkeyList[i].name
		rhk = rhk .. menuItem .. udl_Separator
		ahkFunctions[menuItem] = hotkeyList[i].line
	end
	for i = 1, hotstringNb do
		menuItem = "S [" .. hotstringList[i].line .. "] " .. hotstringList[i].name
		rhs = rhs .. menuItem .. udl_Separator
		ahkFunctions[menuItem] = hotstringList[i].line
	end

	local list = string.gsub(rf .. rl .. rhk .. rhs, udl_Separator .. '$', '')
	return list
end

function ShowAHKFunctions()
	UserListShow(udl_ListAHKFunctions, CreateAHKFunctionList())
end

-- Generic function, only one part is relevant, guess which one... :-)
function OnUserListSelection(lt, choice)
--~ 	print("OnUserListSelection: " .. lt .. ", " .. choice)
	if lt == udl_ListTools then
		local tool = tools[choice]
		tool[2]()
		return true
	elseif lt == udl_SuperAbbr then
		editor:InsertText(-1, abbrList[choice]["template"])
		-- Should locate posChar, remove it and put the caret there
		return true
	elseif lt == udl_ListAHKFunctions then
		CreateAHKFunctionList()
		editor:GotoLine(ahkFunctions[choice] - 1)
		return true
	else
		return false
	end
end

In SciTEUser.properties:

command.name.7.*=&Show AHK Functions
command.7.*=ShowAHKFunctions
command.shortcut.7.*=Ctrl+!
command.subsystem.7.*=3
command.save.before.7.*=2

That means that when you hit Ctrl+!, SciTE will present a list of the functions, labels and hotstuffs. Double-click (or Up/Dn arrows & Enter) on the item of your choice, and you jump there!

Of course, you have to set up the Lua scripting extension properties:

# The path to the Lua extension script
ext.lua.startup.script=$(SciteDefaultHome)/SciTEStartup.lua
# Re-initialize the global scope immediately when the script is saved
ext.lua.auto.reload=1
# The startup script property is checked when SciTE switches buffers
ext.lua.reset=1

Project history:

1.03.000 -- 2006/11/30 (toralf & PL) -- Better detection of hotstuff,
             splitted between hotkeys (and key remapping) and hotstrings.
 1.02.001 -- 2006/11/29 (PL) -- Making labels more tolerant.
 1.02.000 -- 2006/11/25 (PL) -- SciTE version.
 1.01.000 -- 2006/11/24 (PL) -- Minor changes.
 1.00.000 -- 2006/11/23 (PL) -- Creation.

[toralf]

Limitation: I was lazy and didn't tried to match only legal hotkeys and such. It will just list all lines with double : char in them, including occurrences in strings (not in comments nor continuation sections, though).

Is it difficlut to add this?
I have test the script and it has false positives on these lines

Run, hh %AHKManualFile%::/docs/misc/Styles.htm#Common
  Run, hh %AHKManualFile%::/docs/misc/Styles.htm#%Type%

[PhiLho]

Yes, alas. See all the stuff that can go before the ::, it is difficult to handle all legal cases and reject the others...
I have also false positives on Auto-Syntax-Tidy_v10.ahk and of course ListAHKFunctions.lua itself...
It is not a major problem for the main use case for which it was designed: to present a list of points of interest and allow to jump to it. Because the user will spot these (quite rare) exceptions and will ignore them.

[toralf]

I have also false positives on Auto-Syntax-Tidy_v10.ahk

That's why I want to improve it. I also hope that RegEx will improve performance against the current algorithm.
Can't the left of the "::" be matched against illegal chars?

[PhiLho]

I also hope that RegEx will improve performance against the current algorithm.
Can't the left of the "::" be matched against illegal chars?

:?: Current algorithm already uses regular expressions.
What are illegal chars? Most chars can be found in hotkeys...
It would take a whole sub-algorithm to handle the legal cases.
Be free to try to write it... :-) I won't work on it.

[toralf]

You are right it is not that easy. But I guess I got something:

test =
(
  Run, hh %AHKManualFile%::/docs/misc/Styles.htm#Common   ;<- false 
  Run,hh %AHKManualFile%::/docs/misc/Styles.htm#Common    ;<- false
  ::                                                   ;<- false
  Enter & Pause::
  Pause:: Pause
  ^Esc:: ExitApp
  Left::
  Left Up::
  Left Down::      ;<- false
  ^+l::
  ^+v::
  ^+,::
  
  :B:toralf::
  :B0:toralf::
  :B1:toralf::   ;<- false  
  :O:toralf::
  :O0:toralf::
  :O 0:toralf::   ;<- false
  :SI:toralf::
  :SP:toralf::
  :SE:toralf::
  :SZ:toralf::   ;<- false 
  :*?BCK-1OP1RSIZ:toralf::
  :*?B5C2KaO1P-1R1SLZ1:toralf::    ;<- false
)

Loop, Parse, Test, `n
{
  line := A_LoopField
  If RegExMatch(line, "^\s*[^ \t:]+?( & [^ \t:]+?| [uU][pP])?::")
      String = %String%`nHotkey -> %Line%
  Else If RegExMatch(line, "i)^\s*:(\*0?|\?0?|B0?|C[01]?|K(-1|\d+)|O0?|P\d+|R0?|S[IPE]|Z0?)*:.+?::")
      String = %String%`nHotString -> %Line%
  Else
      String = %String%`nnothing -> %Line%
}
MsgBox, done:%String%

Edit: I think I solved it.
Can someone find a false positives?

[foom]

Looks nice but
x & y UP::
is not recognised.
Edit: Btw why don't you make the hotkey regex caseless too?

@PhiLho

Dou you have a list of all your scripts somewhere? Or do you plan on creating one? Maybe like goyyahs tips and trick but without the eyecandy. Only a list of all includefiles would be cool.

[PhiLho]

Good. I slightly changed the first expression: "i)^\s*(?!msg|tool|file)[^\s:]+?(?:\s+&\s+[^\s:]+?)?(?:\s+up)?::"
[EDIT] Changed to take in account foom's remark...
[EDIT] MsgBox is wreaking havoc... So is ToolTip, FileAppend, etc.
[EDIT] I eliminate MsgBox, ToolTip, FileAppend. Who will make a more complete list? Eg. ControlSet.

Else If (RegExMatch(line, "^\s*(?P<Name>.+?)::", hotstuff) > 0)
    { If InStr(FirstWord,"::"){
      If (InStr(FirstWord,"::") = 0) {     ;line is start of a subroutine
MsgBox ::
msgbox up:: No longer False positive!
tooltip & foo:: No longer False positive!
a =::
<^>!m::MsgBox You pressed AltGr+m.
<^<!m::MsgBox You pressed LeftControl+LeftAlt+m.
LControl & RAlt::MsgBox You pressed AltGr itself.
*ScrollLock::Run Notepad
~RButton & C::MsgBox You pressed C while holding down the right mouse button.
Numpad0 & Numpad2::Run Notepad
Numpad0   &   Numpad2::Run Notepad
~*Escape::
x  &   y     up::MsgBox

	a::b

[PhiLho]

Dou you have a list of all your scripts somewhere? Or do you plan on creating one?

I maintain one for my own use, with vague plans to generate an HTML version. I use AutoHotkey.net to transfer this list back and forth between my job and my home, so you can find it there: <!-- m -->http://autohotkey.ne...ArticleList.txt<!-- m -->
I suppose that's better than nothing... :-)

[foom]

Thanks for the list. Its pretty impressive.

Btw the hotkeys in the autoexecute section of ListAHKFunctions.ahk prevent the script from running.

#SingleInstance Force
#NoEnv

; !#c::Reload    ;<- here is where the script stops.
; #c::
script =
scriptName =

[toralf]

I just tested the labels. I couldn't believe what you wrote in your code, but it's true this is a legal statement

GoSub, ][#@$?=!#~öäü?ߧ$&/(){[]}³µ<>|^°\*+':;_-.

Even a ";" and ":" are allowed. So it is more tolerant then you thought.

"^\s*(?P<Name>[^\s,:;```%]+): ; Label are very tolerant...

[toralf]

And this is a leagal function call

MsgBox, % []#@$?_()

[PhiLho]

Btw the hotkeys in the autoexecute section of ListAHKFunctions.ahk prevent the script from running.

No, they don't... Just hit Win+C to get a file chooser. Thus you can test on several files without re-running the script...

Even a ";" and ":" are allowed. So it is more tolerant then you thought.

Yes, I should have made an error in my test code, I updated it and will update the lexer:

labels = !"#$`%&'()*+,-./:;<=>?@[\]^_``{|}~
Loop Parse, labels
{
	c := A_LoopField
	If (IsLabel(c . "L"))
	{
		Gosub %c%L
	}
	Else If (IsLabel("L" . c . "L"))
	{
		Gosub L%c%L
	}
	Else
	{
		invalid = %invalid%%c%
	}
}
MsgBox Tested: %labels%`nValid: %r%`nInvalid: %invalid%
Return

!L:
"L:
#L:
$L:
%L:
&L:
'L:
(L:
)L:
*L:
+L:
; Not label ,L:
-L:
.L:
/L:
; Not label :L:
L:L:
; Not label ;L:
L;L:
<L:
=L:
>L:
?L:
@L:
[L]L:
^L:
_L:
; Not label `L:
{L:
|L:
}L:
~L:
	r = %r%%c%
Return

[EDIT] I updated the sources with the following expression: "^%s*([^%s,:;`][^%s,:`]*):%s*"
It has limitations: it won't accept labels with double-colon (hard to handle) and those starting with an opening parenthesis will wreak havoc. I can probably handle the later by moving up the label detection before detection start of continuation section, but it isn't worth the effort (rare labels!). Hey, it even won't work, as it would detect labels in cont. sections...
It seems to be an obscure corner of syntax anyway...
To start cont. sections, (Join: is rejected, (Join#: - is accepted but the dash is ignored, (Join#: ; has a strange result, and so on...