Lexer and Folding For N++ (Finally!) - [Deprecated]

I am a notepad++ user and was totally disappointed with the userDefined.xml lexer because it has certain limitations that the user cannot overcome, and on top of that adding a language is a pain.

Well... after much troubles I finally imported the AHKLexer for SciTe by PhiLHo (an amazing job done there) to N++.

 

UPDATE:

 

I finally finished my main lexer. I had to learn C++ from scratch for it and that is why it took so long.
 I will be posting it in a few days. I have not developed the N++ Plug-in yet but I will give instructions on how to use my lexer with N++ In the mean time.

I will link to it from here as well but this thread is officially considered deprecated. Any questions/suggestions you might have please direct it to the new Thread.
 

New:
This is a portable version of Notepad++ taken from www.PortableApps.com with the autohotkey lexer working right off the bat.

you can download it from here

N++ 5.8 issues and workaround

The new Scilexer.dll used on 5.8 has some major changes thus making my lexer unusable.

Workaround: Get this file and replace the existing one inside the N++ 5.8 folder.
If you dont want to use this file just go to notepad++ and download the 5.7 version and get the SciLexer.dll file and replace the 5.8 version with it.

Until I rewrite the lexer this is the only work around i know.

Note: There are no noticeable drawbacks, if you find one (like another lexer not working because of the swap) please notify so i can update the post.

Installation:
Replace the plugins and themes folders with the ones provided in the zip file.

If you want to list ahk on the other Themes, run the ahk script created by jethrow located on the themes folder. It will create a subfolder with a modified version of the xml files. Back up old one before replacing.

The Autohotkey language will be listed with all the other languages and you can use the Styler Configurator to change your styles easily.

I included a test file, that should give you an Idea of how things will look when you change the styles.

The latest version can be downloaded here:


-- Deprecated --
New version comming very soon.
It will be posted in a new Topic and linked from here.


* I dont consider the folding error worth fixing anymore since the workaround that i found is pretty good at fixing the issue just simply use the "fold all" option on N++ and then "unfold all" because it seems that N++ is the one having troubles calculating the folds. You can use the hotkeys Alt + 0 to fold all and Alt+Shift+0 to unfold again, usually that fixes the issues right away.

I hope this helps the community, specially those who are using other editors just because N++ doesnt support AHK atm.
 
Special Thanks to:

Chris + the whole AHK community

Thell Fowler who developed NppExtlexer plugin.
http://two.xthost.info/kmeleon/otros/npp/ExtLex/

PhiLHo for creating this amazing lexer.
http://www.autohotkey.com/forum/viewtopic.php?t=9656

jethrow
For providing the script for adding AHKLexer in to the default themes

-RaptorX

Edited by RaptorX, 21 May 2013 - 09:39 PM.

Very nice! It would also be great if there were a way to switch themes & still have proper coloring - like the built-in languages.

Hi there, I dont use styles myself but I can tell you what is happening.

Each theme has an xml file (on the \themes folder) and the person who created the theme added each language in to that xml file and changed the colors just as they would do with the stylers.xml file.

The cool part of this plugin is that you can go to the stylers configurator menu, select autohotkey and make any changes to the colors as you want.

So just change your theme, go to ahk styler, change the colors to match your theme and you are done.

Hmmm, so your saying we need to tell all the ppl who wrote the theme files to include an AutoHotkey entry :wink: . Perhaps someone should create a script that will read each theme file & create an AutoHotkey entry based on the color schemes of the other languages.

Nice, good to see my lexer used in more Scintilla-compatible editors... Thanks!

PS.: My nickname has two 'h's... See my signature for the reason...

PPS.: I don't plan to work more on this lexer on short term, sorry if there are some pending bugs -- and there are! AHK isn't easy to lex...
Maybe I will attempt to do another lexer in the future (undefined as I lack free time...) with the new lexer framework coming to Scintilla (allowing to keep more context stuff).

Perhaps someone should create a script that will read each theme file & create an AutoHotkey entry based on the color schemes of the other languages.

I used the color scheme from Ruby since it had the most options ( except for ESCAPE SECUENCE, ERROR, & USER DEFINED, which I left as red ). If you run this script from the NP++ themes folder, it will create a subfolder with all the theme files with an AHK entry:

[color=DarkRed]#NoEnv[/color]
[color=DarkRed]SetBatchLines[/color], -1
[color=DarkRed]FileCreateDir[/color], Themes - AHK

[color=DarkRed]Loop[/color], *.xml {
	[color=DarkRed]FileRead[/color], FullXML, % file := [color=RedOrange]A_LoopFileName[/color]
	[color=DarkRed]if[/color] [color=DarkRed]RegExMatch[/color]( FullXML, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"ruby"[/color][color=CornFlowerBlue]".*?LexerType"[/color], RUBY ) ; if there is a Ruby Entry
		[color=DarkRed]GoSub[/color], CreateEntry
	[color=DarkRed]else[/color],
		NoEntry .= [color=RedOrange]A_LoopFileName[/color] [color=CornFlowerBlue]"`n"[/color]
}
[color=DarkRed]if[/color] ( NoEntry )
	[color=DarkRed]MsgBox[/color], 262208, Could Not Create:, % [color=DarkRed]SubStr[/color]( NoEntry, 1, -1 )
[color=DarkRed]Return[/color]


CreateEntry:
{
	[color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"DEFAULT"[/color][color=CornFlowerBlue]".*?bgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], BGCOLOR )
	, [color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"IDENTIFIER"[/color][color=CornFlowerBlue]".*?fgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], IDENTIFIER )
	, [color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"COMMENTLINE"[/color][color=CornFlowerBlue]".*?fgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], COMMENT )
	, [color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"INSTRUCTION.*?fgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], INSTRUCTION )
	, [color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"STRING"[/color][color=CornFlowerBlue]".*?fgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], STRING )
	, [color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"NUMBER"[/color][color=CornFlowerBlue]".*?fgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], NUMBER )
	, [color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"CLASS NAME"[/color][color=CornFlowerBlue]".*?fgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], CLASS_NAME )
	, [color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"DEF NAME"[/color][color=CornFlowerBlue]".*?fgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], REGEX )
	, [color=DarkRed]RegExMatch[/color]( RUBY, [color=CornFlowerBlue]"si)name="[/color][color=CornFlowerBlue]"REGEX"[/color][color=CornFlowerBlue]".*?fgColor="[/color][color=CornFlowerBlue]"\K.{6}"[/color], DEF_NAME )

	template =
	( join`r`n
		<LexerType name=[color=CornFlowerBlue]"AutoHotkey"[/color] desc=[color=CornFlowerBlue]"AutoHotkey"[/color] ext=[color=CornFlowerBlue]""[/color] excluded=[color=CornFlowerBlue]"no"[/color]>
			<WordsStyle name=[color=CornFlowerBlue]"DEFAULT"[/color] styleID=[color=CornFlowerBlue]"0"[/color] fgColor=[color=CornFlowerBlue]"%IDENTIFIER%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"COMMENTLINE"[/color] styleID=[color=CornFlowerBlue]"1"[/color] fgColor=[color=CornFlowerBlue]"%COMMENT%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"COMMENTBLOCK"[/color] styleID=[color=CornFlowerBlue]"2"[/color] fgColor=[color=CornFlowerBlue]"%COMMENT%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"1"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"ESCAPE SECUENCE"[/color] styleID=[color=CornFlowerBlue]"3"[/color] fgColor=[color=CornFlowerBlue]"FF0000"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"SYNOPERATOR"[/color] styleID=[color=CornFlowerBlue]"4"[/color] fgColor=[color=CornFlowerBlue]"%INSTRUCTION%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"1"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"EXPOPERATOR"[/color] styleID=[color=CornFlowerBlue]"5"[/color] fgColor=[color=CornFlowerBlue]"%INSTRUCTION%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"1"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"STRING"[/color] styleID=[color=CornFlowerBlue]"6"[/color] fgColor=[color=CornFlowerBlue]"%STRING%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"2"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"NUMBER"[/color] styleID=[color=CornFlowerBlue]"7"[/color] fgColor=[color=CornFlowerBlue]"%NUMBER%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"1"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"IDENTIFIER"[/color] styleID=[color=CornFlowerBlue]"8"[/color] fgColor=[color=CornFlowerBlue]"%IDENTIFIER%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"LABEL"[/color] styleID=[color=CornFlowerBlue]"10"[/color] fgColor=[color=CornFlowerBlue]"%CLASS_NAME%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"1"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"VARIABLES"[/color] styleID=[color=CornFlowerBlue]"9"[/color] fgColor=[color=CornFlowerBlue]"%NUMBER%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"VARREF-KW"[/color] styleID=[color=CornFlowerBlue]"19"[/color] fgColor=[color=CornFlowerBlue]"%NUMBER%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"1"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"ERROR"[/color] styleID=[color=CornFlowerBlue]"20"[/color] fgColor=[color=CornFlowerBlue]"FF0000"[/color] bgColor=[color=CornFlowerBlue]"%IDENTIFIER%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] />
			<!--Keywords -->
			<WordsStyle name=[color=CornFlowerBlue]"FLOWCONTROL"[/color] styleID=[color=CornFlowerBlue]"11"[/color] fgColor=[color=CornFlowerBlue]"%INSTRUCTION%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"1"[/color] keywordClass=[color=CornFlowerBlue]"0"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"COMMANDS"[/color] styleID=[color=CornFlowerBlue]"12"[/color] fgColor=[color=CornFlowerBlue]"%INSTRUCTION%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"1"[/color] keywordClass=[color=CornFlowerBlue]"1"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"FUNCTIONS"[/color] styleID=[color=CornFlowerBlue]"13"[/color] fgColor=[color=CornFlowerBlue]"%REGEX%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] keywordClass=[color=CornFlowerBlue]"2"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"DIRECTIVES"[/color] styleID=[color=CornFlowerBlue]"14"[/color] fgColor=[color=CornFlowerBlue]"%INSTRUCTION%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] keywordClass=[color=CornFlowerBlue]"3"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"BUTTONS/KEYS"[/color] styleID=[color=CornFlowerBlue]"15"[/color] fgColor=[color=CornFlowerBlue]"%IDENTIFIER%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"3"[/color] keywordClass=[color=CornFlowerBlue]"4"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"BUILT-IN VARIABLES"[/color] styleID=[color=CornFlowerBlue]"16"[/color] fgColor=[color=CornFlowerBlue]"%NUMBER%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"2"[/color] keywordClass=[color=CornFlowerBlue]"5"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"SPECIAL PARAMETERS"[/color] styleID=[color=CornFlowerBlue]"17"[/color] fgColor=[color=CornFlowerBlue]"%DEF_NAME%"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] keywordClass=[color=CornFlowerBlue]"6"[/color] />
			<WordsStyle name=[color=CornFlowerBlue]"USER DEFINED"[/color] styleID=[color=CornFlowerBlue]"18"[/color] fgColor=[color=CornFlowerBlue]"FF0000"[/color] bgColor=[color=CornFlowerBlue]"%BGCOLOR%"[/color] fontName=[color=CornFlowerBlue]""[/color] fontStyle=[color=CornFlowerBlue]"0"[/color] keywordClass=[color=CornFlowerBlue]"7"[/color] />
		</LexerType>
	)
	NewXML := [color=DarkRed]RegExReplace[/color]( FullXML, [color=CornFlowerBlue]"m`ai)(</LexerType>\r\n)(^\s+</LexerStyles>)"[/color], [color=CornFlowerBlue]"$1`t`t"[/color] template [color=CornFlowerBlue]"`r`n$2"[/color] )
	[color=DarkRed]FileDelete[/color], Themes - AHK\%file% ; delete file if it already exists
	[color=DarkRed]FileAppend[/color], %NewXML%, Themes - AHK\%file%
	[color=DarkRed]Return[/color]
}

Here's my Auto-Completion File ( right click - save as ). This goes in %NP++ Dir%\plugins\APIs

Maybe I will attempt to do another lexer in the future ...

I would recommend including [] as a SYNOPERATOR. Also, it would be nice if % wasn't grouped with the other SYNOPERATORs. IMO, it would be nice to have this colored the same as the variables, but different than the other SYNOPERATORS - like autoit variables. Plus, it would be nice if a single % was highlighted for forced expressions. Just some thoughts .

Also, I've had some issues with the Folding - particularly when using one-true-brace.

Amazing job my friend!
and agree there with the % operator.

PS: @PhiLho sorry for the misspelling, I will be fixing that soon.

I just found this yesterday, and I love it! Sadly, as it stands, I can't really use it. I've gotten very used to the alternate folding (;*** & ;*) in the user defined language that comes with AHK.

When I tried switching over to curly brackets, it starts messing with how scripts need to be laid out. I guess because it interprets them as separate blocks.

I noticed that there are issues mentioned with the folding, so I don't know if this would just add to it... but I thought I may as well ask. Would it be possible to add the alternate folding method to this?

Already fixed up the coloring to go with my custom theme before I noticed this... gotta have a pure black background!

Definitely agree with the % operator.

Many thanks to everyone involved with making this lexer!

I just found this yesterday, and I love it! Sadly, as it stands, I can't really use it. I've gotten very used to the alternate folding (;*** & ;*) in the user defined language that comes with AHK.

When I tried switching over to curly brackets, it starts messing with how scripts need to be laid out. I guess because it interprets them as separate blocks.

I noticed that there are issues mentioned with the folding, so I don't know if this would just add to it... but I thought I may as well ask. Would it be possible to add the alternate folding method to this?

Already fixed up the coloring to go with my custom theme before I noticed this... gotta have a pure black background!

Definitely agree with the % operator.

Many thanks to everyone involved with making this lexer!

Im glad you are trying it... today I will be messing with the lexer again, I will see if I can include the ;*** and ;* as comment and as folders, because is not only you that is used to them

I will try to fix the % operator but cant promise much on that one.
updates will be shown here on this topic.

****Update****

I added ";***" and ";*" to the comment block section, now they behave like "/*" and "*/".

Be careful though, because they are not treated as comment blocks by the ahk parser, so you must add ";" to each line between them.

;***
This would cause an error while trying to run or compile your code
because these are not comment blocks
;*

;***
; This would behave normally
; and it would also fold because those two symbols are treated 
; as comment blocks by *Notepad++* but NOT by AutoHotkey
;
; I recommend using the toggle block comment from the context menu
; to convert blocks of code to commented lines very quickly.
;*

I added the Auto Complete and the Auto Theme files provided by jethrow
I fixed some misspellings as well, nothing big.
Updated the test file as well.

Uploaded to the same link

****Update****

...
Uploaded to the same link

Does not work here - linked zip-file has a size of "0" Bytes here ...

****Update****

...
Uploaded to the same link

Does not work here - linked zip-file has a size of "0" Bytes here ...

Because im right now working on it :lol:

Give me a few minutes I will post the updates.

I would recommend including [] as a SYNOPERATOR. ...

In the meantime:

**v1.2 Changelog**
-Updated test file
-Added '[' and ']' as SYNOPERATORS as requested
-Added "format folders", you can use ';+' and ';-' or ';{' and ';}' to fold lines of codes that are going to be executed but you want to hide for readability... I use it for hiding the includes, hotkeys and basically any piece of code that i want it to be folded at a specific time, see the test file notes for examples.

;+ [Includes]
#Include c:\a_file.ahk
#Include c:\another_file.ahk
;-

;{ [Directives]
#NoEnv
#SingleInstance Force
SetBatchLines -1
;}

;+--> ;[Hotkeys]
Pause::Pause
Esc::ExitApp
;-

; Note that AHK treat those as comments so you can use whatever funky strings you want after the ;+ or ;{
; you can add another ; for getting the green coloring of comments if you want.

Download Link:
AHKExtLexer 1.2

I'm glad to see you're updating this. Maybe it can get released with AHK when all the bugs are worked out . Since I don't know how to modify the lexar, I cannot offer help; but here are some issues I noticed:

[*:2qhg8hh0]The [] don't highlight as they should -

; correct
[color=red][][/color]
collection("A")[color=red][[/color]5[color=red]][/color]
; doesn't highlight
array[5]

[*:2qhg8hh0]A colon before a word gets highlighted as a label:

label:
:label

[*:2qhg8hh0]IMO, the block comment coloring using ;*** & ;* is misleading, even though it's documented.

Yes I will try to keep it up to date, or at least modify it until the people using it feels is ok. But I say I will TRY because I know crap about C++... :lol:

I am following the logic, copy pasting until everything makes sense, give it a try and voila. So dont feel that you cant help.

--> Note to anybody that codes in C++, If you have free time please lend me a hand to make this little bit better <--

Ok, Regarding the "[]" I think I know where the problem is, the ")" is ending the formatting, thats why it works fine on "collection("A")[5]" but as "Array" is a word and it is not clearing the default format the "[]" is not colorizing.

I think I will be able to fix that one pretty soon.

I am not sure but I think there was a case in AHK that you could call a label with :Label... probably for menus or something like that, I want to believe that PHiLho allowed that on purpose and that it is not a mistake, if I dont find the relevant information (or he says that it was a mistake) then I will do my best to fix it.

EDIT:
--Confirmed--
I think it is to allow formating this:
"Menu, tray, add, This Menu Item Is A Submenu, :MySubmenu"

Concerning the ";***" and ";*" they are used by notepad++ by default on the User Defined Languages for allowing folding and some people have gotten used to them even though they really do not mean much to ahk.

I didnt expect much people to use that since you already have the /* */ and ; for commenting, they are just there to satisfy people that are already used to fold with comments with them, but probably I will follow the advice and make them similar to ;+ and ;- which even thought they fold, they do not color the code between them. It would be safer.

Thanks for pointing those bugs out, it helps improving this lexer.