Working on a complete wrapper for the Scintilla control. Requires Scintilla.dll 32-bit or 64-bit.
This aims to do what Notepad++ and SciTE does with being able to define your own syntax highlighting, and (hopefully) provide easier direct access to the capabilities of Scintilla. I have found the layout of SciTE config files to be a bit much for my preference. This project aims to bypass the middle man so that the user can define the format of the config file themselves without sacrificing speed.
Eventually I will include support for the SciTE Lexer.dll since there really is no need to double-tap that functionality and the languages it already handles. All of my user-defined syntax highlighting can easily be disabled with ctl.CustomSyntaxHighlighting := false, and then you can continue to use the Scintilla class framework to wield fine-grained control over the Scintilla control.
This project aims to be the foundation for my next project which will be the successor to CallTipsForAll. This "foundation" will simply be flexibly defined syntax highlighting at compiled DLL speeds, and will contain the framework to easily implement word lists, call tips, auto-complete, and most if not all of the other features Scintilla offers.
The current version of the DLL is quite sufficient if all you need is flexible syntax coloring (and up to 8 keyword lists). I plan to expand this further, but this will require making a different DLL. The now "old" DLL will be archived in the repo. If you intend to use the CustomLexer.dll and you find issues, please post them in this thread or open an issue on GitHub. I plan to continue supporting this DLL for at least a little while longer.
Stats:
Scintilla version: 5.1.1
test-script.ahk size: 2.16 MB
Current avg load time: 2.2 secs
- Using 7 word lists
- Using case insensitivity (minor performance penalty)
Highlights:
define braces used (default is ()[]{}<>) and coloring
define punctuation (mostly used as operators and separators) and coloring
define line comment sequence (ie ;, //, and others) and coloring
define block comment sequences (ie /* */, --[[ ]], and others) and coloring
other syntax highlighting includes numbers, hex numbers, "string", 'string', selection colors
"`"" and '`'' are properly handled, as well as "``" and '``'
define the escape character as needed (default is ` -> Chr(96))
quick and easy access to change the coloring/background of the editor, default text, number margin, and caret
includes a CustomLexer.dll (and source) to process custom highlighting at compiled speeds
persistent brace matching/coloring, including persistent mismatched brace style for easy-to-see mismatched braces
deleting a brace makes it's previous match red, and commenting out a brace makes it's match red (currently line comments only)
typed braces are initially red, until you type the correct matching brace
typing an opening " or ' or /* won't cascade through the entire document and change the style
pressing ENTER in a /* block comment */ extends the block comment and styles text accordingly
(block comment toggling not yet implemented as a hotkey)
use up to 8 keyword lists and assign colors (see example) - check DefaultTheme() method for setting colors/style
For more details see the GitHub repo docs.
Any feedback is welcome. I'm just trying to make a solid foundation for doing more custom text editor stuff with scintilla that is able to break free from the SciTE established framework. Knowing other people's thoughts or uses will just help make this project better.
Word lists were previously not finished to the level I had intended. That is now fixed (ie. word coloring during typing, no crashing, etc). This is probably the final version of this DLL. I will keep it on the repo as a general means of just "coloring words". The next version will be more aware of the structure of the document and the syntax... closer to a true Lexer. The "keyword lists" will be replace by... something else, not sure what yet. But it will define the structure of the language so that doc parsing and coloring can happen at the same time.
added word highlighting (currently only 8 lists of any length)
updated ctl.DefaultTheme() method to setup 3 keyword list colors
added ctl.SetKeywords() method
added ctl.CaseSense proerty
added keyword lists for AHK (functions, built-in vars, flow control words) - not checked for completeness (just testing)
updated DLL code to take a keyword list and CaseSense value
currently loading test-script.ahk in about 1.55 secs
updated readme to include brief instructions for setting keywords (script example is more verbose)
2021/08/27
fixed exception thrown 0x00000005 when deleting all text
pressing ENTER inside /* block comment */ now extends the block comment and styles text accordingly
removed some old lines of code
updated OP
updated README on GitHub
2021/08/26
disabled LayoutCache in the example for more consistent fast loading
added a mechanism to disable unnecessary document parsing when loading
reworked README and consolidated some docs
2021/08/25
added C code source and DLL to handle styling
much improved performance on large documents
added AHK funcs to do the same operations as the DLL
search for: "benchmarking for C code equivalent in AHK" - not completely equivelant currently
added Italic, Bold, Underline properties to "custom" styles
This was technically already possible, but is now more accessible when using the "cust" class. See docs.
Updated docs (not much - but the new info is good to know).
Added a beefy test script (thanks to robodesign)
experimentation is on-going!
Features:
Brace coloring is done as you edit the doc. Mismatched braces are persistently red.
Deleting a brace, or commenting it out, colors it's matching brace red.
All elements in the new "cust" subclass are colored according to user settings.
Be sure to check the DefaultOpt() method for what settings are currently set for the script demo.
NOTE: Some settings are also set at the top of the script.
2021/08/06
updated to Scintilla 5.1.1
Custom syntax highlighting basic framework is now complete:
- see StylingRoutine() and DeleteRoutine() methods
- currently numbers, strings, comments, braces, and punctuation are colored
- not much customization available yet, unless you edit the class directly
- I plant to implement a framework that will make defining specific colors for specific groups/types of characters
cleaned up code
implemented SCI_GETDIRECTSTATUSFUNCTION in place of SCI_GETDIRECTFUNCTION
ctl.Status property now pulls the status from what was returned using the new DirectStatusFunction (should be faster when checking status)
removed old AutoBraceMatching()
removed Events class, I'm just putting these events in as methods of the class
performance is quite decent, but needs to be broken up to prioritize user viewed lines first
refined pasting large chunks of text (getting 6s load time on ~2MB of text)
initial coloring is strings, comments, and braces only (in that order) on the first go
other syntax elements are colord per-screen when scrolling, or per-line when typing
2021/06/21
tested and working on a137
2021/05/17
added ctl.Target.* subclass, aka Searching category (forgot to add this from a week ago)
added ctl.Target.Text for getting the matched text
removed ctl.Margin.DefaultMinWidth / .WidthOffset
Now calculation of number margin is more dynamic based on font size
ctl.Event.MarginWidth(), with only first two params, can be used to autosize the number margin on-demand
moved ctl.Style.TextWidth() to ctl.TextWidth()
added "DefaultOpt" and "DefaultTheme" options for Gui.AddScintilla(), see example
added ctl.AutoBraceMatch (false by default), uses style 40 and 41 by default (DefaultTheme)
added ctl.Event.BraceMatch() ... must be used in callback, don't use when ctl.AutoBraceMatch := true
2021/05/11
moved example back to top, now initiating class with "(Scintilla)" for the example (see comments)
fixed some instances of blank optional method parameters not functioning as intended
added Doc category (Multiple Views)
officially crossed off "Other Settings" category (it was done a while ago)
reorded several properties and methods on the main Scintilla class prepping for documentation
added ctl.AutoSizeNumberMargin (defaults to false)
added ctl.Events category for packaged functions/methods, intended to assist with common tasks
added ctl.Events.MarginWidth(margin, size) - for on-demand or usage within user callback (don't use this method directly in callback when ctl.AutoSizeNumberMargin := true)
added ctl.Margin.MinWidth / .DefaultMinWidth / .WidthOffset (mostly used in ctl.Events.MarginWidth())
2021/05/08
changed BufferAlloc() to Buffer()
reworked handling of adding .AddScintilla() to Gui class
added some more properties/methods
moved example below the class because of new class init process
2021/05/05
added .wm_messages() callback per control created (attached to the Gui control obj)
added SCNotification struct as a sub-class
added callback property to Gui control object ... [ callback(ctl, scn) ]
added several static objs for listing constants
updated and tested offsets for SCNotification struct (13/22 offsets verified for x86 and x64)
added a few more SCI_\* funcs
moved some Scintilla control customizations out of main class into a func in the example
Added .Lookup() and .GetFlags() static methods for easier interal workings
If (event = "Modified")
{
LineCount := ctl.Lines
ctl.Margin.ID := 0
LineMarginWidthNow := ctl.Margin.Width
; quick and dirty hard code
LineMarginWidthFit := (StrLen(LineCount) * 11) + 5
if (LineMarginWidthNow != LineMarginWidthFit)
ctl.Margin.Width := LineMarginWidthFit
}
But that isn't responsive to font style and zoom levels. It would help if the library had SCI_TEXTWIDTH because its return value is font and zoom responsive.
@neogna2
Thanks! I am definitely planning on adding a base-line set of functions to do stuff like this, as well as properly manage brace highlighting (or some version of it - i have ideas)
moved example back to top, now initiating class with "(Scintilla)" for the example (see comments)
fixed some instances of blank optional parameters not functioning as intended
added Doc category (Multiple Views)
officially crossed off "Other Settings" category (it was done a while ago)
reorded several properties and methods on the main Scintilla class prepping for documentation
added ctl.AutoSizeNumberMargin (defaults to false)
added ctl.Events category for packaged functions/methods, intended to assist with common tasks
added ctl.Events.MarginWidth(margin, size) - for on-demand or usage within user callback (don't use this method directly in callback when ctl.AutoSizeNumberMargin := true)
added ctl.Margin.MinWidth / .DefaultMinWidth / .WidthOffset (mostly used in ctl.Events.MarginWidth())
removed ctl.Margin.DefaultMinWidth / .WidthOffset
Now calculation of number margin is more dynamic based on font size
ctl.Event.MarginWidth(), with only first two params, can be used to autosize the number margin on-demand
moved ctl.Style.TextWidth() to ctl.TextWidth()
added "DefaultOpt" and "DefaultTheme" options for Gui.AddScintilla(), see example
added ctl.AutoBraceMatch (false by default), uses style 40 and 41 by default (DefaultTheme)
added ctl.Event.BraceMatch() ... must be used in callback, don't use when ctl.AutoBraceMatch := true
I'm working on laying the ground-work for a basic lexer written in AHK to do the basics like comments, syntax punctuation, quoted strings, brace matching, etc.
This will then be extendable by the user callback, and can optionally be disabled and the user can completely control how this works as desired.
Custom syntax highlighting basic framework is now complete:
- see StylingRoutine() and DeleteRoutine() methods
- currently numbers, strings, comments, braces, and punctuation are colored
- not much customization available yet, unless you edit the class directly
- I plant to implement a framework that will make defining specific colors for specific groups/types of characters
cleaned up code
implemented SCI_GETDIRECTSTATUSFUNCTION in place of SCI_GETDIRECTFUNCTION
ctl.Status property now pulls the status from what was returned using the new DirectStatusFunction (should be faster when checking status)
removed old AutoBraceMatching()
removed Events class, I'm just putting these events in as methods of the class
performance is quite decent, but needs to be broken up to prioritize user viewed lines first
refined pasting large chunks of text (getting 6s load time on ~2MB of text)
initial coloring is strings, comments, and braces only (in that order) on the first go
other syntax elements are colord per-screen when scrolling, or per-line when typing
added AHK funcs to do the same operations as the DLL
search for: "benchmarking for C code equivalent in AHK" - not completely equivelant currently
added Italic, Bold, Underline properties to "custom" styles
This was technically already possible, but is now more accessible when using the "cust" class. See docs.
Updated docs (not much - but the new info is good to know).
Added a beefy test script (thanks to robodesign)
experimentation is on-going!
Features:
Brace coloring is done as you edit the doc. Mismatched braces are persistently red.
Deleting a brace, or commenting it out, colors it's matching brace red.
All elements in the new "cust" subclass are colored according to user settings.
Be sure to check the DefaultOpt() method for what settings are currently set for the script demo.
NOTE: Some settings are also set at the top of the script.
In VS Code some language plugins have lots of features beyond syntax highlighting and intellisense, like global symbol renaming, "stepwise" intellisense that auto jumps between parameter fields, calltips, jump to definitions, ...
Thanks for the links! I will definitely look into that. I wasn't thinking LSP initially, but I'll give it a try and see if I can make that work.
What exactly is "global symbol renaming"? I know what intellisense is, but what is "stepwise" intellisense?
I did plan on trying to implement recognition for when the cursor is in a particular parameter of a function. I tried that in CallTipsForAll and didn't quite get there. Another limitation I ran into was trying to parse an object tree, so that object members could be discovered. I plan to dive back into that and try it again.
I have a bit to learn to compete with VSCode and use LSP, but I'm certainly willing to learn and try
EDIT: At a minimum I'll be recreating CallTipsForAll functionality and trying to extend the capability of how code and objects are linked.
EDIT2: I'm reading up on LSP stuff now, and wow that looks like a big project. It seems that the application needs to perform its own parsing the content, if I'm not mistaken. I'm already planning to implement word lists in my CustomLexer.dll to facilitate recognizing specific words for coloring. If I can't fit it into the DLL, then I'll do this in the Scintilla AHK class. Hopefully I'll find the right starting point to be able to implement LSP, but I could certainly use help. I'm still trying to understand many parts of what I'm reading just with implementing a server, let alone designing an implementation for AHK. Still reading...
What exactly is "global symbol renaming"? I know what intellisense is, but what is "stepwise" intellisense?
I don't know if there are better terms for this. With global symbol renaming I meant e.g. updating all variable names in one go, see first GIF here https://marketplace.visualstudio.com/items?itemName=thqby.vscode-autohotkey2-lsp
With stepwise intellisense I meant for example you start to type "Mou" and get suggestion "MouseClick". Press enter and the suggestions continue step by step to help you fill out that function's arguments. In contrast to autosuggestion features that only suggest a single string and then ends.
I'm reading up on LSP stuff now, and wow that looks like a big project.
Yeah I was only mentioning this idea because I was checking out various LSP plugins for VS Code and saw some similarity between features there and your goals with CallTipsForAll, but I don't know if LSP for Scintilla controls is at all feasible to make at this point. Way above my coding level for sure. I also don't want to lure you into stretching yourself too thin seeing you've got several awesome projects going here already.
Thanks for that short explanation with the GIF. That cleared it up. That looks very doable to implement (for global symbol updates), though it will certainly be later down the road after prior functions/features that support it are done first.
I think I follow what you mean with "stepwise". Sounds like a minor modification to how CallTipsForAll (CTFA) used to work, but instead showing a preview of parameters before auto filling?
I plan to remake the AHK v2 parser that was part of CTFA and improve upon it. Mostly to make it more complete.
I'm doing a decent job so far of keeping load times low, but I still need to implement the ability for the script to monitor coloring progress (for coloring and parsing syntax) so that I can query progress and avoid a mysterious undefined wait time for large documents (that is probably my next task).
Yeah I was only mentioning this idea because I was checking out various LSP plugins for VS Code and saw some similarity between features there and your goals with CallTipsForAll, ... I also don't want to lure you into stretching yourself too thin seeing you've got several awesome projects going here already.
No worries about "stretching me thin". If I can't do it, I either won't, or I'll slow down until I can.
Even if the end result it isn't LSP based, I still plan to expand on what CTFA did.
I'll keep reading through the links you gave and other stuff I find. If I can wrap my brain around setting up an LSP server then that will be the biggest step toward LSP compatibility.
@TheArkive, have you tried adding a minimap sidebar to a Scintilla control? Minimaps are useful for long text document but I'm pondering what the best approach to make it would be.
One idea is to have another Scintilla control with the same text but minimal font and track the caret or scroll position of the main Scintilla control and color X lines around the caret in the minimap. Not sure if that would cause performance issues with very large documents. Another idea is to let non-Scintilla code make a "thumbnail" image like representation of the source at some interval (a rough match is enough) and use that as minimap. Like what this for rust seems to do.
Notepad++ has a minimap (called "document map") but Window Spy reports its control as Static (not Scintilla), don't know how it is implemented. Found a mention of it here https://sourceforge.net/p/scintilla/feature-requests/1409/ but that's about it.
I have contemplated that. But that is further down the road. I'm still at the point of writing a lexer system purely in ahk, which is also tricky to maintain good performance on large documents.
It would have to be implemented as more of an external thing, a bitmap in a static control, as you saw with WindowSpy.
If it's poorly coded, it will perform poorly Of course doing this purely in ahk is and getting good performance is also tricky, especially with large documents.
Download on GitHub
I have a problem - you keep adding excellent features faster than I'm able to try them 
Of course doing this purely in ahk is and getting good performance is also tricky, especially with large documents.