Beta version pre-v1.0.93: optimized file I/O and #Warn

Update: AutoHotkey_L v1.0.95.00 has been released.

ac and I have developed some new features to assist with debugging scripts. These can be enabled by using the new #Warn directive:

#Warn [v1.0.93.00+]

Enables or disables warnings for selected load-time or run-time conditions that may be indicative of developer errors, such as typos or missing "global" declarations.

#Warn WarningType [, WarningMode]

This is intended to be a developer tool that helps speed up script development by making such errors easily discoverable.

This feature is conceptually related to Visual Basic's "Option Explicit" setting and Perl's "Strict" pragma, though it only provides added-on warnings, rather than requiring any up-front changes to the script (such as explicit declarations for all variables).

[Read more on the #Warn directive.]

Some relatively low-level changes were required to make this possible, but behaviour of existing scripts shouldn't be affected. Note that if warnings are enabled, warnings may be given for certain situations which are valid (but not necessarily good style). For example:
[*:w9dugi2v]Code like If !x or If x= is often used to determine if x needs to be initialized. In particular, this was necessary for static vars in older builds since static initializers did not support expressions. As these uses of x will generate a warning if x has not been initialized, it is recommended to explicitly initialize x to "" or 0, where appropriate.

[*:w9dugi2v]Sometimes a DllCall output parameter (with * or P suffix) is used to initialize a variable. However, since only the programmer and the function itself know if the input value is important, a warning is still raised if the variable was not previously initialized. To avoid this, the variable can be initialized inline: DllCall("SomeFunc", "IntP", outVar := 0).

[*:w9dugi2v]When used alone on a line, x += 1 treats blank values as zero. Because of this, some (working) scripts don't explicitly initialize x and will therefore receive a warning. Since this behaviour is likely to change in v2 (for consistency), users should get used to explicitly initializing x.Additionally, this release includes extensive changes to optimize file I/O, as in the previous test release:

To address some performance issues recently brought up here, I've made some extensive changes to the file reading and writing routines used by FileAppend, Loop Read, the File Object and script parser. Since these changes are so extensive, rather than immediately releasing a new version I thought I should first give everyone the opportunity to (cautiously) test it, and to catch any issues I may have missed.

Even just testing the scripts you use regularly may help, particularly if they contain non-ASCII characters.

(Download link removed; get v1.0.95 instead)
Updated 2011-03-06: Mostly unrelated bug-fixes.
Updated 2011-03-07: #If A_PriorHotkey; improved auto-concat; Run *Verb now supports params.
Updated 2011-03-08: Fixed (x="" and y).

Thanks for this . Currently I have no problems so far (scripts running: S4AHK toolbar, TillaGoto, tools, etc). By the way, I've just updated the S4AHK git repo to support the new #Warn directive.

However, I'm worried by the fact that not all uses of uninitialized variables are error prone, such as this code:

MakeList(obj)
{
    for each, item in obj
        list .= item ","
    StringTrimRight, list, list, 1
    return list
}

Do you have any comments on that?

fincs,

Note that if warnings are enabled, warnings may be given for certain situations which are valid (but not necessarily good style).

Use of concat-assign with an uninitialized variable might be an error, therefore a warning should be given. I would consider it an acceptable drawback of using #Warn. Making exceptions for specific expression operators could in theory affect the performance of all operators, even with #Warn disabled.

As implied by the quote above, it's a matter of style: it is simple enough to initialize list to "", and it's a good habit to be in.

Thanks for the updates. I used this beta version with my normal scripts at work yesterday and didn't notice any script behavior being affected.

Though I don't foresee myself using #Warn too often, I do like the concept. Additionally, though I'm not sure if you're open to suggestions here, I would suggest extending the warnings to optionally warn about Auto-concat code as well (as opposed to limiting Auto-concat).

Again, thanks for the updates .

I would echo what a4u said. For AHK it seems best to allow the user to control error checking, because most projects are small and may not need it. I think the ideal would be that #Warn catches as much as possible, and without it everything is just handled in whatever way produces the most intuitive behavior without errors. That’s essentially what AHK does now (just handles things).
Personally, I would like to see all blank values in math operations treated as zero while the directive would still notify you to initialize them. Same with auto-concat: have #Warn provide notifications, but don’t restrict its behavior. Actually, I would like to see it handle at least one case it currently doesn’t. A quoted string followed by Pre-increment/decrement seems an unambiguous concatenation.

Lexikos, thanks for your continued efforts.
I downloaded and tested AutoHotkey109300b1. Scripts that I use daily are working normally. Just one thing is a little bit confusing - I downloaded AutoHotkey109300b1.zip, extracted it, put it in the right place, but

MsgBox % A_AhkVersion

shows 1.0.92.02. So am I using v1.0.93.00b1 or 1.0.92.02? I guess the first one, cause there is no "This line does not contain a recognized action." message when I run #Warn All, Off

Results for FileAppend test script I posted here are now:
0.042 seconds - average (tested 20 times)

I would suggest extending the warnings to optionally warn about Auto-concat code as well (as opposed to limiting Auto-concat).

I wouldn't use it, but I'll consider adding it. It does seem like a good alternative.

Personally, I would like to see all blank values in math operations treated as zero while the directive would still notify you to initialize them.

I'll consider it. I think the main reason for removing that behaviour was for error-detection (i.e. if you get a blank result, the input was non-numeric). #Warn is a partial solution for that, but what about non-empty, non-numeric values, or empty values that come from functions (e.g. when NumGet detects an invalid parameter)?

In any case, I'd prefer not to change the default behaviour until v2.

A quoted string followed by Pre-increment/decrement seems an unambiguous concatenation.

There are other cases, like "x" &y or "x" -y. I had looked into changing these, and there was some reason I chose not to. I don't recall the reason, so I'll reconsider.

So am I using v1.0.93.00b1 or 1.0.92.02?

It's "pre-v1.0.93." I generally don't bump the version number manually; it's done by script when I do a full release.

I just realised that the default "env" behaviour (i.e. when #NoEnv is not present) combined with #Warn interferes with certain legitimate methods of initializing a variable. For instance:

; #NoEnv
#Warn All
VarSetCapacity(foo, 16, 0)  ; Warning.
init(bar)  ; Warning.
init(ByRef var) {
    var := 1
}

This is because the contents of foo and bar are retrieved in a way that may trigger an "uninitialized" warning if no environment variable by that name exists. When #NoEnv is present or the variables have contents, this is skipped and the variable reference itself is used.

The first and simplest solution would be for #Warn to automatically enable #NoEnv. However, then #Warn would cause a change in behaviour which might seem counterintuitive if it is being used temporarily to debug a script without #NoEnv. Even if the script uses environment variables only unintentionally, having #Warn change the behaviour could be confusing.

The second solution would be almost as simple and would be transparent to the user, but may marginally affect code size and performance. I won't go into details.

There's also the "non-solution" - force the user to choose between adding #NoEnv, removing #Warn, or explicitly (and redundantly) initializing those variables to avoid the warning.

Note that none of this is of relevance to v2, since #NoEnv will be the mandatory default.

Any thoughts?

I like option 1: #Warn forces #NoEnv. Everyone should be used to #NoEnv by now, especially the savvier crowd which is likely to use #Warn. The documentation mentions the possibility of #NoEnv becoming the default, and it's also been in Template.ahk for years. A script using environment variables unintentionally seems like exactly the sort of behavior that #Warn should draw attention to.

A script using environment variables unintentionally seems like exactly the sort of behavior that #Warn should draw attention to.

Agreed.

A script using environment variables unintentionally seems like exactly the sort of behavior that #Warn should draw attention to.

If we choose option 1, #Warn will not draw attention to unintentional use of env vars; it will prevent it entirely. If someone's script doesn't work because they've unintentionally used an env var, then they add #Warn and it starts working correctly, I think there are two possible outcomes:
[*:1z2ia0ns]The user knows #Warn enables #NoEnv, connects the dots and realizes their mistake.
[*:1z2ia0ns]The user is utterly confused that the script now works and no warning is given.There's also the possibility that a user will use #Warn while developing their script, then remove it to distribute the script and they or another user will find that it no longer works.

On the other hand, changing #Warn to always warn - even when an environment variable is implicitly being accessed - would be in line with both your statement and my goal of keeping #Warn as strictly a debugging tool. Now that I think about it, it seems like the best option.

The documentation mentions the possibility of #NoEnv becoming the default,

At this point it's only a matter of time; it is in Chris' v2 document and is one easy change that I look forward to making.

I've updated the download with several bug fixes (unrelated to my previous post). Read the commit history if you're curious.

If we choose option 1, #Warn will not draw attention to unintentional use of env vars; it will prevent it entirely.

Some clarification: an environment variable is retrieved automatically when the variable is blank, not necessarily uninitialized. If #NoEnv is applied automatically by #Warn, a variable initialized as temp := "" won't correspond to an environment variable and won't cause a warning since it has been initialized. If #Warn is then removed, temp may resolve to the environment variable. Thus, #Warn will have made it magically work without ever alerting the user to a possible error.

Although I think it could be a good idea to give the "uninitialized" warning even when an environment variable is accessed (after all, it really wasn't initialized), that unfortunately won't solve the problem I outlined earlier. (That is, the warning which pops up when we're trying to pass the variable to init() or VarSetCapacity() to initialize it.)

What do people think about a warning merely for the absence of #NoEnv (when #Warn All is used)? Or perhaps a new warning specifically for when an environment variable is accessed (which would cover the temp := "" case)?

Wow Lexikos! Amazing improvements in File I/O speeds. I did some simple tests where I gave 5 seconds to write as much data at 80 characters per line. In the previous AHK_L build I got about 30,000. In C++ using <fstream> I got about 110,000. With this new version of AHK_L I got about 140,000.

Edit: All numbers in kilobytes.

Actually, I would like to see [auto-concat] handle at least one case it currently doesn’t. A quoted string followed by Pre-increment/decrement seems an unambiguous concatenation.

It was simple, so I implemented the necessary changes. If the next non-whitespace character after the literal string is one of "+-*&~!" and not followed by "=", it is treated as though there is a concat operator between the string and symbol. (I believe this covers all of the unary operators.)

I've also changed #If to temporarily update A_PriorHotkey and A_TimeSincePriorHotkey, and added support for parameters with Run verbs. (For example, Run *RunAs %A_ScriptFullPath% /foo.) See my first post for the updated beta.

Thanks for the updates . I haven't done extensive testing, but I did notice the following (bug?):

x:="", y:=1

MsgBox, % y and x="" ; 1

MsgBox, % x="" and y ; empty (shows as 1 in v1.0.92.02)

MsgBox, % (x="") and y ; 1