AutoHotkey Community

It is currently May 27th, 2012, 12:21 am

All times are UTC [ DST ]




Post new topic Reply to topic  [ 15 posts ] 
Author Message
PostPosted: December 3rd, 2009, 5:38 pm 
Offline
User avatar

Joined: May 5th, 2007, 7:24 pm
Posts: 1240
Location: Seville, Spain
Just run this (I understand it's for performance but at least it should be documented):
Code:
Loop, 2
   func()

func()
{
   ; Note no static bugvar!
   static switchvar = 0
   if switchvar
   {
      ; Make the variable show its hidden contents
      NumPut(Asc("H"), bugvar, 0, "UChar")
      MsgBox % bugvar
   }else ; This function was ran for the first time
   {
      ; Initialize the variable
      bugvar = _idden variable contents reappear!
      switchvar = 1
   }
}

_________________
fincs
Highly recommended: AutoHotkey_L (see why) (all my code snippets require it)
Formal request to polyethene - I support the unity of the AutoHotkey community
Get SciTE4AutoHotkey v3.0.00 (Release Candidate)
[My project list]


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 3rd, 2009, 6:52 pm 
Offline

Joined: October 7th, 2006, 4:50 pm
Posts: 3157
Location: MN, USA
I get a blank message box (running the latest AHK version).


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 3rd, 2009, 7:17 pm 
Offline

Joined: April 8th, 2009, 8:23 pm
Posts: 3036
Location: Rio de Janeiro - RJ - Brasil
jaco0646 wrote:
I get a blank message box (running the latest AHK version).

Same here.

_________________
"Read the manual. Read it again. Search the forum.
Try something before asking. Show what you've tried.
"
Image
Antonio França
My stuff: Google Profile


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 3rd, 2009, 9:43 pm 
Offline
User avatar

Joined: May 5th, 2007, 7:24 pm
Posts: 1240
Location: Seville, Spain
I'm also using the vanilla version of AutoHotkey v1.0.48.05 (no Unicode/_L stuff involved). Am I missing something here?
Image

_________________
fincs
Highly recommended: AutoHotkey_L (see why) (all my code snippets require it)
Formal request to polyethene - I support the unity of the AutoHotkey community
Get SciTE4AutoHotkey v3.0.00 (Release Candidate)
[My project list]


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 3rd, 2009, 10:43 pm 
Offline

Joined: June 8th, 2006, 9:38 pm
Posts: 307
It's the #NoEnv directive that's causing this to happen :shock:


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 4th, 2009, 12:12 am 
Offline

Joined: October 7th, 2006, 4:50 pm
Posts: 3157
Location: MN, USA
Confirmed.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 4th, 2009, 3:05 am 
Offline

Joined: April 8th, 2009, 8:23 pm
Posts: 3036
Location: Rio de Janeiro - RJ - Brasil
#NoEnv wrote:
A future version of AutoHotkey might make this behavior the default.

:|

_________________
"Read the manual. Read it again. Search the forum.
Try something before asking. Show what you've tried.
"
Image
Antonio França
My stuff: Google Profile


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 4th, 2009, 6:17 am 
Offline

Joined: October 1st, 2005, 9:55 pm
Posts: 775
Location: Texas, USA
Unless I'm missing something, this is a documented condition (feature) and it includes local variables that are defined within functions. fincs just exposed a means of getting at memory that has not been released. You can duplicate this "problem" with global variables. Just create a global variable to something with a length that is less than 64 bytes, set the variable to null, and then use fincs' trick for reallocating the memory. Something like this:

Code:
#NoEnv
gVar=_idden va0iable con0ents reap0ear!56789012345678901234567890123
VarSetCapacity(gVar,0)
NumPut(Asc("H"),gVar,0,"UChar")
MsgBox % gvar

For all (OK, most) practical purposes, the variable does not exist when you use it again. The string length is zero and if you test the variable, i.e. If MyVariable, it will test as False. All string commands such as StringLeft, StringLower StringMid, etc. are not affected.

Them be my thoughts...


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 4th, 2009, 7:02 am 
Offline

Joined: March 27th, 2008, 2:14 pm
Posts: 700
What's happening here is that the first character of the discarded variable is set to NULL, and since ahk's strings are null-terminated this essentially makes the string a length of 0. By overwriting the null character with another character, the remaining string is visible again until the next null character. (which is now at the end of the first string because, as jballi mentioned, it's length is < 64 bytes and it is never really freed in the true sense)

I think.

:P

Edit:

"All string commands such as StringLeft, StringLower StringMid, etc. are not affected."

This, too can be explained by the fact that ahk keeps a separate record of the length of the string in a given variable, which is set to 0 when cleared. All the string commands refer to this value for validating values, which can save it a lot of time vs going through the entire string to find the null char every time someone uses a string command.

_________________
Scripts - License


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 4th, 2009, 9:51 am 
Offline

Joined: October 17th, 2006, 4:15 pm
Posts: 7502
Location: Australia
What do you want to happen when you use NumPut on an "uninitialized" variable? Variables are not guaranteed to be zero-initialized unless you specifically do it yourself.

Roland wrote:
It's the #NoEnv directive that's causing this to happen
If #NoEnv is not present, AutoHotkey looks for an environment variable when the variable's internal "length" field is 0. Even if there is no environment variable, it does not use the variable's actual contents in that case. #NoEnv itself is not in any way responsible for the effect demonstarted by fincs and jballi.

OT: #NoEnv is important for performance because without it, every time you evaluate an empty variable, the script has to search for an environment variable by the same name. Even having this mechanism present (with or without #NoEnv) negatively affects performance of all scripts, though perhaps not noticably.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 4th, 2009, 5:19 pm 
Offline
User avatar

Joined: May 5th, 2007, 7:24 pm
Posts: 1240
Location: Seville, Spain
Lexikos wrote:
What do you want to happen when you use NumPut on an "uninitialized" variable? Variables are not guaranteed to be zero-initialized unless you specifically do it yourself.

Just look at this case (I only want it to be documented!):
Code:
DecodeBase64(data){
   DllCall("crypt32\CryptStringToBinaryA", "uint", &data, "uint", 0, "uint", 1, "uint", 0, "uint*", len, "uint", 0, "uint", 0)
   VarSetCapacity(d_data, len)
   DllCall("crypt32\CryptStringToBinaryA", "uint", &data, "uint", 0, "uint", 1, "str", d_data, "uint*", len, "uint", 0, "uint", 0)
   ; This function doesn't work properly if you don't uncomment the NumPut() line.
   ; The scripter may be puzzled to see remnants from the last function result if
   ; the base64-encoded data doesn't contain a termination NULL (the most common case):
   ;NumPut(0, d_data, len, "UChar")
      
   return d_data
}

_________________
fincs
Highly recommended: AutoHotkey_L (see why) (all my code snippets require it)
Formal request to polyethene - I support the unity of the AutoHotkey community
Get SciTE4AutoHotkey v3.0.00 (Release Candidate)
[My project list]


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 4th, 2009, 7:07 pm 
Offline

Joined: October 17th, 2006, 4:15 pm
Posts: 7502
Location: Australia
I think you missed my point.

"All local variables start of blank each time the function is called," but this only means a null-terminator is set at the beginning of the variable. Any content following the null-terminator is initially undefined, so relying on its value is asking for trouble. Documenting that local variables might "retain" their contents between calls would be counterproductive since the contents are still undefined in other situations.

On the other hand, it might be worth clarifying how variables are "initialized". VarSetCapacity hints at it:
VarSetCapacity wrote:
FillByte: This parameter is normally omitted, in which case the memory of the target variable is not initialized (instead, the variable is simply made blank as described above).

Local variables are freed or cleared when the function returns in the same way as VarSetCapacity(var,0), so the following applies:
VarSetCapacity wrote:
For performance reasons, freeing a variable whose previous capacity was between 1 and 63 might have no effect because its memory is of a permanent type.
However, it only might have no effect, so you can't rely on it. If d_data extends beyond 63 bytes, it becomes non-permanent and will be freed when the function returns. Undocumented: In all subsequent calls it will receive non-permanent memory. VarSetCapacity will allocate exactly the length requested (plus 1) and will null-terminate in the correct position every time. This behaviour can be forced:
Code:
DecodeBase64(data){
    static s
    if !s  ; To demonstrate it only needs to be done once.
        s := VarSetCapacity(d_data,64), VarSetCapacity(d_data,0)
    ...
}

Btw, even if the variable's contents are freed, there's no guarantee that the same memory won't be allocated to that variable (or a different variable) at some later point. A variable could hypothetically even be allocated different memory with contents identical to what it had previously.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 4th, 2009, 10:35 pm 
Offline

Joined: October 7th, 2006, 4:50 pm
Posts: 3157
Location: MN, USA
First off, thanks to jballi, infogulch, and Lexikos. Your explanations were all very informative. I learned something today. 8)

Lexikos wrote:
What do you want to happen when you use NumPut on an "uninitialized" variable?
I think the issue is not about changing the behavior of NumPut, but about how variables should be referenced (e.g. after Numput has been used).

Lexikos wrote:
...it does not use the variable's actual contents in that case.
A variable's actual contents should never be used when its internal length field is 0. The message box should always show blank, regardless of #NoEnv.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 5th, 2009, 4:16 am 
Offline

Joined: October 17th, 2006, 4:15 pm
Posts: 7502
Location: Australia
jaco0646 wrote:
A variable's actual contents should never be used when its internal length field is 0. The message box should always show blank, regardless of #NoEnv.
If the internal length field is inaccurate, why should it be used over the contents of the variable? It is usually disregarded in simple cases:
Code:
t = the test
NumPut(0,t,3,"uchar"), VarSetCapacity(t,-1), NumPut(Asc(" "),t,3,"uchar")
MsgBox 0, % StrLen(t), % t
Changing the behaviour to always respect the internal length field (for consistency) would require a copy to be made of the arg where ordinarily it could get away with using the string directly. In other words, the performance benefit of the internal length field could be negated.

While I was researching this, I noticed that some areas use the internal length field (indirectly) and then assume the arg is null-terminated at that position. In particular, parsing loops allocate a buffer based on the "length" of the arg, then disregard that length and do a simple string copy:
Code:
size_t space_needed = ArgLength(2) + 1;  // +1 for the zero terminator.
...
strcpy(buf, ARG2); // Make the copy.
This can cause stack or heap corruption:
Code:
VarSetCapacity(t,40000,3)
NumPut(0,t,0,"char"), VarSetCapacity(t,-1), NumPut(3,t,0,"char")
Loop, Parse, t
{}
Parsing loops use the stack for anything <= 40000 characters and the heap (malloc) otherwise.
jaco0646 wrote:
I think the issue is not about changing the behavior of NumPut, but about how variables should be referenced (e.g. after Numput has been used).
Right. I meant "after" vs "when".


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: December 5th, 2009, 5:03 am 
Offline

Joined: October 7th, 2006, 4:50 pm
Posts: 3157
Location: MN, USA
Lexikos wrote:
...the performance benefit of the internal length field could be negated.
Ok, I can accept that. Performance trumps all. :) I also was not aware that the internal length could be updated manually, which at least offers a bit of control over the behavior.


Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 15 posts ] 

All times are UTC [ DST ]


Who is online

Users browsing this forum: No registered users and 5 guests


You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group