AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Out of memory Error (??)

 
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Bug Reports
View previous topic :: View next topic  
Author Message
Joy2DWorld



Joined: 04 Dec 2006
Posts: 541
Location: Galil, Israel

PostPosted: Mon Jun 25, 2007 4:41 am    Post subject: Out of memory Error (??) Reply with quote

Code:
a := "1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"

text := "START" . a . a . a . a . a . a . a . a . a . a . "100" .   a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "100" .  a . a . a . a . a . a . a . a . a . a . "STILL GOOD!!!!!!"



Likely i've just overlooked something simple...

but should this really generate an "out of memory" error message ??
_________________
Joyce Jamce
Back to top
View user's profile Send private message
Lexikos



Joined: 17 Oct 2006
Posts: 4473
Location: Qld, Australia

PostPosted: Mon Jun 25, 2007 5:01 am    Post subject: Reply with quote

There are too many operations in the expression. If you split it up into multiple lines, it works fine.
Quote:
Each variable may contain up to 64 MB of text
Back to top
View user's profile Send private message Visit poster's website
Joy2DWorld



Joined: 04 Dec 2006
Posts: 541
Location: Galil, Israel

PostPosted: Mon Jun 25, 2007 5:21 am    Post subject: Reply with quote

thanks for the input, but

1. What syntax rule has been violated ? ie. where do you see a limitation in the number of operations allowed in an expression ???

2. notice that if you change the variable a to, for example "*" (instead of the current number sequence). the number of operations stays exactly the same.. and no error is generated...

Code:
a = *


ie. the result is not as simple as the "number of operations" involved...

(and, if not very obvious, we are not even close to the 64M limit.. more like around 12,000 bytes....)
_________________
Joyce Jamce
Back to top
View user's profile Send private message
engunneer



Joined: 30 Aug 2005
Posts: 7698
Location: Germany (but I only speak English)

PostPosted: Mon Jun 25, 2007 5:39 am    Post subject: Reply with quote

this is certainly confusing. If I make a 50 chars, I can build a string up to 7000ish chars long. If a is 100 chars, I can only build to 13400ish chars

VarSetCapacity doesn't help.
_________________
Unless noted, all code is UNTESTED.
Answers Here: 1.(Loops, Viruses, etc.) 2.Search 3.RTFM 4.Ask for Help.
PMs will be ignored unless you are hiring me.
Back to top
View user's profile Send private message Visit poster's website
PhiLho



Joined: 27 Dec 2005
Posts: 6723
Location: France (near Paris)

PostPosted: Mon Jun 25, 2007 10:02 am    Post subject: Reply with quote

Looks like you have encountered a hard limit in script.
It is not the length of the line (I can use a longer variable name) but somehow the number of variable uses.
Beyond 125 references, we get this out of memory error, even in older versions of AHK. If the string is longer, the number of allowed variables decreases.
I fear it will fall in the "obscure bugs not worth documenting", as one rarely create such super-long script lines.
Unless Chris finds out the problem and fix it.
_________________
vPhiLho := RegExReplace("Philippe Lhoste", "^(\w{3})\w*\s+\b(\w{3})\w*$", "$1$2")
Back to top
View user's profile Send private message Visit poster's website
Titan



Joined: 11 Aug 2004
Posts: 5043
Location: /b/

PostPosted: Mon Jun 25, 2007 10:04 am    Post subject: Reply with quote

I think the maximum number of items allowed in expressions is 100 (#define MAX_EXPR_MEM_ITEMS 100?) which is why it generates an error - stack has not enough memory.
_________________
Chat (IRC)PlusNetScriptsIronAHK Contact by email not private message.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Joy2DWorld



Joined: 04 Dec 2006
Posts: 541
Location: Galil, Israel

PostPosted: Mon Jun 25, 2007 10:21 am    Post subject: Reply with quote

Quote:
Looks like you have encountered a hard limit in script.
It is not the length of the line (I can use a longer variable name) but somehow the number of variable uses.
Beyond 125 references, we get this out of memory error, even in older versions of AHK. If the string is longer, the number of allowed variables decreases.
I fear it will fall in the "obscure bugs not worth documenting", as one rarely create such super-long script lines.
Unless Chris finds out the problem and fix it.



hmmm. gosh.. if that is the case... why is it "OBSCURE". this is the 6th 'obscure' bug i've run in to... trying to finish the *SAME* ahk function [which took me 3 hours to code, and going on 3 weeks to 'debug'...(is there a debugger i should be using and am not ??)]. How many "obscure" bugs are there ? hundreds, thousands ??? if so... the langauge really is not .. well...

Wish there was included in the docs at least reference/guidance so that the language had NO 'obscure' bugs... if 125 (or 100 ?) is the reference per expression limit... easy enough to include that fact in the expression doc...
_________________
Joyce Jamce


Last edited by Joy2DWorld on Mon Jun 25, 2007 10:42 am; edited 2 times in total
Back to top
View user's profile Send private message
Titan



Joined: 11 Aug 2004
Posts: 5043
Location: /b/

PostPosted: Mon Jun 25, 2007 10:31 am    Post subject: Reply with quote

Documenting everything is unnecessary, remember it's only Chris who writes it all. 100+ expression items is an exceptional case, and bad design. You can split the expression to one assignment and another concat.
_________________
Chat (IRC)PlusNetScriptsIronAHK Contact by email not private message.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Joy2DWorld



Joined: 04 Dec 2006
Posts: 541
Location: Galil, Israel

PostPosted: Mon Jun 25, 2007 11:13 am    Post subject: Reply with quote

Quote:
Documenting everything is unnecessary, remember it's only Chris who writes it all. 100+ expression items is an exceptional case, and bad design. You can split the expression to one assignment and another concat.


1. Really really like AHK... but it's hidden limits are starting to frustrate. Would be 100% fine if they were just documented.

2. Have no complaint for undocumented bugs Chris didn't see, etc. and am glad to help figure out limit that was not known, but have a HUGE COMPLAINT AND FRUSTRATION if limitations are known and intentionally left out of the documentation. For goodness sakes, there is a section on expression memory, etc. sure hope suggestion that including the known limit is a HUGE BENEFIT to AHK... the docs are critical...

excuse me if I vent a bit about this repeating call to sweep bugs under the carpet as if there was some shame to provide helpful documentation:: including the known limitations of the langauge.

IF YOU DOCUMENT THE LIMIT IT IS NOT A BUG, The programmer is warned, has understanding of the limits, and can enjoy coding bug-free...

if you hide, sweep under the carpet, and try to 'minimize' the impact of the language's limits in the name of 'shorter docs'.... my own suggestion is you harm in the long term what you've built... Short docs are great!... can express the limit is the tiniest possible words...



3. Every new innovation is an 'exceptional case', the only language that doesn't allow & promote "exceptional cases" is a dead one. You can only get so much out of the 'tried and true old ways' for so long...

4. How in the world can you say what is good or bad design without looking at the full code/flow/outline ?? Moreover, good/bad is a relative concept, good / bad in relationship to WHAT!!! without knowng the purpose of the code construct, how could you possible even have a basis from which to judge 'good vs. bad'.

5. Obviously there are a million ways to 'work around' the issue, but that is not a reason just to IGNORE IT.

(6. if not obvious, the sample code was *not* the actual code, but a work-up to show the bug/issue/problem.)
_________________
Joyce Jamce
Back to top
View user's profile Send private message
Titan



Joined: 11 Aug 2004
Posts: 5043
Location: /b/

PostPosted: Mon Jun 25, 2007 11:35 am    Post subject: Reply with quote

You're starting to sound like Laszlo... everything is actually documented, you can find all technical notes in the source code. The docs is written as a reference for both newbie non-programmers and experts so Chris decided that a simplistic formal tone would be best understood by everybody. In all fairness you have to exercise reasonable judgement of your own, writing megs of documentation takes a lot of time and energy which should really be spent on developing AutoHotkey itself.
_________________
Chat (IRC)PlusNetScriptsIronAHK Contact by email not private message.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Joy2DWorld



Joined: 04 Dec 2006
Posts: 541
Location: Galil, Israel

PostPosted: Mon Jun 25, 2007 12:06 pm    Post subject: Reply with quote

Quote:
You're starting to sound like Laszlo... everything is actually documented, you can find all technical notes in the source code. The docs is written as a reference for both newbie non-programmers and experts so Chris decided that a simplistic formal tone would be best understood by everybody. In all fairness you have to exercise reasonable judgement of your own, writing megs of documentation takes a lot of time and energy which should really be spent on developing AutoHotkey itself.


Thanks for the compliment...

there is a logical falacy in the argument you've put forth. If briefly documenting all of the known bugs/limitations would take MEGS of documentation, that means there are literally thousands upon thousands of such known bugs... NOT a very pleasant thought.


if, instead there are few some yet undocumented 'limits' ... the added docs will be a few lines here and there....


moreover, the TONE of the docs is not the issue, am all for a 'simplistic formal tone' that everybody can understand. For example "Expressions can contain up to 100 references" is something even a beginner can understand.... (if indeed that is the case and determined not worth fixing).
_________________
Joyce Jamce
Back to top
View user's profile Send private message
Lexikos



Joined: 17 Oct 2006
Posts: 4473
Location: Qld, Australia

PostPosted: Mon Jun 25, 2007 12:19 pm    Post subject: Reply with quote

@Titan: Is there a comment documenting the maximum number of concatenations or variable references per expression, or is it not that explicit?


The source of this limitation seems to be (near the top of Line::ExpandExpression()):
Code:
#define MAX_EXPR_MEM_ITEMS 100 // Hard to imagine using even a few in a typical script, let alone 100.
char *mem[MAX_EXPR_MEM_ITEMS]; // No init necessary.  In most cases, it will never be used.

I'm not sure exactly what this array is used for, or when it is used. I think the size of the variables affects when this array gets used - i.e. with many small concatenations, it may never be used, or may not use all 100 "slots".
(Edit: My guess is it is used as temporary storage for the results of operations within an expression.)
(Edit2: I missed Titan's previous post regarding MAX_EXPR_MEM_ITEMS; It seems to use/fill two other buffers before it resorts to using mem[], so 100 isn't exactly the maximum - it's more like the worst-case maximum.)


With the script in your first post, AHK reaches the maximum (100) items in this array, then aborts the script/thread when it gets to the 101st. (The relevant code is around line 2186 of script_expression.cpp as of v1.0.47.)

I think 100 concatenations per expression/line would be a safe (and reasonable) limit. I'd like to hear what it is you think you need that many concatenations for...


Interestingly, if you change it to use traditional %var% refs, there are two definite (but reasonable) limitations:
1. Maximum 512 variable references. This is documented (in the source code, as I recall.)
2. Maximum variable size, which can be changed via #MaxMem.

It also performs considerably faster.
Quote:
For example "Expressions can contain up to 100 references" is something even a beginner can understand.... (if indeed that is the case and determined not worth fixing).

Is there a practical use for 100 references or concatenations in a single expression? Considering how rarely anyone would encounter this limitation, I can see why it might be left out of the documentation. Perhaps it's so trivial that the users would be better off not knowing it...
Quote:
If briefly documenting all of the known bugs/limitations would take MEGS of documentation, that means there are literally thousands upon thousands of such known bugs...

Titan also said "you have to exercise reasonable judgement"... Under my reasonable judgement, "megs" was an exaggeration not to be taken literally. Wink


Last edited by Lexikos on Mon Jun 25, 2007 2:10 pm; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website
Joy2DWorld



Joined: 04 Dec 2006
Posts: 541
Location: Galil, Israel

PostPosted: Mon Jun 25, 2007 1:16 pm    Post subject: Reply with quote

something like this,

if you want to manage callbacks, or messages, etc.. which come as new 'threads', variable management becomes a critical part of the coding logic.

Where you have multiple connections, (say, 4 spreadsheets and 2 word docs, with 2 web connections = 8 connections), expressions (obviously, can be recoded around any limitation issues), can look like:

RESPONDER := (FACTOR7 ? (BASE3_fACTOR > lower_limit2 ? local2 : local 3) : (base4_factor7 < limit7 ) ? LOCAL l4v3 : LOCAL8) || (FACTOR7B ? (BASE3D_fACTOR > limit8 ? local3 : local7) : (base4_factor7C < var32) ? LOCAL3d : LOCAL8f) ... etc. (this is already what... 20 references )...

moreover, sometimes how we want to Join is controlled by the content itself, vrb := a . ( x > vd ? b : c ) . ( v < etrd ? zy : yw ) . ... etc.
and result := (a > b ? a : b ) . ( a > c ? a2 : c2).... is sure a legit and nice way to build a result string. etc.).. and when re-combining multiple connections into a single var, sure is easier to just do it all in one place... but if we have 15 vars returned on 8 connections... that violates the 100 reference rule... (and ERRORS OUT!! AHK) (but only when the vars are large enough to... so intermittent error)... depending upon the var content!! ugggg!!
_________________
Joyce Jamce
Back to top
View user's profile Send private message
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10667

PostPosted: Mon Jun 25, 2007 6:57 pm    Post subject: Reply with quote

I didn't even know it was possible to hit the MAX_EXPR_MEM_ITEMS limit; but apparently having a lot of concat operators is a way to do it. In the next release, the limit will be doubled, which will allow the example in the topmost post to work (that example actually needs less than a 10% increase).

This change shouldn't affect performance because all it does is use a little more stack memory during execution, which is strictly temporary memory -- it won't raise the program's memory utilization (except in extreme cases where a script does deep recursion).

Thanks for reporting it.
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Bug Reports All times are GMT
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group