[Wish] longer continuation sections/lines still Topic is solved

Discuss the future of the AutoHotkey language
swagfag
Posts: 4464
Joined: 11 Jan 2017, 17:59

[Wish] longer continuation sections/lines still

Post by swagfag » 17 Nov 2020, 21:41

in a114 the 16k continuation section/line char limit was removed.
The following limits have been removed by utilizing dynamic allocations:
  • Maximum line or continuation section length of 16,383 characters.
the new limit now is ~32k chars
test code(add/erase "a"s one at a time)

SimpleHeap is used to process the script's text during loading. eventually, it caps out at

Code: Select all

#define BLOCK_SIZE (32 * 1024 * sizeof(TCHAR)) // Relied upon by Malloc() to be a multiple of 4.
// 65536 bytes, 32768 wchars incl. null terminator

...
void* SimpleHeap::Malloc(size_t aSize) // aSize in bytes
// This could be made more memory efficient by searching old blocks for sufficient
// free space to handle <size> prior to creating a new block.  But the whole point
// of this class is that it's only called to allocate relatively small objects,
// such as the lines of text in a script file.  The length of such lines is typically
// around 80, and only rarely would exceed 1000.  Trying to find memory in old blocks
// seems like a bad trade-off compared to the performance impact of traversing a
// potentially large linked list or maintaining and traversing an array of
// "under-utilized" blocks.
{
	if (aSize < 1 || aSize > BLOCK_SIZE) // <<<<<<
		return NULL;

User avatar
kczx3
Posts: 1315
Joined: 06 Oct 2015, 21:39

Re: [Wish] longer continuation sections/lines still

Post by kczx3 » 18 Nov 2020, 13:46

When would you have a need for this long of a continuation section?

swagfag
Posts: 4464
Joined: 11 Jan 2017, 17:59

Re: [Wish] longer continuation sections/lines still

Post by swagfag » 18 Nov 2020, 20:13

shortly before writing this thread i had a need for such a continuation section

User avatar
kczx3
Posts: 1315
Joined: 06 Oct 2015, 21:39

Re: [Wish] longer continuation sections/lines still

Post by kczx3 » 18 Nov 2020, 21:40

You seriously took my question of “when” literally? I’m just trying to understand a use case of having that much text in a continuation section.

swagfag
Posts: 4464
Joined: 11 Jan 2017, 17:59

Re: [Wish] longer continuation sections/lines still

Post by swagfag » 19 Nov 2020, 03:09

i dont know about a general use case, mine was:
  • i run my vim buffers as ahk scripts
  • i wanna do some throwaway text processing
  • i dont wanna pipe my data into a file ill probably forget to delete when im done
  • the old v1 limit was removed and there isnt a documented new one, i know - lets paste the data into a string
  • wonder why the damn script wont launch
  • spend time figuring out if its my vim setup's, the data's or ahk's fault
  • spend time singlestepping through ahk's source

User avatar
nnnik
Posts: 4483
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: [Wish] longer continuation sections/lines still

Post by nnnik » 20 Nov 2020, 04:38

i dont wanna pipe my data into a file ill probably forget to delete when im done
Sounds like a you problem.

That being said the new limits should be documented rather than hidden away.
Recommends AHK Studio

User avatar
tank
Posts: 2856
Joined: 28 Sep 2013, 22:15
Facebook: charlie.simmons.7334
Google: ttnnkkrr
GitHub: ttnnkkrr
Location: Irving TX
Contact:

Re: [Wish] longer continuation sections/lines still

Post by tank » 21 Nov 2020, 14:28

nnnik wrote:
20 Nov 2020, 04:38
Sounds like a you problem
so is every other reason someone comes here. ....
is there some reason to not use an include with tge text?

im reminded of the paradigm in programing of separating content, code, and display
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
https://www.facebook.com/ahkscript.org
If you have forum suggestions please submit a pull request
Check Out WebWriter
Thanks Tank :thumbup:

User avatar
kczx3
Posts: 1315
Joined: 06 Oct 2015, 21:39

Re: [Wish] longer continuation sections/lines still

Post by kczx3 » 21 Nov 2020, 15:17

I guess the way I see it is if you’ve got that much text then you should probably just read it in from a file.

swagfag
Posts: 4464
Joined: 11 Jan 2017, 17:59

Re: [Wish] longer continuation sections/lines still

Post by swagfag » 21 Nov 2020, 17:12

im not sure whether lexikos realized there was a new limit(now imposed by another part of the codebase) when he made the change. he might have, and he might have in fact purposefully left it undocumented(in case he intends to change it later down the road) but this seems like an unlikely scenario. either way, had it been documented, obviously this thread wouldnt have existed

Helgef
Posts: 4486
Joined: 17 Jul 2016, 01:02
Contact:

Re: [Wish] longer continuation sections/lines still

Post by Helgef » 29 Nov 2020, 09:30

SimpleHeap::Malloc is used for a lot of things other than continuation sections. so this limit will also apply to other things too. For example, the number of tokens in an expression. It should probably be fixed at some point.

Cheers.

lexikos
Posts: 7511
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: [Wish] longer continuation sections/lines still

Post by lexikos » 30 Jan 2021, 02:44

It should probably redirect to malloc in the case of large allocations, which I think should be some amount significantly below BLOCK_SIZE - or maybe just when aSize > sLast->mSpaceAvailable. Any allocation (below the limit) that won't fit into the remainder of the current block causes a new block to be allocated, leaving the remainder of the current block permanently unused.

lexikos
Posts: 7511
Joined: 30 Sep 2013, 04:07
GitHub: Lexikos

Re: [Wish] longer continuation sections/lines still  Topic is solved

Post by lexikos » 07 May 2021, 03:18

This was fixed by v2.0-a123 (commit 0ae12156d).

swagfag
Posts: 4464
Joined: 11 Jan 2017, 17:59

Re: [Wish] longer continuation sections/lines still

Post by swagfag » 08 May 2021, 10:29

lexikos wrote:
30 Jan 2021, 02:44
It should probably redirect to malloc in the case of large allocations
i think calls to alloca should be reviewed(unless theres a specific reason for using them)
i can now trigger a stack overflow (for example here https://github.com/Lexikos/AutoHotkey_L/blame/d87b942f0f84a9593e633bfcd21eb91677d25cf5/source/script.cpp#L4816 and maybe in other places) if a long enough continuation section is used

Post Reply

Return to “AutoHotkey v2 Development”