Variadic varrefs and __Enum arity arg?

Get help with using AutoHotkey (v2 or newer) and its commands and hotkeys
sirksel
Posts: 224
Joined: 12 Nov 2013, 23:48

Variadic varrefs and __Enum arity arg?

30 Aug 2021, 09:50

I've been refactoring libraries for beta1, and have been implementing using variadic varrefs to replace my prior uses of byref var1, byref var2, ..., etc. Before varrefs were added, we couldn't use arrays of byref params, but we now can. As I was toying with this, I created a grouped/chunked enumerator of arrays (which requires one other library function):

Code: Select all

rpt(f, i) {   ;repeat f(a_index) i times
  loop i
    f(a_index)
}
grp(xs)   ;grouped enum of xs array; size of group matches arity
  => {__enum : (_,a) => (len := xs.length // a,  i := 0,  (rs*) 
    => (++i > len) ? 0 : (rpt(j => (%rs[j]% := xs[i*a-(a-j)]), a), 1))}
This enum gives you back, on each iteration, as many items at a time as you have variables in the for..in loop. For example...

Code: Select all

a := strsplit('abcdefghijklmnop')
m := ''
for q,r,s in grp(a)
  m .= q r s '`n'
for q,r,s,t in grp(a)
  m .= q r s t '`n'
msgbox m
But then it dawned on me that I might not even need the __enum(arity) method at all, since (1) now functions themselves can be called directly as enumerators, and (2) in the case of a function like this, the arity is evident from the length of the varref variadic received. So I made a second version that returns a function directly, rather than an object:

Code: Select all

grpx(xs)   ;grouped enum of xs array; size of group matches variadic len
  => (i:=0,  (rs*) => (a := rs.length,  (++i > xs.length//a) ? 0 
    : (rpt(j => (%rs[j]% := xs[i*a-(a-j)]), a), 1)))
I suppose that when comparing grpx vs grp, there's an extra .length resolution and a bit of integer division at each iteration rather than up front. Is there any other advantage of using the object vs function return (in other words, to prefer grp over grpx) aside from this? I'm not as familiar as I probably should be with what happens behind the scenes with these enum calls. I appreciate any thoughts you all might have...
swagfag
Posts: 6222
Joined: 11 Jan 2017, 17:59

Re: Variadic varrefs and __Enum arity arg?

30 Aug 2021, 10:53

invoking an object's __Enum() method is the only way to retrieve
the number of variables passed to the for-loop.
https://lexikos.github.io/v2/docs/Objects.htm#__Enum

ull have to benchmark, per for-loop statement encountered:
  • is creating an object and calling __Enum() once more expensive...
  • or is invoking .Length as many times as there are elements in the array?

using an actual enumerator(and not written in the worse-than-regex-barf style)

Code: Select all

Chunk(InputArray) {
	ElementEnum := InputArray.__Enum(1)

	GetNumberOfVars(DummyThis, NumberOfVars) {
		ChunkEnum(VarRefs*) {
			Loop NumberOfVars
				if !ElementEnum(VarRefs[A_Index])
					return false

			return true
		}

		return ChunkEnum
	}

	return {__Enum: GetNumberOfVars}
}
sirksel
Posts: 224
Joined: 12 Nov 2013, 23:48

Re: Variadic varrefs and __Enum arity arg?

30 Aug 2021, 11:14

I appreciate the thoughts, @swagfag. I like the changes you made in your version. And...
the worse-than-regex-barf style
You correctly guessed the subtitle of my functional programming textbook!
Thanks again for the help.
swagfag
Posts: 6222
Joined: 11 Jan 2017, 17:59

Re: Variadic varrefs and __Enum arity arg?

30 Aug 2021, 11:47

sirksel wrote:
30 Aug 2021, 11:14
You correctly guessed the subtitle of my functional programming textbook!
the one u learnt from or the one ure in the process of writing? i would hope not the... know what, nevermind lol
lexikos
Posts: 9679
Joined: 30 Sep 2013, 04:07
Contact:

Re: Variadic varrefs and __Enum arity arg?

02 Sep 2021, 03:54

swagfag wrote:
30 Aug 2021, 10:53
invoking an object's __Enum() method is the only way to retrieve the number of variables passed to the for-loop.
Doesn't sirksel's code already demonstrate the other way to retrieve the number of variables?

Code: Select all

showsthenumberofvariables(thevariables*) {
    MsgBox thevariables.length
    return false
}
for a, b, c in showsthenumberofvariables {
    ; no iterations
}
__Enum is primarily intended for objects which have other purposes besides being passed to a for-loop.

If you are using a function, typically it's a function object/closure that you initialize by calling some other function, which serves the purpose of __Enum. The one thing that other function cannot do is initialize based on the number of for-loop variables. Without the __Enum parameter, an enumerator that needs to know how many variables there are before initializing would need to delay until the first call, putting initialization and iteration in the same function.

The debugger invokes __Enum(2) to list the object's items.
swagfag
Posts: 6222
Joined: 11 Jan 2017, 17:59

Re: Variadic varrefs and __Enum arity arg?

02 Sep 2021, 04:16

lexikos wrote:
02 Sep 2021, 03:54
Doesn't sirksel's code already demonstrate the other way to retrieve the number of variables?
:roll: sure, i guess. by which point uve already begun iterating...
lexikos
Posts: 9679
Joined: 30 Sep 2013, 04:07
Contact:

Re: Variadic varrefs and __Enum arity arg?

03 Sep 2021, 03:58

In my example, the loop does not iterate, so there literally is no point at which iteration has begun, and yet the number of for-loop variables was retrieved.

If what you mean is "by which point the enumerator function has already been called"... so what?

As I said, you would need to delay initialization until the first call. __Enum might allow the code to be structured more cleanly and/or efficiently in some situations.

It is inane to say that __Enum is the only way, especially immediately prior to implying that it might not even be the most efficient.
swagfag
Posts: 6222
Joined: 11 Jan 2017, 17:59

Re: Variadic varrefs and __Enum arity arg?

03 Sep 2021, 18:51

fair enough
so can u demo how u effectively "delay initialization until the first call" for "an enumerator that needs to know how many variables there are before initializing (would need to delay until the first call, putting initialization and iteration in the same function.)"?

i managed to do it using static, which works fine as long as there's only 1 for-loop accessing the iterator at a time:

Code: Select all

loopAsManyTimesAsThereAreVars(VarRefs*) {
	static i := 0
	if !i
		i := VarRefs.Length + 1

	return --i
}

for a, b, c in loopAsManyTimesAsThereAreVars
	MsgBox A_Index
; needs to show:
; MsgBox 1
; MsgBox 2
; MsgBox 3
and is completely disastrous when multiple for-loops attempt to do so:

Code: Select all

for a, b, c in loopAsManyTimesAsThereAreVars
{
	MsgBox 'Outer loop index: ' A_Index

	if (A_Index = 2)
	{
		for a, b, c, d, e in loopAsManyTimesAsThereAreVars
			MsgBox 'Inner loop index: ' A_Index
	}
}
; needs to show:
; MsgBox Outer loop index: 1
; MsgBox Outer loop index: 2
;  MsgBox Inner loop index: 1
;  MsgBox Inner loop index: 2
;  MsgBox Inner loop index: 3
;  MsgBox Inner loop index: 4
;  MsgBox Inner loop index: 5
; MsgBox Outer loop index: 3

; instead shows(WRONG):
; MsgBox Outer loop index: 1
; MsgBox Outer loop index: 2
;  MsgBox Inner loop index: 1
; MsgBox Outer loop index: 3
; MsgBox Outer loop index: 4
; MsgBox Outer loop index: 5
the goals are:
  • to not create additional objects(otherwise u could just use the return {__Enum: ...} pattern)
  • to not access any object properties(otherwise u could just query VarRefs.Length instead). i guess accessing VarRefs.Length at least once to initialize whatever ure using for storage is required, so 1 time is permitted
  • to not have it misbehave when multiple forloops are involved
lexikos
Posts: 9679
Joined: 30 Sep 2013, 04:07
Contact:

Re: Variadic varrefs and __Enum arity arg?

04 Sep 2021, 04:00

@swagfag
In short, you're asking me to demonstrate how to initialize during the first call to the enumerator function, with some additional contrived requirements.

otherwise u could just use the return {__Enum: ...} pattern
Is that some kind of problem? What bearing does this have on the ultimate goal? It is always possible to use the {__Enum: ...} pattern, whether or not the use of that pattern brings any advantages.

to not create additional objects
What are additional objects? If you compare grp and grpx closely, you should find that grp creates not only an additional object with an __Enum method, but an additional closure as well:
  • grp(xs) -> {__Enum} -> (_,a) captures xs -> (rs*) captures xs, a, i
  • grpx(xs) -> (rs*) captures xs, i
So in this case, the non-__Enum method creates fewer objects.

to not access any object properties(otherwise u could just query VarRefs.Length instead).
What bearing does this requirement have on the topic? It seems your goal is not to demonstrate initialization, but to perform micro-optimization. In that case, the requirement should be to meet some performance criteria, not to avoid one or two specific things that you perceive as adding overhead.

I assume this requirement came about because grpx accesses rs.length on each iteration whereas grp does not. But did you notice that grpx also accesses xs.length on every iteration whereas grp does it up front? They are not directly comparable, and these differences are not inherent to the use or disuse of __Enum.

i managed to do it using static, which works fine as long as there's only 1 for-loop accessing the iterator at a time
To support nested for-loops, each for-loop must have its own state, not a single shared state. The example using static demonstrates only that static should not be used for this; it has nothing to do with whether you use __Enum. There are generally two ways to ensure each loop gets its own enumerator state:
  • Explicitly create a new state, such as by calling a function (e.g. ObjOwnProps, grpx and grp).
  • Use __Enum.
Regardless of whether you use __Enum, there are multiple ways to represent or store the state, including:
  • Use a closure.
  • Use a bound function (for mutable state you would bind an object or VarRef).
  • The classic way: create an object with a Next (v1) or Call (v2) method.
There are also "dirtier" methods, such as relying on the value of A_Index or the for-loop variables.


Although the majority of enumerators require at least one object per enumeration, you can avoid creating additional objects by utilizing a closure, as with grpx. If you want to avoid accessing properties more than once, you can do the same thing as with static i, but using a non-static captured variable instead.

Code: Select all

loopAsManyTimesAsThereAreVars() {
	i := 0
	next(VarRefs*) {
		if !i
			i := VarRefs.Length + 1
		return --i
	}
	return next
}

for a, b, c in loopAsManyTimesAsThereAreVars() {
	MsgBox 'Outer loop index: ' A_Index
	if (A_Index = 2) {
		for a, b, c, d, e in loopAsManyTimesAsThereAreVars()
			MsgBox 'Inner loop index: ' A_Index
	}
}
or

Code: Select all

loopAsManyTimesAsThereAreVars() => (
	i := 0,
	(VarRefs*) => i ? --i : i := VarRefs.Length
)

for a, b, c in loopAsManyTimesAsThereAreVars() {
	MsgBox 'Outer loop index: ' A_Index
	if (A_Index = 2) {
		for a, b, c, d, e in loopAsManyTimesAsThereAreVars()
			MsgBox 'Inner loop index: ' A_Index
	}
}

If you want to pass the same object directly to multiple for-loops but give each loop its own state, you generally need __Enum. For example (putting aside the original goal of demonstrating initialization), this can be used with or without ():

Code: Select all

loopAsManyTimesAsThereAreVars() => loopAsManyTimesAsThereAreVars
loopAsManyTimesAsThereAreVars.__Enum := (f,n) => (*) => n--
sirksel
Posts: 224
Joined: 12 Nov 2013, 23:48

Re: Variadic varrefs and __Enum arity arg?

09 Sep 2021, 16:44

@lexikos and @swagfag, thanks so much for the ideas. Just reading your discussion gives me a better understanding not only of the enumerators, but of closures too. And please don't think I'm writing in this style just to obfuscate, or otherwise use a function where I should use for..in. A lot of this is just a simplified proof-of-concept of how to accomplish certain things in v2, especially as it relates to functional programming. I'm actually using v2 in a class on functional programming.

I have encountered some behavior I can't explain related to the original grp function we've been discussing. I'm wondering if you can help me. I'm noticing that its returned object isn't working as I expected when passed as a function object.

N.B. Please don't get hung up that there is no need for a for2 or mb2 function or that rev is a weird reverser that doesn't do much... I know that. These are just simple examples taking the place of more complex variadic functions which took an enumerator as a parameter (assume for the moment, always a real object with an __Enum property, as opposed to just a callable). I'm sure this is just an error I'm making in thinking about the closures or calls or upvars. I was able to simplify the behavior I didn't expect down to the following:

Code: Select all

rpt(f, i) {   ;repeat f(a_index) i times (UNCHANGED)
  loop i
      f(a_index)
}

grp(xs)   ;grouped enum of xs array; size of group matches arity (UNCHANGED)
  => {__enum : (_,a) => (len := xs.length // a,  i := 0,  (rs*) 
    => (++i > len) ? 0 : (rpt(j => (%rs[j]% := xs[i*a-(a-j)]), a), 1))}
    
for2(enm, act) {  ;a sample function receiving an enumerator
  for a,b in enm
    act(a,b)
}

a := strsplit('abcde')
mb2(x,y) => msgbox(x ',' y)  ;sample action of arity 2

for q,r in grp(a)  ;this DOES work as expected
  mb2(q,r)
for2(grp(a), mb2)   ;this DOES NOT work... but why?

rev(xs)   ;a quick-and-dirty reverse enum generator of similar structure
  => {__enum : (_,a) => (len := xs.length,  i := 0,  (rs*)
    => (++i > len) ? 0 : (%rs[1]% := i, %rs[2]% := xs[-i], 1))}

for2(rev(a), mb2)   ;this DOES work... but why?
Except for a few simplifications and not calling rpt, this rev function produces the same kind of object (I think) as grp. Like I said, I'm just looking for why the difference in the behavior, not necessarily different code that does exactly this (which is an oversimplified example).

Many thanks for your thoughts and discussion!
swagfag
Posts: 6222
Joined: 11 Jan 2017, 17:59

Re: Variadic varrefs and __Enum arity arg?

10 Sep 2021, 07:32

its some kind of a logical bug that fails to consider fat-arrow functions. in specific cases, this v1-era code: https://github.com/Lexikos/AutoHotkey_L/blame/3808407a7c8871c0e87288a739f7ff317caaab7b/source/script_expression.cpp#L404-L419
frees correctly prepared result_token(ie in %rs[j]% := xs[i*a-(a-j)], the thing u got from xs[i*a-(a-j)] and are about to assign to %rs[j]%), which:
  • if it was a String, it simply gets freed(which is where the empty string then comes from)
  • if it was an Object, its refcount gets decremented(havent checked if thats actually a problem, but it probably is)
  • if it was anything else, nothing happens since .free() wouldnt do anything(which is why if ur array contains Ints or Floats, theyre displayed just fine)
minimal example is:

Code: Select all

#Requires AutoHotkey v2.0-beta.1

funcWrappingFor()

funcWrappingFor() {
	; forloop HAS to be inside a function, probably because 
	; creating a scope has something to do with the bug
	for firstVar in enumFunc
		MsgBox '<' firstVar '>'
	until A_Index = 3 ; this is here just to prevent endless loops
}

; direct enumerator function
; formal args HAVE to accept the VarRefs themselves,
; NOT dereference them ByRef ie &firstVarRef
enumFunc(firstVarRef) {
	; "normally"-declared functions are unaffected...
	innerFunc() {
		%firstVarRef% := '123456789' ; works
	}

	; ..."fat arrow shorthand"-declared functions have problems with Strings(and maybe Objects)...
	innerFunc() => (%firstVarRef% := '123456789') ; broken

	; ...as do anonymous functions...
	innerFunc := () => (%firstVarRef% := '123456789') ; broken

	; ...unless ure not assigning strings(or Objects, maybe)...
	innerFunc() => (%firstVarRef% := 123456789) ; works
	innerFunc := () => (%firstVarRef% := 12345.6789) ; works

	; ...and unless the last thing to return is NOT an(/any?) expression.
	innerFunc() => (%firstVarRef% := '123456789', 'last thing not an expression') ; works
	innerFunc := () => (%firstVarRef% := '123456789', 'last thing not an expression') ; works

	; call it
	innerFunc()

	return true
}
swagfag
Posts: 6222
Joined: 11 Jan 2017, 17:59

Re: Variadic varrefs and __Enum arity arg?

10 Sep 2021, 09:00

lexikos wrote:
04 Sep 2021, 04:00
In short, you're asking me to demonstrate how to initialize during the first call to the enumerator function, with some additional contrived requirements.
contrived how? the requirements are as relevant as they can be, to this specific context - the one in which we're trying to figure out which enumerator pattern has the advantage over the other, and also if there exists a different one that outdoes either
What are additional objects?
i dont remember exactly. i think i was thinking about regular {} Objects and VariadicArgCaptureLists* Objects
the requirement should be to meet some performance criteria, not to avoid one or two specific things that you perceive as adding overhead.
yes and no. yes, u should judge them by their performance. no, u shouldnt disregard specific things uve identified as adding overhead - if im trying to do better than the implementation the object-accesses once, it stands to reason i should be weary of a proposed one that does so, say, 3 times...right, all things equal
I assume this requirement came about because grpx accesses rs.length on each iteration whereas grp does not.
the requirements came about because of the the different ways that each respective function uses to fetch the number of for-loop parameters:
  • one has to create an Object, but does so only once per for-loop({_Enum: ...})
  • another doesnt have to do that, but instead has to query an object property on every loop iteration(VarRefs.Length)
  • a third one only has to query an object property once, but then has to check if a variable is set on every loop iteration, and manage its state, and is generally unusable with concurrent for-loops(static)
  • a fourth one, same as above but without the unusability downsides(ur later suggestion, captured local var)
But did you notice that grpx also accesses xs.length on every iteration whereas grp does it up front? They are not directly comparable, and these differences are not inherent to the use or disuse of __Enum.
i didnt, but that doesnt matter. i wasnt going to bench with these functions anyway
tldr:
  • {__Enum: ...} is worst for tiny arrays(< 6 elements). alright for arrays < 10 elements. then its the best
  • Func(VarRefs.Length) is best for tiny arrays(< 3 elements). then its bad
  • static is the bestest for arrays < 10 elements, but we dont talk about static here
  • static reimplemented as a local var closure(Lexikos suggestion) is middle of the road for all cases
  • and for the best of all worlds, u can have the hybrid, but u have to write 2 enumerators...
image.png
image.png (51.75 KiB) Viewed 1535 times

Code: Select all

#Requires AutoHotkey v2.0-beta.1

newArr(arraySize) {
	Arr := []
	Arr.Length := arraySize
	return Arr
}

Funcs := [
	withObject,
	withFunc,
	withFunc_Closure,
	withFunc_Static,
	withObjectFunc_Hybrid,
]

Arrays := [
	newArr(1), 
	newArr(2), 
	newArr(3), 
	newArr(4), 
	newArr(5), 
	newArr(6), 
	newArr(7), 
	newArr(8), 
	newArr(9), 
	newArr(1e1), 
	newArr(1e2), 
	newArr(1e3), 
	newArr(1e4), 
	newArr(1e5), 
]

bench(Funcs, [Arrays[1]], 100, 1) ; warmup
ToolTip('warmup over, bench started...')

MsgBox A_Clipboard := bench(Funcs, Arrays, 5000, 5) ; 5 trials, 5sec each, take AVG

bench(FuncsToBench, Arrays, trialMaxDurationMs, numTrials) {
	results := 'Func Name,Duration (ms),Array Size,'
	Loop numTrials
		results .= '#' A_Index ','
	results .= 'AVG # for-loops`n'

	for fn in FuncsToBench
	{
		for Arr in Arrays
		{
			results .= fn.Name ',' trialMaxDurationMs ',' Arr.Length ','

			loopsTotal := 0
			Loop numTrials
			{
				t0 := A_TickCount
				numLoops := 0
				Loop
				{
					for a, b, c in fn(Arr)
					{
					}

					++numLoops
				} until (A_TickCount - t0 > trialMaxDurationMs)

				loopsTotal += numLoops
				results .= numLoops ','
			}

			results .= Integer(loopsTotal // numTrials) '`n'
			ToolTip(results)
		}
	}

	return results
}

withObject(Arr) {
	arraySize := Arr.Length

	GetNumberOfVars(_, NumberOfVars) {
		Enum(VarRefs*) {
			a := NumberOfVars ; simulate access
			
			return arraySize--
		}

		return Enum 
	}

	return {__Enum: GetNumberOfVars}
}

withFunc(Arr) {
	arraySize := Arr.Length

	Enum(VarRefs*) {
		a := VarRefs.Length ; simulate access
		
		return arraySize--
	}

	return Enum 
}

withFunc_Closure(Arr) {
	arraySize := Arr.Length

	a := 0
	Enum(VarRefs*) {
		if !a
			a := VarRefs.Length ; simulate access
	
		return arraySize--
	}

	return Enum 
}

withFunc_Static(Arr) {
	arraySize := Arr.Length

	Enum(VarRefs*) {
		static a := 0
		if !a
			a := VarRefs.Length ; simulate access

		if !arraySize--
			a := 0 ; reset static
	
		return arraySize
	}

	return Enum
}

withObjectFunc_Hybrid(Arr) {
	arraySize := Arr.Length

	GetNumberOfVars(_, NumberOfVars) {
		Enum(VarRefs*) {
			a := NumberOfVars ; simulate access
			
			return arraySize--
		}

		return Enum 
	}

	EnumFunc(VarRefs*) {
		a := VarRefs.Length ; simulate access
		
		return arraySize--
	}

	return (arraySize < 6) ? EnumFunc : {__Enum: GetNumberOfVars}
}

/*
Func Name              Duration (ms)  Array Size  #1       #2       #3       #4       #5       AVG # for-loops
withObject             5000           1           1945254  1927993  1914452  1934812  1949967  1934495
withObject             5000           2           1715788  1720068  1717613  1719687  1719882  1718607
withObject             5000           3           1540577  1538180  1541053  1539713  1541252  1540155
withObject             5000           4           1389593  1394472  1392556  1393690  1390978  1392257
withObject             5000           5           1266118  1262982  1257349  1265592  1270341  1264476
withObject             5000           6           1164437  1166979  1157481  1167635  1164327  1164171
withObject             5000           7           1075677  1079365  1078728  1072994  1071582  1075669
withObject             5000           8           1000771  1003487  1001706  1002484  997316   1001152
withObject             5000           9           938057   937072   936967   938156   941527   938355
withObject             5000           10          880451   883577   881237   882131   883315   882142
withObject             5000           100         135998   134723   135735   135872   136007   135667
withObject             5000           1000        14339    14398    14301    14249    13939    14245
withObject             5000           10000       1383     1411     1407     1423     1449     1414
withObject             5000           100000      145      145      145      145      145      145
withObject             5000           1000000     15       15       15       15       15       15
withFunc               5000           1           2358656  2368269  2363562  2367670  2361705  2363972
withFunc               5000           2           1953775  1957537  1959411  1958857  1960417  1957999
withFunc               5000           3           1671344  1660686  1652057  1674942  1674306  1666667
withFunc               5000           4           1458979  1462303  1464115  1462477  1464035  1462381
withFunc               5000           5           1293908  1299761  1296535  1299488  1298116  1297561
withFunc               5000           6           1162969  1163854  1164234  1163870  1165825  1164150
withFunc               5000           7           1053497  1056317  1057986  1057101  1055997  1056179
withFunc               5000           8           965234   964995   969477   968101   967548   967071
withFunc               5000           9           885244   891154   893406   894549   888854   890641
withFunc               5000           10          825738   829026   829437   828780   829262   828448
withFunc               5000           100         110597   111028   111026   111141   110986   110955
withFunc               5000           1000        11472    11508    11490    11495    11494    11491
withFunc               5000           10000       1151     1154     1154     1153     1154     1153
withFunc               5000           100000      116      116      116      116      116      116
withFunc               5000           1000000     12       12       12       12       12       12
withFunc_Closure       5000           1           2266266  2250779  2267224  2258062  2271677  2262801
withFunc_Closure       5000           2           1915476  1920723  1920870  1921718  1920954  1919948
withFunc_Closure       5000           3           1665136  1662285  1663446  1658205  1662838  1662382
withFunc_Closure       5000           4           1462293  1466876  1465566  1467628  1465893  1465651
withFunc_Closure       5000           5           1312952  1311915  1315187  1308963  1312281  1312259
withFunc_Closure       5000           6           1185100  1187727  1185962  1187346  1184642  1186155
withFunc_Closure       5000           7           1080150  1078473  995850   1084052  1086630  1065031
withFunc_Closure       5000           8           997085   997059   997852   998051   997282   997465
withFunc_Closure       5000           9           922990   923059   923527   923119   923079   923154
withFunc_Closure       5000           10          858603   860231   858653   858379   858465   858866
withFunc_Closure       5000           100         119406   119590   119696   119580   119609   119576
withFunc_Closure       5000           1000        12418    12234    12441    12455    12450    12399
withFunc_Closure       5000           10000       1250     1251     1251     1251     1250     1250
withFunc_Closure       5000           100000      125      126      125      126      126      125
withFunc_Closure       5000           1000000     13       13       13       13       13       13
withFunc_Static        5000           1           2989978  2993045  2993040  2991846  2993648  2992311
withFunc_Static        5000           2           2357272  2350085  2365413  2361718  2354536  2357804
withFunc_Static        5000           3           1949946  1953482  1949752  1946265  1944928  1948874
withFunc_Static        5000           4           1661389  1668290  1669043  1668704  1668422  1667169
withFunc_Static        5000           5           1451022  1450487  1451926  1454942  1453144  1452304
withFunc_Static        5000           6           1288045  1287649  1289947  1289790  1289189  1288924
withFunc_Static        5000           7           1155457  1158915  1156198  1158667  1158138  1157475
withFunc_Static        5000           8           1049142  1050203  1051504  1049505  1050186  1050108
withFunc_Static        5000           9           949967   961002   961082   948501   951971   954504
withFunc_Static        5000           10          883523   884000   879867   884722   887377   883897
withFunc_Static        5000           100         106491   106886   106818   106712   106820   106745
withFunc_Static        5000           1000        11331    11355    11352    11353    9728     11023
withFunc_Static        5000           10000       1135     1135     1138     1140     1139     1137
withFunc_Static        5000           100000      114      114      112      114      114      113
withFunc_Static        5000           1000000     12       12       12       12       12       12
withObjectFunc_Hybrid  5000           1           2096476  2104881  2100353  2101428  2103146  2101256
withObjectFunc_Hybrid  5000           2           1773099  1778219  1778230  1778600  1780194  1777668
withObjectFunc_Hybrid  5000           3           1536806  1542578  1543294  1543702  1543174  1541910
withObjectFunc_Hybrid  5000           4           1354059  1359342  1357492  1358700  1358918  1357702
withObjectFunc_Hybrid  5000           5           1212985  1214865  1215366  1214763  1215610  1214717
withObjectFunc_Hybrid  5000           6           1107775  1108367  1107501  1107945  1105473  1107412
withObjectFunc_Hybrid  5000           7           1026888  1025412  1025139  1029148  1026659  1026649
withObjectFunc_Hybrid  5000           8           954440   956861   946605   958297   956809   954602
withObjectFunc_Hybrid  5000           9           898004   898529   898561   898641   898269   898400
withObjectFunc_Hybrid  5000           10          841376   844905   844309   843332   842813   843347
withObjectFunc_Hybrid  5000           100         134898   135269   135150   134939   135240   135099
withObjectFunc_Hybrid  5000           1000        14362    14392    14389    14391    14349    14376
withObjectFunc_Hybrid  5000           10000       1445     1448     1448     1442     1445     1445
withObjectFunc_Hybrid  5000           100000      145      145      145      145      145      145
withObjectFunc_Hybrid  5000           1000000     15       15       15       15       15       15
lexikos
Posts: 9679
Joined: 30 Sep 2013, 04:07
Contact:

Re: Variadic varrefs and __Enum arity arg?

11 Sep 2021, 05:19

swagfag wrote:
10 Sep 2021, 09:00
the requirements are as relevant as they can be,
I said they were contrived, not irrelevant. I said it based on (possibly false) assumptions that are not relevant to the discussion, and I take it back.

Now I would describe the two requirements as misguided and counter-productive.
yes and no.
Actually, it's just "yes".
yes, u should judge them by their performance.
How can you judge the performance of a technique without implementing it? To implement and test the performance of an approach which accesses properties, one would have to ignore the requirement to avoid accessing properties.
no, u shouldnt disregard specific things uve identified as adding overhead
I wasn't telling you to disregard the observation that property access is slower than variable access or that creating objects has a cost. I was telling you that judging the performance of the solution based on whether it does those things is not valid. Every other part of the algorithm has a cost as well, and you will never know what the cost is if you exclude those possibilities from the beginning.
tldr:
It appears you are making generalizations based on a single synthetic test. Based on past experience and the relative simplicity of the methods being tested, I would say that these results could easily change when the methods are used in a different context, with different data or with minor changes. They appear to be close enough that I would accept all methods as being equal in regard to performance. Not that I was concerned about performance.
I wrote:__Enum might allow the code to be structured more cleanly and/or efficiently in some situations.
lexikos
Posts: 9679
Joined: 30 Sep 2013, 04:07
Contact:

Re: Variadic varrefs and __Enum arity arg?

11 Sep 2021, 06:39

swagfag wrote:
10 Sep 2021, 07:32
its some kind of a logical bug that fails to consider fat-arrow functions.
Keep in mind that a fat arrow function always returns the result of its expression; i.e.

Code: Select all

	innerFunc() {
		return %firstVarRef% := '123456789' ; broken
	}
The condition mActionType == ACT_EXPRESSION only applies to full function definitions, because the expression in a fat arrow function is always part of an ACT_RETURN line.

The issue is due to an optimization which assumes any non-alias local variable being returned from a function is about to be freed. Instead of making and returning a copy of the string, the original string is detached from the variable and returned directly. It must be detached because otherwise it would be freed before the caller reads it (and it could be freed twice).

Thanks for reducing the test case. It can also be reproduced as follows:

Code: Select all

f1
f1() {
    VarSetStrCapacity(&v, 64)  ; The optimization has this side-effect only when mHowAllocated == ALLOC_MALLOC.
    f2(&v)
    MsgBox v
}
f2(r) {
    return %r% := '123456789'  ; The result of this expression is v (actually, a VarRef for which the real v is an alias).
}
Returning a parameter declared with & will not cause this issue, because in that case the immediate result of the expression is the parameter (mType == VAR_ALIAS), not its target.

The local variable being freed need not necessarily belong to a different function, but it would be very unusual for a function to take a reference to its own local variable and then return its value via a double-deref.

Code: Select all

MsgBox 'Return value "' f(&r) '"`n'
    .  'Ref value "' %r% '"'  ; Blank unless 'return %rr%' is commented out.
f(&rr) {
    VarSetStrCapacity(&v, 64)
    v := '123456789'
    rr := &v
    return %rr%
}
return v would be unaffected because v itself is actually an alias for a VarRef (an implementation detail), and aliases are excluded from the optimization.

Global and static variables are unaffected because their value is returned without being detached (for correct behaviour and efficiency).

Return to “Ask for Help (v2)”

Who is online

Users browsing this forum: Google [Bot] and 41 guests