Byref Array elements

[braunbaer]

I just stumbled over a very painful limitation of AHK:

Known limitations:
It is not possible to pass properties of objects (such as foo.bar), Clipboard or other built-in variables to a function by reference. Instead, the function acts as though ByRef was omitted.

Is there a plan to remove this limitation? Not necessarily for the clipboard and for built-in variables, but at least for array elements and object properties.

[lexikos]

Yes and no.

No:

Given the phrase "ByRef was omitted" is present in your quote, I assume you are using AutoHotkey v1, for which I have no plan to remove any limitations or add any new features.

Variables and properties are distinctly different. In the expression f(v), the variable reference v directly corresponds to a Var structure, which is the actual C++ object which stores the value, type information, name and other attributes of a script variable. ByRef works because the function receives the address of this structure, not the value contained by the variable.

By contrast, foo.bar is more like a function call. The property is called and it returns a value, so that value is all the function gets. The property might not actually exist; maybe it is implemented by a property getter/setter or __get/__set, or maybe foo is a COM object and its type and implementation are unknown (could even be in a different process).

What can you do with a reference to a variable? Basically, retrieve its value or assign it a new value. So if we create an object which supports those operations, in theory it could be used in place of an actual variable reference. A reference to a property, in the most abstract case, can be just an object combining the target object and property name, such as:

Code: Select all

#requires AutoHotkey v1.1
class PropertyReference {
    value {
        get {
            return this.target[property]
        }
        set {
            return this.target[property] := value
        }
    }
    __new(target, property) {
        this.target := target
        this.property := property
    }
}

But in reality, it's not so simple to substitute an object for a Var - it would take a lot of work to permit this, especially with the v1 code base. Var does more than just accept a value for assignment. For instance, sometimes the code does the equivalent of VarSetCapacity and then writes into the variable directly, for efficiency.

And there are bigger problems.

For foo.bar to be passed by reference, the "compiler" needs to produce code to create a reference instead of evaluating (calling) the property. Consider these hypothetical options:

1. When used in a ByRef parameter, produce code to create a reference; otherwise produce code to evaluate the property.
2. Always produce code to create a reference and determine at runtime whether to evaluate the reference or pass it.

Option 1 requires the compiler to be able to determine when the value is being passed by reference. For a simple function call like f(foo.bar), the compiler knows what the parameters of f are, and can choose whether to evaluate or reference the property. But since you're wanting to pass a property of an object to a ByRef parameter, I assume at some point you'll want to pass it to a ByRef parameter of a method. How do you know whether %f%(foo.bar) or my.method(foo.bar) should take a reference? It is generally not possible to determine until each call is made. It can vary as the target of the call might not always be the same function.

Option 2 increases complexity and overhead. Whether a parameter is ByRef can only be determined at the last possible moment before the function's body is executed, after the object is invoked and the method lookup completed. When it is determined that a reference wasn't needed after all, it has to be evaluated. That means the parameter conversion code needs to be capable of executing script to dereference the reference. It also needs to manage any memory or object that was allocated, to be freed after the function is done with it. This would be a lot of work for very little gain.


Yes:

v2.0-a128 and later have a significantly different kind of ByRef. Instead of passing a naked variable and having the function maybe, implicitly take the variable by reference instead of its value, the caller must create a variable reference (VarRef) explicitly with &var and pass it to the function.

&var is identified at load time as making a reference to var, not evaluating it. It doesn't matter whether a parameter can be identified as ByRef before the function or method is called, because even if it's not ByRef, x.y(&var) will still pass the reference, not the value of var. What the function does with the reference is up to the function. &foo.bar is currently an error (& requires a variable), but it could be extended to construct an object representing a reference to the property.

Even without syntax support, a method such as foo.ref("bar") could in theory construct an object representing a reference to the property, and in reality it can construct a VarRef. For example:

Code: Select all

#Requires AutoHotkey v2.0-a132 ;(might work on earlier builds if you remove this line)

foo := Reffer()
foo.bar := "baz"  ; A normal property, so far.
MsgBox foo.bar    ; baz
f(foo.ref('bar')) ; Pass foo.bar by reference.
MsgBox foo.bar    ; BAZ

f(&a) {
    a := StrUpper(a)
}

class Reffer {
    ref(name) {
        desc := this.GetOwnPropDesc(name)
        if desc.HasProp('value')
            this.DefineProp(name, make_ref(desc))
        else if !desc.HasProp('get') || !desc.get.HasProp('ref')
            throw Error("Invalid property for ref", -1, name)
        return desc.get.ref
        make_ref(desc) {
            local v := desc.DeleteProp('value')
            desc.get := (this)        => v
            desc.set := (this, value) => v := value
            desc.get.ref := &v  ; Attach the VarRef to the property getter.
            return desc
        }
    }
}

In the example above, the property is redefined so that it actually corresponds to a real script variable; the same one referenced by .ref (and a in f()). However, there's currently no way for the assignment in f() to trigger a property setter, for instance. Accessing a ByRef parameter can only store or retrieve a value from a real script variable.

The plan is for VarRef to be replaced with a more generalized mechanism (and then make the & operator utilize it), such as an object with a specific property; maybe ref[], since COM object wrappers with the VT_BYREF flag already use this syntax to retrieve or set values. A fair bit more work is needed before this can become a reality, as there is still a fair amount of code that expects to deal with real script variables (Var with a capital V), and might use it in ways other than directly assigning a preexisting value (such as setting the variable's capacity and using it as a buffer to write a string).

[braunbaer]

Thank you for these thorough explanations.
While I was aware it would not be so easy in non typed language as AHK, I was not aware how complicated it really would be

I think it's a problem that the only type of array AHK knows is the associative array. In many applications you just need a linear array, but as there exists nothing similar to a type definition, the gain would not be so big.

[lexikos]

AutoHotkey v2 has the Array type, which is a linear array.

AutoHotkey v1's object has the same functionality, and iirc performs nearly as well, though lacks some interface clarity due to mixing various concepts.

In any case, I don't see how the type of array is a problem in the context of this topic.

[braunbaer]

In any case, I don't see how the type of array is a problem in the context of this topic.

You are right, directly in the contet of the original topic, the array type is irrelevant. Typed languages in general allow a more efficient implementation.