Passing Arrays ByRef to COM API

[Dulus]

I am writing a script to automate a third-party application using its COM API. I have fully working VBA code, and partially working AHK code. But I am stuck at the point when the API wants to return an array ByRef - I get "Error: 0x800706F4 - A null reference pointer was passed to the stub.". I have successfully used ComVar() to pass variables ByRef, so I assume the problem is with objects. I was hoping this recent thread would address my problem, but it says "ByRef is not supported". Full API documentation is here.

Basically, my question comes down to, How do I translate VBA:

Code: Select all

Dim NumberItems As Long
Dim ObjectType() As Long
Dim ObjectName() As String
ret = SapModel.SelectObj.GetSelected(NumberItems, ObjectType, ObjectName)

into AHK? This doesn't work:

Code: Select all

NumberItems := ComVar(4) ; long
ObjectType := ComVar(0x2004) ; array of long
ObjectName := ComVar(0x2008) ; array of strings
ret := SapModel.SelectObj.GetSelected(NumberItems.ref, ObjectType.ref, ObjectName.ref)

[Coco]

What are those arguments passed to ComVar? Are those meant to be var(COM) types or values for the variables? If it's the latter, assignment should be something like: NumberItems := ComVar(), NumberItems[] := 4

[Dulus]

Sorry, I forgot I was using the modified version of ComVar that lets me pass the Type as an argument (otherwise the API returns an Type Mismatch error). The API expects a empty array, fills it, and returns it ByRef.

The original ComVar is here: https://github.com/cocobelgica/AutoHotk ... ComVar.ahk
The one I am using is: http://ahkscript.org/docs/commands/ComObjActive.htm
The variable types are described here: http://ahkscript.org/docs/commands/ComObjType.htm

[Coco]

Can you try not passing the type and see if it works?:

Code: Select all

NumberItems := ComVar() ; long
ObjectType := ComVar() ; array of long
ObjectName := ComVar() ; array of strings
ret := SapModel.SelectObj.GetSelected(NumberItems.ref, ObjectType.ref, ObjectName.ref)

By the way, about 2004 and 2008, are these meant to be VT_ARRAY|VT_R4 and VT_ARRAY|VT_BSTR respectively? Shouldn't the values be 8196 and 8200. Disregard if I'm wrong..

[Dulus]

ComVar() is equivalent to ComVar(0xC), which returns a Type Mismatch error. I have successfully used ComVar(4), ComVar(0xB), etc when passing non-arrays ByRef to the API. These functions also return Type Mismatch errors if I use just ComVar(). That is why I think it is a problem with object handling.

Oops, 2004 and 2008 were typos for 0x2004 and 0x2008. The rest of my post holds true.

[Coco]

hmm, perhaps you need not use ComVar(). have you tried just passing normal variables?

[Dulus]

Yes. I just tried again, and got the Type Mismatch error again. I need at least ComObjParameter to pass the correct type. Maybe the problem is that ComObjArray (used in ComVar) ignores VT_ARRAY?

I just tried this, which gets past the error point above, but then crashes AHK:

Code: Select all

SapObject := ComObjActive("SAP2000v15.SapObject")
SapModel := SapObject.SapModel

NumberItems := ComObjArray(0x4, 1)
NumberItemsRef := ComObjParameter(0x4004, ComObjValue(NumberItems))
ObjectType := ComObjArray(0x4, 10)
ObjectTypeRef := ComObjParameter(0x6004, ComObjValue(ObjectType))
ObjectName := ComObjArray(0x8, 10)
ObjectNameRef := ComObjParameter(0x6008, ComObjValue(ObjectName))

SapModel.SelectObj.GetSelected(NumberItemsRef, ObjectTypeRef, ObjectNameRef)

msgbox % NumberItems
for k, v in ObjectName
	msgbox %k%, %v%

For comparison, this function works as expected:

Code: Select all

IsSelected(Name, Type="") {
	Type := T2Type(Type) ; this is SAP type (point, line), not variable type (boolean, long)
	selc := ComVar(0xB)
	error := 0
	selected := 0
	Loop, Parse, Type, `,
		error += SapModel[A_LoopField].GetSelected(Name, selc.ref)
		, selected |= -1*selc[]
	Return selected
}

but I can't practically loop through all elements to see if they are selected.

[lexikos]

Sorry, I forgot I was using the modified version of ComVar that lets me pass the Type as an argument

The original version allowed this.

The original ComVar is here: https://github.com/cocobelgica/AutoHotk ... ComVar.ahk

This is a modified version, made to work with AutoHotkey v1 and an obsolete v2 alpha. ComVar was originally distributed in the documentation. I believe the current version in the documentation is identical to the original except for "ComObject" vs "ComObjParameter".

Generally you will want to make a ComVar of VT_VARIANT, then store a value (such as an array) inside it.

Maybe the problem is that ComObjArray (used in ComVar) ignores VT_ARRAY?

I don't think it's valid to have an array of arrays, but you can have an array of VT_VARIANT, and each variant can contain an array.

NumberItems := ComObjArray(0x4, 1)
NumberItemsRef := ComObjParameter(0x4004, ComObjValue(NumberItems))

This is what you're doing:

Code: Select all

NumberItems := ComObjArray(0x4, 1)
NumberItemsRef := ComObjParameter(0x4004, 1)

For types with the VT_BYREF flag, you need to pass the address of a buffer, not the value itself.

You might like to try the VT_BYREF support in this test build. For VT_BYREF|VT_R4, I think something like this would work:

Code: Select all

VarSetCapacity(r4, 4, 0), ref := ComObject(0x4004, &r4), ref[] := initial_value
;...
value := ref[]

"ComObject" is the same as "ComObjParameter" (in v1; in v2 only the former works).

For the current AutoHotkey version, you could use NumPut(initial_value, r4, "float") and NumGet(r4, "float") (for VT_R4 specifically) instead of ref[].

However, I can't say whether the API you're using really wants VT_BYREF|VT_R4.

[Dulus]

Thanks for your time, and your clarifications. Back to my original question, I think know what the API wants (because it's stated in the documentation and because it works in VBA): an array of long (or string, boolean, whatever) passed by reference.

I did see your test build, which is why I finally posted this question, but I didn't find any documentation for ComObject, so my tests with it didn't work. I was definitely ignorant that 'For types with the VT_BYREF flag, you need to pass the address of a buffer, not the value itself.' So to follow your suggestion:

Code: Select all

; Create a 4bype variable named r4
VarSetCapacity(r4, 4, 0)
; Get a reference to this variable with a type of VT_BYREF|VT_R4
ref := ComObject(0x4004, &r4)
; Assign an initial value to the variable
; This is the bonus feature that only works in the test build
ref[] := initial_value

I don't see any use of arrays here. Am I missing something? My problem is with passing arrays, not variables byref. I have successfully used things such as VT_BYREF|VT_BOOL in AHK.

; First I test with a function that expects a non-array boolean by reference. Here "738" is the name of a PointObj that may or may not be selected. The API function does not expect an initial value, so I commented it out:

Code: Select all

VarSetCapacity(selc, 4, 0), selcRef := ComObject(0x400B, &selc) ;, selcRef[] := initial_value
SapModel.PointObj.GetSelected("738", selcRef)
msgbox % -selcRef[]

; This works as expected.

; Then I test with a function that expects arrays:

Code: Select all

VarSetCapacity(NumberItems, 4, 0), NumberItemsRef := ComObject(0x4004, &NumberItems) ;, NumberItemsRef[] := initial_value
VarSetCapacity(ObjectType, 4, 0), ObjectTypeRef := ComObject(0x4004, &ObjectType) ;, ObjectTypeRef[] := initial_value
VarSetCapacity(ObjectName, 4, 0), ObjectNameRef := ComObject(0x4008, &ObjectName) ;, ObjectNameRef[] := initial_value

SapModel.SelectObj.GetSelected(NumberItemsRef, ObjectTypeRef, ObjectNameRef)

NumberItemsValue := NumberItemsRef[]
ObjectTypeValue := ObjectTypeRef[]
ObjectNameValue := ObjectNameRef[]

msgbox % NumberItemsValue
for k, v in ObjectTypeValue
	msgbox %k%, %v%

; This does not work.

; This is what I originally had working (the only code in this post not requiring your test build):

Code: Select all

	arr := ComObjArray(0xB, 1)
	DllCall("oleaut32\SafeArrayAccessData", "ptr", ComObjValue(arr), "ptr*", arr_data)
	ref := ComObjParameter(0x4000|0xB, arr_data)
	SapModel.PointObj.GetSelected("738", ref)
	Msgbox % -arr[0]

; Interestingly, your suggestion also works without VarSetCapacity:

Code: Select all

selcRef := ComObject(0x400B, &selc) ;, selcRef[] := initial_value
SapModel.PointObj.GetSelected("738", selcRef)
msgbox % -selcRef[]

; Maybe if I set it to an array instead of a variable?

Code: Select all

selc := ComObjArray(0xB, 1), selcRef := ComObject(0x400B, &selc) ;, selcRef[] := initial_value
SapModel.PointObj.GetSelected("738", selcRef)
msgbox % -selcRef[]

; That worked as well. Back to the array API function:

Code: Select all

NumberItems := "", NumberItemsRef := ComObject(0x4004, &NumberItems) ;, NumberItemsRef[] := initial_value 
ObjectType := ComObjArray(0x4, 1), ObjectTypeRef := ComObject(0x4004, &ObjectType) ;, ObjectTypeRef[] := initial_value
ObjectName := ComObjArray(0x8, 1), ObjectNameRef := ComObject(0x4008, &ObjectName) ;, ObjectNameRef[] := initial_value
msgbox ok
SapModel.SelectObj.GetSelected(NumberItemsRef, ObjectTypeRef, ObjectNameRef)
msgbox crashes ahk before getting here

NumberItemsValue := NumberItemsRef[]
ObjectTypeValue := ObjectTypeRef[]
ObjectNameValue := ObjectNameRef[]

msgbox % NumberItemsValue
for k, v in ObjectTypeValue
	msgbox %k%, %v%

I tried a couple variations on the parameters is this last setup and they all crashed AHK.

[lexikos]

I didn't find any documentation for ComObject

See "ComObj...()" in the help file. As I said, "ComObject" is the same as "ComObjParameter" in v1, but "ComObject" also works in the current v2 alpha.

I don't see any use of arrays here.

That's because there isn't any.

NumberItems := ComObjArray(0x4, 1)
NumberItemsRef := ComObjParameter(0x4004, ComObjValue(NumberItems))

This is what you're doing:

I was wrong. 0x4004 is a combination of VT_BYREF and VT_R4; i.e. a variable, not an array. My mind reconciled this by reinterpreting ComObjArray(0x4, 1) as ComObject(0x4, 1), which is definitely not equivalent.

The actual error is that you are treating a SAFEARRAY pointer as the address of a VT_R4 variable, because you're missing the VT_ARRAY flag; VT_BYREF | VT_ARRAY | VT_R4 = 0x6004.

; Interestingly, your suggestion also works without VarSetCapacity:

What does selc contain? If it hasn't been initialized, don't do that. Uninitialized variables contain a pointer to a shared empty string, which you would be overwriting via COM. Writing a true (non-zero) value would make the program unstable, like this example:

Code: Select all

f := A_IsUnicode ? "wcscpy" : "strcpy"
DllCall("msvcrt\" f, "str", target, "str", "!")
; "Error:  This DllCall requires a prior VarSetCapacity. The program is now unstable and will exit."

DllCall actually detects the condition even if it was caused by prior code. For example:

Code: Select all

selcRef := ComObject(0x400B, &selc)
NumPut(-1, ComObjValue(selcRef), "short")  ; Simulate a COM API using ByRef to store 'true' in selc.
DllCall("MulDiv", "int", 1, "int", 1, "int", 1)
; "Error:  This DllCall requires a prior VarSetCapacity. The program is now unstable and will exit."

Code: Select all

selc := ComObjArray(0xB, 1), selcRef := ComObject(0x400B, &selc)

I can't see how that would ever work. &selc returns the address of the ComObject (wrapper object). If someone writes to selcRef, they will overwrite the first two bytes of the wrapper object's virtual function table pointer. I'd be surprised if even -selcRef[] worked afterward, but if it did, it would probably just produce the default value for VT_BOOL.

To pass an array by value, you would just pass selc itself. I think a reference to an array would be created like this:

Code: Select all

arr := ComObjArray(element_type, length)
VarSetCapacity(parr, A_PtrSize), NumPut(ComObjValue(arr), parr)
arr_ref := ComObject(0x6000 | element_type, &parr)

[Dulus]

Thank you, that worked! I used 1 for all lengths, and the API expanded the array as required. And by changing 0x6000 to 0x4000, it worked for non-arrays as well.

For my understanding, why did the ComVar function not require the address of the array?

Code: Select all

arr := ComObjArray(element_type, length)
DllCall("oleaut32\SafeArrayAccessData", "ptr", ComObjValue(arr), "ptr*", parr)
arr_ref := ComObject(0x4000|element_type, parr)

I tried creating a similar function that could return reference to COM objects, including arrays. It didn't work, but I'm not going to worry about it.

Code: Select all

APIFuctionExpectingArray(ComRef(ObjectType := ComObjArray(0xB, 1) , 0))

ComRef(ComObj, noarray=1) { ; set noarray to 0 for arrays
	VarSetCapacity(ptr, A_PtrSize), NumPut(ComObjValue(ComObj), ptr)
	Return ComObject(0x4000|(ComObjType(ComObj)-noarray*0x2000), &ptr)
}

[lexikos]

0x4000|element_type means a pointer to the type of data indicated by element_type. Why would it require the address of an array?

In C, an array is just a series of data structures arranged contiguously in memory. parr points to the first element of such an array. Script-compatible COM APIs expect a SAFEARRAY structure, which describes the length, type and other characteristics of the data, and is what ComObjValue(ComObjArray(...)) returns a pointer to.

Your ComRef function can't work because ptr is a local variable. Local variables are freed when the function returns.

[Dulus]

I just meant, why doesn't ComObject() require an & in ComVar(), when it does require one in your working solution.

Still trying to replace ComVar:

Code: Select all

ComVar(Type=0xC)
{
    static base := { __Get: "ComVarGet", __Set: "ComVarSet", __Delete: "ComVarDel" }
    ; Create an array of type Type.
    arr := ComObjArray(Type & 0x3F, 1)
    ; Create a pointer to the array.
    VarSetCapacity(ptr, A_PtrSize), NumPut(ComObjValue(arr), ptr)
    ; Store the array and an object which can be used to pass the VARIANT ByRef.
    return { ref: ComObjParameter(0x4000|Type, &ptr), ptr: ptr, _: arr, base: base }
}
ComVarGet(cv, p=0) { ; Called when script accesses an unknown field.
    if p is integer
        return cv._[p]
}
ComVarSet(cv, v, p=0) { ; Called when script sets an unknown field.
    if p is integer
        return cv._[p] := v
}
ComVarDel(cv) { ; Called when the object is being freed.
    return cv.ptr := ""
}

Returns 'The parameter is incorrect.'

[lexikos]

Like I said, 0x4000|Type means a pointer (i.e. reference) to a value. You are passing a pointer (i.e. reference) to a variable containing an array, which would only be valid if Type is a combination of 0x2000 (VT_ARRAY) and some other type.

ComObject() accepts a type code and value. What kind of value depends entirely on the type code. &var would only be used if var contains the value and you are passing the value by reference. ComVar() doesn't use &address-of because the value is not stored in a variable, but in an array. The array allows the script to retrieve the value without having to know how - AutoHotkey can do it automatically based on information contained by the array.

[Dulus]

Great! Thanks for all your explanations.