jeeswg's benchmark tests

[jeeswg]

@boiler: The issue is closed, I've put a moratorium on discussing this issue, within this thread, until the 17th of March. The original suggestion to move the thread was done in a way that 'lacked neutrality', and I cannot accept having to operate under these conditions, of insult, threats to modify forum posts, and then potential badgering by other people, the derailment of the thread, and pointless arguments. If someone is desperate to discuss it, they can send me a direct message. I respect your point of view, but I do not want this thread to be about anything other than benchmark tests for the time being. Ta dah! Let's resume normality in this thread.

[boiler]

@jeeswg: You are making it more dramatic than what it is: a simple and civil discussion on where this thread belongs. You may want to consider it closed, but it is still located in Tutorials and this is an open forum thread. After it is moved or decided that it stays here, the mods could delete any related posts if they would like.

I'm not PMing you because having a private discussion doesn't serve the purpose of determining where this thread belongs. This is the appropriate place for this discussion.

[jeeswg]

It's not so much the original suggestion post that is the problem, it's that it's part of a package, a trend, the cause of which I know not. I don't view the suggestion post as innocent, or done in good faith, or as having anything to do with moving the thread, but as part of serving some curious goal. On principle, if someone goes about trying to get things done in that manner, they should not be allowed to succeed. My simple solution, let the debate happen at a future point, if people want it. In my view we have a situation where someone latches on to a cause, an argument, that they think they can win, when they have no interest or care in the matter, and combine making arguments with making personal attacks wherever possible. I have tried to build bridges behind the scenes. With all kindness, I would suggest people not to challenge me in this way. Thanks for your comments boiler. [EDIT:] I would advise anyone considering posting here to only post about matters relevant to benchmark tests.

[jeeswg]

Someone I correspond with, asked me about Helgef's proposal.
Btw we had joked about q:: on at least one previous occasion, Helgef and I.

Code: Select all

;jeeswg original
q::
DllCall("QueryPerformanceFrequency", Int64P,vQPF)
DllCall("QueryPerformanceCounter", Int64P,vQPC1)
Sleep, 1000
DllCall("QueryPerformanceCounter", Int64P,vQPC2)
MsgBox, % Clipboard := (((vQPC2-vQPC1)/vQPF)*1000)
return

;jeeswg recommendation to Helgef if he doesn't like the use of q:: or QueryPerformanceFrequency
;add semicolons to the top 2 lines:
;note: including a hotkey label is to aid newbies, to show where a hotkey label would go, and to prevent a script from executing without the user having explicitly asked it to
;q::
;DllCall("QueryPerformanceFrequency", Int64P,vQPF)
DllCall("QueryPerformanceCounter", Int64P,vQPC1)
Sleep, 1000
DllCall("QueryPerformanceCounter", Int64P,vQPC2)
MsgBox, % Clipboard := (((vQPC2-vQPC1)/vQPF)*1000)
return

;perhaps the other comments would result in this:
;which I don't believe adds much clarity, plus it introduces a dependency, which can easily be omitted by accident, or which can cause duplicate function definition errors
;q::
;DllCall("QueryPerformanceFrequency", Int64P,vQPF)
vQPC1 := QueryPerformanceCounter()
Sleep, 1000
vQPC2 := QueryPerformanceCounter()
Clipboard := (((vQPC2-vQPC1)/vQPF)*1000)
MsgBox, % Clipboard
return

QueryPerformanceCounter()
{
	DllCall("QueryPerformanceCounter", Int64P,vQPC)
	return vQPC
}

;note:
;I could change this:
MsgBox, % Clipboard := (((vQPC2-vQPC1)/vQPF)*1000)
;to this:
MsgBox, % Clipboard := ((vQPC2-vQPC1)/vQPF)*1000
;which looks clearer,
;however, if you add strings to either side of the calculations, it's clearer if you use parentheses to mark the beginning/end of the calculations
MsgBox, % Clipboard := vText1 (((vQPC2-vQPC1)/vQPF)*1000) vText2

In short, I already made numerous considerations re. my code and how to present it. If someone makes a suggestion to improve the code, and I say no, or I don't think so, it doesn't mean I'm lazy, or I don't care about the code. Within the calculus of my considerations, I already made the best choices for reasons I can justify. That said, sometimes I do agree with suggestions and change my code accordingly. Helgef is welcome to outline his vision for the script.

[derz00]

I don't know how this "QueryPerformace..." stuff works. But I think what Helgef meant was

Code: Select all

DllCall("QueryPerformanceFrequency", Int64P,vQPF)
q:: ; except for the minor annoyance of needing to press q

rather than Having QPF in the hotkey subroutine.

This looks prettier to me...

Code: Select all

; "ASSUMED CODE" here

DllCall("QueryPerformanceFrequency", Int64P,vQPF)
vQPC1 := QPC()
Sleep, 1000
Clipboard := (vQPC2 - QPC()) / vQPF * 1000
MsgBox, % Clipboard
return

QPC() {
	DllCall("QueryPerformanceCounter", Int64P,vQPC)
	return vQPC
}

[jeeswg]

- Thanks for the example, derz00.
- I've added the QueryPerformanceFrequency line, to the 'ASSUME ALWAYS' code in my OP.
- Btw I usually like to include the dll name in DllCall lines, but for clarity, I omitted it in these examples. The things I do for my users. The hundreds of little considerations that I consider. And do I get any appreciation for all that effort?

Code: Select all

DllCall("QueryPerformanceFrequency", Int64P,vQPF)
DllCall("kernel32\QueryPerformanceFrequency", Int64P,vQPF)

- Btw having the QueryPerformanceFrequency line within the examples (where it can be commented out or not copied), and not within the 'ASSUME ALWAYS' code, was again done for a good reason, you can get good results with my examples, even if you completely omit the 'ASSUME ALWAYS' code. The subroutines work by themselves, without having to put stuff in the auto-execute section or having to include functions.
- Hmm, the one change I could, in theory, make. (Other than putting the QPF function in the 'ASSUME ALWAYS' code, which I've already done.) #SeeIDoListen.

Code: Select all

MsgBox, % Clipboard := ((vQPC2-vQPC1) / vQPF * 1000)
MsgBox, % Clipboard := (vQPC2-vQPC1) / vQPF * 1000

- One of the reasons for using separate variables, and not using a function within the calculation, is that in some examples, when I do scripts, I store all of the times in variables or arrays, and present them only at the end. See. So much thought, put, in.
- Btw for some tasks, e.g. this, regular forum threads don't lend themselves well to what I want to do. Perhaps it would be better if I could have some mod-like/admin-like privileges for a few threads, e.g. to have a locked thread, with one post per example, and a separate discussion thread, that I link to in the OP. Otherwise everything gets cluttered with (interesting and important, but) random posts.

[nnnik]

Well if he says that it is WIP then it can stay. Also if it is related to another tutorial of his I don't want to be too strict.
We would have to move a lot of posts if we would react like this towards every topic.
I just want you to keep in mind to at least fully describe the test cases and the results you are sharing here - generally speaking I feel that any tutorial should be understandeable without having to run any code.

[guest3456]

It's not so much the original suggestion post that is the problem, it's that it's part of a package, a trend, the cause of which I know not. I don't view the suggestion post as innocent, or done in good faith, or as having anything to do with moving the thread, but as part of serving some curious goal.

"The cause of which you know not."

The 'cause' has been explained to you numerous times, by numerous different people. You cannot claim any "curious goal" when there are multiple unbiased posters all saying the same thing: your posts are rambling, they have no stated intent, and half the time they make no sense and are just bullet points of unrelated topics. You start many threads and they all look the same. Everyone tells you to clean things up and clarify your thoughts, and you take it as an attack. You get banned from stackoverflow and you wonder why. Then you claim that stackoverflow are just too strict. And now you claim that the AHK forums are strict. No pal. It's all you. That's the common denominator.

You have useful stuff to contribute. If you would just listen to the input instead of fighting it, and actually change your ways, everything would be fine.

[jeeswg]

- Thank you for your posts.
- @nnnik:

I feel that any tutorial should be understandable without having to run any code.

That's a great point, I'll add that to my Thoughts on AutoHotkey thread.
- @guest3456: I found this critical message more interesting to read than other critical messages that you have written, because you have added more detail than usual.
- [EDIT:] I had some thoughts for nnnik and guest3456, which I might restore in some form, but I have since had some new information sent to me, so we'll wait and see.

[Helgef]

The point is I'm trying to keep everything as uncluttered and stand-alone as possible, by keeping everything self-contained and avoiding the use of functions

We obviously have different views, I respect yours, you should have it as you please of course.

And q:: is cool, very cool.

q:: ? :

---

Now, back to topic. TRUNCATE STRING. That is a useful function JEE_StrTruncate, it has its applications, but your test doesn't showcase it. I think it is important to note what is going on, the two test snippets do different things, and it is not really meaningful to compare them the way the test does. A quick explaination of the tests,
Test 1, substr

• VarSetCapacity(out, x, 1), allocates and fill memory on each iteration.
• out := substr(out, ...), copies the specified substring to a new buffer and discards the old. Consequently, on the next iteration, a new buffer of size x needs to be allocated and filled with 1....

Test 2, JEE_StrTruncate

• VarSetCapacity(out, x, 1), allocates and fill memory on the first iteration. On subsequent iterations, it only fills the memory, because, VarSetCapacity: for performance reasons, it shrinks the variable only when RequestedCapacity is 0. and the memory location being operated upon is not changing in this test.
• JEE_StrTruncate(byref out, ...),
• NumPut(0, out), Zero terminates out at the specified position, that is, writes 0 to two bytes.
• VarSetCapacity(out, -1), updates the internally stored string length of out, the length is found by searching from the start of the buffer until the zero from the numput above is found.

In summary, the first method truncates the string by copying the substring to a new buffer. The second method only null terminates and updates the string length. You should use the first method when you do not want to append to the string, and the second when you do want to append to the string, since the second test preserves the allocated memory, making appending to the string faster. The second method can also be used when you just need the truncated string temporarily, and memory isn't an issue.

To make the test more meaningful,

Code: Select all

JEE_StrTruncate(vOutput, vLen-1000), vOutput := ""

This way, you will see the difference between out:=substr(out) and JEE_StrTruncate.

Hopefully, in v2, we will be able to specify the new length of the string without searching.

---

For similar reasons, your varsetcapacity loop test is not meaningful.

For the byte swap tests, all the varsetcapacity doesn't need to be in the loop, in the function from which the code comes, the src and dest are static variables.

Cheers.

[jeeswg]

- @Helgef: Your comments are interesting, and I was aware of potential VarSetCapacity issues, and was trying to handle them, but it is very very fiddly. Even reading your posts multiple times, it's still hard to work out how exactly to implement it all, and to be sure that it's completely correct.
- If you have the time at some point, I would recommend that you post revised versions of those VarSetCapacity benchmark tests. Also, I would welcome seeing your vision for the basic Sleep benchmark test example that I provided. Cheers.

[Helgef]

it is very very fiddly.

More likely, my description is poor . I might consider your comments later.

Cheers.

[jeeswg]

Haha no, you've done a good job, it's one of those things that is just really fiddly to describe, and that I'd illustrate with an example in a tutorial. Although I could try and fix it myself, I'd then want you to double-check that I'd done it right. Hence it seems logical enough for you to fix what I've already done, versus fix my attempt at a fix. Got your fix of fixes? Anyhow, I will do some code tests myself. Btw q:: is qool.

[Helgef]

Although I could try and fix it myself

I'm not sure I follow, specifically what do you want to fix?

bbl

[jeeswg]

You said that there were some issues with one/two of my benchmark tests, I would like to rewrite the code and post it. However, I was suggesting you might like to rewrite the code and post it, as well as your perfect dream version of how the Sleep benchmark test should look. If that's alright. Cheers.

[Helgef]

Well, for the truncate example, I think clearing the string in the second loop suffices, that is JEE_StrTruncate(vOutput, vLen-1000), vOutput := "", then VarSetCapacity(vOut...) will do the same amount of work in both loops. Alternatively, you can remove varsetcap from both loops and do vOutput2 := subStr(vOutput,...) in the first loop, and no vOutput := "" in the second. Also note, if the size of a variable is less than 4092 bytes, you should use varsetcapacity(var, 0) instead of var := ""

For the varsetcapacity loop, that is

Code: Select all

Loop, % vNum
	VarSetCapacity(RECT, 16, 0)

you could do

Code: Select all

Loop, % vNum
	RECT%A_Index% := 0
Dllcall(qpc)
Loop, % vNum
	VarSetCapacity(RECT%A_Index%, 16, 0)
Dllcall(qpc)	

(I Guess)
Also,

Code: Select all

VarSetCapacity(vData1, 4)
, VarSetCapacity(vData2, 4)

could be moved outside of the loop in the byte swap test, it is not very significant. And numget/put(&var) is slower than numget/put(var), I believe, you could bench it

---

I do not have a dream, I wouldn't have said anything if it was done the way derz00 suggested though, but imo, you could move everything, including the msgbox into a function, eg,

Code: Select all

q() ; start measurement
;test code
q() ; end and show result.

where q() would have the frequency and the start-tic, as static variables. That would un-clutter the code.

Cheers.

[jeeswg]

- Thanks for your comments Helgef, I will reinvestigate the truncate string benchmark tests, and provide an updated version as a new post. You could do likewise if you wanted to. I was looking forward to seeing your truncate and sleep benchmark tests.
- Btw are you in agreement re. the overall result, that the StrTruncate function can be much faster than other methods? And that also it could be a useful function for general use?
- I had wondered if there could be some sneaky way to crop the start of a variable, by resetting where the variable points to. Or to create a new variable specifying the start of the string.

[Helgef]

It is faster, but does something else than substr, it is useful in special cases, not in general. As you said, it can only cut from the beginning of the string.

Cheers.

[nnnik]

Results from comparing the speed of several ways of calling a function with a long string
The String is inputted in several different ways and several different functions are called.
The functions return the first character of the string.
The string consists of 4194304 As.
In order to get more reliable results the call to each function is repeated 1000 times.
String in Variable - function with byRef: 1.516004 µs
String as the result of an expression - function with byRef: 9274.337790 µs
String from an object - function with byRef: 10869.383398 µs
String from a variable, variable from an object - function with byRef: 5527.088376 µs
String from a variable, variable from an object - function with byRef: 7136.554132 µs
Object containing the string - function accessing the object: 7349.535131 µs
Object containing the string - function accessing the object using an pointer to the string: 1.187566 µs

Code: Select all

#NoEnv
SetBatchLines, -1
DllCall( "QueryPerformanceFrequency", Int64P ,vQPF )

OutPut := "Results from comparing the speed of several ways of calling a function with a long string`nThe String is inputted in several different ways and several different functions are called.`nThe functions return the first character of the string.`n"

String := "A"
StrLenExp := 22
Loop %StrLenExp%
	String .= String
OutPut .= "The string consists of " . 2**StrLenExp . " As.`n"

testTimes := 1000
OutPut .= "In order to get more reliable results the call to each function is repeated " . testTimes . " times.`n"


DllCall( "QueryPerformanceCounter", Int64P, vQPC1)
Loop % testTimes
	fn( String )
DllCall( "QueryPerformanceCounter", Int64P, vQPC2 )
OutPut .= "String in Variable - function with byRef: " ( vQPC2 - vQPC1 )*1000/vQPF " µs`n"

DllCall( "QueryPerformanceCounter", Int64P, vQPC1 )
Loop % testTimes
	fn( (String . "" ) )
DllCall( "QueryPerformanceCounter", Int64P, vQPC2 )
OutPut .= "String as the result of an expression - function with byRef: " ( vQPC2 - vQPC1 )*1000/vQPF " µs`n"

obj := [String]              ;emulate an object that contains the string
obj.2 := obj.GetAddress( 1 ) ;it's like a small string class that already has the pointer to the string stored

DllCall( "QueryPerformanceCounter", Int64P, vQPC1 )
Loop % testTimes
	fn( obj.1 )
DllCall( "QueryPerformanceCounter", Int64P, vQPC2 )
OutPut .= "String from an object - function with byRef: " ( vQPC2 - vQPC1 )*1000/vQPF " µs`n"

DllCall( "QueryPerformanceCounter", Int64P, vQPC1 )
Loop % testTimes {
	test := obj.1
	fn( test )
	VarSetCapacity( test, 0 )
}
DllCall( "QueryPerformanceCounter", Int64P, vQPC2 )
OutPut .= "String from a variable, variable from an object - function with byRef: " ( vQPC2 - vQPC1 )*1000/vQPF " µs`n"

DllCall( "QueryPerformanceCounter", Int64P, vQPC1 )
Loop % testTimes
	fn2( obj )
DllCall( "QueryPerformanceCounter", Int64P, vQPC2 )
OutPut .= "String from a variable, variable from an object - function with byRef: " ( vQPC2 - vQPC1 )*1000/vQPF " µs`n"

DllCall( "QueryPerformanceCounter", Int64P, vQPC1 )
Loop % testTimes
	fn3( obj )
DllCall( "QueryPerformanceCounter", Int64P, vQPC2 )
OutPut .= "Object containing the string - function accessing the object: " ( vQPC2 - vQPC1 )*1000/vQPF " µs`n"

DllCall( "QueryPerformanceCounter", Int64P, vQPC1 )
Loop % testTimes
	fn4( obj )
DllCall( "QueryPerformanceCounter", Int64P, vQPC2 )
OutPut .= "Object containing the string - function accessing the object using an pointer to the string: " ( vQPC2 - vQPC1 )*1000/vQPF " µs`n"

Msgbox % ClipBoard := OutPut 


fn( byref var ) {
	return subStr( var, 1, 1 )
}
fn2( obj ) {
	return subStr( obj.1, 1, 1 )
}
fn3( byref obj ) {
	return subStr( obj.1, 1, 1 )
}
fn4( obj ) {
	return chr( NumGet( obj.2, "UShort" ) )
}

[jeeswg]

- I do benchmark tests to *save* time. You're welcome to convert them all to that style, combining them in one post. But seriously though, well done, that's a great example. Thanks nnnik.
- More and more though, I'm thinking that the examples should either be in separate posts, or in one massive post. So I might need a new thread. I'm thinking about it. I.e. to separate the tests and the discussions. Cheers.