AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Advanced Benchmark

 
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Scripts & Functions
View previous topic :: View next topic  
Author Message
Titan



Joined: 11 Aug 2004
Posts: 5009
Location: imaginationland

PostPosted: Sun May 13, 2007 4:44 pm    Post subject: Advanced Benchmark Reply with quote

This function times the running length of each code snippet with very high precision. It more advanced than the standard benchmark as it calculates deviation and ranks code on performance.

qpc2(Code|Source [, "pre"|"inc"])
Code: AutoHotkey code, if this is a file its contents will be read and used
Options: use "pre" to include the code before the test (i.e. initializing variables) or "inc" will save the code at the end of the script (i.e. for deleting temporary files). Leave blank to test the code

result := qpc2([Iterations])
Output in TSV format


Example:
Code:
qpc2("n := 2.5", "pre") ; set variable
qpc2("x := Floor(n)") ; using floor function
qpc2("y := n // 1") ; integer divide
qpc2("z := SubStr(n, 1, InStr(n, ""."") - 1)") ; modify string
MsgBox, % qpc2()

Which gives...
Code:
Test   Time   Deviation   Rank
1      91      -0.02      74%
2      216      0.10      37%
3      32      -0.08      91%

Iterations          14662
Standard deviation   0.09


Download
_________________

RegExReplace("irc.freenode.net/autohotkey", "^(?=(.(?=[\0-r\[]*((?<=\.).))))(?:[c-\x73]{2,8}(\S))+((2)|\b[^\2-]){2}\D++$", "$u3$1$3$4$2")


Last edited by Titan on Sun May 13, 2007 5:02 pm; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website
Laszlo



Joined: 14 Feb 2005
Posts: 3942
Location: Pittsburgh

PostPosted: Sun May 13, 2007 4:55 pm    Post subject: Reply with quote

I'd give it a try, if you posted a link to download…
Back to top
View user's profile Send private message
Titan



Joined: 11 Aug 2004
Posts: 5009
Location: imaginationland

PostPosted: Sun May 13, 2007 5:03 pm    Post subject: Reply with quote

lol thanks.
_________________

RegExReplace("irc.freenode.net/autohotkey", "^(?=(.(?=[\0-r\[]*((?<=\.).))))(?:[c-\x73]{2,8}(\S))+((2)|\b[^\2-]){2}\D++$", "$u3$1$3$4$2")
Back to top
View user's profile Send private message Visit poster's website
Laszlo



Joined: 14 Feb 2005
Posts: 3942
Location: Pittsburgh

PostPosted: Sun May 13, 2007 5:16 pm    Post subject: Reply with quote

It works, fine, but could you give a little more info:
- how does it work, what does it do?
- how many iterations does it perform and why that many?
- what is the Rank?
Back to top
View user's profile Send private message
Titan



Joined: 11 Aug 2004
Posts: 5009
Location: imaginationland

PostPosted: Sun May 13, 2007 6:02 pm    Post subject: Reply with quote

Like the previous version it repeats each given code to get the average length of time they take to complete. The number is calculated with the systems performance frequency, such that the integral of resulting execution times is loosely normalized with 1.8 seconds. Since this is random, depending on CPU load and thread interruption, an efficiency ratio (rank) is calculated to express deviation as a percentage. Standard deviation just tells you the range of variation; a value of <0.01 means the methods are almost identical performance wise and any could be used, but for anything greater than 0.03 you should use the method with the highest rank to increase the efficiency of your script.
_________________

RegExReplace("irc.freenode.net/autohotkey", "^(?=(.(?=[\0-r\[]*((?<=\.).))))(?:[c-\x73]{2,8}(\S))+((2)|\b[^\2-]){2}\D++$", "$u3$1$3$4$2")
Back to top
View user's profile Send private message Visit poster's website
Titan



Joined: 11 Aug 2004
Posts: 5009
Location: imaginationland

PostPosted: Fri May 18, 2007 11:01 pm    Post subject: Reply with quote

In the latest update I fixed an error that caused some information to be deleted from the results.

Here is another example which I used to test my XPath function:

Code:
qpc2("XPath.ahk", "inc") ; include XPath functions
qpc2("FileAppend, <test><var>val</var></test>, test.xml", "pre") ; create temporary XML file
qpc2("IniWrite, val, test.ini, test, var", "pre") ; create temporary INI file
qpc2("xr1 := XmlDoc(""test.xml"")", "pre") ; method 1: test preloaded XML document
qpc2("x1 := XPath(xr1, ""/test/var"")")
qpc2("x2 := XPath(xr2 := XmlDoc(""test.xml""), ""/test/var"")") ; method 2: test loading and parsing
qpc2("IniRead, x3, test.ini, test, var") ; method 3: test INI reading
qpc2("FileDelete, test.xml", "inc") ; delete temporary files...
qpc2("FileDelete, test.ini", "inc")
MsgBox, % qpc2()

_________________

RegExReplace("irc.freenode.net/autohotkey", "^(?=(.(?=[\0-r\[]*((?<=\.).))))(?:[c-\x73]{2,8}(\S))+((2)|\b[^\2-]){2}\D++$", "$u3$1$3$4$2")
Back to top
View user's profile Send private message Visit poster's website
toralf



Joined: 31 Jan 2005
Posts: 3842
Location: Bremen, Germany

PostPosted: Sat May 26, 2007 2:38 pm    Post subject: Reply with quote

Dear Titan,

Because of the path functions we worked on I looked at your benchmark script. But it is kind of hard to decipher. I tried it with your XPath test script, but that doesn't give much infomation on what is tested either. Could you please add some comments to the benchmark code and give a clearer information on how it is supposed to be used. For me it doesn't make sense to use it if I do not know what is measured and what the values mean and how I have to interpret them.

Thanks. I think the script can be of great help. If it only would be easier to understand.
_________________
Ciao
toralf
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Titan



Joined: 11 Aug 2004
Posts: 5009
Location: imaginationland

PostPosted: Sat May 26, 2007 2:59 pm    Post subject: Reply with quote

Sure, I'll explain the example in my first post. It tests to see which method is the fastest to get an integer from a float. The first call sets a variable to 2.5 (using the "pre" option) which will be the subject of the test. The second call uses the floor function, then an integer divide and finally a SubStr method to exclude anything past the '.' character. The last call returns the results. The Rank column determines which method is best, in this case the SubStr method came out as 91% which means its superior to the other methods. It also had a relative time count of 32 which is much lower 91 or 216. If you want to know how the rank is calculated read my response to Laszlo.

This is what I used to compare our absolute path functions:

Code:
qpc2("AbsPathFX.ahk", "inc") ; include the file with our two functions
qpc2("Path1 = \\server.com\user\Files\Docs\Code\AHK\SmartGui\no_commit\icons_dev", "pre") ; init. first variable
qpc2("Path2 = ..\..\..\SciTERelativePathector\includes\", "pre") ; and the second one
qpc2("AbsolutePath(path1, path2)") ; test your function
qpc2("RelToAbs(path1, path2)") ; then mine
MsgBox, % qpc2() ; then show results


One result was:
Code:
Test   Time   Deviation   Rank
1      184       0.01      47%
2      162      -0.01      54%

Iterations          2462
Standard deviation   0.02

However with such a low deviation it can be considered that both are virtually identical. Since results are susceptible to slight fluctuations from external factors such as thread scheduling (especially at a microsecond level) it's recommended you repeat the test two or three times and ignore anomalies.
_________________

RegExReplace("irc.freenode.net/autohotkey", "^(?=(.(?=[\0-r\[]*((?<=\.).))))(?:[c-\x73]{2,8}(\S))+((2)|\b[^\2-]){2}\D++$", "$u3$1$3$4$2")
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Scripts & Functions All times are GMT
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group