AutoHotkey Community

atnbueno · Joined: 24 Mar 2007 Posts: 26

Hello all.

I've been testing the new UPX 2.93 beta for a while and today I started wondering how could I make the AHK compiler to use the new LZMA compression.

First I replaced upx.exe with a (compiled) AHK script to find out the parameters for UPX that the compiler uses. Then I simply put the new UPX as _upx.exe and wrote this other script (upx--ultra-brute.ahk):

corrupt · Joined: 29 Dec 2004 Posts: 2393

Chris · Site Admin Joined: 02 Mar 2004 Posts: 10467

Thanks for posting your findings. When a stable version is released, AutoHotkey should probably be updated to include it.

By the way, the reason AutoHotkey still includes a 1.x version of UPX is that it compresses just as well as the 2.x series for most/all compiled scripts, while providing a UPX.exe that's much smaller. This reduces the size of the AutoHotkey installer, which gives faster downloads.

atnbueno · Joined: 24 Mar 2007 Posts: 26

Hi again.

Just today I noticed a stable version of UPX, 3.00, is out (dated april 27th).

Changelog: http://upx.sf.net/upx-news.txt
_________________
Regards,
Antonio

Chris · Site Admin Joined: 02 Mar 2004 Posts: 10467

Thanks for the news. I or someone else should probably measure how much better the compression is with UPX 3.0 with the new --lzma option. Also, I have some concern about launch/startup time: if we only save a few KB, but AutoHotkey.exe and compiled scripts take 10 times longer to launch, it might not be worth it.

atnbueno · Joined: 24 Mar 2007 Posts: 26

Well, the compression numbers haven't changed from the ones in the original post.

About the startup time (decompression), I have no idea on how to measure it, but according to Wikipedia's article on LZMA...

Chris · Site Admin Joined: 02 Mar 2004 Posts: 10467

Thanks for running the tests.

atnbueno · Joined: 24 Mar 2007 Posts: 26

Hi again.

As mentiones before, --ultra-brute option in UPX gives the smallest possible size, but only after testing 72 variations. The --brute option is slightly better (36 combinations). Waaay too slow.

After diving into UPX source code I found the best (size-wise) options to compress AutoHotkeySC.bin (without additional files) is --filter=73 --lzma. It's my understanding that these would be the options to use with any script that doesn't include additional files.

A longer but quasi-optimal possibility (for several scripts I've tried, many of them including additional files) is --all-filters --lzma, that performs 9 tests. The undocumented option --fileinfo gives information on what method/filter combination was used to compress a file. The translation to command line options is not inmediate, but it may give some clues.

In case anyone wants to perform their own tests, this is my UPX-testing setup (in the compiler/ folder):
1) Rename "classical" UPX to upx1.exe
2) Download UPX 3.0 and put it in compiler/ with the name upx3.exe
3) Put this into a upx.ini file

Chris · Site Admin Joined: 02 Mar 2004 Posts: 10467

Great post! I'll try to apply your findings soon.

Chris · Site Admin Joined: 02 Mar 2004 Posts: 10467

UPX's LZMA method produced 70% slower launch speeds for AutoHotkey.exe on my system. Although my CPU is a few years old, I opted to avoid LZMA on AutoHotkey.exe; but I did apply it to compiled scripts, which reduces their size by about 16 KB. Here are the settings being used in today's v1.0.46.15:

Titan · Joined: 11 Aug 2004 Posts: 5041 Location: imaginationland

I did a benchmark test of my own and got some interesting results. Comparison and time graphs show that compressing greatly reduces performance although size is almost halved. I concluded that using an AutoHotkey.exe uncompressed and compiled scripts with level 9 compression give the most appropriate size to speed ratios.
_________________

RegExReplace("irc.freenode.net/ahk", "^(?=(.(?=[\0-r\[]*((?<=\.).))))(?:[c-\x73]{2,8}(\S))+((2)|\b[^\2-]){2}\D++$", "$u3$1$3$4$2")

atnbueno · Joined: 24 Mar 2007 Posts: 26

Maybe it's that I'm slow today but I don't understand your data. De. Time is Decompress/Time but Decompress is Original Size-Size. What are De. Time and Decompress then? And what is the unit for Time? (Which is what? just the decompression time or the time to execute the script?)

Also I'm not clear on why you get a better size with "best" than with "brute".

I tried to answer myself looking at the script but, as I say, today I feel slower than usual Rolling Eyes

_________________
Regards,
Antonio

Titan · Joined: 11 Aug 2004 Posts: 5041 Location: imaginationland

'Decompress' lists how many bytes were saved after compression and 'De. Time' is how long it took (in seconds) for my computer to decompress the exes. 'Best' gave better results than Brute as it has more options (--filter=73 --compress-icons=0) whereas Brute simply tried lzma and other algorithms with their default settings. Time is measured in Performance Counts, like my Advanced Benchmark script. I have over 850 gigs of storage so size doesn't matter to me, I'd rather performance instead.
_________________

RegExReplace("irc.freenode.net/ahk", "^(?=(.(?=[\0-r\[]*((?<=\.).))))(?:[c-\x73]{2,8}(\S))+((2)|\b[^\2-]){2}\D++$", "$u3$1$3$4$2")

Chris · Site Admin Joined: 02 Mar 2004 Posts: 10467

Titan · Joined: 11 Aug 2004 Posts: 5041 Location: imaginationland

Yes, mine only tests for launch time. I assumed script performance would be the same once AutoHotkey is loaded in memory. The test is repeated 25 times for an average so each version should have been cached at one point. Level 9 is '-9' whereas best is '--best --filter=73 --compress-icons=0 --no-lzma'. Since the former is much faster and only around 20kb larger I'd switch to that instead.

Personally I'll be using a decompressed AutoHotkey.exe for performance. I rarely ever need to compile my scripts so size is not a concern.
_________________

RegExReplace("irc.freenode.net/ahk", "^(?=(.(?=[\0-r\[]*((?<=\.).))))(?:[c-\x73]{2,8}(\S))+((2)|\b[^\2-]){2}\D++$", "$u3$1$3$4$2")