AutoHotkey Community

Hello all.

I've been testing the new UPX 2.93 beta for a while and today I started wondering how could I make the AHK compiler to use the new LZMA compression.

First I replaced upx.exe with a (compiled) AHK script to find out the parameters for UPX that the compiler uses. Then I simply put the new UPX as _upx.exe and wrote this other script (upx--ultra-brute.ahk):


Once compiled I got a upx--ultra-brute.exe which I renamed to upx.exe and voilá! Extreme compression for AHK.

If you try it and find unbearable the exhaustive compression tests, write lzma in the script instead of ultra-brute. The result is slightly worse but still better than with UPX 2.03

To put it in numbers, these are the sizes of a tiny script:

• Uncompressed: 398 KB
• UPX 2.03 (uses --best): 210 KB
• UPX 2.93 (using --lzma): 197 KB
• UPX 2.93 (using --ultra-brute): 195 KB

I know, I know, too much work for 15 KB

well, yes but thanks for the tip .

Thanks for posting your findings. When a stable version is released, AutoHotkey should probably be updated to include it.

By the way, the reason AutoHotkey still includes a 1.x version of UPX is that it compresses just as well as the 2.x series for most/all compiled scripts, while providing a UPX.exe that's much smaller. This reduces the size of the AutoHotkey installer, which gives faster downloads.

Hi again.

Just today I noticed a stable version of UPX, 3.00, is out (dated april 27th).

Changelog: http://upx.sf.net/upx-news.txt

_________________
Regards,
Antonio

Thanks for the news. I or someone else should probably measure how much better the compression is with UPX 3.0 with the new --lzma option. Also, I have some concern about launch/startup time: if we only save a few KB, but AutoHotkey.exe and compiled scripts take 10 times longer to launch, it might not be worth it.

Well, the compression numbers haven't changed from the ones in the original post.

About the startup time (decompression), I have no idea on how to measure it, but according to Wikipedia's article on LZMA...

and the (still not updated) UPX overview...

And last October, in the SF forums...


[EDIT]
As a raw and unscientific first look at the startup times I ran 1000 times the following script in different forms:

The results:

• script (unpacked AutoHotKey.exe) --> 251.391s
• script (original AutoHotKey.exe) --> 250.844s
• script (UPX3-repacked AutoHotKey.exe) --> 278.953s
• compiled using UPX 1.25 --> 231.875s
• compiled using UPX 3.00 --> 265.750s

[/EDIT]

_________________
Regards,
Antonio

Thanks for running the tests.

Hi again.

As mentioned before, --ultra-brute option in UPX gives the smallest possible size, but only after testing 72 variations. The --brute option is slightly better (36 combinations). Waaay too slow.

After diving into UPX source code I found the best (size-wise) options to compress AutoHotkeySC.bin (without additional files) is --filter=73 --lzma. It's my understanding that these would be the options to use with any script that doesn't include additional files.

A longer but quasi-optimal possibility (for several scripts I've tried, many of them including additional files) is --all-filters --lzma, that performs 9 tests. The undocumented option --fileinfo gives information on what method/filter combination was used to compress a file. The translation to command line options is not inmediate, but it may give some clues.

In case anyone wants to perform their own tests, this is my UPX-testing setup (in the compiler/ folder):
1) Rename "classical" UPX to upx1.exe
2) Download UPX 3.0 and put it in compiler/ with the name upx3.exe
3) Put this into a upx.ini file

4) Compile this script (which I call fakeupx.ahk):

5) Rename fakeupx.exe to upx.exe

This way you can easily change the command line used when compiling your scripts.

With all this, an aggressive use of ResHacker and the MSVCRT-requiring AutoHotkeySC.bin, I got my script down to 174KB from the original 212KB

I think I got a tiny little bit obsessed with this

_________________
Regards,
Antonio

Great post! I'll try to apply your findings soon.

UPX's LZMA method produced 70% slower launch speeds for AutoHotkey.exe on my system. Although my CPU is a few years old, I opted to avoid LZMA on AutoHotkey.exe; but I did apply it to compiled scripts, which reduces their size by about 16 KB. Here are the settings being used in today's v1.0.46.15:

Thanks again for discovering the command line switches "--filter=73" and "--fileinfo".

I did a benchmark test of my own and got some interesting results. Comparison and time graphs show that compressing greatly reduces performance although size is almost halved. I concluded that using an AutoHotkey.exe uncompressed and compiled scripts with level 9 compression give the most appropriate size to speed ratios.

_________________
GitHub • Scripts • IronAHK • Contact by email not private message.

Maybe it's that I'm slow today but I don't understand your data. De. Time is Decompress/Time but Decompress is Original Size-Size. What are De. Time and Decompress then? And what is the unit for Time? (Which is what? just the decompression time or the time to execute the script?)

Also I'm not clear on why you get a better size with "best" than with "brute".

I tried to answer myself looking at the script but, as I say, today I feel slower than usual

_________________
Regards,
Antonio

'Decompress' lists how many bytes were saved after compression and 'De. Time' is how long it took (in seconds) for my computer to decompress the exes. 'Best' gave better results than Brute as it has more options (--filter=73 --compress-icons=0) whereas Brute simply tried lzma and other algorithms with their default settings. Time is measured in Performance Counts, like my Advanced Benchmark script. I have over 850 gigs of storage so size doesn't matter to me, I'd rather performance instead.

_________________
GitHub • Scripts • IronAHK • Contact by email not private message.

By performance, I interpret your results to measure strictly the time it takes to launch AutoHotkey.exe, not the performance of loops and commands inside the script (which is identical in my testing). My original results showed that AutoHotkey.exe takes 21% longer to launch when compressed with its current settings vs. being uncompressed. Even so, I think compression is worthwhile because it probably halves the time AutoHotkey.exe takes to launch when uncached (which is often the case for casual users, who typically launch their main script then don't run AutoHotkey.exe again for a long time). The smaller AutoHotkey.exe size also makes it a bit more portable for size-constrained media.

By the way, isn't "level 9" compression the same as the "--best" switch?

Yes, mine only tests for launch time. I assumed script performance would be the same once AutoHotkey is loaded in memory. The test is repeated 25 times for an average so each version should have been cached at one point. Level 9 is '-9' whereas best is '--best --filter=73 --compress-icons=0 --no-lzma'. Since the former is much faster and only around 20kb larger I'd switch to that instead.

Personally I'll be using a decompressed AutoHotkey.exe for performance. I rarely ever need to compile my scripts so size is not a concern.

_________________
GitHub • Scripts • IronAHK • Contact by email not private message.