AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Compiled script (ahk2exe) smallest size vs. best performance
Goto page 1, 2  Next
 
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> General Chat
View previous topic :: View next topic  

Should compiled scripts be reduced in size by 9% even if a small amount of performance is lost?
Yes
0%
 0%  [ 0 ]
No
92%
 92%  [ 26 ]
Other
7%
 7%  [ 2 ]
Total Votes : 28

Author Message
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10467

PostPosted: Wed Dec 01, 2004 2:04 pm    Post subject: Compiled script (ahk2exe) smallest size vs. best performance Reply with quote

Currently, both AutoHotkey.exe and compiled scripts are compiled in C++ to favor performance over EXE size. This increases the size of each EXE by about 15 KB (and this increase will get larger over time as more code is added).

The performance loss of not doing this would probably not be noticable in a typical script. However, the loss might be more significant for CPU intensive scripts (math loops, etc).

Note: If compiled scripts are optimized for small size but AutoHotkey.exe is kept optimzed for performance, there is a miniscule chance that scripts that rely on precise timing will behave differently in compiled form due to their slightly lower performance. This might be another reason to leave things as they are.

On a related note, I've recently been thinking of other ways to reduce the size of compiled scripts. One way is to offer an optional scaled down feature-set that omits syntax checking (5 KB), keyboard/mouse hooks (40 KB?), and maybe the GUI feature (15 KB?) However, I have some doubt that these small savings are worth the confusion of an extra compiler option, not to mention the extra effort to maintain two different AutoHotkeySC.bin files.

Ideally, scripts should be compiled to include code only for the commands they actually use. However, this seems too difficult and impractical to be worth it.

Comments are welcome.


Last edited by Chris on Mon Mar 27, 2006 3:16 am; edited 1 time in total
Back to top
View user's profile Send private message Send e-mail
Titan



Joined: 11 Aug 2004
Posts: 5049
Location: imaginationland

PostPosted: Wed Dec 01, 2004 4:49 pm    Post subject: Re: Compiled script (ahk2exe) smallest size vs. best perform Reply with quote

Chris wrote:
Ideally, scripts should be compiled to include code only for the commands they actually use. However, this seems too difficult and impractical to be worth it.

This would be a great addition if possible. I really couldn't choose between Speed and Performance, couldn't there be a line of code we could put in to choose wether the compiled script would be smaller or peform better? (e.g. #Compress1) Question
Back to top
View user's profile Send private message Visit poster's website
Nemroth



Joined: 07 Sep 2004
Posts: 262
Location: France

PostPosted: Wed Dec 01, 2004 6:23 pm    Post subject: Reply with quote

For my point of view, there is no such difference of size between a large and a short compiled script (it seem there is a common minimum size), and they are usually (for me) small in size. So I prefer to gain in performance rather than in size.
Back to top
View user's profile Send private message
deguix



Joined: 26 Aug 2004
Posts: 72
Location: Everett - MA, USA

PostPosted: Wed Dec 01, 2004 10:19 pm    Post subject: Reply with quote

For me, using this language (AutoHotkey) the performance matters in most of the aspects of a script, like hotkeys... And that's the main reason I'm here. Smile
_________________
Working now on:
NSIS Wiki (NSIS)
Back to top
View user's profile Send private message
jonny



Joined: 13 Nov 2004
Posts: 3004
Location: Minnesota

PostPosted: Wed Dec 01, 2004 10:48 pm    Post subject: Reply with quote

15kb isn't much to worry about. A slight decrease in performance that affects regularly used scripts, that is. I don't care much about the size of my scripts (compiled or not), but the performance can sometimes make a big difference.
Back to top
View user's profile Send private message
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10467

PostPosted: Wed Dec 01, 2004 11:36 pm    Post subject: Re: Compiled script (ahk2exe) smallest size vs. best perform Reply with quote

Titan wrote:
couldn't there be a line of code we could put in to choose wether the compiled script would be smaller or peform better? (e.g. #Compress1) Question
Offering such an option now would save perhaps 25 KB (15 KB by changing compiler settings, 5 KB from removing syntax checking, and another 5 KB from other non-essential things if I could find any). But then I would need to include (and maintain for all future versions) two different bin files, which would also increase the size of the installer. It doesn't seem worth it just for 25 KB. But I'll certainly be on the look out for better ideas since you're not the only one who has asked for smaller EXEs.

And thanks everyone else for your comments and votes. More discussion is welcome of course.
Back to top
View user's profile Send private message Send e-mail
sleepyy35



Joined: 22 Jul 2004
Posts: 194
Location: cedar city UT

PostPosted: Thu Dec 02, 2004 12:23 pm    Post subject: Reply with quote

If I understand this rite we loose some performance and the exe size will go down
I personally like the small compile size of my scripts but i think performance is more important Laughing Laughing
_________________
^sleepy^
Back to top
View user's profile Send private message Send e-mail Visit poster's website Yahoo Messenger MSN Messenger
Rajat



Joined: 28 Mar 2004
Posts: 1717

PostPosted: Thu Dec 02, 2004 1:53 pm    Post subject: Reply with quote

i'd have voted for size had the cut been bigger... like bringing the size down to nearly 100 or lesser.
_________________
Back to top
View user's profile Send private message
Guest






PostPosted: Sun Dec 12, 2004 10:19 am    Post subject: Re: Compiled script (ahk2exe) smallest size vs. best perform Reply with quote

Chris wrote:
Ideally, scripts should be compiled to include code only for the commands they actually use. However, this seems too difficult and impractical to be worth it.

...Exactly! I've wondered why my compiled scripts are just another copy of AutoHotkey.exe...152 KB...even when the only command is MsgBox, a C program compiled with one MessageBox could be 5 KB or less. If you distributed a bunch of compiled AHK scripts, that would very quickly get excessive in file size...each one including the code to do everything AHK can do, even when it only does one thing.

Chris wrote:
Currently, both AutoHotkey.exe and compiled scripts are compiled in C++

...huh? compiled? it looks more like it's just a simple AutoHotkeySC.bin + Script.ahk = Script.exe, no compiling, but raw tacking onto the end.

Chris wrote:
This increases the size of each EXE by about 15 KB (and this increase will get larger over time as more code is added).

...15 KB for performance isn't the problem, 152 KB for every .exe is.

Chris wrote:
...will behave differently in compiled form due to their slightly lower performance.

...it shouldn't start acting different because it's compiled, if anything compiling should make it work better, not worse.

Chris wrote:
One way is to offer an optional scaled down feature-set that omits syntax checking (5 KB),

...you could syntax check on compile, then omit syntax check in the final .exe, the compiled AHK code isn't going to change after it's compiled...without re-compiling...when you could do another compile-time syntax check.

Chris wrote:
keyboard/mouse hooks (40 KB?), and maybe the GUI feature (15 KB?)

...any feature that the script don't use, should be omitted.

Chris wrote:
However, I have some doubt that these small savings are worth the confusion of an extra compiler option,

...options are good...if people don't know how to use them, they can use the default & that default can either do the most efficient thing (make it as small as possible) or just do what it does now (make it as large as possible).

Chris wrote:
not to mention the extra effort to maintain two different AutoHotkeySC.bin files.

...I don't get where with those options you only have 2 .bin files, you would more logically have a bunch of em with each possible combination of options, where do you get just 2?

If you currently do just tack the .ahk onto the .bin, couldn't you make a .bin for each part of the code...split the 152 KB up into parts...gui.bin, key-mouse.bin (or perhaps key.bin & mouse.bin separate), blah.bin, then tack them together in a way that works. I really need to know how it really does it now to know how I would change it. Does it store the raw .ahk or does it do the run-time pre-process & store that in the .exe...would storing the run-time pre-process have any advantages? If it does raw file tacking-on now, couldn't it be changed to a more gcc-like compile?

Could you include an option to compile an .exe that is dependant on either AutoHotkey.exe or another file that was compiled full? So on my own computer I could compile a 5 KB Script.exe that looks for the code in my AHK install dir? & for the example of distributing multiple compiled AHK's, include one main compiled AHK & have the children dependant on that...like...
    main.exe (152 KB)
    child1.exe (5 KB)
    child2.exe (5 KB)
...main.exe is compiled normally, child1 & 2 are compiled to be dependant on a main.exe in the current dir, with the option of falling back to an AHK installation (AutoHotkey.exe), if one exists. If you can't do the full optimize-all-compiled-AHK's this would at least help on my own computer & be a semi-workable solution for multiple distributed compiled AHK's. Perhaps all compiled AHK's could support 2 command line options by default...
    -ahk-portable
    -ahk-dependant [<on what>]
...the ahk prefix so it don't conflict with any command line the script expects, portable would re-compile that .exe (a 5 KB dependant .exe) to include the AutoHotkey.exe code in all it's 152 KB glory, the other option would reverse that, cut a 152 KB portable .exe into a 5 KB dependant .exe, the <on what> would either be the name of the .exe it should look for or not be there to indicate that it should look for an AHK installation...I just thought of this...telling it to go portable when it's already portable is pointless, so maybe instead of 2 options, just one...
    -ahk-switch
...would take a 5 KB & make it a 152 KB & vice-versa. Going from 5 KB to 152 KB would require either an AHK installation or a fully compiled script, so if some dumb user tried to take the above main.exe & cut it to 5 KB it could give an error saying that there is no AHK installation or fully compiled script found to undo what they are trying to do & it would fail, but if they 1st told child1.exe to go portable, then they could tell main.exe to go dependant, with no error.
Back to top
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10467

PostPosted: Sun Dec 12, 2004 3:29 pm    Post subject: Re: Compiled script (ahk2exe) smallest size vs. best perform Reply with quote

Quote:
...huh? compiled? it looks more like it's just a simple AutoHotkeySC.bin + Script.ahk = Script.exe, no compiling, but raw tacking onto the end.
I meant that the internal version of AutoHotkey.exe contained inside each compiled script is compiled in C++.

Quote:
...you could syntax check on compile, then omit syntax check in the final .exe, the compiled AHK code isn't going to change after it's compiled...without re-compiling...when you could do another compile-time syntax check.
Not a bad idea, but a lot of coding effort since large sections from AutoHotkey.exe would have to be ported to ahk2exe, which uses a different source tree. I'll see if it can be done for a future version.

Quote:
...I don't get where with those options you only have 2 .bin files, you would more logically have a bunch of em with each possible combination of options, where do you get just 2?
One would have everything and the other would omit a set of seldom used features. The reason for having only two is to keep the size of the Installer to a reasonable level (which cuts down on download times, etc.)

Quote:
If you currently do just tack the .ahk onto the .bin, couldn't you make a .bin for each part of the code...split the 152 KB up into parts...gui.bin, key-mouse.bin (or perhaps key.bin & mouse.bin separate), blah.bin, then tack them together in a way that works.
Although I'm sure there are ways to do it, it's way outside my knowledge area. Also, I doubt there's much info on the 'net about it short of carefully studying and applying the code for an open source linker, which would probably take a slow learner like me a week or longer just to understand. Smile

Quote:
I really need to know how it really does it now to know how I would change it. Does it store the raw .ahk or does it do the run-time pre-process & store that in the .exe...would storing the run-time pre-process have any advantages? If it does raw file tacking-on now, couldn't it be changed to a more gcc-like compile?
A compiled script is built in these stages:
1) Merge the .ahk file and all its #Include files into one big temp script.
2) Compress and encrypt the above into the new EXE, which is simply a copy of AutoHotkeySC.bin.
3) Compress the resulting file with UPX.

Quote:
Could you include an option to compile an .exe that is dependant on either AutoHotkey.exe or another file that was compiled full?
Although likely to be a lot of coding effort, it's a good idea for a future version.

Quote:
this would at least help on my own computer
I'll assume there's a reason you compile so many scripts on your own PC rather than just leaving them as .ahk files (perhaps to keep other users from viewing their source code?)

Quote:
-ahk-switch...would take a 5 KB & make it a 152 KB & vice-versa. Going from 5 KB to 152 KB would require either an AHK installation or a fully compiled script, so if some dumb user tried to take the above main.exe & cut it to 5 KB it could give an error saying that there is no AHK installation or fully compiled script found to undo what they are trying to do & it would fail, but if they 1st told child1.exe to go portable, then they could tell main.exe to go dependant, with no error.
Thanks for explaining it all. This will probably come in handy someday when I or someone else implements it.
Back to top
View user's profile Send private message Send e-mail
JSLover



Joined: 20 Dec 2004
Posts: 542
Location: LooseChange911.com... the WTC attacks were done by the US Gov't... the official story is a lie...

PostPosted: Sat Feb 26, 2005 11:19 pm    Post subject: Re: Compiled script (ahk2exe) smallest size vs. best perform Reply with quote

Ok, now I have a new idea...I'm going to have to just explain it & ask anyone if they think it'll work...

In a PE exe file, each "thing" AHK does is a function...so say you have...

Test.exe (before UPX) wrote:
Run=|101010101010101010101010101010101010101010101010101010101010101010101010|
Send=|010101010101010101010101010101010101010101010101010101010101010101010101|
WinSet=|101101101101101101101101101101101101101101101101101101101101101101101101|

...ok, this is supposed to be a simplifed version of what a PE exe would look like (we're assuming a full, 152KB {or more now} AHK exe, with all the functions...but I'm only doing 3 for an example), with each function defined & then the code to do that function after it. For one thing, are the function's sections easily identifiable like this?

My idea, is a program that looks at the .ahk & makes a list of what AHK functions (C++ functions) the .ahk commands result in (like Run & RunWait might be handled by the same C++ function), then with the list of the C++ functions your program actually calls, it could search thru & 0 out all the sections your program never calls...

Test.ahk wrote:
Run, wow.exe

Test.exe (before UPX, after 0'd out) wrote:
Run=|101010101010101010101010101010101010101010101010101010101010101010101010|
Send=|000000000000000000000000000000000000000000000000000000000000000000000000|
WinSet=|000000000000000000000000000000000000000000000000000000000000000000000000|

...so you end up with a file, just as big, but with lots of 0's...why is this better? Because then UPX might be able to compress the 0 sections better. If you can 0 out sections like this & not make it crash, UPX should have a field day (easy time) compressing all the 0 sections.

The key parts for this to work...

1) Does UPX compress "code" or just other parts of the exe?
2) Can the function boundaries be searched for in a (un-UPX'd) PE exe? (if they can't, can markers be added to the C++ code to help?)
3) Does 0ing out sections that will never be called hurt? What would happen if a 0 section accidently got executed? Instead of just 0ing it out it could add the binary code equiv of "return" & 0 out the rest...so if it did get executed only "return" would be executed.
4) The code to do this would need to be added to Ahk2Exe or Ahk2Exe would need to be able to call a 3rd-party program, before calling UPX.
_________________

Home • Click image! • Blog
Back to top
View user's profile Send private message Visit poster's website
Rajat



Joined: 28 Mar 2004
Posts: 1717

PostPosted: Sun Feb 27, 2005 7:09 am    Post subject: Reply with quote

now that's some fab thinking JSLover!
_________________
Back to top
View user's profile Send private message
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10467

PostPosted: Sun Feb 27, 2005 11:21 pm    Post subject: Re: Compiled script (ahk2exe) smallest size vs. best perform Reply with quote

JSLover wrote:
1) Does UPX compress "code" or just other parts of the exe?
I'm pretty sure it does everything (except the icons if you choose to omit them).

Quote:
2) Can the function boundaries be searched for in a (un-UPX'd) PE exe? (if they can't, can markers be added to the C++ code to help?)
That's pretty far outside my area of knowledge. Even if feasible, I suspect it would be difficult to implement.

Quote:
3) Does 0ing out sections that will never be called hurt? What would happen if a 0 section accidently got executed? Instead of just 0ing it out it could add the binary code equiv of "return" & 0 out the rest...so if it did get executed only "return" would be executed.
AutoHotkey isn't currently organized in a way that would allow functions to be surgically zeroed out.

Overall, these are ingenious ideas but I'd don't think I'll be up to the challenge anytime soon. If anyone else has expertise in these areas, feel free to come to the rescue.
Back to top
View user's profile Send private message Send e-mail
[deXter]



Joined: 28 Nov 2004
Posts: 35

PostPosted: Wed May 11, 2005 9:31 pm    Post subject: Reply with quote

A slightly old topic, but I feel that smaller compiled exe's should be more of a priority now. AHK keeps getting better and better, but unfortunately, the filesize just keeps on growing..

I dont know why this discussion has been unnecessarily complicated. It would be quite simple to implement this actually.
Instead of thinking along the lines of performance v/s size, how about a modular approach?
What I propose is to abandon the current AutoIT based exe compiler system entirely, which is where I suppose the limitations lie.

Instead of having one big AutoHotkey.exe, you could have several smaller files (exes/dlls). There could be a small Ahk-main.exe (or something) which would include the core AHK features like hotkeys/hotstrings. You could then group up various features in several dlls/exes. For eg, all the GUI stuff in one dll, image/screen stuff in another and so on. The main exe parses the script as normal, and it simply would call the necessary dll/exe as and when required.

Similarly, the script compiler would parse the script and include only those dlls/exes that would actually be required according to the functions used.

And as for the compilation itself- the main exe, the dlls and the script itself can be simply bundled into a single file. There would be no need to use the AutoIt compiler.
For starters, there are many exe-bundlers out there which can do that (for eg, PEBundle). And a final coating of UPX to top it up.

Thus the changes would all have to be made at the source level itself (which is much simpler), rather than modifying binaries. And I suppose it wouldn't be that difficult either as the code is already present. What you would be needing to work on is the grouping and seperation and of the source files.


If all this is done sucessfully, we could be looking at compiled scripts as small as 50k! (or smaller!)
Back to top
View user's profile Send private message
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10467

PostPosted: Thu May 12, 2005 12:45 am    Post subject: Reply with quote

[deXter] wrote:
Instead of thinking along the lines of performance v/s size, how about a modular approach?

Similarly, the script compiler would parse the script and include only those dlls/exes that would actually be required according to the functions used.
I would like to see AutoHotkey become more modular someday. However, one of the nice things about having it be all contained in one large EXE file is that you can copy just the EXE anywhere, without installing anything, and have almost all the functionality for running scripts. I know this is a minor advantage compared to having smaller compiled scripts.

Quote:
For starters, there are many exe-bundlers out there which can do that (for eg, PEBundle). And a final coating of UPX to top it up.
PEBundle sounds interesting. I wonder if there are any comparable products that are GPL or similarly unrestricted.

Quote:
If all this is done sucessfully, we could be looking at compiled scripts as small as 50k! (or smaller!)
It's a worthwhile goal. The reason I don't assign a high priority to it is that it would be yield comparatively little benefit when weighed against the large investment in time it would be require. For example, adding features such as more GUI control types, wildcards, and scheduling probably have more popular appeal than devoting lots of effort to reducing the size of compiled EXEs.

EDIT: The 9% reduction mentioned at the top of this topic was done several months ago. This is because a series of benchmarks showed that the reduction could be achieved without sacrificing performance. This was achieved by compiling performance-sensitive parts of AutoHotkeySC.bin with "maximize speed", but compiling everything else with “minimize size”.


Last edited by Chris on Mon Mar 27, 2006 3:16 am; edited 1 time in total
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> General Chat All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group