AutoHotkey Community

Costas · Joined: 31 Mar 2009 Posts: 66

I love AHK and have found many uses for it. Recently, I decided to use it to add a header record to a number of HTML files that are automatically generated by another software tool.

AHK works great for this. The catch is that I need to read about 25 input files and some of these input files contain thousands of records.

I quickly learned that my AHK script would work, but it takes a long time to complete (over 30 minutes for all files).

I then did some performance tests.

If I have an input file of 3,000 records, I can read the entire file with AHK in less than one second (just read, no writes)

When I add the FileAppend command to write the records to a new file (to add my 1 new header record) the run time jumps to 30 seconds.

Is there a way to programmatically reduce this time? Am I missing something? Adding faster hardware is not an option due to budget constraints.

Thanks for your ideas,
Costas

Leef_me · Posted: Fri Aug 28, 2009 6:13 pm Post subject:

[VxE] · Joined: 07 Oct 2006 Posts: 3254 Location: Simi Valley, CA

Using a Loop, Read, InputFile, OutputFile and using FileAppend inside the loop to append to the OutputFile will dramatically increase the speed of multiple FileAppends. See the (linked) docs for details.
edit:] link fix, sry about that.
_________________
Ternary (a ? b : c) guide TSV Table Manipulation Library
Post code inside [code][/code] tags!

Costas · Joined: 31 Mar 2009 Posts: 66

Here is the code that I trimmed down for performance testing. After several tests, it appears that the FileAppend is the command that is very time consuming. Any ideas on how to speed this up would be most appreciated.

Thanks,
Costas

;~~~~~~~~~~~ START ~~~~~~~~~~~~~~~~~~~~~~~~

startTime:=A_TickCount

Input_File = C:\Input_File.TXT
Output_File = C:\Output_File.TXT

Filedelete %Output_File%

;~~~~~~~~
; Read input file and add a new record before the <HEAD> record as output file is created

loop, read, %Input_File%

{

IfInString, A_LoopReadLine, <HEAD>
{
FileAppend , Newly-Added-Test-Record `n, %Output_File%
}

FileAppend , %A_LoopReadLine% `n, %Output_File%

}

;~~~~~~~

EndTime:=A_TickCount

TimeInScript:=(A_TickCount-startTime)/1000
Msgbox Time in Script = %TimeInScript% seconds `n`n

EXITAPP

;~~~~~~~~~~~ END ~~~~~~~~~~~~~~~~~~~~~~~~

horntail · Joined: 03 Aug 2009 Posts: 69 Location: UK

test it again with this code, it should be faster and its what [VxE] meant in his post above:

Costas · Joined: 31 Mar 2009 Posts: 66

Horntail,

Thanks for the help, I really appreciate it.

I made this minor change as you suggested and the run time was reduced significantly.

Being a curious person, I would like to understand why this minor change made such a large impact.

Thanks again,
Costas

why_bother_helping · Guest

Costas · Joined: 31 Mar 2009 Posts: 66

Sorry...

I did read what VXE posted, but I just didn't understand. I am still fairly new to AHK. The example that Horntail posted better helped me to understand what to try.

Now I would like to understand the underlying reason why this minor change makes such a big difference.

Thanks,
Costas

Guest · Posted: Sat Aug 29, 2009 1:52 pm Post subject:

Costas · Joined: 31 Mar 2009 Posts: 66

No, I had not read this. When I click on the first of the four links in the post that VXE added, I receive a "404 Page Not Found" error message.

The link that you show works fine and leads to the explanation of why this minor change is going to have such a large impact.

Thanks for the help, I really appreciate it.

Costas