AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Script speed on different computers

 
Reply to topic    AutoHotkey Community Forum Index -> Ask for Help
View previous topic :: View next topic  
Author Message
Uminnsky



Joined: 12 Aug 2009
Posts: 26
PostPosted: Fri Sep 25, 2009 2:56 am    Post subject: Script speed on different computers Reply with quote
Would anyone have any suggestions on why a script would run extremely slow on one computer (XP, 1.6 Ghz Duo) and fast on the other (vista)? The program does some basic text file manipulation, and runs extremely slow when Loop reading a text file with an inner Loop parse routine.

The computer with Vista takes 27 seconds to complete, the XP computer is taking over 20 minutes operating on the same text file, which is seemingly impossible!

I have never noticed a difference between the two before.

The longest part of the code...

Code:
Loop 150
      {
         Loop,parse,fieldlist,`,   
         {
            Num+=%A_loopfield%
            Field:=pcsSplit%Num%
            Fileappend,%field%`,,%A_programfiles%\WIA\Data\MainData1.txt
            Num-=%A_loopfield%
         }
         
         num+=1434
      }
Back to top
View user's profile Send private message
hd0202



Joined: 13 Aug 2006
Posts: 265
Location: Germany
PostPosted: Fri Sep 25, 2009 5:03 am    Post subject: Re: Script speed on different computers Reply with quote
Uminnsky wrote:
The program does some basic text file manipulation
As I know from an earlier post your program does some hundred thousend basic text file manipulation. Your problem on the very slow computer is the disk and the filesystem. Have a look at my post "Performance of file ouput (FileAppend)" to find a better solution.

Hubert
Back to top
View user's profile Send private message
Guest
PostPosted: Fri Sep 25, 2009 5:56 am    Post subject: Re: Script speed on different computers Reply with quote
[quote="hd0202"]
Uminnsky wrote:
Have a look at my post "Performance of file ouput (FileAppend)" to find a better solution.

You could/should give a link when refering to another thread.

For the op, http://www.autohotkey.com/forum/topic45771.html
Back to top
tank



Joined: 21 Dec 2007
Posts: 3700
Location: Louisville KY USA
PostPosted: Fri Sep 25, 2009 12:38 pm    Post subject: Reply with quote
like EVERY application ahk relies on available interupts for cpu time
like EVERY application that relies on data written to a disk the speed of the hard drive is a factor
Like EVERY application currently running processes affect the above

troubleshoot what has changed and you will know why performance is affected
_________________

We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
Back to top
View user's profile Send private message
randallf



Joined: 06 Jul 2009
Posts: 678
PostPosted: Fri Sep 25, 2009 1:55 pm    Post subject: Re: Script speed on different computers Reply with quote
Uminnsky wrote:
Would anyone have any suggestions on why a script would run extremely slow on one computer (XP, 1.6 Ghz Duo) and fast on the other (vista)?


Drive fragmentation and available space.
Back to top
View user's profile Send private message
Uminnsky



Joined: 12 Aug 2009
Posts: 26
PostPosted: Wed Sep 30, 2009 1:48 am    Post subject: Re: Script speed on different computers Reply with quote
hd0202 wrote:
Uminnsky wrote:
The program does some basic text file manipulation
As I know from an earlier post your program does some hundred thousend basic text file manipulation. Your problem on the very slow computer is the disk and the filesystem. Have a look at my post "Performance of file ouput (FileAppend)" to find a better solution.

Hubert


Thanks for referencing this.

I made this switch, the commented was my old code, the speed difference in my situation was 47 seconds down to 18 seconds on the data tested:

Code:
Loop,read, % fflist, % output
         {
            Num+=%A_loopreadline%
            Field:=pcsSplit%Num%
            Fileappend, % field "`,"
            Num-=%A_loopreadline%
         }
         
;~          Loop,parse,fieldlist,`,   
;~          {
;~             Num+=%A_loopfield%
;~             Field:=pcsSplit%Num%
;~             Fileappend,%field%`,,%A_programfiles%\WIA\Data\MainData1.csv
;~             Num-=%A_loopfield%
;~          }


This is pretty huge considering I plan for my program to build a database on thousands of files which would take anywhere from 8-12 hours. Just so I understand better what is going on, by switching from loop parse to loop read I am preventing the file from opening when it appends? Any suggestions on how I should or coudl speed up my next section of code...

Code:
Loop, read, %A_programfiles%\WIA\Data\MainData.csv
   {
      Bob:=A_loopreadline
      Stringsplit, fish, A_loopreadline, `,
      Loop, read, %A_programfiles%\WIA\Data\results.csv               ; Import the results into the Data file
      {
         ifinstring,A_loopreadline, %fish1%`,%fish2%`,%fish9%
         fileappend,%bob%`,%A_loopreadline%`n,%A_programfiles%\WIA\Data\Data.csv
      }   
   }
Back to top
View user's profile Send private message
hd0202



Joined: 13 Aug 2006
Posts: 265
Location: Germany
PostPosted: Wed Sep 30, 2009 7:34 pm    Post subject: Re: Script speed on different computers Reply with quote
For youzr first problem: did you get the same data as before you changed the code? I say NO without seeing any of your data!
Here is my suggestion with some comments:
Code:
; dummyfile.txt must contain minimal 1 record of any content, it is not the data to be processed
; here the output file is opened
Loop,read, duumyfile.txt, %A_programfiles%\WIA\Data\MainData1.txt
(
   Loop 150
   {
      Loop,parse,fieldlist,`,   
      {
         Num+=%A_loopfield%
         Field:=pcsSplit%Num%

;        this command opens, (positions to the end and) appends and closes
;        150 times <the number of fields> the field data
;        ------------------------------------------------------------
;        Fileappend,%field%`,,%A_programfiles%\WIA\Data\MainData1.txt
;        ------------------------------------------------------------
;
;        this command really only appends 150 time <the number of fields>
;        the field data as the file is kept open all the time
;        --------------------
         Fileappend,%field%`,
;        --------------------
;
         Num-=%A_loopfield%
      }
     
      num+=1434
   }
   break  ; the dummyfile loop after the only or first record
}
; here the ouput file is closed

I hope that helps.

Hubert
Back to top
View user's profile Send private message
hd0202



Joined: 13 Aug 2006
Posts: 265
Location: Germany
PostPosted: Wed Sep 30, 2009 7:54 pm    Post subject: Re: Script speed on different computers Reply with quote
And here is my suggestion for your second problem (also not tested as I do not have the data):
Code:
FileRead, results, %A_programfiles%\WIA\Data\results.csv  ; get results into variable with one read

Loop,read, dummyfile.txt, %A_programfiles%\WIA\Data\Data.csv
(
   FileRead, maindata, %A_programfiles%\WIA\Data\MainData.csv  ; get data into variable with one read
   Loop, parse, maindata, `n, `r
   {
      if a_loopfield =
         continue
      Bob:=A_loopfield
      Stringsplit, fish, A_loopfield, `,
      Loop, parse, results, `n, `r
      {
         if a_loopfield =
            continue
         ifinstring,A_loopfield, %fish1%`,%fish2%`,%fish9%
            fileappend,%bob%`,%A_loopfield%`n
      }   
   }
   break
}

It also uses the dummyfile to hold the output file open.

I hope this also helps.

Hubert
Back to top
View user's profile Send private message
entropic



Joined: 21 Dec 2008
Posts: 181
PostPosted: Wed Sep 30, 2009 10:30 pm    Post subject: Reply with quote
Why use FileAppend in the loop? It would be faster if you kept the output in a variable during the loop and write it out in one go afterwards.

Code:

FullText =
Loop 150
      {
         Loop,parse,fieldlist,`,   
         {
            Num += %A_loopfield%
            Field := pcsSplit%Num%
            FullText := FullText . Field . ","
            Num -= %A_loopfield%
         }
         
         num+=1434
      }
FileAppend, %FullText%, %A_programfiles%\WIA\Data\MainData1.txt
Back to top
View user's profile Send private message
Guest
PostPosted: Fri Oct 02, 2009 3:25 am    Post subject: Reply with quote
entropic wrote:
Why use FileAppend in the loop? It would be faster if you kept the output in a variable during the loop and write it out in one go afterwards.

Code:

FullText =
Loop 150
      {
         Loop,parse,fieldlist,`,   
         {
            Num += %A_loopfield%
            Field := pcsSplit%Num%
            FullText := FullText . Field . ","
            Num -= %A_loopfield%
         }
         
         num+=1434
      }
FileAppend, %FullText%, %A_programfiles%\WIA\Data\MainData1.txt


Thanks, this will help me a lot. I have been meaning to research on how the syntax for this would look, but just never got around to it. From some initial testing it is significantly faster, and there are a lot of places I append during loops. Thanks!
Back to top
Display posts from previous:   
Reply to topic    AutoHotkey Community Forum Index -> Ask for Help All times are GMT
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group