AutoHotkey Community
It is currently May 26th, 2012, 12:17 pm

View unanswered posts | View active topics


Board index » AutoHotkey » Support

All times are UTC [ DST ]



Post new topic Reply to topic  Page 1 of 1
  [ 5 posts ] 
  Print view | E-mail friend Previous topic | Next topic 
Author Message
Albireo
PostPosted: September 12th, 2008, 9:51 pm 
Offline

Joined: February 1st, 2006, 4:37 pm
Posts: 365
My Infil looks like: (17 fields and about 30000 rows)

"aaaaa";"abcdes";"asdfghjkl";"asdfdsa";"";"";"32323";....;"234323"
"bbb";"abcde123";"";"";"sdfsa";.......;""

And so on

Why does it take about 90sek on my Thinkpad to run
I think it's about 5500 rows I clean with this function.
After all I wrote a logfile with information / statistic
about the cleaning

Now I have this 8 rules to clean,
maybe It can be more or less :D

Code:
SetBatchLines, -1
Process, Priority,, High
Fields = 17

AllRows := Clean(Infil_Out, Fields)

ExitApp

; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
; - - - - -   Function Clean    - - - - - - - - - - - - - -
; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Clean(Fil_In, FieldNo)
   {
   Global

   Loop parse, Fil_In, `n, `r
      {
      RowOut_%A_Index% := A_LoopField
      ArtRow := RowOut_%A_Index%   ; More easy to use
      StringLen RowLength, ArtRow   
      StringSplit Field_, ArtRow, `;, %A_Space%%A_TAB%`"
      
      ; ----------------------------------------------------------
      ; 1. Clean empty rows -(maybe i don't need this control)
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      If RowLength = 0   {
         Log_1 := Log_1 "Some log 1 info"      
         Count_1 ++
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
      
      ; ----------------------------------------------------------
      ; 2. Clean Rows more then 423 char
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      If RowLength > 423   {
         Log_2 := Log_2 "Some log 2 info"
         Count_2 ++
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
      
      ; ----------------------------------------------------------
      ; 3. Clean Rows with not 17 fields
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      IfNotEqual Field_0, %FieldNo%   {
         Log_3 := Log_3 "Some log 3 info"
         Count_3 ++
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
      
      ; ----------------------------------------------------------
      ; 4. If Field_8 = Empty - Clean
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      If StrLen(Field_8) = 0   {
         Log_4 := Log_4 "Some log 4 info"
         Count_4 ++
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
      
      ; ----------------------------------------------------------
      ; 5. If Field_8 => more than nine chars - Clean
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      If StrLen(Field_8) > 9   {
         Log_5 := Log_5 "Some log 5 info"
         Count_5 ++
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
      
      ; ----------------------------------------------------------
      ; 6. If Field_2 empty - Clean
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      If StrLen(Field_2) = 0   {
         Log_6 := Log_6 "Some log 6 info"
         Count_6 ++
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
      
      ; ----------------------------------------------------------
      ; 7. If Field_1 empty and Field_3 not empty - copy 3 to 1
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      If (StrLen(Field_1)=0) and (StrLen(Field_3)=0)   {
         Log_7 := Log_7 "Some log 7 info"
         Count_7 ++
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
      
      ; ----------------------------------------------------------
      ; 8. If Field_1 more then 13 char - clean
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      StringLen, RowTemp, Field_1
      If RowTemp > 13   {
         Log_8 := Log_8 "Some log 8 info"
         Count_8 ++
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
      
      
      ArtNr ++
      NewRow_%ArtNr% = %ArtRow%
      }
   
   Loop %ArtNr%
      AllRows .= NewRow_%A_Index% . ( A_Index < ArtNr ? "`n" : "" )
   
   Return AllRows   
   }
; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -


Report this post
Top
 Profile  
Reply with quote  
wrecklass
 Post subject:
PostPosted: September 13th, 2008, 1:38 am 
Offline

Joined: March 19th, 2007, 5:12 pm
Posts: 29
Honestly, I'm surprised this takes only 90 seconds. I don't see an obvious way to speed it up. The big problem is that AHK doesn't handle Arrays very efficiently. Your script is creating thousands of individual variables which AHK must keep referenced throughout the process until you output the final variable.

If you REALLY want to speed this up, I'd move to a system that has better array manipulation capabilities. Perl, Python or Ruby would be better choices for this kind of data parsing.


Report this post
Top
 Profile  
Reply with quote  
Albireo
 Post subject:
PostPosted: September 13th, 2008, 9:10 am 
Offline

Joined: February 1st, 2006, 4:37 pm
Posts: 365
(excuse my bad english)
I have done this before with Autohotkey (Export from program).
Then I used Python with convert.
After that I used Autohotkey to import the result to another program.
To the end I want to create some result (maybe an chart or something to show what happend)
I can't use Python to control the autoexport/import from, and to another program.

Today I don't handled any logfile. Rows disappeared "without" control, other rows was modify "without" any information. what happend.....?

Now I thought - I do everything with autohotkey - Maybe Autohotkey have some limit I can't handle or my computer or ..... but, I hope the hole convert / file control doesn't take more than 10-15min. (this is only a small part of the hole convert - all moment). (today it takes about 1,5hour, but I have an old win98 computer. Now I want to do much more thing, have better control)

Have get some help with other part - this part takes about 25% of the entire time

I want to run "everything" before I create the logfile/informations file.
In the header I can have some statistic information and so on.

To handle the logfile I think it was more easy to do everything in the same script.

Thank's anyway

:)


Report this post
Top
 Profile  
Reply with quote  
Albireo
 Post subject:
PostPosted: September 13th, 2008, 9:21 am 
Offline

Joined: February 1st, 2006, 4:37 pm
Posts: 365
I forget to say.
Most of the control ( If....... ) is not true.
(maybe it take same time to do?).


Report this post
Top
 Profile  
Reply with quote  
Albireo
 Post subject:
PostPosted: September 13th, 2008, 6:50 pm 
Offline

Joined: February 1st, 2006, 4:37 pm
Posts: 365
I have solved the "problem"
Now the complete run is taken about 2-3 sek. :lol: :lol:
(insted of 90 seconds)

I have changed the code. To make the logfile is taken longer time (I don't know yet how long that was taken :) )

Insted to give logfile the real value at once I save important information in indexed variables - Finaly I must create the logfile, maybe it goes faster than now.

Code:
SetBatchLines, -1
Process, Priority,, High
Fields = 17

AllRows := Clean(Infil_Out, Fields)

ExitApp

; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
; - - - - -   Function Clean    - - - - - - - - - - - - - -
; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
Clean(Fil_In, FieldNo)
   {
   Global

   Loop parse, Fil_In, `n, `r
      {
      RowOut_%A_Index% := A_LoopField
      ArtRow := RowOut_%A_Index%   ; More easy to use
      StringLen RowLength, ArtRow   
      StringSplit Field_, ArtRow, `;, %A_Space%%A_TAB%`"
     
      ; ----------------------------------------------------------
      ; 1. Clean empty rows -(maybe i don't need this control)
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      If RowLength = 0
         {
         Count_1 ++
         Log1_%Count_1% := ArtRow
         Log1_Row%Count_1% := A_Index
         Continue   ; Jump to Next Row
         }
      ; ----------------------------------------------------------
     
      ; ----------------------------------------------------------
      ; 2. Clean Rows more then 423 char
      ; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
      If RowLength > 423
         {
         Count_2 ++
         Log2_%Count_2% := ArtRow
         Log2_Row%Count_2% := A_Index
         }
      ; ----------------------------------------------------------
     
  And so on

; -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -



Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 1
  [ 5 posts ] 

Board index » AutoHotkey » Support

All times are UTC [ DST ]

Who is online

Users browsing this forum: BrandonHotkey, kkkddd1, Morpheus, RUBn, sks, StepO, Yahoo [Bot] and 23 guests

You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group