AutoHotkey Community

Posted: **19 Jun 2019, 12:33**

I have a string that I create an array from. (like this) .:

InputCSV = 
( LTrim
	`n
	"201","Freddy","4,50"
	"302","Emelie","174,50"
	"203","Clas","9,90"
	
	"904","Johanna","1400,00"
	"205","Erica","1,75"
	
)
InputData := StrSplit(InputCSV, "`n")

MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % InputCSV "`n`n" InputData.Count() "`n- " InputData[1] "`n- " InputData[2] "`n- " InputData[3] "`n- " InputData[4] "`n- " InputData[5] "`n- " InputData[6]
Count = 0
Loop % InputData.Count()
{	If ( StrLen(InputData[A_Index]) < 1 )
		InputData.Delete(A_Index)
	else
		Count += 1
}
MsgBox ,, Rad %A_LineNumber% -> %A_ScriptName%, % InputCSV "`n`n" InputData.Count() "`n- " InputData[1] "`n- " InputData[2] "`n- " InputData[3] "`n- " InputData[4] "`n- " InputData[5] "`n- " InputData[6]

There are empty rows in that string above (InputCSV). These should be deleted - along with the object keys.
But… InputData[1] is empty before InputData.Delete(1) and even after - why?
InputData.Count() is "9" before InputData.Delete() and "5" after (it's correct)

I have made an attempt with what I believe the AHK manual say about Object.delete()
But it doesn't work, as I want.
How can this be solved?

Posted: **19 Jun 2019, 12:48**

Is this what you're trying to do?

Code: Select all

Index:=InputData.Length()
Loop,% InputData.Length()	{
	If( StrLen(InputData[Index]) < 1 )
		InputData.RemoveAt(Index)
	
	Index--
}

Or if you want to save space.

Code: Select all

Loop,% Index:=InputData.Length()
	(StrLen(InputData[Index])< 1)?(InputData.RemoveAt(Index--)):(Index--)

Posted: **19 Jun 2019, 13:41**

Thank you!

Hellbent wrote: ↑
19 Jun 2019, 12:48
... Or if you want to save space...

Save Space? (In the array or AHK-program?)
I get the result I want, in both examples.

What does this instruction do? Index--?

Posted: **19 Jun 2019, 14:20**

Albireo wrote: ↑
19 Jun 2019, 13:41
Thank you!

No problem.

Albireo wrote: ↑
19 Jun 2019, 13:41

Hellbent wrote: ↑
19 Jun 2019, 12:48
... Or if you want to save space...
Save Space? (In the array or AHK-program?)
I get the result I want, in both examples.

Just reduce the number of lines of code.
If it didn't add that in someone else would surly be along shortly to do it lol.

Albireo wrote: ↑
19 Jun 2019, 13:41
What does this instruction do? Index--?

That is so that I can loop through the array backwards.
If you don't, when you remove a element, the one ahead of it gets shifted and fills that now vacant spot and ends up getting missed.

Posted: **20 Jun 2019, 04:21**

If you don't want to store empty lines in the array, don't store them:

Code: Select all

#NoEnv
InputCSV =
(LTrim
	`n
	"201","Freddy","4,50"
	"302","Emelie","174,50"
	"203","Clas","9,90"

	"904","Johanna","1400,00"
	"205","Erica","1,75"

)

; --------------------------------------
InputData := []
DataIndex := 0
For LineIndex, Line In StrSplit(InputCSV, "`n")
{
   LineCount := LineIndex
   If StrLen(Line)
   {
      DataIndex += 1
      InputData[DataIndex] := Line
   }
}
; --------------------------------------

DataLines := ""
For Each, Line In InputData
   DataLines .= "`n" Line
MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % LineCount . "`n" . InputCSV . "`n`n`n" . InputData.Count() . DataLines

Posted: **20 Jun 2019, 16:20**

Thanks for the suggestion. All tips give the same results
Have done an analysis on all three examples.
I can't find any difference in performance with the suggestions from @Hellbent

Two CSV-files are used in the test,
- File1 with 25,622 rows (20 columns and 6172kB) and
- File2 with 34,658 rows (72 columns 6638kB)

The test was done as follows:
- Start the clock
- Load a CSV file
- Create an array
- Stop the clock

The result .: File1.: File2.:
@Helbent .: (94ms) (125ms)
@just me .: (2937ms) (3297ms)

Was very surprised that there was such a big difference.
I don't understand why!

Posted: **20 Jun 2019, 16:31**

The test was done as follows:
- Start the clock
- Load a CSV file
- Create an array
- Stop the clock
...
Was very surprised that there was such a big difference.
I don't understand why!

Nice, but I'm not able to run your 'script'. So I cannot help you.

Posted: **20 Jun 2019, 18:59**

reserve enough capacity for the array to hold 35k rows
push into the array with .Push, not via array index access
if strlen(line) to check whether a line is empty(assuming that was the intention) is wasteful

Posted: **21 Jun 2019, 00:54**

Albireo wrote: ↑
20 Jun 2019, 16:20
Have done an analysis on all three examples.

How about this one ?

Code: Select all

Loop, % InputData.Length()
	(z:=InputData.Pop()) ? (InputData.InsertAt(1,z))

Posted: **21 Jun 2019, 03:37**

swagfag wrote: ↑
20 Jun 2019, 18:59
reserve enough capacity for the array to hold 35k rows...

How do you do it? Something like that? InputData.SetCapacity(35000)

swagfag wrote: ↑
20 Jun 2019, 18:59
...push into the array with .Push, not via array index access
if strlen(line) to check whether a line is empty(assuming that was the intention) is wasteful.

If you do not use StrLen(),how do you know which line is "empty"?
(Even a line that contains only field delimiters is empty and should be removed.)

The suggestion from @SpeedMaster give the following run times .:
File1 .: 3391ms
File2 .: 6313ms

Posted: **21 Jun 2019, 04:06**

Your benchmarking results are less meaningful if you don't show your complete benchmark script.

Posted: **21 Jun 2019, 04:12**

an empty line is "". anything else is by definition not empty. if u have redefined empty to mean something else, then simply checking against "" won't work. a different discriminator is required.

Posted: **21 Jun 2019, 04:47**

I just ran my own benchmark. Reading a CSV file with ~20000 rows of 18 columns (~3 MB) and converting it to an array using my code needs < 100 ms here.

Posted: **21 Jun 2019, 18:25**

Below is the AHK code I used.
To perform the tests I chose the input file, and configured Goto to different test codes

Unfortunately, I will not share the test files, but create your own files. The time differences will also depend on the type of computer the test takes place.

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
#Persistent
#SingleInstance Force
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.

StartTime := A_TickCount
; FileRead InputCSV, c:\temp\FDT_ART_100.txt	; File1
FileRead InputCSV, c:\temp\EXPOART.txt		; File2

Goto Example4	; Example 1-4
ExitApp

Example1:	; Test1 @Helbent
	InputData := StrSplit(InputCSV, "`n")
	Loop % Index := InputData.Length()
		( StrLen(InputData[Index]) < 1 )?(InputData.RemoveAt(Index--)):(Index--)

	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex1 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp


Example2:	; Test @Helbent
	InputData := StrSplit(InputCSV, "`n")
	Index:=InputData.Length()
	Loop,% InputData.Length()	{
		If( StrLen(InputData[Index]) < 1 )
			InputData.RemoveAt(Index)
	Index--
	}
	
	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex2 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp


Example3:	; Test @just me
	InputData := []
	DataIndex := 0
	For LineIndex, Line In StrSplit(InputCSV, "`n")
	{	LineCount := LineIndex
		If StrLen(Line)
		{	DataIndex += 1
			InputData[DataIndex] := Line
		}
	}
	DataLines := ""
	For Each, Line In InputData
		DataLines .= "`n" Line
	
	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex3 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp


Example4:	; Test @SpeedMaster
	InputData := StrSplit(InputCSV, "`n")
	Loop, % InputData.Length()
		(z:=InputData.Pop()) ? (InputData.InsertAt(1,z))

	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex4 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp

Posted: **21 Jun 2019, 19:04**

Albireo wrote: ↑
21 Jun 2019, 18:25
Below is the AHK code I used.
To perform the tests I chose the input file, and configured Goto to different test codes

I ran your test. Here is my script.

Code: Select all

/*

#SingleInstance,Force
SetBatchLines,-1
FileDelete,%A_ScriptDir%\Test File.txt
temp:=""
Loop, 10000	{
	if(Mod(A_Index,4)=0){
		temp.= "`n"
	}else	{
		Temp .= "Some Random Text`n"
		
	}

}
FileAppend,%Temp%,%A_ScriptDir%\Test File.txt
SoundBeep, 500
return
*ESC::ExitApp

*/


#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
#Persistent
#SingleInstance Force
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.

StartTime := A_TickCount
; FileRead InputCSV, c:\temp\FDT_ART_100.txt	; File1
FileRead InputCSV, %A_ScriptDir%\Test File.txt		; File2

Goto Example4	; Example 1-4     ;<--------- ran each 10 times to get fastest and longest times
ExitApp

Example1:	; Test1 @Helbent   ;<-------------------- 31 - 63 ms
	InputData := StrSplit(InputCSV, "`n")
	Loop % Index := InputData.Length()
		( StrLen(InputData[Index]) < 1 )?(InputData.RemoveAt(Index--)):(Index--)

	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex1 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp


Example2:	; Test @Helbent    ;<-------------------- 31 - 93 ms
	InputData := StrSplit(InputCSV, "`n")
	Index:=InputData.Length()
	Loop,% InputData.Length()	{
		If( StrLen(InputData[Index]) < 1 )
			InputData.RemoveAt(Index)
	Index--
	}
	
	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex2 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp


Example3:	; Test @just me       ;<-------------------- 125 - 157 ms
	InputData := []
	DataIndex := 0
	For LineIndex, Line In StrSplit(InputCSV, "`n")
	{	LineCount := LineIndex
		If StrLen(Line)
		{	DataIndex += 1
			InputData[DataIndex] := Line
		}
	}
	DataLines := ""
	For Each, Line In InputData
		DataLines .= "`n" Line
	
	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex3 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp


Example4:	; Test @SpeedMaster     ;<-------------------- 1938 - 2063    ms
	InputData := StrSplit(InputCSV, "`n")
	Loop, % InputData.Length()
		(z:=InputData.Pop()) ? (InputData.InsertAt(1,z))

	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex4 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp

I ran each one 10 times and recorded the fastest and slowest times.

Example 1: 31 - 63 ms
Example 2: 31 - 93 ms
Example 3: 125 - 157 ms
Example 4: 1938 - 2063 ms

***Edit***
Closer inspection of the strlen showed that min length was 1.

New results:
Example 1: average 250 ms
Example 2: average 250 ms
Example 3: average 125 ms

Posted: **22 Jun 2019, 03:44**

@Albireo:

Code: Select all

Example3:	; Test just me
	InputData := []
	DataIndex := 0
	For LineIndex, Line In StrSplit(InputCSV, "`n")
	{	LineCount := LineIndex
		If StrLen(Line)
		{	DataIndex += 1
			InputData[DataIndex] := Line
		}
	}
	DataLines := ""
	For Each, Line In InputData
		DataLines .= "`n" Line

	StopTime := A_TickCount
	CountTime := StopTime - StartTime
	MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Ex3 `nNumber of rows .: " InputData.Count()"`nRunTime .: " CountTime " ms"
ExitApp

You kept the test code rebuilding a string from the array.

Code: Select all

	DataLines := ""
	For Each, Line In InputData
		DataLines .= "`n" Line

It consumes most of the time if the string gets really large.

Posted: **22 Jun 2019, 05:13**

swagfag wrote: ↑
21 Jun 2019, 04:12
an empty line is "". anything else is by definition not empty. if u have redefined empty to mean something else, then simply checking against "" won't work. a different discriminator is required.

I agree with you, but in my world even a row with only field dividers can be empty. These "empty" lines occur when a tab with data in Excel, OpenOffice calc / LibreOffice calc, is processed with formulas into another tab and this tab is exported to a file with CSV structure.
All cells that lack content in the first tab will be empty cells in the second tab and only the field delimiters is exported for this rows.

e.g. Tab 1 contains 3 columns and 200 rows of data. Tab 2 retrieves data from Flik1 with formulas and performs something.
But in Tab 2, there are 500 rows of formulas (with data from Tab 1).
When Tab 2 is exported to a CSV file, the first 200 rows will contain data, while the 300 following rows will only contain field separators. The same thing happens if empty rows are inserted in the data on Tab 1
Because this happens quite often to me, I see these lines as empty and should be handled and skipped in the CSV-file, when the file is read by AHK.

Posted: **22 Jun 2019, 05:55**

Code: Select all

Cleaned := []
Cleaned.SetCapacity(BIG NUMBER)
emptyLineOrJustCommas := "^(?:,*)$"

Loop Parse, csvFile, `n
{
	if !(A_LoopField ~= emptyLineOrJustCommas)
		Cleaned.Push(A_LoopField)
}

Posted: **30 Jun 2019, 05:23**

swagfag wrote: ↑

22 Jun 2019, 05:55

Code: Select all

Cleaned := []
Cleaned.SetCapacity(BIG NUMBER)
emptyLineOrJustCommas := "^(?:,*)$"

Loop Parse, csvFile, `n
{
	if !(A_LoopField ~= emptyLineOrJustCommas)
		Cleaned.Push(A_LoopField)
}

Thanks for your tip, but I don't understand what happens

Specially on line .: emptyLineOrJustCommas := "^(?:,*)$" and maybe if !(A_LoopField ~= emptyLineOrJustCommas)

If I understand correctly, all rows that are empty or just consist of "," (comma) are ignored
But how to control other separators? (like ";" "TAB" or ...)?
Can these be checked in the same code?

AutoHotkey Community

Delete Object key - How?

Delete Object key - How?

Re: Delete Object key - How? Topic is solved

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?

Re: Delete Object key - How?