 |
AutoHotkey Community
Let's help each other out
|
| View previous topic :: View next topic |
| Author |
Message |
CannedCheese
Joined: 21 May 2008
Posts: 85
|
 Posted: Wed Aug 06, 2008 2:21 am Post subject: Sort INI File By Section (w/ Script) - Ye Olde 2D Array Prob |
 |
|
The following is a working slow script that will read the following ini file and sort it alphabetically by username...
PlayerDB_Input.ini
| Code: |
; Save this file as PlayerDB_Input.ini
[Jake]
points=15,599
allowance=$1
salary=$12
completed=12%
earnings=$18,870
date=20080731
[Joe]
points=5,622
allowance=$2
salary=$13
completed=11%
earnings=$8,479
date=20080731
[GMan]
points=40,827
allowance=$4
salary=$57
completed=10%
earnings=$177,055
date=20080731
[Bazooka]
points=7,629
allowance=$2
salary=$15
completed=15%
earnings=$18,257
date=20080731
[d-j]
points=13,580
allowance=$1
salary=$23
completed=6%
earnings=$14,887
date=20080731
[chris]
points=24,284
allowance=$2
salary=$15
completed=11%
earnings=$45,256
date=20080731
[fund]
points=9,996
allowance=$2
salary=$10
completed=19%
earnings=$18,988
date=20080731
[Jake1]
points=15,599
allowance=$1
salary=$12
completed=12%
earnings=$18,870
date=20080731
[Joe1]
points=5,622
allowance=$2
salary=$13
completed=11%
earnings=$8,479
date=20080731
[GMan1]
points=40,827
allowance=$4
salary=$57
completed=10%
earnings=$177,055
date=20080731
[Bazooka1]
points=7,629
allowance=$2
salary=$15
completed=15%
earnings=$18,257
date=20080731
[d-j1]
points=13,580
allowance=$1
salary=$23
completed=6%
earnings=$14,887
date=20080731
[chris1]
points=24,284
allowance=$2
salary=$15
completed=11%
earnings=$45,256
date=20080731
[fund1]
points=9,996
allowance=$2
salary=$10
completed=19%
earnings=$18,988
date=20080731
[Jake1]
points=15,599
allowance=$1
salary=$12
completed=12%
earnings=$18,870
date=20080731
[Joe1]
points=5,622
allowance=$2
salary=$13
completed=11%
earnings=$8,479
date=20080731
[GMan1]
points=40,827
allowance=$4
salary=$57
completed=10%
earnings=$177,055
date=20080731
[Bazooka1]
points=7,629
allowance=$2
salary=$15
completed=15%
earnings=$18,257
date=20080731
[d-j1]
points=13,580
allowance=$1
salary=$23
completed=6%
earnings=$14,887
date=20080731
[chris1]
points=24,284
allowance=$2
salary=$15
completed=11%
earnings=$45,256
date=20080731
[fund1]
points=9,996
allowance=$2
salary=$10
completed=19%
earnings=$18,988
date=20080731
[Jake11]
points=15,599
allowance=$1
salary=$12
completed=12%
earnings=$18,870
date=20080731
[Joe11]
points=5,622
allowance=$2
salary=$13
completed=11%
earnings=$8,479
date=20080731
[GMan11]
points=40,827
allowance=$4
salary=$57
completed=10%
earnings=$177,055
date=20080731
[Bazooka11]
points=7,629
allowance=$2
salary=$15
completed=15%
earnings=$18,257
date=20080731
[d-j11]
points=13,580
allowance=$1
salary=$23
completed=6%
earnings=$14,887
date=20080731
[chris11]
points=24,284
allowance=$2
salary=$15
completed=11%
earnings=$45,256
date=20080731
[fund11]
points=9,996
allowance=$2
salary=$10
completed=19%
earnings=$18,988
date=20080731
[Jake11]
points=15,599
allowance=$1
salary=$12
completed=12%
earnings=$18,870
date=20080731
[Joe11]
points=5,622
allowance=$2
salary=$13
completed=11%
earnings=$8,479
date=20080731
[GMan11]
points=40,827
allowance=$4
salary=$57
completed=10%
earnings=$177,055
date=20080731
[Bazooka11]
points=7,629
allowance=$2
salary=$15
completed=15%
earnings=$18,257
date=20080731
[d-j11]
points=13,580
allowance=$1
salary=$23
completed=6%
earnings=$14,887
date=20080731
[chris11]
points=24,284
allowance=$2
salary=$15
completed=11%
earnings=$45,256
date=20080731
[fund11]
points=9,996
allowance=$2
salary=$10
completed=19%
earnings=$18,988
date=20080731
[Jake111]
points=15,599
allowance=$1
salary=$12
completed=12%
earnings=$18,870
date=20080731
[Joe111]
points=5,622
allowance=$2
salary=$13
completed=11%
earnings=$8,479
date=20080731
[GMan111]
points=40,827
allowance=$4
salary=$57
completed=10%
earnings=$177,055
date=20080731
[Bazooka111]
points=7,629
allowance=$2
salary=$15
completed=15%
earnings=$18,257
date=20080731
[d-j111]
points=13,580
allowance=$1
salary=$23
completed=6%
earnings=$14,887
date=20080731
[chris111]
points=24,284
allowance=$2
salary=$15
completed=11%
earnings=$45,256
date=20080731
[fund111]
points=9,996
allowance=$2
salary=$10
completed=19%
earnings=$18,988
date=20080731
|
This code will sort it and write the sorted INI File to disk...
| Code: |
#SingleInstance Force
#Persistent
BeginningTime := A_TickCount / 1000
inireadfile := "PlayerDB_Input.ini" ; file to read from
iniwritefile := "PlayerD_Sorted.ini" ; file to write to (append to)
filedelete % iniwritefile
FileRead, MyFile, % IniReadFile
SetBatchLines -1
Loop, Parse, MyFile, `n
{
if SubStr(A_LoopField,1,1) = "["
{
NameList := Namelist . SubStr(A_LoopField, 2, StrLen(A_LoopField)-3) . "`n"
}
}
Sort, NameList, U
;msgbox % NameList
loop, Parse, NameList, `n
{
if (StrLen(A_LoopField) < 1)
break
iniread, points,% IniReadFile, % A_LoopField,points
iniwrite, % points, % iniwritefile, % A_LoopField,points
iniread, allowance,% IniReadFile, % A_LoopField,allowance
iniwrite, % allowance, % iniwritefile, % A_LoopFIeld,allowance
iniread, salary,% IniReadFile, % A_LoopField,salary
iniwrite, % salary, % iniwritefile, % A_LoopField,salary
iniread, completed,% IniReadFile, % A_LoopField,completed
iniwrite, % completed, % iniwritefile, % A_LoopField,completed
iniread, earnings,% IniReadFile, % A_LoopField,earnings
iniwrite, % earnings, % iniwritefile, % A_LoopField,earnings
iniread, date,% IniReadFile, % A_LoopField,date,20080731
iniwrite, % date, % iniwritefile, % A_LoopField,date
iF !(A_LoopField = "" )
This_IniList := "[" A_LoopField "]`npoints=" points "`nallowance=" allowance "`nsalary=" salary "`ncompleted" completed "`nearnings=" earnings "`ndate=`" date "`n"
IniList := IniList . This_iniList
}
EndTime := ((A_TickCount / 1000) - BeginningTime)
run notepad.exe %iniwritefile%
msgbox,,Players_Stats,% EndTime " Seconds"
|
The code is reeeeeaaaaaallllllyyyy slow when large files are involved. I've written a different routine which builds an index off of the sort routine, and then attempts to parse the variable that the file is loaded into via this index, but this seems even slower.
There's a few posts on sorting 2D arrays, but I'm curious if anyone has any ideas about applying this to an INI file. I'd love to hear some thoughts and I'll be posting my slow index code when I get home again. |
|
| Back to top |
|
 |
engunneer
Joined: 30 Aug 2005
Posts: 6847
Location: Pacific Northwest, US
|
| Posted: Wed Aug 06, 2008 3:02 am Post subject: |
|
|
sorry to throw a wrench into the works, but why do you need to sort the ini file? that's not normally done. There are a few scripts for importing the entire file quite quickly into your script, then you may access any variable you want.
_________________
 Unless otherwise noted, all code is untested.
Common Answers: 1.(Loops, Viruses, etc.) 2. Search 3.RTFM |
|
| Back to top |
|
|
[VxE]
Joined: 07 Oct 2006
Posts: 1496
|
| Posted: Wed Aug 06, 2008 3:30 am Post subject: |
|
|
I presume that sorting the ini is meant to remove duplicates (no, I have no idea why).
Anyways, here's an overhaul that should be much faster at rebuilding the ini file: | Code: | #SingleInstance Force
#Persistent
sections = points/allowance/salary/completed/earnings/date
BeginningTime := A_TickCount / 1000
inireadfile := A_Desktop "\PlayerDB_Input.ini" ; file to read from
iniwritefile := A_Desktop "\PlayerD_Sorted.ini" ; file to write to (append to)
filedelete % iniwritefile
FileRead, MyFile, % IniReadFile
SetBatchLines -1
Loop, Read, %IniReadFile%, %iniwritefile%
{
Loop, Parse, MyFile, `n, `r
{
If StrLen(A_LoopField) < 3 || InStr(A_LoopField, ";")
continue
StringSplit, loopvar, A_LoopField, =, %A_Space%`t[]
if InStr(A_LoopField, "[")
{
Loop, parse, A_LoopField, `n, []
NameList .= (loopname := Vok(loopvar2 := A_Loopfield)) "`n"
loopvar1 =
}
Array2D%loopname%%loopvar1% := loopvar2
}
Sort, NameList, U
FileAppend, `; %ErrorLevel% entries were removed as duplicates`n`n
Loop, PArse, Namelist, `n
{
If !A_LoopField
continue
name := A_LoopField
FileAppend, % "[" Array2D%name% "]`n"
Loop, Parse, Sections, /
FileAppend, % A_LoopField "=" Array2D%name%%A_LoopField% "`n"
}
break
}
;msgbox % NameList
EndTime := ((A_TickCount / 1000) - BeginningTime)
; run notepad.exe %iniwritefile%
msgbox,,Players_Stats,% EndTime " Seconds"
Vok(name) ; removes characters from the name that make it
{ ; an invalid variable name. This may need some chars added.
Loop, parse, name,, ``~!`%^&*(){}-=+`,./<>?`;:'"\|
newname .= A_LoopField
return newname
} |
_________________
My Home Thread
More Common Answers: [1]. It's in the FAQ [2]. Ternary ( a ? b : c ) guide [3]. Post code inside [code][/code] tags ! |
|
| Back to top |
|
|
CannedCheese
Joined: 21 May 2008
Posts: 85
|
| Posted: Thu Aug 07, 2008 2:41 am Post subject: |
|
|
| engunneer wrote: | | sorry to throw a wrench into the works, but why do you need to sort the ini file? that's not normally done. There are a few scripts for importing the entire file quite quickly into your script, then you may access any variable you want. |
I realize that you shouldn't normally need to sort an INI file, but I wrote an app that was using INI files as a cheap database and a way to avoid searching for duplicates. Problem is, sometimes I want to edit the INI file manually, and I thought it would be nice to keep things alphabetized. I will probably end up moving to a homegrown csv database with each new line containing the data of a particular person. This got me thinking of faster ways to access and write a basic homemade database. However, I love the fact that the INI file structure is very readable-- I figured that there had to be a way to access any variable by importing a file to a variable, only making one call to load the file, and then writing it back after all fields are done being accessed-- or in this case-- after it has been alphabetized. |
|
| Back to top |
|
|
CannedCheese
Joined: 21 May 2008
Posts: 85
|
| Posted: Thu Aug 07, 2008 8:02 am Post subject: |
|
|
@VxE - Your code is so fast it is ridiculous. I had to add A_Space to the end of your Vok function and everything worked great. Sorted a 13k person database in like 3 seconds on my slower work machine. Sick! I think I understand how it works-- I'm wondering if we could run into trouble if we had a name JoeBob as well as a Joe!Bob as the "!" character will be removed, correct? If so, is there a common way to use some kind of method to restore the name to its correct name (I think I use virtually all characters in names, except for maybe tab)? Still, in the 13k I tested it on, I didn't get one duplicate removed (which is good, since there were no duplicates in the source ini file).
Also, doesn't this create (13K names) x (#keys) variables? Before seeing your code, I had an idea to load the file into a variable, parsing it and grabbing every line that began with a "[" (every name), writing an index variable with "Name {tab} FILELINE", sorting it, and then doing a stringsplit on the original variable containing the entire file contents, looking up each value with something like..
| Code: |
#SingleInstance Force
#Persistent
msgbox Press OK to start!
BeginningTime := A_TickCount / 1000
inireadfile := "PlayerDB.ini" ; file to read from
iniwritefile := "PlayerDB_Sorted.ini" ; file to write to (append to)
FileRead, MyFile, % IniReadFile
SetBatchLines -1
Loop, Parse, MyFile, `n
{
if SubStr(A_LoopField,1,1) = "["
{
NameIndexList := NameIndexList . SubStr(A_LoopField, 2, StrLen(A_LoopField)-3) . A_Tab . A_Index . "`n"
}
}
Sort, NameIndexList, U
stringsplit This_File_Line, MyFile, `n ; create a billion variable array consisting of name {tab} index#
loop, parse, NameIndexList, `n ; e.g. Bob {tab} 12`n Jason {tab} 1, etc.
{
StringSplit Index, A_LoopField, %A_Tab%
StringSplit Name, This_file_line%Index2%, %A_Tab%
Index := Index2
Loop {
Key := This_File_Line%Index%
if ((substr(key,1,1) <> "[") || (Index = Index2)) && (strlen(key) > 0)
{
Index := Index2 + A_Index
NewFile := NewFile . Key . "`n"
;ThisSection .= Key . "`n"
}
else Break
}
;Index=
;Index2=
;ThisSection =
}
filedelete % iniwritefile
fileappend, % NewFile, % iniwritefile
EndTime := ((A_TickCount / 1000) - BeginningTime)
;run notepad.exe %iniwritefile%
msgbox,,Players_Stats,% EndTime " Seconds"
|
My code takes like 110 seconds, whereas yours takes 3. Dynamically assigning variable names is something that I've only briefly touched on while learning javascript-- I'll definitely have to look at it. I absolutely love how your code gives me access to any key under any name. My current INI file is now around 13k (of 6 or 7 keys each--not totally fixed # of subkeys for each name) and is slightly over 1 MB. Assuming that this file grows to 100x this, is it still feasible to name all of these variables in memory? I'd love to write or use a routine that works much faster than IniRead.
Really appreciate the help. I'm still amazed at how you've been able to incorporate into 1 line what would have taken me 4 in so many cases. |
|
| Back to top |
|
|
[VxE]
Joined: 07 Oct 2006
Posts: 1496
|
| Posted: Thu Aug 07, 2008 8:35 am Post subject: |
|
|
| CannedCheese wrote: | | Also, doesn't this create (13K names) x (#keys) variables? |
Yes, it does. But even if each variable holds a thousand characters (1Kb), it would still only take about (13000*8*1000 = 104 Mbs). You could scope those vars inside a function so that they only hog memory for a short time. If you have a typical amount of RAM (2G or more), you'll probably never notice the resource usage. One of the basic tenants of programming is that task = time * memory * processor so by using more memory (in the right way) you can reduce time.
I admit that the Vok() function is a hack (a fugly one too). You could rewrite it to translate illegal characters into legal-but-unique substrings (i.e. "{" into "LEGAL_SUBSTRING_OPEN_BRACKET" or "#LSS_OB#"). Since that function is only used to obtain variable name segments, tweaking it shouldn't effect the data.
Anyways, there are some more tech-wise people on these forums who may contradict the particulars of what I've said, but here's a golden rule: working in memory is always faster than accessing the hard disk. That's the biggest reason why my code is faster.
_________________
My Home Thread
More Common Answers: [1]. It's in the FAQ [2]. Ternary ( a ? b : c ) guide [3]. Post code inside [code][/code] tags ! |
|
| Back to top |
|
|
BoBo²
Guest
|
| Posted: Thu Aug 07, 2008 8:58 am Post subject: |
|
|
[OT]
Wouldn't it make sense to convert the INI into a csv? There are AHK [csv functions] available to work with those. Output should work with any data base. Even with the poor-mens-db-AKA-Excel. [CSV-Editor] for easy editing. ...
[/OT] |
|
| Back to top |
|
|
|
|
You can post new topics in this forum
You can reply to topics in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Register
Profile
Log in to check your private messages
Log in
