AutoHotkey Homepage AutoHotkey Community
Let's help each other out
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

DOUBLE-speedUp for very large lists > 1000 items

 
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Scripts & Functions
View previous topic :: View next topic  
Author Message
wOxxOm



Joined: 09 Feb 2006
Posts: 319
PostPosted: Thu Mar 09, 2006 2:42 am    Post subject: DOUBLE-speedUp for very large lists > 1000 items Reply with quote
I was lucky to realize that there is no need to form ListView contents in two steps (first-collecting 3 MatchLists, second-feeding them into LV).
So the solution is to maintain just three integer indices and to insert to ListView directly:
Code:

      ; Advanced Search
      MatchPList1 =
      MatchPList2 =
      MatchPList3 =
      r1:=1   ; following three lines added
      r2:=1
      r3:=1


then in first match incrementing all three indices to shift insertion points for match 1/2/3
Code:

         StringLeft, LText, FName, %Len%

         ;Matching leftmost text
         If CompareString(LText,CurrTextU)=2 ; dllCall wrapper for winApi non-US locale-friendly comparing function
         {  MatchPList1=%MatchPList1%|%CurrItem%
            AddToList(CurrItem,1,r1) ; further expanded
            r1 ++
            r2 ++
            r3 ++
            Continue
         }


then in second match incrementing second & third indices to shift insertion points for match 2 & 3

Code:

         IfEqual, MatchFound, Y
         {  MatchPList2=%MatchPList2%|%CurrItem%
            AddToList(CurrItem,1,r2)
            r2 ++
            r3 ++
            Continue
         }


then in third match we increment only third index to shift insertion point for match 3

Code:

         IfEqual, MatchFound, Y
         {  MatchPList3=%MatchPList3%|%CurrItem%
            AddToList(CurrItem,1,r3)
            r3 ++
            Continue
         }
      }
      ; of course after finishing this loop, we don't need to repopulate listView, so nothing to do ;-):

      IfEqual, Count,
         LV_Delete()
   }

   ;post results
   Results:=LV_GetCount()
   GuiControl,, Results, Results=%Results%
   LV_ModifyCol()


And due to sparsing insertion of items I made a dedicated function for updating ListView:
Code:

compareString(s1,s2) ; 1-less, 2-equal, 3-greater
{ ; LOCALE_USER_DEFAULT=0x400, NORM_IGNORECASE=1, resultEQUAL=2
   return dllCall("CompareString",int,0x400,int,1,str,s1,int,strLen(s1),str,s2,int,strLen(s2))
}

AddToList(item,ResultsGUI=0,row=0)
{  global count,showIcons,mainWnd,ImageListID1,Results
   Count ++
   SplitPath, Item, , FDir, FExt, FNameNoExt
   IfEqual, ShowIcons, 1
   { hIcon:=DllCall("Shell32\ExtractAssociatedIconA", Int,0, Str,Item, UShortP,iIndex)
      DllCall("ImageList_ReplaceIcon", UInt,ImageListID1, Int,-1, UInt,hIcon)
      DllCall("DestroyIcon", Uint, hIcon)
   }

   if (row=0)
      LV_Add("Icon" Count, FNameNoExt, FExt, fsize, mtime, FDir)
   else
      LV_Insert(row,"Icon" Count, FNameNoExt, FExt, fsize, mtime, FDir)
   if (Count<20)
   {  If (Count=1)
         ControlSend, SysListView321, {Down}, %MainWnd%
      else
         LV_ModifyCol()
   }
   if ResultsGUI
      GuiControl,, Results, Results=%Count%
}


And one little speed-up for keyboard interface:-) here it is:
Code:

HotKey,ifWinActive,%mainWnd%
HotKey,~Up,ScrollLineUp
HotKey,~Down,ScrollLineDown
HotKey,~PgUp,ScrollPageUp
HotKey,~PgDn,ScrollPageDown
HotKey,~^Home,ScrollFirst
HotKey,~^End,ScrollLast
HotKey,~*WheelUp,ScrollWheelUp
HotKey,~*WheelDown,ScrollWheelDown
HotKey,^Del,keyControlDel

;............... ripped
LastText=fadsfSDFDFasdFdfsadfsadFDSFDf
;............... ripped

ScrollLineUp:
   ControlSend, SysListView321, {Up}, %MainWnd%
   return
ScrollPageUp:
   ControlSend, SysListView321, {PgUp}, %MainWnd%
   return
ScrollLineDown:
   ControlSend, SysListView321, {Down}, %MainWnd%
   return
ScrollPageDown:
   ControlSend, SysListView321, {PgDn}, %MainWnd%
   return
ScrollFirst:
   ControlSend, SysListView321, {Home}, %MainWnd%
   return
ScrollLast:
   ControlSend, SysListView321, {End}, %MainWnd%
   return
ScrollWheelUp:
   ControlSend, SysListView321
      , % iif(GetKeyState("LShift"),"{PgUp}","{Up}"), %MainWnd%
   return
ScrollWheelDown:
   ControlSend, SysListView321
      , % iif(GetKeyState("LShift"),"{PgDn}","{Down}"), %MainWnd%
   return

iif(expr, a, b)
{
   if (expr)
      return a
   else
      return b
}



So to conclude and summarize benefits:
1. Speed of gathering ListView contents doubled
2. During search process results in GUI are updated on-the-fly (with column resizing for first 20 results - just what is visible on screen) and you will see first results in 1000+ file-box just a millesecond after beginning of search process Wink This means you can decrease reaction delay from 100 to 10 ms AND IT WILL BE GREAT and IT IS!!!! oh yes. try it.
3. Using CompareString we catch all filenames in national codepage characters regardless of letter case in typed string
4. HotKeys now not system-wide, and you may use more: PgUp/Dn, Shift-WheelUp/Down, Ctrl-Home/End Wink
Back to top
View user's profile Send private message Send e-mail Visit poster's website
wOxxOm



Joined: 09 Feb 2006
Posts: 319
PostPosted: Thu Mar 09, 2006 2:58 am    Post subject: Ghhmm my previous estimate was underestimated ;-) 10times!!! Reply with quote
Ghhmm my previous estimate was underestimated Wink 10times(!!!) faster!!! Wink Oh yeah.

P.S. I mean of course search result of about 1000-2000 and more Wink
Back to top
View user's profile Send private message Send e-mail Visit poster's website
robiandi
Guest
PostPosted: Thu Mar 09, 2006 6:46 am    Post subject: Reply with quote
How about a little demo: copy, paste, run !
( a demo with 3 items, for example)
Back to top
wOxxOm



Joined: 09 Feb 2006
Posts: 319
PostPosted: Thu Mar 09, 2006 7:12 am    Post subject: Deferred icon loading implementation by wOxxOm Reply with quote
Now I changed the script in this way:
1. Listbox is fully filled only with fname/fdir/ext (in one pass - see prev.posts). Items beginning with text entered will fetch icon 'cause there are most certainly very few of them.
2. Timer is launched.
3. Timer procedure works for not more than 100ms and on my PC it manages to get about 100 icons during this slice. Proc gets icons, fileSize & date/time, sets listview icon & info, updates columns for first visible 50 results
4. iconCache implemented: icons for TXT/CDR/MP3 and other frequently repeated files with constant icons keep icon handle in memory and not requeried on every file

P.S. on a list with 14'333 lines it finds 10'000 matches in 20.6 secs, with most icons, unique icons get rendered in a 1/4 sec on first page
on the same list (14'333 files) it finds now 18 matches in 1.5 sec
I will debug it carefully now, and if someone needs such capabilities I will post my version here.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Chris
Site Admin


Joined: 02 Mar 2004
Posts: 10471
PostPosted: Thu Mar 09, 2006 10:53 am    Post subject: Reply with quote
Impressive posts. Thanks for presenting your findings with such clarity and organization.
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic    AutoHotkey Community Forum Index -> Scripts & Functions All times are GMT
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum


Powered by phpBB © 2001, 2005 phpBB Group