Count files - fastest way - use Topic is solved

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
Albireo
Posts: 1747
Joined: 16 Oct 2013, 13:53

Count files - fastest way - use

Post by Albireo » 03 Dec 2022, 05:30

Hi!
I want to count files in one directory. (it can be ?00,000 files)
(in this case, no subdirectories, but the files can be on a network drive)
Maybe - The obvious way
(Haven't checked how/if it works on a network drive)

Then I saw - maybe a faster (and better?) method Joel Lipman.Com - (from 2013)
But this doesn't work for me (I don't know why)

Code: Select all

UserFolder:="C:"

-- to return file count in a folder
ItemCount := ComObjCreate("Shell.Application").NameSpace(UserFolder).Items.Count
FileCount := ComObjCreate("Scripting.FileSystemObject").GetFolder(UserFolder).Files.Count
FolderCount := ItemCount - FileCount


-- quick if looking at one directory.
-- just as slow if looping for subdirectories:

Loop, %UserFolder%\*.*, 1, 1
{
        ThisFolder := RegExReplace( A_LoopFileFullPath, "\\$" )
        ItemCount := ComObjCreate("Shell.Application").NameSpace(ThisFolder).Items.Count
        FileCount := ComObjCreate("Scripting.FileSystemObject").GetFolder(ThisFolder).Files.Count
        FolderCount := ItemCount - FileCount
        TotalFileCount += FileCount
        TotalFolderCount += FolderCount
}

Maybe there are even better ways to count files in a directory today?

User avatar
mikeyww
Posts: 26602
Joined: 09 Sep 2014, 18:38

Re: Count files - fastest way - use  Topic is solved

Post by mikeyww » 03 Dec 2022, 06:00

doesn't work for me
Do you have any better problem statement of what actually happens when you run your script, and what should happen instead?

Debugging seems straightforward, such as in a line that displays ThisFolder. Your file paths will not actually end in a backslash, right? You can display the values of your variables to understand what is happening in your loop.

Loop, FilePattern is deprecated. See https://www.autohotkey.com/docs/commands/LoopFile.htm.

Code: Select all

dir := A_Desktop
n := nItems(dir), files := n.files, folders := n.subdirs
Loop, Files, %dir%\*, DR ; Loop through subdirectories
 n := nItems(A_LoopFilePath), files += n.files, folders += n.subdirs
MsgBox, 64, Totals, Files: %files%`n`nFolders: %folders%

nItems(dir) { ; Return number of files & subdirectories
 ; https://www.autohotkey.com/boards/viewtopic.php?p=494290#p494290
 objFolder := ComObjCreate("Scripting.FileSystemObject").GetFolder(dir)
 Return {files: objFolder.Files.Count, subdirs: objFolder.SubFolders.Count}
}
Or:

Code: Select all

dir := A_Desktop
n   := nItems(dir, RECURSE := True)
MsgBox, 64, Totals, % "Folders: " n.subdirs "`n`nFiles: " n.files

nItems(dir, recurse := False) { ; Return number of files & subdirectories
 ; https://www.autohotkey.com/boards/viewtopic.php?p=494290#p494290
 Try objFolder := ComObjCreate("Scripting.FileSystemObject").GetFolder(dir)
 files := objFolder.Files.Count, folders := objFolder.SubFolders.Count
 Loop, Files, % recurse ? dir "\*" : "", DR
  n := nItems(A_LoopFilePath), files += n.files, folders += n.subdirs
 Return {files: files, subdirs: folders}
}

Albireo
Posts: 1747
Joined: 16 Oct 2013, 13:53

Re: Count files - fastest way - use

Post by Albireo » 03 Dec 2022, 18:07

OK!
Loop Files, %UserFolder%\*.*, R (is better - but doesn't works faster.)

I thought this method with ComObjCreate() was the fastest.
(It works on the selected directory, not on subdirectories)

Code: Select all

#SingleInstance Force
SetBatchLines, -1
UserFolder:="C:\temp"
UserFolder := RegExReplace( UserFolder, "\\$")  ; gets rid of trailing slash if required

startTime := A_TickCount
Loop 1000
	FileCount := ComObjCreate("Scripting.FileSystemObject").GetFolder(UserFolder).Files.Count
stopTime := A_TickCount
MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Count file .: " FileCount " pcs`n`nRuntime .: " stopTime - startTime "ms"
The second method below, from the manual, takes half as long to perform.

Code: Select all

#SingleInstance Force
SetBatchLines, -1
UserFolder := "C:\temp"
UserFolder := RegExReplace( UserFolder, "\\$")  ; gets rid of trailing slash if required

startTime := A_TickCount
Loop 1000
{	count := 0
	Loop Files, %UserFolder%\*.*, F
		count++
}
stopTime := A_TickCount
MsgBox ,, Row %A_LineNumber% -> %A_ScriptName%, % "Count .: " count "pcs `nRuntime .: " stopTime - startTime "ms"
But may not work well on a network drive. (I can't test it right now.)

When I try to use the ComObjCreate() method

Code: Select all

Loop Files, %UserFolder%\*.*, R
{
        ThisFolder := RegExReplace( A_LoopFileFullPath, "\\$" )
        ItemCount := ComObjCreate("Shell.Application").NameSpace(ThisFolder).Items.Count
        FileCount := ComObjCreate("Scripting.FileSystemObject").GetFolder(ThisFolder).Files.Count
	FolderCount := ItemCount - FileCount
        TotalFileCount += FileCount
        TotalFolderCount += FolderCount
}
- I have no idea why the program execution crashes at the first ComObjCreate() instruction in the loop.
(Maybe because I don't understand the instruction)

User avatar
mikeyww
Posts: 26602
Joined: 09 Sep 2014, 18:38

Re: Count files - fastest way - use

Post by mikeyww » 03 Dec 2022, 18:10

Or maybe you did not read the answer in my first post.

Albireo
Posts: 1747
Joined: 16 Oct 2013, 13:53

Re: Count files - fastest way - use

Post by Albireo » 04 Dec 2022, 15:20

Excuse me! :?

Now I have tested your suggestions.
The second script is more appealing and considerably faster.

to compare (count only files in one directory)
1) With only Loop - 47ms
2 With simple ComObj - 94ms
3) mikeyww ex.1 - doesn't work without subdirectories - ??ms
4) mikeyww ex.2 - 141ms - Good structure



to compare (count all items in a directory and subdirectories)
The numbers refers to the test scripts above. (10,811 files x 100)
  • 1) from the manual .: 5,546ms
  • 2) Subdirectories doesn't work!
  • 3) @mikeyww ex.1 - 24,768ms
  • 4) @mikeyww ex.2 - 16,109ms
I like ex.2 from @mikeyww
Is it possible to optimize ex2. to go faster

User avatar
mikeyww
Posts: 26602
Joined: 09 Sep 2014, 18:38

Re: Count files - fastest way - use

Post by mikeyww » 04 Dec 2022, 15:21

I have no idea. Try it! Post an answer! :)

wetware05
Posts: 750
Joined: 04 Dec 2020, 16:09

Re: Count files - fastest way - use

Post by wetware05 » 04 Dec 2022, 16:20

hi Albireo.

Do you need to use the number of files, if you just need to know the number, hitting properties on a folder in Explorer tells you the number of files or folders almost instantly.

Albireo
Posts: 1747
Joined: 16 Oct 2013, 13:53

Re: Count files - fastest way - use

Post by Albireo » 08 Dec 2022, 16:07

I have a directory that should be automatically emptied of files every day.
It won't have time to get as many files as today (maybe 500,000 or more).
My idea was to check the number of files before and after the files are deleted. (there may be a small risk with locked files).
My strategy is
  • - count files
  • - delete files
  • - count files
  • - create a log file with statistics of the run.
Have other projects where it's more time critical
My wish was to check what limitations (and possibilities) there are for counting the number of files in a directory and subdirectories locally or on a network drive.

User avatar
andy_BSZY
Posts: 67
Joined: 13 Sep 2023, 09:52

Re: Count files - fastest way - use

Post by andy_BSZY » 06 Oct 2023, 11:17

mikeyww wrote:
03 Dec 2022, 06:00

Code: Select all

dir := A_Desktop
n := nItems(dir), files := n.files, folders := n.subdirs
Loop, Files, %dir%\*, DR ; Loop through subdirectories
 n := nItems(A_LoopFilePath), files += n.files, folders += n.subdirs
MsgBox, 64, Totals, Files: %files%`n`nFolders: %folders%

nItems(dir) { ; Return number of files & subdirectories
 ; https://www.autohotkey.com/boards/viewtopic.php?p=494290#p494290
 objFolder := ComObjCreate("Scripting.FileSystemObject").GetFolder(dir)
 Return {files: objFolder.Files.Count, subdirs: objFolder.SubFolders.Count}
}
Or:

Code: Select all

dir := A_Desktop
n   := nItems(dir, RECURSE := True)
MsgBox, 64, Totals, % "Folders: " n.subdirs "`n`nFiles: " n.files

nItems(dir, recurse := False) { ; Return number of files & subdirectories
 ; https://www.autohotkey.com/boards/viewtopic.php?p=494290#p494290
 Try objFolder := ComObjCreate("Scripting.FileSystemObject").GetFolder(dir)
 files := objFolder.Files.Count, folders := objFolder.SubFolders.Count
 Loop, Files, % recurse ? dir "\*" : "", DR
  n := nItems(A_LoopFilePath), files += n.files, folders += n.subdirs
 Return {files: files, subdirs: folders}
}
Is there any chance for v2 version of code

User avatar
mikeyww
Posts: 26602
Joined: 09 Sep 2014, 18:38

Re: Count files - fastest way - use

Post by mikeyww » 07 Oct 2023, 05:29

Code: Select all

#Requires AutoHotkey v2.0
dir := A_Desktop
n   := nItems(dir, RECURSE := True)
MsgBox 'Folders: ' n.subdirs '`n`nFiles: ' n.files, 'Totals', 64

nItems(dir, recurse := False) { ; Return number of files & subdirectories
 ; https://www.autohotkey.com/boards/viewtopic.php?p=494290#p494290
 If DirExist(dir) {
  objFolder := ComObject('Scripting.FileSystemObject').GetFolder(dir)
  files := objFolder.Files.Count, folders := objFolder.SubFolders.Count
  Loop Files recurse ? dir '\*' : '', 'DR'
   n := nItems(A_LoopFilePath), files += n.files, folders += n.subdirs
 } Else files := folders := 'ERROR'
 Return {files: files, subdirs: folders}
}

User avatar
andy_BSZY
Posts: 67
Joined: 13 Sep 2023, 09:52

Re: Count files - fastest way - use

Post by andy_BSZY » 07 Oct 2023, 10:39

@mikeyww Thank you :thumbup:

Post Reply

Return to “Ask for Help (v1)”