GetFileAttributesEx() - 2020/11/15 AHK v1

[TheArkive]

Thanks to @robodesign for testing, benchmarking, and assistance with optimizations. His work took this already fast function and reduced processing time by several seconds.
Thanks to @@just me for suggesting the "file loop" method. This added more benchmark data for a more complete comparison.

AHK v2 version is posted here. There is also a significant speed hike on AHK v2 as well!

This lib offers 3 benefits:

• Collects all file properties in one call.
• Allows calling any combination of properties.
• 50%+ faster compared to built-in AHK functions when looping through an array of files.
  Calling fewer properties is a bit faster than calling all of them.

NOTE: This is not meant to be used inside a file loop. Only use this with an array ([] or {}) list of files. You will most likely notice a speed difference more when looping through at least hundreds of thousands of files.

Usage:

Code: Select all

Usage:
obj := GetFileAttributesEx(FileName, Properties := "ACWTS")

obj.aTime    obj.size
obj.cTime    obj.attrib (this property is an array[])
obj.wTime

Properties parameter:

• A = accessed time
• C = creation time
• W = write time (modified time)
• T = file attributes
• S = file size

By default, all properties are called.

=======================================================================================
The Script
=======================================================================================

Code: Select all

; AHK v1
; =====================================================================================
; === Example 1 =======================================================================
; =====================================================================================
; #NoEnv
; SetBatchLines, -1 ; required for fastest execution
; fileList := []

; Loop, Files, C:\Windows\*, R
    ; fileList.Push(A_LoopFileFullPath) ; := A_LoopFileFullPath

; MsgBox File list made ... starting benchmark.

; curTime := A_TickCount

; Loop, % fileList.Length() { ; run only one of these
    ; obj := GetFileAttributesEx(fileList[A_Index],"ACWTS") ; ACWTS / CWS
    ; obj := ahk_attribs(fileList[A_Index]) ; single ahk functions
    ; obj := ahk_attribs2(fileList[A_Index]) ; ahk file loop
; }

; ahk_attribs(inFile) { ; normal AHK functions
    ; If !FileExist(inFile)
        ; return
    ; FileGetSize, size, %inFile%
    ; FileGetTime, cTime, %inFile%, C
    ; FileGetTime, wTime, %inFile%, M
    
    ; return {cTime:cTime, wTime:wTime, size:size}
; }

; ahk_attribs2(inFile) {
    ; Loop, Files, inFile
    ; {
        ; result := {  aTime:A_LoopFileTimeAccessed
                  ; ,  cTime:A_LoopFileTimeCreated
                  ; ,  wTime:A_LoopFileTimeModified
                  ; ,   size:A_LoopFileSize
                  ; , attrib:A_LoopFileAttrib}
    ; }
    
    ; return result
; }

; msgbox % "Total: " fileList.Length() "`r`n`r`nElapsed: " (A_TickCount - curTime)/1000

; === Benchmarks ======================================================================
; 280,362 - 8.78s    / 8.83s  calling GetFileAttributesEx(file,"CSW")
; 280,362 - 10.95s   / 10.98s calling GetFileAttributesEx(file,"ACWTS")
; 280,362 - 28.297s  / 28.64s calling same properties ahk_attribs() (CSW)
; 280,337 - 28.09s   / 28.38s calling all properties ahk_attribs2() (ACWTS)
; =====================================================================================
; === Example 2 =======================================================================
; =====================================================================================

; fName := A_ScriptDir "\" A_ScriptName ; current script file
; o := GetFileAttributesEx(fName,"ACWTS")

; attr := ""
; For i, attrib in o.attr
    ; attr .= attrib "`r`n"

; MsgBox % "attributes:`r`n" attr "`r`n"
     ; . "CreationTime:`r`n" o.cTime "`r`n`r`n"
     ; . "AccessTime:`r`n" o.aTime "`r`n`r`n"
     ; . "WriteTime:`r`n" o.wTime "`r`n`r`n"
     ; . "size: " o.size

; ====================================================================================
; GetFileAttributesEx() by TheArkive (requested by robodesign
; - Thanks to robodesign for testing, benchmarking, and assistance with optimizations.
;   His work took this already fast function and knocked of several seconds.
; ====================================================================================
GetFileAttributesEx(inFile,sOptions:="ACWTS") { ; A = access time / C = create time / W = write (modified) time / T = attributes / S = file size
    Static attr := { Archive:0x20 ; https://docs.microsoft.com/en-us/windows/win32/fileio/file-attribute-constants
            , Compressed:0x800, Device:0x40, Directory:0x10, Encrypted:0x4000, Hidden:0x2, integ_stream:0x8000, Normal:0x80, NotContentIndexed:0x2000
            , NoScrubData:0x20000, Offline:0x1000, ReadOnly:0x1, RecallOnDataAccess:0x400000, RecallOnOpen:0x40000, ReparsePoint:0x400, SparseFile:0x200
            , System:0x4, Temporary:0x100, Virtual:0x10000}
    
    ; Static attr := { ReadOnly:0x1 ; short list
            ; , Archive:0x20, System:0x4, Hidden:0x2, Normal:0x80, Directory:0x10, Offline:0x1000, Compressed:0x800, Temporary:0x100, Encrypted:0x4000}
    
    Static bSize := (A_PtrSize=8)?40:36
    VarSetCapacity(bFileAttribs,bSize,0) ; https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfileattributesexw
    r := DllCall("GetFileAttributesExW","Str","\\?\" inFile,"Int",0,"Ptr",p2 := &bFileAttribs) ; (A_IsUnicode ? "W" : "A") ; (A_IsUnicode ? "\\?\" : "")
    
    AttrList := [], aTime := "", cTime := "", wTime := "", fileSize := "", fileSize := 0
    Loop, Parse, sOptions
    {
        Switch A_LoopField {
            Case "T":
                iAttribs := NumGet(bFileAttribs,"UInt")
                For attrib, value in attr
                    If (value & iAttribs)
                        AttrList[A_Index] := attrib
            Case "C": cTime := FileTimeToSystemTime(p2+4)
            Case "A": aTime := FileTimeToSystemTime(p2+12)
            Case "W": wTime := FileTimeToSystemTime(p2+20)
            Case "S": fileSize := (NumGet(bFileAttribs,28,"UInt") << 32) | NumGet(bFileAttribs,32,"UInt")
        }
    }
    
    return {attr:AttrList, cTime:cTime, aTime:aTime, wTime:wTime, size:fileSize}
}

FileTimeToSystemTime(ptr) {         
    VarSetCapacity(SYSTEMTIME,16,0) ; https://docs.microsoft.com/en-us/windows/win32/api/minwinbase/ns-minwinbase-systemtime
    r := DllCall("FileTimeToSystemTime","Ptr",ptr,"Ptr",&SYSTEMTIME) ; https://docs.microsoft.com/en-us/windows/win32/api/timezoneapi/nf-timezoneapi-filetimetosystemtime
    
    VarSetCapacity(SYSTIME2,16,0)
    r := DllCall("SystemTimeToTzSpecificLocalTime","Ptr",0,"Ptr",&SYSTEMTIME,"Ptr",&SYSTIME2) ; https://docs.microsoft.com/en-us/windows/win32/api/timezoneapi/nf-timezoneapi-systemtimetotzspecificlocaltime
    
    year := NumGet(SYSTIME2,"UShort"),                      hour := Format("{:02d}",NumGet(SYSTIME2,8,"UShort"))
    month := Format("{:02d}",NumGet(SYSTIME2,2,"UShort")),  minute := Format("{:02d}",NumGet(SYSTIME2,10,"UShort"))
    day := Format("{:02d}",NumGet(SYSTIME2,6,"UShort")),    second := Format("{:02d}",NumGet(SYSTIME2,12,"UShort"))
    
    return year month day hour minute second
}

Example #1 is for benchmarking.

Example #2 is a basic usage example.

Benchmark results:

• 280,362 files - 8.78s / 8.83s calling GetFileAttributesEx(file,"CSW")
• 280,362 files - 10.95s / 10.98s calling GetFileAttributesEx(file,"ACWTS")
• 280,362 files - 28.297s / 28.64s calling same properties ahk_attribs() (CSW)

[robodesign]

Thank you very much for this awesome function! Kudos!

I implemented it in my image viewer, QPV, and I was able to optimize a function that once run on 352 000 files took about 50 seconds. It now takes about 17 seconds. AHK's built-in functions for retrieving individually file properties (eg. dates, file size, attributes) are darn slow...

Best regards, Marius!

[just me]

AHK's built-in functions for retrieving individually file properties (eg. dates, file size, attributes) are darn slow...

What about the Loop, Files, ... special variables?

Code: Select all

#NoEnv
SetBatchLines, -1
Loop, Files, %A_ScriptFullPath%
   FO := {AT: A_LoopFileTimeAccessed
        , CT: A_LoopFileTimeCreated
        , MT: A_LoopFileTimeModified
        , Attr:   A_LoopFileAttrib
        , Size:   A_LoopFileSize}
Attributes := "Attributes: "  . FO.Attr . "`n"
            . "File size: "   . FO.Size . "`n"
            . "Created: "     . FO.CT . "`n"
            . "Modified: "    . FO.MT . "`n"
            . "Last access: " . FO.AT
MsgBox, 0, %A_ScriptFullPath%, %Attributes%

[TheArkive]

@just me
Of course, not trying to make wild claims. This is not meant to be used in a file loop at all. This is for a file array only, where the file list may not be sequential in a way that a file loop would require.

EDIT: And of course the testing done was with a few hundred thousand files.

I'll update the OP.

[just me]

@TheArkive,

as shown by my example you can use the File, Loop method for a single file too.

[TheArkive]

@just me
I have to admit, I didn't think of that

Now I'm curious. I'll run some tests.

[TheArkive]

@just me thanks for posting. More benchmarking data is always good.

It turns out a file loop is a bit faster than a file array loop while using the built in AHK FileGet... functions to get the file properties.

Code: Select all

280,362 - 28.297s  / 28.64s calling ahk_attribs()  (Created/Modified/Size)
280,337 - 28.09s   / 28.38s calling ahk_attribs2() (Accessed/Created/Modified/Attributes/Size)

Still seems that GetFIleAttributesEx() offers a considerable speed hike:

Code: Select all

280,362 - 10.95s   / 10.98s calling GetFileAttributesEx(file,"ACWTS") (Accessed/Created/Modified/Attributes/Size)

I have updated the OP to show the new function and results. I'd be happy to see what results anyone else gets

[robodesign]

AHK's built-in functions for retrieving individually file properties (eg. dates, file size, attributes) are darn slow...

What about the Loop, Files, ... special variables?

Code: Select all

#NoEnv
SetBatchLines, -1
Loop, Files, %A_ScriptFullPath%
   FO := {AT: A_LoopFileTimeAccessed
        , CT: A_LoopFileTimeCreated
        , MT: A_LoopFileTimeModified
        , Attr:   A_LoopFileAttrib
        , Size:   A_LoopFileSize}
Attributes := "Attributes: "  . FO.Attr . "`n"
            . "File size: "   . FO.Size . "`n"
            . "Created: "     . FO.CT . "`n"
            . "Modified: "    . FO.MT . "`n"
            . "Last access: " . FO.AT
MsgBox, 0, %A_ScriptFullPath%, %Attributes%

Example array:
imgArray[1] := imgpath1
imgArray[2] := imgpath2
imgArray[3] := imgpath3


Yes, I use that trick when I open folders to scan for image files. But I use the function coded by TheArkive when I have an already existing array of files from various folders and I lack file properties details [size, dates]. It would be slow as heck to take my array and force a file loop on each of them [ I do not know if this is what you suggest ]. I used initially filegettime and filegetsize, but that is darn slower than TheArkive's function.

Best regards, Marius.

[just me]

@TheArkive,

confirmed. I didn't expect Loop, Files, ... to be so slow.

Thanks for testing.

[robodesign]

Hehe, i did some quick tests as well on 1.38 million image files:

collectFileInfos() - elapsed time: 7m 30s - Files Loop on single file
collectFileInfos() - elapsed time: 3m 15s - ahk built-in FileGetTime, FileGetSize
collectFileInfos() - elapsed time: 1m 15s - TheArkive function

Best regards, Marius.

[toralf]

Good morning,

Is there any chance to extract the date the picture was taken?
Windows explorer has the option to show this date as an extra column. In the past I have used exiftool to extract the date.

[robodesign]

Good morning,

Is there any chance to extract the date the picture was taken?
Windows explorer has the option to show this date as an extra column. In the past I have used exiftool to extract the date.

This function is only concerned with the basic file properties (size, date created /modified and others), not image file metadata or other types of metadata.

To extract the date taken property from an image, use gdi+. My library edition contains the required functions for this, but we are going off-topic.

Best regards, Marius.

[TheArkive]

@toralf

The DLL functions used in this script may not do this. I'll have to go back and read them again. I'm pretty sure you are talking about EXIF tags.

I'll see what I can find out though.

Edit: Hopefully robodesign's script will do what you need until then (if I am successful).

[AHKStudent]

Is there a way to use this to get info if the file has a digital certificate?

[robodesign]

Is there a way to use this to get info if the file has a digital certificate?

This function does not do this.

Best regards, Marius.

[carno]

It would be great if this script also offered the full path of the file.

[TheArkive]

@carno
You actually must provide the full path of the file in order to actually get the attributes. These DLL calls don't actually accept relative paths (i tried), so providing the full path as a property isn't useful in this library.