7 replies to this topic

[just me]

As Lexikos statet here the following is a bug and will be fixed:

Within the writing script you may check File.Pos against File.Length. Whenever File.Pos exceeds File.Length data are buffered internally.

There is an issue related to this. If you open a file in append mode and File.Write() is buffered, the buffer won't be written to disk immediatly when you set File.Length := File.Pos. It remains in the buffer instead and will be appended later to the new end-of-file. This should be fixed soon.

#NoEnv
FilePath := A_ScriptDir . "\Test.txt"
ToWrite := ""
Loop, 512
   ToWrite .= "A"
FO := {}
FileDelete, %FilePath%
HFile := FileOpen(FilePath, "a")
FO.Insert({Pos: HFile.Pos, Len: HFile.Length, Cmd: "FileOpen"})
BW := HFile.Write(ToWrite)
FO.Insert({Pos: HFile.Pos, Len: HFile.Length, Cmd: "File.Write()"})
HFile.Length := HFile.Pos
FO.Insert({Pos: HFile.Pos, Len: HFile.Length, Cmd: "File.Length := File.Pos"})
HFile.__Handle
FO.Insert({Pos: HFile.Pos, Len: HFile.Length, Cmd: "File.__Handle"})
HFile.Close()
Gui, Color, F0F0F0
Gui, Margin, 20, 20
Gui, Add, ListView, w400 r5 Grid, File.Pos|File.Length|After
For K, O In FO
   LV_Add("", O.Pos, O.Len, O.Cmd)
Loop, % FO.MaxIndex()
   LV_ModifyCol(A_Index, "AutoHdr")
Gui, Show, , File - bytes written = %BW%
Return

GuiClose:
GuiEscape:
ExitApp

The file object is using an own internal buffer of 8188 bytes for I/O-buffering. If an application opens a LOG file on start and closes it on exit (which is good practice in other programming languages), over 100 records will be buffered when the records have an average length of 80 bytes. In case of a crash all the records will be lost. So AHK should offer an option for unbuffered I/O, e.g. an additional option "U"(nbuffered) for FileOpen.

[Lexikos]

Thanks for reporting it.

The file object is using an own internal buffer of 8188 bytes for I/O-buffering.

The buffer is 8192 bytes (8 KB), but some space is reserved to allow for a 4-byte UTF-8 character, UTF-16 surrogate pair or `r`n sequence at the end of the buffer.

So AHK should offer an option for unbuffered I/O, e.g. an additional option "U"(nbuffered) for FileOpen.

The text-writing routine relies heavily on the buffer, so a truly unbuffered mode would not be feasible. An option to flush the buffer after each Write() call will be added, eventually. Read buffering can't be disabled for practical reasons, but can only become a problem for non-seeking devices (such as pipes or console buffers).

Related: FileOpen() handle support is broken

[just me]

The buffer is 8192 bytes (8 KB), but some space is reserved to allow for a 4-byte UTF-8 character, UTF-16 surrogate pair or `r`n sequence at the end of the buffer.

I've seen this, but for all my testing on U32 exactly 8188 bytes are written for File.Write() as soon as the amount of buffered bytes exceeds 8187.

#NoEnv
FilePath := A_ScriptDir . "\Test.txt"
ToWrite := ""
Loop, 8188
   ToWrite .= "A"
HFILE := FileOpen(FilePath, "w")
BytesWritten := HFILE.Write(ToWrite)
FileGetSize, SizeB, %FilePath%
FileRead, Content, %FilePath%
BytesRead := StrLen(Content)
FileGetSize, SizeA, %FilePath%
Gui, Margin, 20, 20
Gui, Add, Text, xm Section, HFILE.Pos
Gui, Add, Text, y+5, HFILE.Length
Gui, Add, Text, y+5, Bytes read
Gui, Add, Text, y+5, Bytes written
Gui, Add, Text, y+5, FileGetSize before FileRead
Gui, Add, Text, y+5, FileGetSize after FileRead
Gui, Add, Text, ys, % HFILE.Pos
Gui, Add, Text, y+5, % HFILE.Length
Gui, Add, Text, y+5, %BytesRead%
Gui, Add, Text, y+5, %BytesWritten%
Gui, Add, Text, y+5, %SizeB%
Gui, Add, Text, y+5, %SizeA%
Gui, Show, , File.Write()
Return
GuiCLose:
GuiEscape:
HFILE.Close()
ExitApp

Is this intended behavior?

(BTW: Where is Lexikos?)

[Lexikos]

Yes.

some space is reserved

[just me]

Wouldn't it be better in matters of file system performance to write complete sectors/clusters only? And why do you "keep free" up to 4 bytes on disk?

[Lexikos]

Wouldn't it be better in matters of file system performance to write complete sectors/clusters only?

8 KB is the figure that worked best in my benchmarks.

And why do you "keep free" up to 4 bytes on disk?

I don't. You must have misunderstood something.

[just me]

I don't. You must have misunderstood something.

Probably. What I think (without any expert knowledge) is:
[*:34edyb4n]The common NTFS cluster size is 4096 bytes strictly net (this may be the reason for misunderstanding).
[*:34edyb4n]8188 bytes need one complete cluster and a second one filled with 4092 bytes.
[*:34edyb4n]When the next buffer will be written, NTFS has to read the second cluster of the first buffer, add the first four bytes of data, and rewrite it.
[*:34edyb4n]After that one complete cluster is written and a second one containing 4088 bytes.
[*:34edyb4n]And so on, and so on.Probably wrong, isn't it?

[Lexikos]

This commit fixes the issues with File.Length.