Format not enough precise

Report problems with documented functionality
Archimede
Posts: 491
Joined: 25 Nov 2021, 09:49
Location: Switzerland / Italy

Format not enough precise

Post by Archimede » 03 Dec 2022, 10:49

This example
fNum := 100.22
sTest2 := Format( "{:03.15f}", fNum )
returns sTest = 100.21999999

User avatar
Xtra
Posts: 2750
Joined: 02 Oct 2015, 12:15

Re: Format not enough precise

Post by Xtra » 03 Dec 2022, 12:50

Code: Select all

fNum := 100.22
sTest2 := Format( "{:03.15f}", fNum )
MsgBox % sTest2    ; 100.220000000000000
Maybe you were mistaken you say:
returns sTest = 100.21999999
when the var name should be sTest2

Above example works fine for me.

Archimede
Posts: 491
Joined: 25 Nov 2021, 09:49
Location: Switzerland / Italy

Re: Format not enough precise

Post by Archimede » 03 Dec 2022, 17:53

Hallo.
My last example line is not a program line, but a descriptive text, where I reported sTest instead sTest2, sorry.
The program lines are only the first 2 lines.
I controlled the value in the variable list of the developing environment.
I test it many time.
It seem me the problem is no constant: many times before it seem works well, but I found the same problem with the negative value -100.22.
It seem me that is no a constant problem.

Tensai
Posts: 29
Joined: 12 Dec 2019, 14:15

Re: Format not enough precise

Post by Tensai » 03 Dec 2022, 19:30

I do not believe this is a bug.
From the Docs:
https://lexikos.github.io/v2/docs/Concepts.htm#pure-numbers
Note: There are some decimal fractions which the binary floating-point format cannot precisely represent, so a number is rounded to the closest representable number. This may lead to unexpected results. For example:

Code: Select all

MsgBox 0.1 + 0           ; 0.10000000000000001
MsgBox 0.1 + 0.2         ; 0.30000000000000004
MsgBox 0.3 + 0           ; 0.29999999999999999
MsgBox 0.1 + 0.2 = 0.3   ; 0 (not equal)
One strategy for dealing with this is to avoid direct comparison, instead comparing the difference. For example:

Code: Select all

MsgBox Abs((0.1 + 0.2) - (0.3)) < 0.0000000000000001
Also maybe worth mentioning, I get different results from different ahk versions.
Hope this helps! :)

Archimede
Posts: 491
Joined: 25 Nov 2021, 09:49
Location: Switzerland / Italy

Re: Format not enough precise

Post by Archimede » 05 Dec 2022, 05:20

Yes, thank you.
I solved the problem, but I think the problem must be solved into the language.
:-)

lexikos
Posts: 9583
Joined: 30 Sep 2013, 04:07
Contact:

Re: Format not enough precise

Post by lexikos » 05 Dec 2022, 21:56

There's nothing to "solve into the language" unless you want to wish for decimal-based floating-point numbers, instead of the binary format understood by your CPU and used by most programming languages by default.

Post Reply

Return to “Bug Reports”