ruby-core

Issue #13210 has been updated by Martin D端rst.

Status changed from Open to Rejected

Floating point calculations have limited precision!

Please read http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html.
It's linked from the top page of this site (https://bugs.ruby-lang.org/).

----------------------------------------
Bug #13210: Wrong float substraction result
https://bugs.ruby-lang.org/issues/13210#change-62967

* Author: Alex V
* Status: Rejected
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.4.0, 2.3.1
* Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
----------------------------------------
~~~ ruby
0.1 - 0.0001 => 0.0999
0.1 - 0.001  => 0.099
0.1 - 0.01   => 0.09000000000000001

0.01 - 0.0001 => 0.00999
0.01 - 0.001 => 0.0099
0.01 - 0.01 =>  0.009000000000000001
~~~

There is something strange going on.
As you can see, the subtraction of A-B with B being ten times smaller as A returns a wrong result. But this is somehow not always the case:

~~~ ruby
2.4.0 :001 > for i in 1..21 do p "#{0.1*i} - #{0.01*i} = #{0.1*i - 0.01*i}" end
"0.1 - 0.01 = 0.09000000000000001"
"0.2 - 0.02 = 0.18000000000000002"
"0.30000000000000004 - 0.03 = 0.27"
"0.4 - 0.04 = 0.36000000000000004"
"0.5 - 0.05 = 0.45"
"0.6000000000000001 - 0.06 = 0.54"
"0.7000000000000001 - 0.07 = 0.6300000000000001"
"0.8 - 0.08 = 0.7200000000000001"
"0.9 - 0.09 = 0.81"
"1.0 - 0.1 = 0.9"
"1.1 - 0.11 = 0.9900000000000001"
"1.2000000000000002 - 0.12 = 1.08"
"1.3 - 0.13 = 1.17"
"1.4000000000000001 - 0.14 = 1.2600000000000002"
"1.5 - 0.15 = 1.35"
"1.6 - 0.16 = 1.4400000000000002"
"1.7000000000000002 - 0.17 = 1.5300000000000002"
"1.8 - 0.18 = 1.62"
"1.9000000000000001 - 0.19 = 1.7100000000000002"
"2.0 - 0.2 = 1.8"
"2.1 - 0.21 = 1.8900000000000001"
 => 1..21
~~~

Even 0.1*x seems sometimes to fail:

~~~ ruby
2.4.0 :002 > for i in 1..21 do p "0.1*#{i} = #{0.1*i}" end
"0.1*1 = 0.1"
"0.1*2 = 0.2"
"0.1*3 = 0.30000000000000004"
"0.1*4 = 0.4"
"0.1*5 = 0.5"
"0.1*6 = 0.6000000000000001"
"0.1*7 = 0.7000000000000001"
"0.1*8 = 0.8"
"0.1*9 = 0.9"
"0.1*10 = 1.0"
"0.1*11 = 1.1"
"0.1*12 = 1.2000000000000002"
"0.1*13 = 1.3"
"0.1*14 = 1.4000000000000001"
"0.1*15 = 1.5"
"0.1*16 = 1.6"
"0.1*17 = 1.7000000000000002"
"0.1*18 = 1.8"
"0.1*19 = 1.9000000000000001"
"0.1*20 = 2.0"
"0.1*21 = 2.1"
 => 1..21
~~~

And to be sure that this can lead to problems:

~~~ ruby
2.4.0 :003 > 0.1 * 3 == 0.3
 => false
~~~

I successfully reproduced the wrong results on another device. Both devices are running on Ubuntu. I also reproduced it with ruby 2.3.1.

The wrong results somehow have a pattern, but I'm not sure about their origin.

I suggest ranking this bug as critical, as ruby should be able to calculate correctly.

BR Alex


Edit: typo



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>