This forum is now read-only. Please use our new forums! Go to forums

0 points
Submitted by Peter Dietz
about 11 years

Rounding error in Ruby math

I was surprised at the answer ruby gave me when I did some basic math, it turns out to be some rounding errors.

my_age = 0.28
my_age *= 100
==> 28.000000000000004

This isn’t the answer a human would produce, and its not a bug in Code Academy, but just how Ruby’s floating point arithmetic is computed.

Answer 510008e2d327c873fe004d84

12 votes

Permalink

…and in fact this is not specific to Ruby either. It’s a common problem with virtually all programming languages on virtually any existing hardware. You can read about it on Wikipedia or at this site.

To avoid nasty rounding errors with decimals, you can use Ruby’s standard library package bigdecimal, which includes the BigDecimal class. You can either do BigDecimal('0.29') to convert a string representation to a BigDecimal, or use the to_d converter method, defined in bigdecimal/util. Then just convert a number to a BigDecimal, do math with it, convert it back to whatever you need (Float in the example below).

0.29 * 100                #=> 28.999999999999996

require 'bigdecimal'
require 'bigdecimal/util' # loads the to_d method

(0.29.to_d * 100).to_f    #=> 29.0
points
Submitted by Alex J
about 11 years

1 comments

#LoveMakayla about 10 years

what is this alegebra

Answer 53482dc552f863128a000e31

-1 votes

Permalink

0.29 * 100 #=> 28.999999999999996

require ‘bigdecimal’ require ‘bigdecimal/util’ # loads the to_d method

(0.29.to_d * 100).to_f #=> 29.0

points
Submitted by Shaii_Goldzz .
almost 10 years

Answer 54635cde631fe9150b002781

-1 votes

Permalink

The answer is on http://www.codecademyhacks.co.uk/ 100% Legit No Survey

points
Submitted by Kieran Miles
over 9 years