Prev: optparser question
Next: Newbie
From: Neve Le on 30 Jun 2010 08:06 Siep Korteling wrote: >> Xavier Noria wrote: >> That 15 is hardcoded in the definition of Float#to_s. > > Yes, and exposing more digits is not accurate: > The float is bogus after the 16th digit. Working with big numbers here, it seems that precision is only guaranteed to 15 digits on either side of the decimal. When a whole number >15 digits is divided the decimal is approximated, and anything >17 digits prints a float of n.0 no matter how it's "%n.nf" formatted. Is the hardcoded 15 just a "safe" limit? Would I be able to recompile ruby with a higher definition? If possible, how high could I go?  Posted via http://www.rubyforum.com/.
From: Brian Candler on 30 Jun 2010 08:32 Neve Le wrote: > Is the hardcoded 15 just a "safe" limit? Would I be able to recompile > ruby with a higher definition? If possible, how high could I go? No, it's a limit of IEEE doubleprecision floatingpoint representation. If you want more digits of precision, use BigDecimal.  Posted via http://www.rubyforum.com/.
From: Giampiero Zanchi on 30 Jun 2010 08:44 http://www.rubyforum.com/topic/208840#new hope this help  Posted via http://www.rubyforum.com/.
From: brabuhr on 30 Jun 2010 08:48 On Wed, Jun 30, 2010 at 8:06 AM, Neve Le <nevele(a)gmx.com> wrote: > Siep Korteling wrote: >>> Xavier Noria wrote: >>> That 15 is hardcoded in the definition of Float#to_s. >> >> Yes, and exposing more digits is not accurate: >> The float is bogus after the 16th digit. > > Working with big numbers here, it seems that precision is only > guaranteed to 15 digits on either side of the decimal. When a whole > number >15 digits is divided the decimal is approximated, and anything >>17 digits prints a float of n.0 no matter how it's "%n.nf" formatted. http://rubydoc.org/core/classes/Float.html "Float objects represent real numbers using the native architectures doubleprecision floating point representation." http://en.wikipedia.org/wiki/Double_precision_floatingpoint_format You could try BigDecimal.
From: Javier Goizueta on 30 Jun 2010 10:56 You may also want to take a look at http://flt.rubyforge.org/ It's an implementation of arbitrary precision floating point numbers (decimal and binary). If you don't need it to be fast it has some advantages over BigDecimal.  Posted via http://www.rubyforum.com/.

Pages: 1 Prev: optparser question Next: Newbie 