Prev: optparser questionNext: Newbie From: Neve Le on 30 Jun 2010 08:06 Siep Korteling wrote:>> Xavier Noria wrote: >> That 15 is hard-coded in the definition of Float#to_s. > > Yes, and exposing more digits is not accurate: > The float is bogus after the 16th digit. Working with big numbers here, it seems that precision is only guaranteed to 15 digits on either side of the decimal. When a whole number >15 digits is divided the decimal is approximated, and anything >17 digits prints a float of n.0 no matter how it's "%n.nf" formatted. Is the hard-coded 15 just a "safe" limit? Would I be able to recompile ruby with a higher definition? If possible, how high could I go? -- Posted via http://www.ruby-forum.com/. From: Brian Candler on 30 Jun 2010 08:32 Neve Le wrote:> Is the hard-coded 15 just a "safe" limit? Would I be able to recompile > ruby with a higher definition? If possible, how high could I go? No, it's a limit of IEEE double-precision floating-point representation. If you want more digits of precision, use BigDecimal. -- Posted via http://www.ruby-forum.com/. From: Giampiero Zanchi on 30 Jun 2010 08:44 http://www.ruby-forum.com/topic/208840#new hope this help -- Posted via http://www.ruby-forum.com/. From: brabuhr on 30 Jun 2010 08:48 On Wed, Jun 30, 2010 at 8:06 AM, Neve Le wrote:> Siep Korteling wrote: >>> Xavier Noria wrote: >>> That 15 is hard-coded in the definition of Float#to_s. >> >> Yes, and exposing more digits is not accurate: >> The float is bogus  after the 16th digit. > > Working with big numbers here, it seems that precision is only > guaranteed to 15 digits on either side of the decimal. When a whole > number >15 digits is divided the decimal is approximated, and anything >>17 digits prints a float of n.0 no matter how it's "%n.nf" formatted. http://ruby-doc.org/core/classes/Float.html "Float objects represent real numbers using the native architectures double-precision floating point representation." http://en.wikipedia.org/wiki/Double_precision_floating-point_format You could try BigDecimal. From: Javier Goizueta on 30 Jun 2010 10:56 You may also want to take a look at http://flt.rubyforge.org/ It's an implementation of arbitrary precision floating point numbers (decimal and binary). If you don't need it to be fast it has some advantages over BigDecimal. -- Posted via http://www.ruby-forum.com/.  |  Pages: 1 Prev: optparser questionNext: Newbie