From: david mainzer on

Dear Python-User,


today i create some slides about floating point arithmetic. I used an
example from

http://docs.python.org/tutorial/floatingpoint.html

so i start the python shell on my linux machine:

dm(a)maxwell $ python
Python 2.6.5 (release26-maint, May 25 2010, 12:37:06)
[GCC 4.3.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> >>> sum = 0.0
>>> >>> for i in range(10):
.... sum += 0.1
....
>>> >>> sum
0.99999999999999989
>>> >>>
But thats looks a little bit wrong for me ... i must be a number greater
then 1.0 because 0.1 = 0.100000000000000005551115123125782702118158340454101562500000000000
in python ... if i print it.

So i create an example program:

sum = 0.0
n = 10
d = 1.0 / n
print "%.60f" % ( d )
for i in range(n):
print "%.60f" % ( sum )
sum += d

print sum
print "%.60f" % ( sum )


-------- RESULTs ------
0.100000000000000005551115123125782702118158340454101562500000
0.000000000000000000000000000000000000000000000000000000000000
0.100000000000000005551115123125782702118158340454101562500000
0.200000000000000011102230246251565404236316680908203125000000
0.300000000000000044408920985006261616945266723632812500000000
0.400000000000000022204460492503130808472633361816406250000000
0.500000000000000000000000000000000000000000000000000000000000
0.599999999999999977795539507496869191527366638183593750000000
0.699999999999999955591079014993738383054733276367187500000000
0.799999999999999933386618522490607574582099914550781250000000
0.899999999999999911182158029987476766109466552734375000000000
1.0
0.999999999999999888977697537484345957636833190917968750000000

and the jump from 0.50000000000000*** to 0.59999999* looks wrong
for me ... do i a mistake or is there something wrong in the
representation of the floating points in python?

my next question, why could i run

print "%.66f" % ( sum )

but not

print "%.67f" % ( sum )

can anybody tell me how python internal represent a float number??


Best and many thanks in advanced,
David

From: Mark Dickinson on
On Jul 7, 1:05 pm, david mainzer <d...(a)tu-clausthal.de> wrote:
> Dear Python-User,
>
> today i create some slides about floating point arithmetic. I used an
> example from
>
> http://docs.python.org/tutorial/floatingpoint.html
>
> so i start the python shell on my linux machine:
>
> dm(a)maxwell $ python
> Python 2.6.5 (release26-maint, May 25 2010, 12:37:06)
> [GCC 4.3.4] on linux2
> Type "help", "copyright", "credits" or "license" for more information.>>> >>> sum = 0.0
> >>> >>> for i in range(10):
>
> ...     sum += 0.1
> ...>>> >>> sum
> 0.99999999999999989
>
> But thats looks a little bit wrong for me ... i must be a number greater
> then 1.0 because 0.1 = 0.100000000000000005551115123125782702118158340454101562500000000000
> in python ... if i print it.

So you've identified one source of error here, namely that 0.1 isn't
exactly representable (and you're correct that the value stored
internally is actually a little greater than 0.1). But you're
forgetting about the other source of error in your example: when you
do 'sum += 0.1', the result typically isn't exactly representable, so
there's another rounding step going on. That rounding step might
produce a number that's smaller than the actual exact sum, and if
enough of your 'sum += 0.1' results are rounded down instead of up,
that would easily explain why the total is still less than 1.0.

>
> So i create an example program:
>
> sum = 0.0
> n = 10
> d = 1.0 / n
> print "%.60f" % ( d )
> for i in range(n):
>     print "%.60f" % ( sum )
>     sum += d
>
> print sum
> print "%.60f" % ( sum )
>
> -------- RESULTs ------
> 0.100000000000000005551115123125782702118158340454101562500000
> 0.000000000000000000000000000000000000000000000000000000000000
> 0.100000000000000005551115123125782702118158340454101562500000
> 0.200000000000000011102230246251565404236316680908203125000000
> 0.300000000000000044408920985006261616945266723632812500000000
> 0.400000000000000022204460492503130808472633361816406250000000
> 0.500000000000000000000000000000000000000000000000000000000000
> 0.599999999999999977795539507496869191527366638183593750000000
> 0.699999999999999955591079014993738383054733276367187500000000
> 0.799999999999999933386618522490607574582099914550781250000000
> 0.899999999999999911182158029987476766109466552734375000000000
> 1.0
> 0.999999999999999888977697537484345957636833190917968750000000
>
> and the jump from 0.50000000000000*** to 0.59999999* looks wrong
> for me ... do i a mistake or is there something wrong in the
> representation of the floating points in python?

Look at this more closely: you're adding

0.500000000000000000000000....

to

0.1000000000000000055511151231257827021181583404541015625

The *exact* result is, of course

0.6000000000000000055511151231257827021181583404541015625

However, that's not a number that can be exactly represented as a C
double (which is how Python stores floats internally). This value
falls between the two (consecutive) representable values:

0.59999999999999997779553950749686919152736663818359375

and

0.600000000000000088817841970012523233890533447265625

But of these two, the first is closer to the exact value than the
second, so that's the result that you get.

You can convince yourself of these results by using the fractions
module to do exact arithmetic:

>>> from fractions import Fraction
>>> tenth = Fraction.from_float(0.1)
>>> half = Fraction.from_float(0.5)
>>> point_six = Fraction.from_float(0.6) # 0.599999999999
>>> point_six_plus = Fraction.from_float(0.6 + 2**-53) # next float up, 0.6000000
>>> sum = tenth + half # exact value of the sum
>>> point_six < sum < point_six_plus # lies between point_six and point_six_plus
True
>>> sum - point_six < point_six_plus - sum # but it's closer to point_six
True


> my next question, why could i run
>
> print "%.66f" % ( sum )
>
> but not
>
> print "%.67f" % ( sum )

That's a historical artefact resulting from use of a fixed-length
buffer somewhere deep in Python's internals. This restriction is
removed in Python 2.7 and Python 3.x.

> can anybody tell me how python internal represent a float number??

In CPython, it's stored as a C double, which typically means in IEEE
754 binary64 format. (Of course, since it's a Python object, it's not
just storing the C double itself; it also has fields for the object
type and the reference count, so a Python float typically takes 16
bytes of memory on a 32-bit machine, and 24 bytes on a 64-bit
machine.)

--
Mark
From: Thomas Jollans on
On 07/07/2010 02:05 PM, david mainzer wrote:
> today i create some slides about floating point arithmetic. I used an
> example from
>
> http://docs.python.org/tutorial/floatingpoint.html
>
> so i start the python shell on my linux machine:
>
> dm(a)maxwell $ python
> Python 2.6.5 (release26-maint, May 25 2010, 12:37:06)
> [GCC 4.3.4] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>>>>> sum = 0.0
>>>>>>> for i in range(10):
> ... sum += 0.1
> ...
>>>>>>> sum
> 0.99999999999999989
>>>>>>>
> But thats looks a little bit wrong for me ... i must be a number greater
> then 1.0 because 0.1 = 0.100000000000000005551115123125782702118158340454101562500000000000
> in python ... if i print it.

The simple fact of the matter is: floating point arithmetic isn't
accurate. This has nothing to do with Python, it's the fault of your
processor's floating point handling. It's good enough in most cases, but
you should never rely on a floating-point number to have exactly a
certain value. It won't work. To generalize your example a bit:

>>> def test_floating_product(a, b):
.... sum = 0
.... for _ in range(int(b)):
.... sum += a
.... return sum, a * int(b), sum == a * b
....
>>> test_floating_product(0.1, 1)
(0.1, 0.1, True)
>>> test_floating_product(0.1, 10)
(0.9999999999999999, 1.0, False)
>>> test_floating_product(0.2, 4)
(0.8, 0.8, True)
>>> test_floating_product(0.2, 5)
(1.0, 1.0, True)
>>> test_floating_product(0.2, 6)
(1.2, 1.2000000000000002, False)
>>>


> -------- RESULTs ------
> 0.100000000000000005551115123125782702118158340454101562500000
> 0.000000000000000000000000000000000000000000000000000000000000
> 0.100000000000000005551115123125782702118158340454101562500000
> 0.200000000000000011102230246251565404236316680908203125000000
> 0.300000000000000044408920985006261616945266723632812500000000
> 0.400000000000000022204460492503130808472633361816406250000000
> 0.500000000000000000000000000000000000000000000000000000000000
> 0.599999999999999977795539507496869191527366638183593750000000
> 0.699999999999999955591079014993738383054733276367187500000000
> 0.799999999999999933386618522490607574582099914550781250000000
> 0.899999999999999911182158029987476766109466552734375000000000
> 1.0
> 0.999999999999999888977697537484345957636833190917968750000000
>
> and the jump from 0.50000000000000*** to 0.59999999* looks wrong
> for me ... do i a mistake or is there something wrong in the
> representation of the floating points in python?

the difference is almost exactly 0.1, so that looks okay

>
> my next question, why could i run
>
> print "%.66f" % ( sum )
>
> but not
>
> print "%.67f" % ( sum )

I can run either... with Python 3.1. Using 2.6, I get a nice error message:

>>> "%.129f" % 0.1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: formatted float is too long (precision too large?)

There just isn't anything like 67 decimals of information available.
Having that information wouldn't help you a bit.

basically, floating point number are stored in the format

N * (2 ** E)

And use a lot of guesswork. as E gets larger, the precision decreases.
Rounding errors occur at the last few decimals, in either direction,
depending on the numbers.


>
> can anybody tell me how python internal represent a float number??
>
From: Christian Heimes on
> can anybody tell me how python internal represent a float number??

It's an IEEE 754 double precision float on all hardware platforms that
support IEEE 754 semantics. Python follows the C99 standards for double
and complex numbers.

Christian

From: Philip Semanchuk on

On Jul 7, 2010, at 9:08 AM, Thomas Jollans wrote:

> On 07/07/2010 02:05 PM, david mainzer wrote:
>> today i create some slides about floating point arithmetic. I used an
>> example from
>>
>> http://docs.python.org/tutorial/floatingpoint.html
>>
>> so i start the python shell on my linux machine:
>>
>> dm(a)maxwell $ python
>> Python 2.6.5 (release26-maint, May 25 2010, 12:37:06)
>> [GCC 4.3.4] on linux2
>> Type "help", "copyright", "credits" or "license" for more
>> information.
>>>>>>>> sum = 0.0
>>>>>>>> for i in range(10):
>> ... sum += 0.1
>> ...
>>>>>>>> sum
>> 0.99999999999999989
>>>>>>>>
>> But thats looks a little bit wrong for me ... i must be a number
>> greater
>> then 1.0 because 0.1 =
>> 0.100000000000000005551115123125782702118158340454101562500000000000
>> in python ... if i print it.
>
> The simple fact of the matter is: floating point arithmetic isn't
> accurate. This has nothing to do with Python, it's the fault of your
> processor's floating point handling. It's good enough in most cases,
> but
> you should never rely on a floating-point number to have exactly a
> certain value. It won't work.

Yes, this might be a good time to review the dense but interesting
document, "What Every Computer Scientist Should Know About Floating-
Point Arithmetic":
http://docs.sun.com/source/806-3568/ncg_goldberg.html


Cheers
Philip