From: Hrvoje Niksic on
"Alf P. Steinbach" <alfps(a)start.no> writes:

> Speedup would likely be more realistic with normal implementation (not
> fiddling with bit-fields and stuff)

I'm not sure I understand this. How would you implement tagged integers
without encoding type information in bits of the pointer value?
From: Alf P. Steinbach on
* Hrvoje Niksic:
> "Alf P. Steinbach" <alfps(a)start.no> writes:
>
>> Speedup would likely be more realistic with normal implementation (not
>> fiddling with bit-fields and stuff)
>
> I'm not sure I understand this. How would you implement tagged integers
> without encoding type information in bits of the pointer value?

A normal tag field, as illustrated in code earlier in the thread.


Cheers & hth.,

- Alf
From: Hrvoje Niksic on
"Alf P. Steinbach" <alfps(a)start.no> writes:

> * Hrvoje Niksic:
>> "Alf P. Steinbach" <alfps(a)start.no> writes:
>>
>>> Speedup would likely be more realistic with normal implementation (not
>>> fiddling with bit-fields and stuff)
>>
>> I'm not sure I understand this. How would you implement tagged integers
>> without encoding type information in bits of the pointer value?
>
> A normal tag field, as illustrated in code earlier in the thread.

Ah, I see it now. That proposal effectively doubles the size of what is
now a PyObject *, meaning that lists, dicts, etc., would also double
their memory requirements, so it doesn't come without downsides. On the
other hand, tagged pointers have been used in various Lisp
implementations for decades, nothing really "baroque" (or inherently
slow) about them.
From: Alf P. Steinbach on
* Hrvoje Niksic:
> "Alf P. Steinbach" <alfps(a)start.no> writes:
>
>> * Hrvoje Niksic:
>>> "Alf P. Steinbach" <alfps(a)start.no> writes:
>>>
>>>> Speedup would likely be more realistic with normal implementation (not
>>>> fiddling with bit-fields and stuff)
>>> I'm not sure I understand this. How would you implement tagged integers
>>> without encoding type information in bits of the pointer value?
>> A normal tag field, as illustrated in code earlier in the thread.
>
> Ah, I see it now. That proposal effectively doubles the size of what is
> now a PyObject *, meaning that lists, dicts, etc., would also double
> their memory requirements, so it doesn't come without downsides.

Whether it increases memory usage depends on the data mix in the program's
execution.

For a program primarily handling objects of atomic types (like int) it saves
memory, since each value (generally) avoids a dynamically allocated object.

Bit-field fiddling may save a little more memory, and is nearly guaranteed to
save memory.

But memory usage isn't an issue except to the degree it affects the OS's virtual
memory manager.

Slowness is an issue -- except that keeping compatibility is IMO a more
important issue (don't fix, at cost, what works).


> On the
> other hand, tagged pointers have been used in various Lisp
> implementations for decades, nothing really "baroque" (or inherently
> slow) about them.

Unpacking of bit fields generally adds overhead. The bit fields need to be
unpacked for (e.g.) integer operations.

Lisp once ran on severely memory constrained machines.


Cheers & hth.,

- Alf
From: Terry Reedy on
Alf P. Steinbach wrote:
> * Hrvoje Niksic:
>> "Alf P. Steinbach" <alfps(a)start.no> writes:
>>
>>> * Hrvoje Niksic:
>>>> "Alf P. Steinbach" <alfps(a)start.no> writes:
>>>>
>>>>> Speedup would likely be more realistic with normal implementation (not
>>>>> fiddling with bit-fields and stuff)
>>>> I'm not sure I understand this. How would you implement tagged
>>>> integers
>>>> without encoding type information in bits of the pointer value?
>>> A normal tag field, as illustrated in code earlier in the thread.
>>
>> Ah, I see it now. That proposal effectively doubles the size of what is
>> now a PyObject *, meaning that lists, dicts, etc., would also double
>> their memory requirements, so it doesn't come without downsides.
>
> Whether it increases memory usage depends on the data mix in the
> program's execution.
>
> For a program primarily handling objects of atomic types (like int) it
> saves memory, since each value (generally) avoids a dynamically
> allocated object.
>
> Bit-field fiddling may save a little more memory, and is nearly
> guaranteed to save memory.
>
> But memory usage isn't an issue except to the degree it affects the OS's
> virtual memory manager.
>
> Slowness is an issue -- except that keeping compatibility is IMO a
> more important issue (don't fix, at cost, what works).

I believe the use of tagged pointers has been considered and so far
rejected by the CPython developers. And no one else that I know of has
developed a fork for that. It would seem more feasible with 64 bit
pointers where there seem to be spare bits. But CPython will have to
support 32 bit machines for several years.

Terry Jan Reedy