Prev: C++ Memory Management Innovation: GC Allocator
Next: pitfalls of string for binary data
From: Lance Diduck on 22 Apr 2008 02:17
On Apr 21, 10:43 pm, marlow.and...(a)googlemail.com wrote:
> On 21 Apr, 19:13, xushiwei <xushiwe...(a)gmail.com> wrote:
> I took a quick look with particular interest
> in what you have to say about scoped allocators.
> It seems to me that there is some overlap with
> what you are doing and the work by Pablo Halpern in
> his N2523 submission to WG21 entitled "The Scoped Allocator
> Model". Seehttp://www.open-std.org/JTC1/SC22/WG21/docs/papers/2008/n2523.pdf
>
> IMO Pablo's proposal is more thought through.
> I recommend you take a look.
Pablo doesnt actually propose any allocator. It only mentions a
technique for applying allocators to containers. The only "scoped
allocator" in there is a wrapper around the system allocator.
Pablos work's doesnt mention anything on how to write allocators that
could be scoped. Rather, that if they could be written then this is
how to pass them from one container to another.
And that is the rub-- writing an allocator that could be "scoped" is
very hard indeed. The stated intent of "scoped allocators" is to
change the semantics of STL "regular type" such that any property that
does not contribute to operator== is a runtime policy, instead of
compile time policy. So as such it has little to do with allocators.
It is just that in std containers the allocator is the only thing that
is a policy that does not constribute to operator==.
So OK, now I have a way to propogate allocators from one container to
another. Now there are two questions 1) write a non trivial allocator
that *could* be scoped, and 2) just what problem am I solving by
passing this allocator instance from one container to the next?

xushiwei's work is far more relevant, in that it actually presents
something useful. It is not ready to be standardized, but I am
personally interested in writing up a few allocators for
standardization


> One final point, it seems to me that there is a need
> to bear thread safety in mind. You just say "it's not
> needed". Can you explain why please?

If you look at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2486.pdf
page three you will see a example of this usage. The idea is this
void * threadfunc(void*){

std::list<X> x;//assume X allocates no memory
x.resize(10000);
}
now, when I start up N threads, the list instances *should* be
independent, but this does not happen. Rather, they contend via their
allocator.
So the idea is this
void * threadfunc2(void*){
SpecialAlloc sa;//grab memory from stack, or in big chunks from
the heap
std::list<X, SpecialAlloc > x(sa);
x.resize(10000);
}
Now each thread is isolated, nor does SpecialAlloc need to be thread
safe.

The difference can be dramatic: See
http://groups.google.com/group/comp.lang.c++.moderated/browse_thread/thread/a1eaab58c5bb390c/b763830e33a45985?hl=en&tvc=2#b763830e33a45985
for a test I did. The formatting is not the best, but you can get the
idea. In this case I used a lock-free allocator, but when I repeated
the test using something like SpecialAlloc, I got better performance
still.


--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: xushiwei on 22 Apr 2008 02:17
On 4��22��, ����10ʱ43��, marlow.and...(a)googlemail.com wrote:
>
> I took a quick look with particular interest
> in what you have to say about scoped allocators.
> It seems to me that there is some overlap with
> what you are doing and the work by Pablo Halpern in
> his N2523 submission to WG21 entitled "The Scoped Allocator
> Model". Seehttp://www.open-std.org/JTC1/SC22/WG21/docs/papers/2008/n2523.pdf
>
> IMO Pablo's proposal is more thought through.
> I recommend you take a look.

Before I post this article, I read submissions to WG21 related to
Allocator, such as n1850, n1509, n2387, n2523(n2446), n2436, n2486,
n2524, n2554.

You mentioned "The Scoped Allocator Model", so I read it though again
(http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2008/n2523.pdf and
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2446.pdf). I
don't find common points between them except the similar name:-) Would
you point out them for me?

"GC Allocator" focus:

1. It's a better "Smart Pointer".
2. Deleting objects manually is no need more if you use "GC
Allocator".
3. Use non-static allocator instances to allocate memory instead of
global new/delete procedures. Then multithreaded locks are OPTIONAL.
You can use explicit locks if you need.

>
> [ huge prose snipped ]
> The prose snipped seems to be a copy of the PDF article.
> I am suprised the moderator did not comment on this.
> It makes your post very long.
>
> > When you creating a GC Allocator, You can use STD_NEW, STD_NEW_ARRAY
> > to new objects.
> > Frankly speaking, I don't like STD_NEW and STD_NEW_ARRAY.
>
> Take a look at Pablo's article to see an alternative
> approach that does not use macros or concepts.
>

In fact, I can use a series of functions instead of STD_NEW. For
instance:

template <typename Type, typename AllocT>
Type* New(AllocT& alloc) // new(alloc) Type
{
void* p = alloc.allocate(sizeof(Type),
DestructTraits<Type>::destruct);
return new(p) Type;
}

template <typename Type, typename AllocT, typename ArgT1>
Type* New(AllocT& alloc, ArgT1 arg1) // new(alloc) Type(arg1)
{
void* p = alloc.allocate(sizeof(Type),
DestructTraits<Type>::destruct);
return new(p) Type(arg1);
}

....

Then you can use New<Type>(alloc, arg1, ...) instead of STD_NEW(alloc,
Type)(arg1, ...).

But I think it is better if I can get type-info when overriding
operator new:

template <typename Type, typename AllocT>
inline void* operator _new(AllocT& alloc)
{
return alloc.allocate(sizeof(Type), DestructTraits<Type>::destruct);
}

template <typename Type, typename AllocT>
inline void* operator _new[](size_t count, AllocT& alloc)
{
return DestructTraits<Type>::allocArrayBuf(alloc, count);
}

When new(alloc) Type(arg1, ...) is called, the C++ compiler will
detect if there is a mataching operator _new<Type>. If there isn't,
then it will try a normal operator new.

> One final point, it seems to me that there is a need
> to bear thread safety in mind. You just say "it's not
> needed". Can you explain why please?

** No Multithreaded Locks **

Why doesn't GC Allocator need multithreaded locks?

Memory allocation = System memory block management + Memory management
algorithm

The underlying system memory block management is provided by OS. It
will optimize large memory block allocation (allocating small objects
is supported, but doesn't need to optimize). And It is thread/process
safe.

Memory management algorithm is provided by C/C++ runtime library, or
other libraries. Memory management algorithms ARE only algorithms.
Most of them are designed to be thread safe.

If we use global new/delete or malloc/free procedures to allocate
memory, thread safe is a MUST (because all threads use these
procedures), not OPTIONAL. But if we use allocator instances to manage
memory, then thread safe becomes OPTIONAL.

Why? Sharing GC Allocator in multi threads is not recommended. It
means that sharing memory between threads is also not recommended. If
you REALLY want to share memory, use new/delete or anything else.

For users of GC Allocator, we suggest that only use ONE BlockPool
instance in ONE thread, and ONE thread may use multiple PRIVATE
"ScopeAlloc" instances (depend on your requirement) to allocate
memory.


Regards,

shiwei xu


--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: Chris Thomasson on 22 Apr 2008 06:19
"xushiwei" <xushiweizh(a)gmail.com> wrote in message
news:8c691ff1-f4c5-48dd-a87a-23957295f5cd(a)m1g2000pre.googlegroups.com...
> On 4��22��, ����10ʱ43��, marlow.and...(a)googlemail.com wrote:
>>
>> I took a quick look with particular interest
>> in what you have to say about scoped allocators.
>> It seems to me that there is some overlap with
>> what you are doing and the work by Pablo Halpern in
>> his N2523 submission to WG21 entitled "The Scoped Allocator
>> Model".
>> Seehttp://www.open-std.org/JTC1/SC22/WG21/docs/papers/2008/n2523.pdf
>>
>> IMO Pablo's proposal is more thought through.
>> I recommend you take a look.
>
[...]
>> One final point, it seems to me that there is a need
>> to bear thread safety in mind. You just say "it's not
>> needed". Can you explain why please?
>
> ** No Multithreaded Locks **
>
> Why doesn't GC Allocator need multithreaded locks?
>
> Memory allocation = System memory block management + Memory management
> algorithm
[...]

Your using per-thread allocation, what's new about that? Anyway, I don't
think I can use your design in any of my programs. Most are based on
message-passing or reader-writer patterns which require the ability to
allocate in one thread and free in another. This is a must. AFAICT, your not
addressing this aspect, therefore, I cannot use your work.



--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: Chris Thomasson on 22 Apr 2008 06:19
"Lance Diduck" <lancediduck(a)nyc.rr.com> wrote in message
news:d2adf943-d612-4682-9315-1e7e94d7053c(a)8g2000hse.googlegroups.com...
> On Apr 21, 10:43 pm, marlow.and...(a)googlemail.com wrote:
>> On 21 Apr, 19:13, xushiwei <xushiwe...(a)gmail.com> wrote:
[...]
>> One final point, it seems to me that there is a need
>> to bear thread safety in mind. You just say "it's not
>> needed". Can you explain why please?
[...]
> Now each thread is isolated, nor does SpecialAlloc need to be thread
> safe.
>
> The difference can be dramatic: See
> http://groups.google.com/group/comp.lang.c++.moderated/browse_thread/thread/a1eaab58c5bb390c/b763830e33a45985?hl=en&tvc=2#b763830e33a45985
> for a test I did. The formatting is not the best, but you can get the
> idea. In this case I used a lock-free allocator, but when I repeated
> the test using something like SpecialAlloc, I got better performance
> still.

What's the difference between xushiwei's work and all of the allocators out
there that are based on per-thread heaps? Answer, xushiwei's design does not
allow you to pass memory between threads, and the others do. This is MAJOR
problem. Please explain how your going to implement message-passing? How are
you going to implement ANY producer/consumer patters using xushiwei's
design? Allocators like StreamFlow and Hoard don't use any atomic operations
or memory barriers to allocate from a per-thread heap, yet they still allow
one to create in thread A and free in thread B.


--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: xushiwei on 22 Apr 2008 06:22
In fact, "GC Allocator" is NOT "GC".

"GC Allocator" focus:

1. It is a better "Smart Pointer".
2. Deleting objects manually is no need if you use "GC Allocator".
3. Use non-static allocator instances to allocate memory instead of
global new/delete procedures. Then multithreads locks are OPTIONAL and
you can use explicit locks if you need.

You can consider "GC Allocator" as a "Smart Pointer" or a "Allocator",
but It isn't a "GC".

Regards,

Shiwei Xu

On 4��22��, ����10ʱ13��, "Chris Thomasson" <cris...(a)comcast.net> wrote:
>
> I just want to know if I can use this proposal to efficiently allocate in
> one thread and free in another. As for the GC aspect, well, that's fine as
> long as there are no mandatory requirements. I personally would think about
> dropping C++ _if_ it "forced" me to use a GC. Even if the GC was
> transparent, I need to be able to manually manage memory most of the time. I
> don't want a GC running under my nose, so-to-speak... IMVHO, that would go
> against the low-level nature of C++. I want to be able to use C++ to create
> state-of-the-art allocators and garbage collectors... OS kernels, device
> drivers and other low-level applications... Its the low-level nature of C++
> that's particularly attractive... This is only my personal take on the
> matter... Indeed.


--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

 |  Next  |  Last
Pages: 1 2 3
Prev: C++ Memory Management Innovation: GC Allocator
Next: pitfalls of string for binary data