Prev: chroot() and popen() won't get along
Next: Memory eating...
From: Casper H.S. Dik on 16 Apr 2010 05:33
jt(a)toerring.de (Jens Thoms Toerring) writes:

>In comp.os.linux.development.apps Jens Thoms Toerring <jt(a)toerring.de> wrote:
>> In comp.unix.programmer Mark Hobley <markhobley(a)hotpop.donottypethisbit.com> wrote:
>> > Can I also protect dynamically created structured arrays by having a mutex
>> > as part of the array definition?

>> > struct buffer {
>> > char *addr;
>> > unsigned int sz;
>> > unsigned int ptr;
>> > unsigned int tail;
>> > pthread_mutex_t buffermx; <--- Can I put the mutex here?
>> > };

>> > buffertable.addr = malloc(tablesize * sizeof(struct buffer));
>> > ^
>> > |
>> > I plan to have a mutex for dynamically allocated structures. This
>> > will enable me to apply a mutex to just one particular element of
>> > the array without affecting other elements. In this case, it is
>> > convenient to be able to bundle the mutex record in with the data
>> > structure, because sometimes the tablesize will be increased via a
>> > realloc call, and this would mean that new mutexes are required to
>> > work with the new elements.

>> Also that is legal - if you can put a 'pthread_mutex_t' in a
>> structure you can put it into an array of structures, be it
>> allocated statically or dynamically.

>Please don't listen to me but to Eric Sosman, I missed the bit
>with the reallocation and only considered a dynamically allo-
>cated array of such structures!

"struct buffer" and the "char *addr" should be allocated in two
different bits of memory; "struct buffer" shouldn't be reallocated but
reallocating *addr should work as long as it doesn't include more
mutexes.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
From: Mark Hobley on 16 Apr 2010 06:40
In comp.os.linux.development.apps Jens Thoms Toerring <jt(a)toerring.de> wrote:

> My impression isn't that 'addr' is meant as part of a linked list
> but as a pointer to an array of structures - the way the alloca-
> tion is done would seem to me to be very strange if really meant
> for an array of chars as the type of 'addr' (char *) is expres-
> sing. What would one do with a char array of a length that's a
> multiple of the length of a structure, given that the amount of
> memory allocated can depend on the compiler and the system being
> used?

Right. They are separate entities. The buffertable is the master record. This
tells us where the buffer information table (of type buffer) is allocated in
memory. The buffer information table can be dynamically expanded. The buffer
information table contains records of dynamically allocated string
manipulation buffers (of type char). The string manipulation buffers can be
resized dynamically using chunks. The buffer information table can be resized
to hold additional records in the event that more buffers are requested than
the table can currently hold.

Master Record ---> Buffer Information Table
(buffertable) (buffer)
Entry 1 ---> String manipulation buffer
(char[])
Entry 2 ---> String manipulation buffer
...
Entry N ---> String manipulation buffer


http://markhobley.yi.org/mlibrary/libbuffers/index.html

Mark.

--
Mark Hobley
Linux User: #370818 http://markhobley.yi.org/

From: Eric Sosman on 16 Apr 2010 09:41
On 4/15/2010 6:53 PM, Rick Jones wrote:
> In comp.unix.programmer Eric Sosman<esosman(a)ieee-dot-org.invalid> wrote:
>> To make it work, you would need to do something awful like
>
>> - make sure all the mutexes in the region are unlocked, and
>> that no other thread will try to lock any of them,
>
> That implies a higher-level lock yes? Perhaps a global read-write
> lock where normally a read lock is taken, but when one wants to mess
> with the structures one grabs the read/write as a write.

Something along those lines, yes. The higher-level lock
protects the array that holds the individual mutexes. Somebody
who wants to lock one of those mutexes does

pthread_rwlock_rdlock(&global_lock);
pthread_mutex_lock(&array[n].mutex);
...
pthread_mutex_unlock(&array[n].mutex);
pthread_rwlock_unlock(&global_lock);

(order is important). To reallocate the whole array one does

pthread_rwlock_wrlock(&global_lock);
for (i = 0; i < count; ++i)
pthread_mutex_destroy(&array[i].mutex);
count = new_count;
array = realloc(array, count * sizeof *array);
for (i = 0; i < count; ++i)
pthread_mutex_init(&array[i].mutex, &attrs);
pthread_rwlock_unlock(&global_lock);

(just a sketch; error-checking omitted for brevity). Seems to
me the phrase "something awful" is apt for this sort of thing ...

Maybe there's a slicker way to do it with semaphores? I'm
not very good at semaphores.

> BTW, for the OP - something similar would be required if one used a
> hash table of mutexes - in that case though you would walk through the
> hash table locking all the mutexes, then you can realloc the
> structure(s) protected by those mutexes, then you can unlock all the
> mutexes and let the other threads do their thing. [...]

Right. In this case, the mutexes themselves never move,
just the things they protect.

--
Eric Sosman
esosman(a)ieee-dot-org.invalid
From: Mark Hobley on 16 Apr 2010 13:15
In comp.unix.programmer Rick Jones <rick.jones2(a)hp.com> wrote:

> That implies a higher-level lock yes? Perhaps a global read-write
> lock where normally a read lock is taken, but when one wants to mess
> with the structures one grabs the read/write as a write.

The buffertable lock is the higher level lock. That means that the
buffertable is being manipulated.

I haven't actually implement locks on the buffers themselves, but I am thinking
that I could add support for multiple processes writing to or reading from a
single buffer (packet fashion). I don't need this yet, but I thought this might
be required in future.

If need be, I could just apply a single mutex for all buffers. This would
simplify the code, but this would mean that threads would have to wait, even
if the buffer that they are using is not being used elsewhere, although I
uess that the delays would only be minimal.

Another option could be to use linked lists for the mutexes, but I think the
overhead for this too great for the benefits that it provides. In all
probability there will probably only be one thread using the buffer library at
any given time.

Is there a way of applying a shared lock to a mutex? I don't necessarily need
an exclusive lock for reading processes. Reading processes could apply
a shared lock, preventing a write process from obtaining an exclusive lock
and updating the buffer until the reading process has finished.

Mark.

> Or you could nest
> under a read/write lock - but it means having a read/write lock's
> pathlength added to each access.

I don't know what that means. If it means that I am manipulating pathnames,
then the buffer library is required to be implemented before any string
manipulation code, because the string manipulation code will make use of the
libbuffers library in order to obtain a workspace to manipulate the strings.

Mark.

--
Mark Hobley
Linux User: #370818 http://markhobley.yi.org/

From: Rainer Weikusat on 16 Apr 2010 14:19
markhobley(a)hotpop.donottypethisbit.com (Mark Hobley) writes:
> In comp.unix.programmer Rick Jones <rick.jones2(a)hp.com> wrote:

[...]

> Is there a way of applying a shared lock to a mutex? I don't necessarily need
> an exclusive lock for reading processes. Reading processes could apply
> a shared lock, preventing a write process from obtaining an exclusive lock
> and updating the buffer until the reading process has finished.

Yes. This is a called an rwlock, the type begin pthread_rwlock_t and
using the ptrhead_rwlock_* functions.

>> Or you could nest
>> under a read/write lock - but it means having a read/write lock's
>> pathlength added to each access.
>
> I don't know what that means.

It means that the code path (sequence of sucessive IP/PC values)
necessary to perform an access would include the rwlock code.
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: chroot() and popen() won't get along
Next: Memory eating...