From: Peter Zijlstra on
On Fri, 2009-09-04 at 16:51 +1000, npiggin(a)suse.de wrote:
> +static void file_list_lock_all(void)
> +{
> + int i;
> + int nr = 0;
> +
> + for_each_possible_cpu(i) {
> + spinlock_t *lock;
> +
> + lock = &per_cpu(files_cpulock, i);
> + spin_lock_nested(lock, nr);
> + nr++;
> + }
> +}

Same here, this'll make lockdep explode on >8-way. And create large spin
times on large smp boxen.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Peter Zijlstra on
On Fri, 2009-09-04 at 16:51 +1000, npiggin(a)suse.de wrote:
> Improve scalability of files_lock by adding per-cpu, per-sb files lists,
> protected with per-cpu locking. Effectively turning it into a big-writer
> lock.

What I did was fine-grain locking the double linked list so that you can
delete items without hitting a global lock.

For addition I added per-cpu list-heads that would be spliced onto the
global list once in a while.

Granted, the code was a tad involved... and hch wanted to get rid of
these lists, which is of course a much better solution.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Nick Piggin on
On Mon, Sep 28, 2009 at 03:24:08PM +0200, Peter Zijlstra wrote:
> On Fri, 2009-09-04 at 16:51 +1000, npiggin(a)suse.de wrote:
> > Improve scalability of files_lock by adding per-cpu, per-sb files lists,
> > protected with per-cpu locking. Effectively turning it into a big-writer
> > lock.
>
> What I did was fine-grain locking the double linked list so that you can
> delete items without hitting a global lock.
>
> For addition I added per-cpu list-heads that would be spliced onto the
> global list once in a while.
>
> Granted, the code was a tad involved... and hch wanted to get rid of
> these lists, which is of course a much better solution.

I did see that of course, and I sent you a critique of it... I
didn't think it was appropriate for reasons I can't remember off
hand (either overly complex for the same task, or had a scalability
problem).

files_lock I would love to see go away completely, and in fact
depending on progress of work to that end, these patches may never
need to be merged. The problem I have is:

1. I don't want to significantly change data structures or cause
avoidable reductions in potential expressiveness of the data
structures we have. (I don't want someone to complain that my
patches suck because they want to be able to traverse files).

2. I need to take out this lock otherwise it become the choke
point and hides the rest of the progress on the rest of the
scalability work.

Again, I think brlock is not such a terrible thing for contention
especially if we're looking at umount slowpath... For this guy
actually though, the read side can probably be turned into RCU
traversal quite easily, I _think_.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/