futex: FUTEX_LOCK with optional adaptive spinning
in [Kernel]
|
From: Peter Zijlstra on 6 Apr 2010 11:30 On Mon, 2010-04-05 at 13:23 -0700, Darren Hart wrote: > Lastly, I should mention that these results all underperform a simple > pthread_mutex_lock()/pthread_mutex_unlock() pair. I'm looking into why but felt > I was overdue in sharing what I have to date. A test comparing this to a > sched_yield() style userspace spinlock would probably be more appropraite. This really should be able to out-perform a regular pthread_mutex_lock() we saw a significant performance gain when we added adaptive spins to the kernel mutex implementation, so I'd expect a gain on the futex one as well. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Darren Hart on 6 Apr 2010 11:30 Alan Cox wrote: > On Tue, 06 Apr 2010 15:35:31 +0200 > Peter Zijlstra <peterz(a)infradead.org> wrote: > >> On Tue, 2010-04-06 at 16:28 +0300, Avi Kivity wrote: >>> Yes, but that's the best case for spinning. You could simply use a >>> userspace spinlock in this case. >> Userspace spinlocks are evil.. they should _never_ be used. > > Thats a gross and inaccurate simplification. For the case Avi is talking > about spinning in userspace makes sense in a lot of environments. Once > you've got one thread pinned per cpu (or gang scheduling >-) ) there are > various environments where it makes complete and utter sense. Hi Alan, Do you feel some of these situations would also benefit from some kernel assistance to stop spinning when the owner schedules out? Or are you saying that there are situations where pure userspace spinlocks will always be the best option? If the latter, I'd think that they would also be situations where sched_yield() is not used as part of the spin loop. If so, then these are not our target situations for FUTEX_LOCK_ADAPTIVE, which hopes to provide a better informed mechanism for making spin or sleep decisions. If sleeping isn't part of the locking construct implementation, then FUTEX_LOCK_ADAPTIVE doesn't have much to offer. Thanks, -- Darren Hart IBM Linux Technology Center Real-Time Linux Team -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Darren Hart on 6 Apr 2010 11:40 Peter Zijlstra wrote: > On Tue, 2010-04-06 at 07:47 -0700, Ulrich Drepper wrote: >> On Tue, Apr 6, 2010 at 01:48, Peter Zijlstra <peterz(a)infradead.org> >> wrote: >>> try >>> spin >>> try >>> syscall >> This is available for a long time in the mutex implementation >> (PTHREAD_MUTEX_ADAPTIVE_NP mutex type). It hasn't show much >> improvement if any. There were some people demanding this support for >> as far as I know they are not using it now. This is adaptive >> spinning, learning from previous calls how long to wait. But it's >> still unguided. There is no way to get information like "the owner >> has been descheduled". > > That's where the FUTEX_LOCK thing comes in, it does all those, the above > was a single spin loop to amortize the syscall overhead. > > I wouldn't make it any more complex than a single pause ins, syscalls > are terribly cheap these days. And yet they still seem to have a real impact on the futex_lock benchmark. Perhaps I am just still looking at pathological cases, but there is a strong correlation between high syscall counts and really low iterations per second. Granted this also correlates with lock contention. However, when using the same period and duty-cycle I find that a locking mechanism that makes significantly fewer syscalls also significantly outperforms one that makes more. Kind of handwavy stilly, I'll have more numbers this afternoon. -- Darren Hart IBM Linux Technology Center Real-Time Linux Team -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Peter Zijlstra on 6 Apr 2010 11:40 On Tue, 2010-04-06 at 08:33 -0700, Darren Hart wrote: > Peter Zijlstra wrote: > > On Tue, 2010-04-06 at 07:47 -0700, Ulrich Drepper wrote: > >> On Tue, Apr 6, 2010 at 01:48, Peter Zijlstra <peterz(a)infradead.org> > >> wrote: > >>> try > >>> spin > >>> try > >>> syscall > >> This is available for a long time in the mutex implementation > >> (PTHREAD_MUTEX_ADAPTIVE_NP mutex type). It hasn't show much > >> improvement if any. There were some people demanding this support for > >> as far as I know they are not using it now. This is adaptive > >> spinning, learning from previous calls how long to wait. But it's > >> still unguided. There is no way to get information like "the owner > >> has been descheduled". > > > > That's where the FUTEX_LOCK thing comes in, it does all those, the above > > was a single spin loop to amortize the syscall overhead. > > > > I wouldn't make it any more complex than a single pause ins, syscalls > > are terribly cheap these days. > > And yet they still seem to have a real impact on the futex_lock > benchmark. Perhaps I am just still looking at pathological cases, but > there is a strong correlation between high syscall counts and really low > iterations per second. Granted this also correlates with lock > contention. However, when using the same period and duty-cycle I find > that a locking mechanism that makes significantly fewer syscalls also > significantly outperforms one that makes more. Kind of handwavy stilly, > I'll have more numbers this afternoon. Sure, but I'm still not sure why FUTEX_LOCK ends up making more syscalls than FUTEX_WAIT based locking. Both should only do the syscall when the lock is contended, both should only ever do 1 syscall per acquire, right? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 6 Apr 2010 12:10
On 04/06/2010 06:28 PM, Darren Hart wrote: > Alan Cox wrote: >> On Tue, 06 Apr 2010 15:35:31 +0200 >> Peter Zijlstra <peterz(a)infradead.org> wrote: >> >>> On Tue, 2010-04-06 at 16:28 +0300, Avi Kivity wrote: >>>> Yes, but that's the best case for spinning. You could simply use a >>>> userspace spinlock in this case. >>> Userspace spinlocks are evil.. they should _never_ be used. >> >> Thats a gross and inaccurate simplification. For the case Avi is talking >> about spinning in userspace makes sense in a lot of environments. Once >> you've got one thread pinned per cpu (or gang scheduling >-) ) there are >> various environments where it makes complete and utter sense. > > Hi Alan, > > Do you feel some of these situations would also benefit from some > kernel assistance to stop spinning when the owner schedules out? Or > are you saying that there are situations where pure userspace > spinlocks will always be the best option? > > If the latter, I'd think that they would also be situations where > sched_yield() is not used as part of the spin loop. If so, then these > are not our target situations for FUTEX_LOCK_ADAPTIVE, which hopes to > provide a better informed mechanism for making spin or sleep > decisions. If sleeping isn't part of the locking construct > implementation, then FUTEX_LOCK_ADAPTIVE doesn't have much to offer. IMO the best solution is to spin in userspace while the lock holder is running, fall into the kernel when it is scheduled out. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |