Anomalous performance increase from mutex profiling

On Mon, 17 Apr 2006, Kris Kennaway wrote:

> On Mon, Apr 17, 2006 at 01:14:40AM -0400, Kris Kennaway
wrote:
>
>> * Our best guess is that mutex profiling is doing
something that
>> reduces contention on this very heavily contended
mutex (unp), but I'd
>> like to know what is happening precisely so I can
maybe make use of
>> it.
>>
>> Can anyone think of what may be happening that
I've missed?
>
> I think it is just doing effectively the same thing as
my exponential
> spin backoff patch, namely introducing delays with the
effect of
> reducing common memory accesses.  When I turn the
maximum spin backoff
> limit *way* up (from 1600 to 51200) I get performance
that slightly
> exceeds what I see from mutex profiling alone (adding
mutex profiling
> again on top of this gives a small further increase,
but only a few %
> and so probably achievable by further increasing the
backoff limit).
>
> A limit of 51200 is not an appropriate default since it
penalizes the
> common case of light to moderate contention.  The point
is that here
> basically all 12 CPUs are spinning on a single lock
> (kern/uipc_usrreq.c:308), so it's massively
over-contended and all we
> can do is mitigate the effects of this.
>
> On this system, the maximum supersmack performance
(3700 queries/sec)
> comes when there are only 6 clients, so (as jasone
eloquently put it)
> with 10 clients the difference between 2300 queries/sec
(with absurdly
> high backoff limits or mutex profiling) and 1450/sec
(with reasonable
> backoff limits) is the difference between
"slow" and "ass slow".

Please excuse if this is a stupid question - but might using
MCS or
QOLB locks in this situation be useful?
_______________________________________________
freebsd-smpfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-smp
To unsubscribe, send any mail to
"freebsd-smp-unsubscribefreebsd.org"

On Mon, Apr 17, 2006 at 03:54:07AM -0400, Surer Dink wrote:
> On Mon, 17 Apr 2006, Kris Kennaway wrote:
> 
> > On Mon, Apr 17, 2006 at 01:14:40AM -0400, Kris
Kennaway wrote:
> >
> >> * Our best guess is that mutex profiling is
doing something that
> >> reduces contention on this very heavily
contended mutex (unp), but I'd
> >> like to know what is happening precisely so I
can maybe make use of
> >> it.
> >>
> >> Can anyone think of what may be happening that
I've missed?
> >
> > I think it is just doing effectively the same
thing as my exponential
> > spin backoff patch, namely introducing delays with
the effect of
> > reducing common memory accesses.  When I turn the
maximum spin backoff
> > limit *way* up (from 1600 to 51200) I get
performance that slightly
> > exceeds what I see from mutex profiling alone
(adding mutex profiling
> > again on top of this gives a small further
increase, but only a few %
> > and so probably achievable by further increasing
the backoff limit).
> >
> > A limit of 51200 is not an appropriate default
since it penalizes the
> > common case of light to moderate contention.  The
point is that here
> > basically all 12 CPUs are spinning on a single
lock
> > (kern/uipc_usrreq.c:308), so it's massively
over-contended and all we
> > can do is mitigate the effects of this.
> >
> > On this system, the maximum supersmack performance
(3700 queries/sec)
> > comes when there are only 6 clients, so (as jasone
eloquently put it)
> > with 10 clients the difference between 2300
queries/sec (with absurdly
> > high backoff limits or mutex profiling) and
1450/sec (with reasonable
> > backoff limits) is the difference between
"slow" and "ass slow".
> 
> Please excuse if this is a stupid question - but might
using MCS or
> QOLB locks in this situation be useful?

What are they?

Kris

On 4/17/06, Kris Kennaway <krisobsecurity.org> wrote:
> On Mon, Apr 17, 2006 at 03:54:07AM -0400, Surer Dink
wrote:
> > Please excuse if this is a stupid question - but
might using MCS or
> > QOLB locks in this situation be useful?
>
> What are they?

Mellor-Crummy Scott:
http://www.cs.rochester.edu/u/scott/papers/1991_A
SPLOS_sync.pdf

An overview comparing various possible optimizations for a
few lock
types, including MCS and QOLB:
ftp://ftp.cs.utexas.edu/pub/dburger/papers/ISCA97_qolb.pdf

I believe the QOLB proposal only suggested hardware
modificaition for
performance improvement, but could be implemented entirely
in software
- the overheads are high, but offer substantial performance
benefit in
high contention situations.  MCS is based on QOSB, however
fully
implemented in software.

There is also a proposal for changing lock to MCS
dynamically, however
I have not read it:
ftp://ftp.cag.lcs.mit.edu/pub/papers/pdf/reactive.pdf
_______________________________________________
freebsd-smpfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-smp
To unsubscribe, send any mail to
"freebsd-smp-unsubscribefreebsd.org"