mutex fault and no symbol table on recent Dom0 and Xen

Hi,

I'm having difficulties with recent Dom0 kernel.  It
crashes
with following message.

   Mutex error: mutex_vector_exit: exiting unheld spin
mutex

I tried Dom0 from both 20070627 and 20070701.  Both crashed
because of mutex error.  Here is a crash log from 0627. 
Not
sure why, but db didn't show symbol table.

----
Mutex error: mutex_vector_exit: exiting unheld spin mutex

lock address : 0x00000000c0971620
current cpu  :                  0
current lwp  : 0x00000000cae21d80
owner field  : 0x0000000000000b00 wait/spin:	0/1

panic: lock error
Stopped in pid 0.2 (system) at  0xc04b5e54:     popl   
%ebp
db> bt
?(c08591c9,cae45ef4,cae45ee8,c0434437,b) at 0xc04b5e54
?(c083756d,c0836356,c06ea070,c083633c,c0971620) at
0xc04343c5
?(0,c0971620,c08cb50c,c06ea070,c083633c) at 0xc042f9f1
?(c0971620,c06ea070,c083633c,c148a500,975aa0) at 0xc0410c9c
?(c0971620,1,cae45f88,c06b932c,c148a500) at 0xc0410e7f
?(0,0,cae45fb8,c04c6026,0) at 0xc0426bf7
?(0,cae45fa8,0,f,fffff000) at 0xc04ee2cf
?(0,cad96bdc,0,cae45fec,c04ee2db) at 0xc0103c8d
?(f,cad96bdc,cad96b94,0,ffffffff) at 0xc04c6627
--- switch to interrupt stack ---
?(cad96bdc,0,11,31,ca000011) at 0xc01024e0
?(c0407e30,cae21d80,cad96c6c,c0407ed5,cae21d80) at
0xc010416b
?(cae21d80,cad96c68,c040f9fe,0,cae21d80) at 0xc04b8585
?(cae21d80,0,c01001e7,c01001df,c01001e7) at 0xc0407ed5
db> reboot
syncing disks... Mutex error: mutex_vector_enter: locking
against myself

lock address : 0x00000000c096fed0
current cpu  :                  0
current lwp  : 0x00000000cae21d80
owner field  : 0x0000000000010b00 wait/spin:	0/1

panic: lock error
Stopped in pid 0.2 (system) at  0xc04b5e54:     popl   
%ebp
db>
----

Another problem is that db doesn't recognize kernel symbol
table like above log.  I put kernel to boot at
/stand/20070701/netbsd-XEN3_DOM0 and hard link it to
/netbsd
for symbol table reading.  This worked fine before with
Xen3.0.4 and Dom0 at May.  Is there anything I need to do
for recent kernel or Xen?  Thanks.

-- Kazushi
Millihelen, adj:
	The amount of beauty required to launch one ship.

On Wed, Aug 08, 2007 at 06:23:06PM +0200, Christoph Egger
wrote:
> 
> > On Wed, Aug 08, 2007 at 10:29:45AM +0200,
Christoph Egger wrote:
> > > 
> > > Good news: The symbol table is back again.
The fix is upstream
> > > in xen-unstable. The elf code in Xen got
reworked between Xen 3.0.4
> > > and Xen 3.1. During that rework, elf symbol
loading got broken in the
> > way
> > > that BSD_SYMTAB got still parsed, but not
handled.
> > > This is handled in xen-unstable again.
> > 
> > this is great, thanks ! Do you know if this can
easily be applied to
> > the stable branch ?
> 
> I think so. Just take the changesets 15672, 15678 and
15679.
> These changesets touches files that haven't been
changed
> otherwise since Xen 3.1.
> 
> 
> 
> The following links show you the changelog and the
diffs.
> 
> 15672:
> http://xenbits.xensource.com/xen-unstable.hg/rev/88
bb0d305308
> 
> 15678:
> http://xenbits.xensource.com/xen-unstable.hg/rev/78
be123257fa
> 
> 15679:
> http://xenbits.xensource.com/xen-unstable.hg/rev/7e
e30cf72fd3
> 
> 15672 is my original patch a little modified by
XenSource.
> 15678 fixed a bug introduced via modifications from
XenSource.
> 15679 is a cleanup by XenSource and tested by me on
both Dom0 and DomU.
> 
> All three changesets merged together results in a
relativ
> small patch.

Thanks ! I've been able to apply it to Xen 3.1.0. I updated
the xenkernel3
to build the kernel from source, with this patch.

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--

> On Wed, Aug 08, 2007 at 06:23:06PM +0200, Christoph
Egger wrote:
> > 
> > > On Wed, Aug 08, 2007 at 10:29:45AM +0200,
Christoph Egger wrote:
> > > > 
> > > > Good news: The symbol table is back
again. The fix is upstream
> > > > in xen-unstable. The elf code in Xen got
reworked between Xen 3.0.4
> > > > and Xen 3.1. During that rework, elf
symbol loading got broken in
> the
> > > way
> > > > that BSD_SYMTAB got still parsed, but
not handled.
> > > > This is handled in xen-unstable again.
> > > 
> > > this is great, thanks ! Do you know if this
can easily be applied to
> > > the stable branch ?
> > 
> > I think so. Just take the changesets 15672, 15678
and 15679.
> > These changesets touches files that haven't been
changed
> > otherwise since Xen 3.1.
> > 
> > 
> > 
> > The following links show you the changelog and the
diffs.
> > 
> > 15672:
> > http://xenbits.xensource.com/xen-unstable.hg/rev/88
bb0d305308
> > 
> > 15678:
> > http://xenbits.xensource.com/xen-unstable.hg/rev/78
be123257fa
> > 
> > 15679:
> > http://xenbits.xensource.com/xen-unstable.hg/rev/7e
e30cf72fd3
> > 
> > 15672 is my original patch a little modified by
XenSource.
> > 15678 fixed a bug introduced via modifications
from XenSource.
> > 15679 is a cleanup by XenSource and tested by me
on both Dom0 and DomU.
> > 
> > All three changesets merged together results in a
relativ
> > small patch.
> 
> Thanks ! I've been able to apply it to Xen 3.1.0. I
updated the
> xenkernel3 to build the kernel from source, with this
patch.

The patch-a* patches you added to xenkernel3-nb1 are also in
xen-unstable.
(AFAIR I mailed you, Manuel, when I got the Xen-kernel
building on
NetBSD.) See changeset 15177 for the proof 

In short: When Xen 3.2 is out, no additional patches are
necessary
to build the xen-kernel - except patch-aa if you continue to
build
a *debug* Xen kernel.


> -- 
> Manuel Bouyer <bouyerantioche.eu.org>
>      NetBSD: 26 ans d'experience feront toujours la
difference
> --

-- 
Greetings,

Christoph

On Sat, Aug 18, 2007 at 02:24:12PM +0200, Christoph Egger
wrote:
> 
> 
> > Thanks ! I've been able to apply it to Xen 3.1.0.
I updated the
> > xenkernel3 to build the kernel from source, with
this patch.
> 
> The patch-a* patches you added to xenkernel3-nb1 are
also in xen-unstable.
> (AFAIR I mailed you, Manuel, when I got the Xen-kernel
building on
> NetBSD.) See changeset 15177 for the proof 

Yes I remember it. 

> 
> In short: When Xen 3.2 is out, no additional patches
are necessary
> to build the xen-kernel - except patch-aa if you
continue to build
> a *debug* Xen kernel.

That's what I expected, yes (altough I didn't look at the
-unstable
sources, because I don't know how to get a full tree from
the Xen
repository :(. I'll keep patch-aa around though, a debug
kernel is
usefull when doing developement.

-- 
Manuel Bouyer, LIP6, Universite Paris VI.          
Manuel.Bouyerlip6.fr
     NetBSD: 26 ans d'experience feront toujours la
difference
--

> On Sat, Aug 18, 2007 at 02:24:12PM +0200, Christoph
Egger wrote:
> > 
> > 
> > > Thanks ! I've been able to apply it to Xen
3.1.0. I updated the
> > > xenkernel3 to build the kernel from source,
with this patch.
> > 
> > The patch-a* patches you added to xenkernel3-nb1
are also in
> xen-unstable.
> > (AFAIR I mailed you, Manuel, when I got the
Xen-kernel building on
> > NetBSD.) See changeset 15177 for the proof 
> 
> Yes I remember it. 
> 
> > 
> > In short: When Xen 3.2 is out, no additional
patches are necessary
> > to build the xen-kernel - except patch-aa if you
continue to build
> > a *debug* Xen kernel.
> 
> That's what I expected, yes (altough I didn't look at
the -unstable
> sources, because I don't know how to get a full tree
from the Xen
> repository :(. I'll keep patch-aa around though, a
debug kernel is
> usefull when doing developement.

Install devel/mercurial from pkgsrc, then type

hg clone http://x
enbits.xensource.com/xen-unstable.hg

This fetches the xen-unstable tree to your disk in a
subdirectory
xen-unstable.hg. If you want a different subdirectory name,
then
a) specify it as third argument to hg or b) simply rename
the subdirectory.

Oh, and "man hg" is useful, too. 



-- 
Greetings,

Christoph

On Mon, Aug 20, 2007 at 01:01:15PM +0200, Christoph Egger
wrote:
> > That's what I expected, yes (altough I didn't look
at the -unstable
> > sources, because I don't know how to get a full
tree from the Xen
> > repository :(. I'll keep patch-aa around though, a
debug kernel is
> > usefull when doing developement.
> 
> Install devel/mercurial from pkgsrc, then type
> 
> hg clone http://x
enbits.xensource.com/xen-unstable.hg
> 
> This fetches the xen-unstable tree to your disk in a
subdirectory
> xen-unstable.hg. If you want a different subdirectory
name, then
> a) specify it as third argument to hg or b) simply
rename the subdirectory.
> 
> Oh, and "man hg" is useful, too. 

Ho, I though mercurial was binary-only. Maybe I mixed it
with some other
system. Thanks !

-- 
Manuel Bouyer, LIP6, Universite Paris VI.          
Manuel.Bouyerlip6.fr
     NetBSD: 26 ans d'experience feront toujours la
difference
--

> On Mon, Aug 20, 2007 at 01:01:15PM +0200, Christoph
Egger wrote:
> > > That's what I expected, yes (altough I didn't
look at the -unstable
> > > sources, because I don't know how to get a
full tree from the Xen
> > > repository :(. I'll keep patch-aa around
though, a debug kernel is
> > > usefull when doing developement.
> > 
> > Install devel/mercurial from pkgsrc, then type
> > 
> > hg clone http://x
enbits.xensource.com/xen-unstable.hg
> > 
> > This fetches the xen-unstable tree to your disk in
a subdirectory
> > xen-unstable.hg. If you want a different
subdirectory name, then
> > a) specify it as third argument to hg or b) simply
rename the
> subdirectory.
> > 
> > Oh, and "man hg" is useful, too. 
> 
> Ho, I though mercurial was binary-only. Maybe I mixed
it with some other
> system. Thanks !

Once you are involved in mercurial, you will find kdiff3 [1]
very useful
to merge files. Unfortunately, this is not available via
pkgsrc
(or I can't find it). Can someone package it, please?

Thanks in advance.

[1] http://kdiff3.sourcefo
rge.net/


-- 
Greetings,

Christoph

> On Mon, 20 Aug 2007 13:19:21 +0200
> "Christoph Egger" <Christoph_Eggergmx.de> wrote:
> 
> > 
> > Once you are involved in mercurial, you will find
kdiff3 [1] very
> > useful to merge files. Unfortunately, this is not
available via pkgsrc
> > (or I can't find it). Can someone package it,
please?
> > 
> I see it in devel/kdiff3.

Thanks for this information. I obviously looked into the
wrong places.

>  I've never tried it -- are its advantages
> over diff3 (part of the base system) sufficiently great
to warrant
> dragging in all of kde? 

hmm... I think, this question is very much bound to a
person.
When it comes to resolve merge conflicts, I personally
prefer kdiff3.

Resolving conflicts with mercurial (which uses diff3) is
different
as with cvs.
You can configure mercurial with editing .hg/hgrc (.hg
exists in the
toplevel directory of the repository). Add a section

[ui]
merge = /usr/bin/diff3

to use diff3 (which is the default anyway)
or use

[ui]
merge = <path-to-kdiff3>/kdiff3

to use kdiff3.


-- 
Greetings,

Christoph

On Mon, 20 Aug 2007 13:19:21 +0200
"Christoph Egger" <Christoph_Eggergmx.de> wrote:

> 
> Once you are involved in mercurial, you will find
kdiff3 [1] very
> useful to merge files. Unfortunately, this is not
available via pkgsrc
> (or I can't find it). Can someone package it, please?
> 
I see it in devel/kdiff3.  I've never tried it -- are its
advantages
over diff3 (part of the base system) sufficiently great to
warrant
dragging in all of kde? 



		--Steve Bellovin, http://www.cs.columbi
a.edu/~smb

On Mon, 20 Aug 2007, Steven M. Bellovin wrote:

> On Mon, 20 Aug 2007 13:19:21 +0200
> "Christoph Egger" <Christoph_Eggergmx.de> wrote:
>
>>
>> Once you are involved in mercurial, you will find
kdiff3 [1] very
>> useful to merge files. Unfortunately, this is not
available via pkgsrc
>> (or I can't find it). Can someone package it,
please?
>>
> I see it in devel/kdiff3.  I've never tried it -- are
its advantages
> over diff3 (part of the base system) sufficiently great
to warrant
> dragging in all of kde?

 	For a merge tool I've found meld very useful. Of course
you have
 	to pull in python, but I've certainly found it worth it.
Almost
 	as good as BeyondCompare (a Windows app).

-- 
 		David/absolute       -- www.NetBSD.org: No hype required
--