x86 changes

Hi,

I just realised that I am not subscribed to this list.. Is
anyone willing to
try a Xen kernel with this patch?

	http://www.netbsd.
org/~ad/x86.diff

Among other things, it:

- makes the i386 kernel compile with pcc (well almost, a
number of files
  are problematic)
- reduces amd64 GENERIC.MP kernel by ~130kb, similar for the
others I guess
- makes x86 bus_space and bus_dma more LKM friendly
- shares a few more files between the x86 ports

Thanks,
Andrew

On Friday 21 September 2007 00:41:46 Andrew Doran wrote:
> Hi,
>
> I just realised that I am not subscribed to this list..
Is anyone willing
> to try a Xen kernel with this patch?
>
> 	http://www.netbsd.
org/~ad/x86.diff

In xenfunc.h, you have this:

+vaddr_t
+rcr2(void)
+{
+
+	return 0;	/* XXX Why? */
+}

How about this:

#ifdef MULTIPROCESSOR
#define curvcpu()    (/* Multiprocessor support not yet
implemented */)
#else
#define curvcpu()    (0)
#endif

vaddr_t
rcr2(void)
{
#ifdef XEN3
       int v = curvcpu();
       return
HYPERVISOR_shared_info->vcpu_info[v].arch.cr2;
#else
       __PRINTK(("XXX rcr2 not supportedn"));
       return 0;
#endif
}

Did you try to run a NetBSD/Xen Dom0 and a NetBSD/Xen DomU
?
Xen emulates some instructions such as rdmsr/wrmsr for Dom0,
but not
for DomU.



> Among other things, it:
>
> - makes the i386 kernel compile with pcc (well almost,
a number of files
>   are problematic)
> - reduces amd64 GENERIC.MP kernel by ~130kb, similar
for the others I guess
> - makes x86 bus_space and bus_dma more LKM friendly
> - shares a few more files between the x86 ports
>
> Thanks,
> Andrew

On Thu, Sep 20, 2007 at 11:41:46PM +0100, Andrew Doran
wrote:
> Hi,
> 
> I just realised that I am not subscribed to this list..
Is anyone willing to
> try a Xen kernel with this patch?
> 
> 	http://www.netbsd.
org/~ad/x86.diff
> 
> Among other things, it:
> 
> - makes the i386 kernel compile with pcc (well almost,
a number of files
>   are problematic)
> - reduces amd64 GENERIC.MP kernel by ~130kb, similar
for the others I guess
> - makes x86 bus_space and bus_dma more LKM friendly
> - shares a few more files between the x86 ports

there is a missing
#include "opt_xen.h" in i386/cpufunc.S
without it the kenrel doesn't build

A XEN3_DOM0 kernel fails with:
Loaded initial symtab at 0xc09875e0, strtab at 0xc09ea82c, #
entries 23879
panic: kernel diagnostic assertion "doing_shutdown ||
curlwp != NULL || (map->flags &
VM_MAP_INTRSAFE)" failed: file
"/dsk/l1/misc/bouyer/current/src/sys/uvm/uvm_map.c"
;, line 1078
Stopped at      netbsd:breakpoint+0x1:  ret

This is before the copyright string is printed; I never get
the db> prompt
(maybe it's too early at this point to have a full ddb
working).

A XEN3_DOMU kernel doesn't start, it seems to be looping
before console
is working.

A XEN2_DOM0 fails in the same way a XEN3_DOM0 kernel.

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--

On Sun, Sep 23, 2007 at 01:36:29PM +0200, Manuel Bouyer
wrote:

> On Thu, Sep 20, 2007 at 11:41:46PM +0100, Andrew Doran
wrote:
> > Hi,
> > 
> > I just realised that I am not subscribed to this
list.. Is anyone willing to
> > try a Xen kernel with this patch?
> > 
> > 	http://www.netbsd.
org/~ad/x86.diff
> > 
> > Among other things, it:
> > 
> > - makes the i386 kernel compile with pcc (well
almost, a number of files
> >   are problematic)
> > - reduces amd64 GENERIC.MP kernel by ~130kb,
similar for the others I guess
> > - makes x86 bus_space and bus_dma more LKM
friendly
> > - shares a few more files between the x86 ports
> 
> there is a missing
> #include "opt_xen.h" in i386/cpufunc.S
> without it the kenrel doesn't build
> 
> A XEN3_DOM0 kernel fails with:
> Loaded initial symtab at 0xc09875e0, strtab at
0xc09ea82c, # entries 23879
> panic: kernel diagnostic assertion "doing_shutdown
|| curlwp != NULL || (map->flags &
VM_MAP_INTRSAFE)" failed: file
"/dsk/l1/misc/bouyer/current/src/sys/uvm/uvm_map.c"
;, line 1078
> Stopped at      netbsd:breakpoint+0x1:  ret

It looks like curlwp was not set in cpu_info_primary;
another problem was
that curlwp was not being set in cpu_switchto(). I've fixed
both of these.

Thanks,
Andrew

On Fri, Sep 21, 2007 at 10:23:27AM +0200, Christoph Egger
wrote:

> On Friday 21 September 2007 00:41:46 Andrew Doran
wrote:
> > Hi,
> >
> > I just realised that I am not subscribed to this
list.. Is anyone willing
> > to try a Xen kernel with this patch?
> >
> > 	http://www.netbsd.
org/~ad/x86.diff
> 
> In xenfunc.h, you have this:
> 
> +vaddr_t
> +rcr2(void)
> +{
> +
> +	return 0;	/* XXX Why? */
> +}
> 
> How about this:
> 
> #ifdef MULTIPROCESSOR
> #define curvcpu()    (/* Multiprocessor support not yet
implemented */)
> #else
> #define curvcpu()    (0)
> #endif
> 
> vaddr_t
> rcr2(void)
> {
> #ifdef XEN3
>        int v = curvcpu();
>        return
HYPERVISOR_shared_info->vcpu_info[v].arch.cr2;
> #else
>        __PRINTK(("XXX rcr2 not
supportedn"));
>        return 0;
> #endif
> }

I think that makes sense. I'll have a look.
 
> Did you try to run a NetBSD/Xen Dom0 and a NetBSD/Xen
DomU ?
> Xen emulates some instructions such as rdmsr/wrmsr for
Dom0, but not
> for DomU.

I posted here since I had no xen installation. . I've
since got a dom0
created, I'll try with a domU in the next day or so.

Thanks,
Andrew

On Wed, 26 Sep 2007 20:36:55 +0100 Andrew Doran wrote:

 AD> On Sun, Sep 23, 2007 at 01:36:29PM +0200, Manuel
Bouyer wrote:

 >> there is a missing
 >> #include "opt_xen.h" in i386/cpufunc.S
 >> without it the kenrel doesn't build
 >> 
 >> A XEN3_DOM0 kernel fails with:
 >> Loaded initial symtab at 0xc09875e0, strtab at
0xc09ea82c, # entries 23879
 >> panic: kernel diagnostic assertion
"doing_shutdown || curlwp != NULL || (map->flags
& VM_MAP_INTRSAFE)" failed: file
"/dsk/l1/misc/bouyer/current/src/sys/uvm/uvm_map.c"
;, line 1078
 >> Stopped at      netbsd:breakpoint+0x1:  ret

 AD> It looks like curlwp was not set in
cpu_info_primary; another problem was
 AD> that curlwp was not being set in cpu_switchto().
I've fixed both of these.

It looks like http://www.netbsd.
org/~ad/x86.diff is still old patch. Could you
point where I can find patch with your last fixes to try?

Thanks.

-- 
Mikolaj Golub

On Thursday 27 September 2007 09:37:14 Mikolaj Golub wrote:
> On Wed, 26 Sep 2007 20:36:55 +0100 Andrew Doran wrote:
>
>  AD> On Sun, Sep 23, 2007 at 01:36:29PM +0200,
Manuel Bouyer wrote:
>  >> there is a missing
>  >> #include "opt_xen.h" in
i386/cpufunc.S
>  >> without it the kenrel doesn't build
>  >>
>  >> A XEN3_DOM0 kernel fails with:
>  >> Loaded initial symtab at 0xc09875e0, strtab
at 0xc09ea82c, # entries
>  >> 23879 panic: kernel diagnostic assertion
"doing_shutdown || curlwp !=
>  >> NULL || (map->flags &
VM_MAP_INTRSAFE)" failed: file
>  >>
"/dsk/l1/misc/bouyer/current/src/sys/uvm/uvm_map.c"
;, line 1078 Stopped
>  >> at      netbsd:breakpoint+0x1:  ret
>
>  AD> It looks like curlwp was not set in
cpu_info_primary; another problem
> was AD> that curlwp was not being set in
cpu_switchto(). I've fixed both of
> these.
>
> It looks like http://www.netbsd.
org/~ad/x86.diff is still old patch. Could
> you point where I can find patch with your last fixes
to try?

I think, AD committed this patch. So try -current. 

Christoph

On Thu, 27 Sep 2007 09:45:45 +0200 Christoph Egger wrote:

 >> It looks like http://www.netbsd.
org/~ad/x86.diff is still old patch. Could
 >> you point where I can find patch with your last
fixes to try?

 CE> I think, AD committed this patch. So try -current.


Oh, yes, I see it in current. Thaks.

-- 
Mikolaj Golub

On Thu, 27 Sep 2007 12:59:46 +0300 Mikolaj Golub wrote:

 MG> On Thu, 27 Sep 2007 09:45:45 +0200 Christoph Egger
wrote:

 >>> It looks like http://www.netbsd.
org/~ad/x86.diff is still old patch. Could
 >>> you point where I can find patch with your
last fixes to try?

 CE>> I think, AD committed this patch. So try
-current. 

 MG> Oh, yes, I see it in current. Thaks.

I have tried today current kernel. It crashes when starting
xend:

> Mounting all filesystems...
> Clearing temporary files.
> Starting xend.
> uvm_fault(0xc654ba20, 0, 2) -> 0xe
> fatal page fault in supervisor mode
> trap type 6 code 2 eip c04c908d cs 9 eflags 10246 cr2 0
ilevel b
> panic: trap
> syncing disks... Mutex error: mutex_vector_enter:
locking against myself
> 
> lock address : 0x00000000c090d740
> current cpu  :                  0
> current lwp  : 0x00000000c7de7700
> owner field  : 0x0000000000010b00 wait/spin:           
    0/1
> 
> panic: lock error
> 
> dumping to dev 0,1 offset 940022
> dump 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37
36 35 34 33 32 31 30 29 28 27 26 25 2d
> 
> 
> rebooting...
> (XEN) Domain 0 shutdown: rebooting machine.
> 

(gdb) disassemble 0xc04c908d
Dump of assembler code for function mutex_enter:
0xc04c9080 <mutex_enter+0>:     mov    0x4(%esp),%edx
0xc04c9084 <mutex_enter+4>:     mov    %fs:0x18,%ecx
0xc04c908b <mutex_enter+11>:    xor    %eax,%eax
0xc04c908d <mutex_enter+13>:    cmpxchg
%ecx,0x0(%edx)
0xc04c9091 <mutex_enter+17>:    jne,pn 0xc04106c0
<mutex_vector_enter>
0xc04c9098 <mutex_enter+24>:    ret    
0xc04c9099 <mutex_enter+25>:    lea    0x0(%esi),%esi
End of assembler dump.

Unfortunately I can't provide trace from core dump:

> savecore: can't find device 1383/1646

Could someone say what this message means and how I can get
core?

-- 
Mikolaj Golub

On Thu, Sep 27, 2007 at 03:21:02PM +0300, Mikolaj Golub
wrote:
> 
> On Thu, 27 Sep 2007 12:59:46 +0300 Mikolaj Golub
wrote:
> 
>  MG> On Thu, 27 Sep 2007 09:45:45 +0200 Christoph
Egger wrote:
> 
>  >>> It looks like http://www.netbsd.
org/~ad/x86.diff is still old patch. Could
>  >>> you point where I can find patch with
your last fixes to try?
> 
>  CE>> I think, AD committed this patch. So try
-current. 
> 
>  MG> Oh, yes, I see it in current. Thaks.
> 
> I have tried today current kernel. It crashes when
starting xend:
> 
> > Mounting all filesystems...
> > Clearing temporary files.
> > Starting xend.
> > uvm_fault(0xc654ba20, 0, 2) -> 0xe
> > fatal page fault in supervisor mode
> > trap type 6 code 2 eip c04c908d cs 9 eflags 10246
cr2 0 ilevel b
> > panic: trap
> > syncing disks... Mutex error: mutex_vector_enter:
locking against myself
> > 
> > lock address : 0x00000000c090d740
> > current cpu  :                  0
> > current lwp  : 0x00000000c7de7700
> > owner field  : 0x0000000000010b00 wait/spin:      
         0/1
> > 
> > panic: lock error
> > 
> > dumping to dev 0,1 offset 940022
> > dump 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39
38 37 36 35 34 33 32 31 30 29 28 27 26 25 2d


I get:
uvm_fault(0xc656b0d8, 0, 2) -> 0xe
fatal page fault in supervisor mode
trap type 6 code 2 eip c04d1fcd cs 9 eflags 10246 cr2 0
ilevel b
kernel: supervisor trap page fault, code=0
Stopped in pid 522.1 (python2.4) at    
netbsd:mutex_enter+0xd: cmpxchgl        %ecx,0(%edx)
db> show registers
ds          0x11
es          0x11
fs          0x31
gs          0x11
edi         0
esi         0xffffffff
ebp         0xc840f844
ebx         0xbbafc910
edx         0x8
ecx         0xc7e54c40
eax         0
eip         0xc04d1fcd  mutex_enter+0xd
cs          0x9
eflags      0x10246
esp         0xc840f818
ss          0x11
netbsd:mutex_enter+0xd: cmpxchgl        %ecx,0(%edx)
db> tr             
mutex_enter(0,bbafc910,c6561e00,4,c7e54c40) at
netbsd:mutex_enter+0xd
cpu_switchto(c7e54c40,c0d6ac50,c840f8dc,c0482518,0) at
netbsd:cpu_switchto+0x81
sleepq_block(0,0,c0843d88,0,c0d6ac58) at
netbsd:sleepq_block+0x8e
biowait(c0d6ac50,0,0,c840fa5c,c840fa44) at
netbsd:biowait+0x54
genfs_getpages(c840fa94,1000,c840fabc,c03c58eb,c06f6e00) at
netbsd:genfs_getpage
s+0xf4f
VOP_GETPAGES(c7e1f000,18000,0,c840fbcc,c840fbd8) at
netbsd:VOP_GETPAGES+0x52
uvm_fault_internal(c656b0d8,bbafc000,1,0,0) at
netbsd:uvm_fault_internal+0xfae
trap() at netbsd:trap+0x405
--- trap (number 6) ---
0xbbafc910:
db> 

Any idea ?

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--