Domain 0 kernel, DAC960 and SMP

Is the scenario below true for Xen3?

I understand SMP support on a per kernel basis is currently
unavailable
but does the Xen3 hypervisor shedule to all available CPUs?

My initial perception was that the domain0 kernel is
required to identify
both CPUs but that obviously isn't the case for at least
Xen2, going by the
response to the previous post.

Please excuse any ignorance on my part. I have only been
playing with Xen for
the past week, I can't wait to consolidate the majority of
my servers into one box. 
I'm looking forward to a smaller power bill  (and a
happier wife!).

Thanks for any info.

Best regards,

Sarton O'Brien


Manuel Bouyer <bouyer <at> antioche.eu.org>
writes:
> 
> On Sun, May 28, 2006 at 03:46:20PM -0600, Tillman
Hodgson wrote:
> > > With Xen2, multiple CPUs can't be used by a
single domain, but if you
> > > have multiple domains running, the hypervisor
will schedule them on
> > > different CPUs.  The assignement is done on
domain startup, you can't
> > > move a domain from one CPU to another once
started.
> > 
> > Does this demonstrate what you were saying?:
> > 
> >  Name              Id  Mem(MB)  CPU  State 
Time(s)  Console
> >  Domain-0           0       63    0  r----   
297.1        
> >  dom1               7      127    1  -b---    
11.8    9607
> 
> Yes.

On Fri, Jul 21, 2006 at 11:47:25AM +1000, Sarton O'Brien
wrote:
> Is the scenario below true for Xen3?
> 
> I understand SMP support on a per kernel basis is
currently unavailable
> but does the Xen3 hypervisor shedule to all available
CPUs?
> 
> My initial perception was that the domain0 kernel is
required to identify
> both CPUs but that obviously isn't the case for at
least Xen2, going by the
> response to the previous post.
> 
> Please excuse any ignorance on my part. I have only
been playing with Xen for
> the past week, I can't wait to consolidate the
majority of my servers into one box. 
> I'm looking forward to a smaller power bill  (and a
happier wife!).

Xen dom0 doensn't need to support SMP to have Xen itself
use multiple
CPUs. AFAIK, with Xen3 physical CPUs are still statically
allocated
to virtual CPUs, but one domain can use multiple CPUs. But
right now,
NetBSD doesn't support this. Worse, you may have to disable
Xen SMP
do use a NetBSD dom0, because of interrupt routing issues.

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--

On Fri, Jul 21, 2006 at 11:47:25AM +1000, Sarton O'Brien
wrote:
> Is the scenario below true for Xen3?
> 
> I understand SMP support on a per kernel basis is
currently unavailable
> but does the Xen3 hypervisor shedule to all available
CPUs?
> 
> My initial perception was that the domain0 kernel is
required to identify
> both CPUs but that obviously isn't the case for at
least Xen2, going by the
> response to the previous post.
> 
> Please excuse any ignorance on my part. I have only
been playing with Xen for
> the past week, I can't wait to consolidate the
majority of my servers into one box. 
> I'm looking forward to a smaller power bill  (and a
happier wife!).

Xen dom0 doensn't need to support SMP to have Xen itself
use multiple
CPUs. AFAIK, with Xen3 physical CPUs are still statically
allocated
to virtual CPUs, but one domain can use multiple CPUs. But
right now,
NetBSD doesn't support this. Worse, you may have to disable
Xen SMP
do use a NetBSD dom0, because of interrupt routing issues.

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--

On Fri, Jul 21, 2006 at 11:20:46PM +0200, Manuel Bouyer
wrote:
> Xen dom0 doensn't need to support SMP to have Xen
itself use multiple
> CPUs. AFAIK, with Xen3 physical CPUs are still
statically allocated
> to virtual CPUs, but one domain can use multiple CPUs.
But right now,
> NetBSD doesn't support this. Worse, you may have to
disable Xen SMP
> do use a NetBSD dom0, because of interrupt routing
issues.

Being relatively new to SMP, does diabling apic prevent the
use of SMP?

I have not defined 'vcpu' yet xentop is reporting that
there are 2.

I'm finding all functions OK so long as I set noapic.

The problem exhibited when apic was enabled was with the
onboard bge-driver
based network card (watchdog timer resetting ... blah blah).

Other than that, a mknod for the missing device and
installing the XML
parser has got everything up and running.

One other question, does the DOMU_INSTALL kernel require
access to the cdrom?
and if so, can it be emulated sufficiently with a built
iso-image used in
place of a physical device?

Thank-you very much for your help. I can't wait to see this
port mature and
I'm sure I speak for more than my self when I say your hard
work is greatly
appreciated.

Sarton

On Fri, Jul 21, 2006 at 11:20:46PM +0200, Manuel Bouyer
wrote:
> Xen dom0 doensn't need to support SMP to have Xen
itself use multiple
> CPUs. AFAIK, with Xen3 physical CPUs are still
statically allocated
> to virtual CPUs, but one domain can use multiple CPUs.
But right now,
> NetBSD doesn't support this. Worse, you may have to
disable Xen SMP
> do use a NetBSD dom0, because of interrupt routing
issues.

Being relatively new to SMP, does diabling apic prevent the
use of SMP?

I have not defined 'vcpu' yet xentop is reporting that
there are 2.

I'm finding all functions OK so long as I set noapic.

The problem exhibited when apic was enabled was with the
onboard bge-driver
based network card (watchdog timer resetting ... blah blah).

Other than that, a mknod for the missing device and
installing the XML
parser has got everything up and running.

One other question, does the DOMU_INSTALL kernel require
access to the cdrom?
and if so, can it be emulated sufficiently with a built
iso-image used in
place of a physical device?

Thank-you very much for your help. I can't wait to see this
port mature and
I'm sure I speak for more than my self when I say your hard
work is greatly
appreciated.

Sarton

On Sat, Jul 22, 2006 at 03:33:03PM +1000, Sarton O'Brien
wrote:
> On Fri, Jul 21, 2006 at 11:20:46PM +0200, Manuel Bouyer
wrote:
> > Xen dom0 doensn't need to support SMP to have Xen
itself use multiple
> > CPUs. AFAIK, with Xen3 physical CPUs are still
statically allocated
> > to virtual CPUs, but one domain can use multiple
CPUs. But right now,
> > NetBSD doesn't support this. Worse, you may have
to disable Xen SMP
> > do use a NetBSD dom0, because of interrupt routing
issues.
> 
> Being relatively new to SMP, does diabling apic prevent
the use of SMP?

I think it does not, but on my systems I've not been able
to get
Xen3 working SMP at all with NetBSD/dom0 yet.

> 
> I have not defined 'vcpu' yet xentop is reporting
that there are 2.
> 
> I'm finding all functions OK so long as I set noapic.
> 
> The problem exhibited when apic was enabled was with
the onboard bge-driver
> based network card (watchdog timer resetting ... blah
blah).

Yes, typical interrupt routing issue.

> 
> Other than that, a mknod for the missing device and
installing the XML
> parser has got everything up and running.
> 
> One other question, does the DOMU_INSTALL kernel
require access to the cdrom?

Do, it can do a network install too.

> and if so, can it be emulated sufficiently with a built
iso-image used in
> place of a physical device?

Yes this should work.

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--

On Sat, Jul 22, 2006 at 03:33:03PM +1000, Sarton O'Brien
wrote:
> On Fri, Jul 21, 2006 at 11:20:46PM +0200, Manuel Bouyer
wrote:
> > Xen dom0 doensn't need to support SMP to have Xen
itself use multiple
> > CPUs. AFAIK, with Xen3 physical CPUs are still
statically allocated
> > to virtual CPUs, but one domain can use multiple
CPUs. But right now,
> > NetBSD doesn't support this. Worse, you may have
to disable Xen SMP
> > do use a NetBSD dom0, because of interrupt routing
issues.
> 
> Being relatively new to SMP, does diabling apic prevent
the use of SMP?

I think it does not, but on my systems I've not been able
to get
Xen3 working SMP at all with NetBSD/dom0 yet.

> 
> I have not defined 'vcpu' yet xentop is reporting
that there are 2.
> 
> I'm finding all functions OK so long as I set noapic.
> 
> The problem exhibited when apic was enabled was with
the onboard bge-driver
> based network card (watchdog timer resetting ... blah
blah).

Yes, typical interrupt routing issue.

> 
> Other than that, a mknod for the missing device and
installing the XML
> parser has got everything up and running.
> 
> One other question, does the DOMU_INSTALL kernel
require access to the cdrom?

Do, it can do a network install too.

> and if so, can it be emulated sufficiently with a built
iso-image used in
> place of a physical device?

Yes this should work.

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--

> > On Fri, Jul 21, 2006 at 11:20:46PM +0200, Manuel
Bouyer wrote:
> > > Xen dom0 doensn't need to support SMP to
have Xen itself use multiple
> > > CPUs. AFAIK, with Xen3 physical CPUs are
still statically allocated
> > > to virtual CPUs, but one domain can use
multiple CPUs. But right now,
> > > NetBSD doesn't support this. Worse, you may
have to disable Xen SMP
> > > do use a NetBSD dom0, because of interrupt
routing issues.

I've been running with Xen3 SMP for a few days now and have
only experienced
one reproducable problem (which is now the only problem
since completely
configuring the system).

Occaisionally I/O will just stop (scp, console, demesg, you
name it) but 
lauching xentop and closing it (opening and closing, opening
and closing) 
prods it to continue on.

Not sure if this is any help, but thought I'd mention it. I
thought it was
strange that xentop could coax it into continuing. The only
kernel
flags I am using are noapic and acpi=off (I'll be adding
nosmp to that
shortly .

Sarton

> > On Fri, Jul 21, 2006 at 11:20:46PM +0200, Manuel
Bouyer wrote:
> > > Xen dom0 doensn't need to support SMP to
have Xen itself use multiple
> > > CPUs. AFAIK, with Xen3 physical CPUs are
still statically allocated
> > > to virtual CPUs, but one domain can use
multiple CPUs. But right now,
> > > NetBSD doesn't support this. Worse, you may
have to disable Xen SMP
> > > do use a NetBSD dom0, because of interrupt
routing issues.

I've been running with Xen3 SMP for a few days now and have
only experienced
one reproducable problem (which is now the only problem
since completely
configuring the system).

Occaisionally I/O will just stop (scp, console, demesg, you
name it) but 
lauching xentop and closing it (opening and closing, opening
and closing) 
prods it to continue on.

Not sure if this is any help, but thought I'd mention it. I
thought it was
strange that xentop could coax it into continuing. The only
kernel
flags I am using are noapic and acpi=off (I'll be adding
nosmp to that
shortly .

Sarton