i386 with PAE or AMD64 on PowerEdge with 4G RAM

Good afternoon,

I just receive 2 PowerEdge servers (a 1950 and a 860) both
with 4G of 
RAM. I installed FreeBSD 6.2 Release i386 on both of them. 
Unfortunately, only 3,5G is recognized on the 860 and 3,3G
on the 1950.
dmesg on 860:
real memory  = 3757834240 (3583 MB)
avail memory = 3678318592 (3507 MB)

I am facing a difficult decision. Should I use i386 with PAE
enabled in 
the kernel (I read a lot of warnings using it) or should I
go with 
AMD64? Which branch should I follow?

These servers will be front-end/back-end MySQL(with
replication) and 
Apache servers with BIND, Postfix, Dovecot, PF.

There is the detailled configuration of the servers:
PowerEdge 1950
Xeon 5110
4G RAM
PERC 5 Raid controller (mfi)
Dual Broadcom 5708 (bce)

PowerEdge 860
Xeon 3070
4G RAM
LSI Logic Raid controller (mpt)
Dual Broadcom 5750 (bge)

Thanks a lot for your advice,

Martin
_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"

On Mon, Jun 18, 2007 at 01:03:44PM -0400, Martin Turgeon
wrote:
>  I just receive 2 PowerEdge servers (a 1950 and a 860)
both with 4G of RAM. I 
>  installed FreeBSD 6.2 Release i386 on both of them.
Unfortunately, only 3,5G 
>  is recognized on the 860 and 3,3G on the 1950.
>  dmesg on 860:
>  real memory  = 3757834240 (3583 MB)
>  avail memory = 3678318592 (3507 MB)
> 
>  I am facing a difficult decision. Should I use i386
with PAE enabled in the 
>  kernel (I read a lot of warnings using it) or should I
go with AMD64? Which 
>  branch should I follow?

Based on what I've read from some of the porters and
miscellaneous
others, generally-speaking there's too many issues with
amd64 (in the
sense of 32-bit vs. 64-bit compatibility -- not the fault of
the kernel
or otherwise) to consider it worth switching to.

I personally don't run 64-bit OSes because most developers
still use
32-bit machines and don't have a way to develop/test in
64-bit
environments.

That said, I'd recommend you stick with i386 + PAE, simply
for
guaranteed application compatibility.

You'll lose the amount of RAM you're seeing due to PAE
addressing for
PCI address space.  I can dig you up a usage map (broken
down by how
much is taken up by each portion; PCI, ACPI, etc.) if you
want one.
It's for SuperMicro systems, but the general idea applies to
most
everything.

-- 
| Jeremy Chadwick                                    jdc at
parodius.com |
| Parodius Networking                           http://www.parodius.com/
|
| UNIX Systems Administrator                      Mountain
View, CA, USA |
| Making life hard for others since 1977.                 
PGP: 4BD6C0CB |

_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"

In <20070618180813.GA13003eos.sc1.parodius.com>,
Jeremy Chadwick <koitsuFreeBSD.org> typed:
> On Mon, Jun 18, 2007 at 01:03:44PM -0400, Martin
Turgeon wrote:
> >  I just receive 2 PowerEdge servers (a 1950 and a
860) both with 4G of RAM. I 
> >  installed FreeBSD 6.2 Release i386 on both of
them. Unfortunately, only 3,5G 
> >  is recognized on the 860 and 3,3G on the 1950.
> >  dmesg on 860:
> >  real memory  = 3757834240 (3583 MB)
> >  avail memory = 3678318592 (3507 MB)
> > 
> >  I am facing a difficult decision. Should I use
i386 with PAE enabled in the 
> >  kernel (I read a lot of warnings using it) or
should I go with AMD64? Which 
> >  branch should I follow?
> 
> Based on what I've read from some of the porters and
miscellaneous
> others, generally-speaking there's too many issues with
amd64 (in the
> sense of 32-bit vs. 64-bit compatibility -- not the
fault of the kernel
> or otherwise) to consider it worth switching to.

If you need to run 32-bit apps on amd64 FreeBSD, you're
pretty much
SOL at this point. You'll have to build the the
infrastructure to
install your apps by hand. On the other hand, 64-bit FreeBSD
is fairly
solid, and most of the apps I need run as well on amd64
FreeBSD as
they do on i386 FreeBSD.

My last major project - the ETL code for the world largest
linux-based
Oracle database with 7x24 availability - has been running on
x86_64
linux since day 1, over two years ago. With the exclusion of
oracle,
it's built entirely on FOSS apps or custom code.

> I personally don't run 64-bit OSes because most
developers still use
> 32-bit machines and don't have a way to develop/test in
64-bit
> environments.

I find that extremely ironic. I've spent most of the last
two days
trying to put together a Linux system with python 2.5 (or
later) and
lxml 1.2 (or later), because I need to add an oracle library
to it.
While both FreeBSD and darwin ports (where I do development)
have all
the appropriate bits except oracle, the Linux distros don't
have any
of them in their packaging systems.  The precompiled
versions of lxml
available are either 64-bit Linux or 32-bit Windows.
Unfortunately, I
have to have the 32-bit version since the linux dev box is
running on
VM software that won't run 64-bit code.

> That said, I'd recommend you stick with i386 + PAE,
simply for
> guaranteed application compatibility.

If you are going to be using standard protocols to
communicate over
the network, then the issue isn't compatability so much as
availability - the apps you need may not be available for
amd64, or
may not work reliably if they are.

On the other hand, something that nobody ever seems to point
out is
that the same CPU is noticably faster running amd64 code
than i386
code. Probably has something to do with the amd64 mode
having twice as
many registers. If performance is an issue, it might be
worth your
while to see if your critical applications are available for
the
amd64.

	<mike
-- 
Mike Meyer <mwmmired.org>		http://www.mired
.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more
information.
_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"

On Mon, Jun 18, 2007 at 02:35:59PM -0400, Mike Meyer wrote:
> While both FreeBSD and darwin ports (where I do
development) have all
> the appropriate bits except oracle, the Linux distros
don't have any
> of them in their packaging systems.

It might be nice to point that out to Oracle, as a way to
try to sell
FreeBSD as a platform.

mcl
_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"

In <20070618184802.GC30616soaustin.net>, Mark
Linimon <linimonlonesome.com> typed:
> On Mon, Jun 18, 2007 at 02:35:59PM -0400, Mike Meyer
wrote:
> > While both FreeBSD and darwin ports (where I do
development) have all
> > the appropriate bits except oracle, the Linux
distros don't have any
> > of them in their packaging systems.
> It might be nice to point that out to Oracle, as a way
to try to sell
> FreeBSD as a platform.

I've started looking for a contact....

	Thanks,
	<mike
-- 
Mike Meyer <mwmmired.org>		http://www.mired
.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more
information.
_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"

2007/6/18, Jeremy Chadwick <koitsufreebsd.org>:
>
> On Mon, Jun 18, 2007 at 01:03:44PM -0400, Martin
Turgeon wrote:
> >  I just receive 2 PowerEdge servers (a 1950 and a
860) both with 4G of
> RAM. I
> >  installed FreeBSD 6.2 Release i386 on both of
them. Unfortunately, only
> 3,5G
> >  is recognized on the 860 and 3,3G on the 1950.
> >  dmesg on 860:
> >  real memory  = 3757834240 (3583 MB)
> >  avail memory = 3678318592 (3507 MB)
> >
> >  I am facing a difficult decision. Should I use
i386 with PAE enabled in
> the
> >  kernel (I read a lot of warnings using it) or
should I go with AMD64?
> Which
> >  branch should I follow?
>
> Based on what I've read from some of the porters and
miscellaneous
> others, generally-speaking there's too many issues with
amd64 (in the
> sense of 32-bit vs. 64-bit compatibility -- not the
fault of the kernel
> or otherwise) to consider it worth switching to.
>
> I personally don't run 64-bit OSes because most
developers still use
> 32-bit machines and don't have a way to develop/test in
64-bit
> environments.
>
> That said, I'd recommend you stick with i386 + PAE,
simply for
> guaranteed application compatibility.


My setup is fairly standard (as I described), should I
expect problem with
64 bit version of these programs?


You'll lose the amount of RAM you're seeing due to PAE
addressing for
> PCI address space.  I can dig you up a usage map
(broken down by how
> much is taken up by each portion; PCI, ACPI, etc.) if
you want one.
> It's for SuperMicro systems, but the general idea
applies to most
> everything.


I'm not sure to understand what you mean by that. Are you
saying that PAE
will eat the 500M that should be available?

--
> | Jeremy Chadwick                                   
jdc at parodius.com |
> | Parodius Networking                           http://www.parodius.com/
|
> | UNIX Systems Administrator                     
Mountain View, CA, USA |
> | Making life hard for others since 1977.              
   PGP: 4BD6C0CB |
>
>
_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"

On Mon, Jun 18, 2007 at 05:15:30PM -0400, Martin Turgeon
wrote:
> 2007/6/18, Jeremy Chadwick <koitsufreebsd.org>:
> >
> >On Mon, Jun 18, 2007 at 01:03:44PM -0400, Martin
Turgeon wrote:
> >>  I just receive 2 PowerEdge servers (a 1950
and a 860) both with 4G of
> >RAM. I
> >>  installed FreeBSD 6.2 Release i386 on both of
them. Unfortunately, only
> >3,5G
> >>  is recognized on the 860 and 3,3G on the
1950.
> >>  dmesg on 860:
> >>  real memory  = 3757834240 (3583 MB)
> >>  avail memory = 3678318592 (3507 MB)
> >>
> >>  I am facing a difficult decision. Should I
use i386 with PAE enabled in
> >the
> >>  kernel (I read a lot of warnings using it) or
should I go with AMD64?
> >Which
> >>  branch should I follow?
> >
> >Based on what I've read from some of the porters
and miscellaneous
> >others, generally-speaking there's too many issues
with amd64 (in the
> >sense of 32-bit vs. 64-bit compatibility -- not the
fault of the kernel
> >or otherwise) to consider it worth switching to.
> >
> >I personally don't run 64-bit OSes because most
developers still use
> >32-bit machines and don't have a way to
develop/test in 64-bit
> >environments.
> >
> >That said, I'd recommend you stick with i386 + PAE,
simply for
> >guaranteed application compatibility.
> 
> 
> My setup is fairly standard (as I described), should I
expect problem with
> 64 bit version of these programs?

No.  I'd go with amd64 personally.

Kris
_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"

On Mon, Jun 18, 2007 at 05:15:30PM -0400, Martin Turgeon
wrote:
>  My setup is fairly standard (as I described), should I
expect problem with
>  64 bit version of these programs?

Like I said, I don't run 64-bit OSes because I prefer
compatibility.
Believe me, the instant you run into some quirky problem
with either the
kernel or any of its subsystems, or a third-party program
(from ports or
otherwise), the first thing you'll be told is "it works
for me on i386,
have you tried i386?"

I'm sorry if this sounds condescending or combative, but
it's what I
continually see on other lists.

>  You'll lose the amount of RAM you're seeing due to PAE
addressing for
> > PCI address space.  I can dig you up a usage map
(broken down by how
> > much is taken up by each portion; PCI, ACPI, etc.)
if you want one.
> > It's for SuperMicro systems, but the general idea
applies to most
> > everything.
> 
>  I'm not sure to understand what you mean by that. Are
you saying that PAE
>  will eat the 500M that should be available?

PCI addressing is actually responsible for most of it, but
it's worse
when PAE is in use.  This is one of the many reasons a lot
of people
prefer to run in 64-bit environments.  Taken from a
Supermicro
motherboard manual, documenting the issue (seems their math
may be off
by 2MB ;) ):

http://www.supermicro.com/manuals/motherboard/300
0/MNL-0889.pdf

4. Due to memory allocation to system devices, memory
remaining
available for operational use will be reduced when 4 GB of
RAM is used.
The reduction in memory availability is disproportional.
(Refer to the
following Memory Availability Table for details.)

System Device                   Size      Physical Memory
                                          Remaining
(-Available)
                                          (4GB Total System
Memory)
============================================================
=======
Firmware Hub flash memory       1MB       3.99GB
  (System BIOS)
Local APIC                      4KB       3.99GB
Area Reserved for chipset       2MB       3.99GB
I/O APIC (4 Kbytes)             4KB       3.99GB
PCI Enumeration Area 1          256MB     3.76GB
PCI Express (256 MB)            256MB     3.51GB
PCI Enumeration Area 2          512MB     3.01GB
  (if needed)
  -Aligned on 256-MB boundary-
VGA Memory                      16MB      2.85GB
TSEG                            1MB       2.84GB
------------------------------------------------------------
-------
Memory available to OS and                2.84GB
  other applications
============================================================
=======

-- 
| Jeremy Chadwick                                    jdc at
parodius.com |
| Parodius Networking                           http://www.parodius.com/
|
| UNIX Systems Administrator                      Mountain
View, CA, USA |
| Making life hard for others since 1977.                 
PGP: 4BD6C0CB |

_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"

On Mon, Jun 18, 2007 at 03:10:22PM -0700, Jeremy Chadwick
wrote:
> On Mon, Jun 18, 2007 at 05:15:30PM -0400, Martin
Turgeon wrote:
> >  My setup is fairly standard (as I described),
should I expect problem with
> >  64 bit version of these programs?
> 
> Like I said, I don't run 64-bit OSes because I prefer
compatibility.
> Believe me, the instant you run into some quirky
problem with either the
> kernel or any of its subsystems, or a third-party
program (from ports or
> otherwise), the first thing you'll be told is "it
works for me on i386,
> have you tried i386?"

I think this assertion is false.  amd64 is pretty well
supported and
run by an increasing number of users and developers.

Kris

In <20070618221022.GA17952eos.sc1.parodius.com>,
Jeremy Chadwick <koitsuFreeBSD.org> typed:
> On Mon, Jun 18, 2007 at 05:15:30PM -0400, Martin
Turgeon wrote:
> >  My setup is fairly standard (as I described),
should I expect problem with
> >  64 bit version of these programs?
> Like I said, I don't run 64-bit OSes because I prefer
compatibility.

So you have no first-hand experience with support for 64-bit
OSes.

> Believe me, the instant you run into some quirky
problem with either the
> kernel or any of its subsystems, or a third-party
program (from ports or
> otherwise), the first thing you'll be told is "it
works for me on i386,
> have you tried i386?"
> I'm sorry if this sounds condescending or combative,
but it's what I
> continually see on other lists.

I don't mean to sound condescending either, but I
continually see "it
works for me on <change random config thing>" on
other lists as
well. Linux vs. FreeBSD, 64 vs. 32 bits, LOCALBASE being
something
other than /usr/local, etc. People trying to help try what
you say
doesn't work. If it works for them, they'll latch on to
whatever is
the most obvious thing that's different between your two
systems as
the most likely cause. Sometimes they may be right, but not
always.

I've found support for 64 bit FreeBSD and the applications
in the
ports tree to be nearly indistinguishable from 32 bit
FreeBSD. The
developers are either responsive, and things will get fixed
(or are
already fixed, and you need to update your sources), or they
aren't
responsive, and you'll be stuck trying to fix it yourself.
If the
developer is responsive and you are reasonably capable and
willing to
do some work yourself, whether or not the developer has a 64
bit box
simply isn't an issue. If the developer isn't responsive,
whether
you're running on 32 or 64 bit hardware isn't an issue
either.

For applications, there have been 64 bit Unix boxes around
for a long
time. Especially servers. Anything that's in serious use
almost
certainly had all the 32 vs. 64 bit issues shaken out long
ago.

Yeah, some things are probably so 32-bit dependent they'll
never be
fixed (the X server code in tightvnc comes to mind), but
there are
usually alternative solutions available. Some things are
proprietary,
and aren't available (like Windows codecs). The only way to
figure out
where your applications fit on the list is to try them and
see.

	<mike
-- 
Mike Meyer <mwmmired.org>		http://www.mired
.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more
information.
_______________________________________________
freebsd-hardwarefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hard
ware
To unsubscribe, send any mail to
"freebsd-hardware-unsubscribefreebsd.org"