Re: cooling door

> Can someone please, pretty please with sugar on top,
explain the point
> behind high power density? 

maybe.

> Raw real estate is cheap (basically, nearly free).

not in downtown palo alto.  now, you could argue that
downtown palo alto
is a silly place for an internet exchange.  or you could
note that conditions
giving rise to high and diverse longhaul and metro fiber
density, also give
rise to high real estate costs.

> Increasing power density per sqft will *not* decrease
cost, beyond
> 100W/sqft, the real
estate costs are a tiny portion of total cost. Moving
> enough air to cool 400 (or, in your case, 2000) watts
per square foot is
> *hard*.

if you do it the old way, which is like you said, moving
air, that's always
true.  but, i'm not convinced that we're going to keep doing
it the old way.

> I've started to recently price things as "cost per
square amp". (That is,
> 1A power, conditioned, delivered to the customer rack
and cooled). Space
> is really irrelevant - to me, as colo provider, whether
I have 100A going
> into a single rack or 5 racks, is irrelevant. In fact,
my *costs*
> (including real estate) are likely to be lower when the
load is spread
> over 5 racks. Similarly, to a customer, all they care
about is getting
> their gear online, and can care less whether it needs
to be in 1 rack or
> in 5 racks.
> 
> To rephrase vijay, "what is the problem being
solved"?

if you find me 300Ksqft along the caltrain fiber corridor in
the peninsula
where i can get 10mW of power and have enough land around it
for 10mW worth
of genset, and the price per sqft is low enough that i can
charge by the
watt and floor space be damned and still come out even or
ahead, then please
do send me the address.

On Sat, Mar 29, 2008 at 06:54:02PM +0000, Paul Vixie wrote:
> 
> > Can someone please, pretty please with sugar on
top, explain the point
> > behind high power density? 

Customers are being sold blade servers on the basis that
"it's much
more efficient to put all your eggs in one basket"
without being told
about the power or cooling requirements and how not a whole
lot of
datacenters really want/are able to support customers
installing 15
racks of blade servers in one spot with 4x 230V/30A
circuits
each. (Yes, I had that request.)

Customers don't want to pay for the space. They forget that
they still
have to pay for the power and that that charge also includes
a fee for
the added load on the UPS as well as the AC to get rid of
the heat.

While there are advantages to blade servers, a fair number
of sales
are to gullable users who don't know what they're getting
into, not
those who really know how to get the most out of them. They
get sold
on the idea of using blade servers, stick them into S&D,
Equinix, and
others and suddenly find out that they can only fit 2 in a
rack
because of the per-rack wattage limit and end up having to
buy the
space anyway. (Wether it's extra racks or extra sq ft or
meters, it's
the same problem.)

Under current rules for most 3rd party datacenters, one of
the
principle stated advantages, that of much greater density,
is
effectively canceled out.

> > Increasing power density per sqft will *not*
decrease cost, beyond
> > 100W/sqft, the real
estate costs are a tiny portion of total cost. Moving
> > enough air to cool 400 (or, in your case, 2000)
watts per square foot is
> > *hard*.

(Remind me to strap myself to the floor to keep from
becoming airborne
by the hurricane force winds while I'm working in your
datacenter.)

Not convinved of the first point but experience is limited
there. For
the second, I think the practical upper bound for my
purposes is
probably between 150 and 200 watts per sq foot. (Getting
much harder
once you cross the 150 watt mark.) Beyond that, it gets
quite
difficult to supply enough cool air to the cabinet to keep
the
equipment happy unless you can guarentee a static load and
custom
design for that specific load. (And we all know that will
never
happen.) And don't even talk to me about enclosed cabinets
at that
point.

> if you do it the old way, which is like you said,
moving air, that's always
> true.  but, i'm not convinced that we're going to keep
doing it the old way.

One thing I've learned over the various succession of
datacenter /
computer room builds and expansions that I've been involved
in is that
if you ask the same engineer about the right way to do
cooling in
medium and large scale datacenters (15k sq ft and up),
you'll probably
get a different oppinion every time you ask the question.
There are
several theories of how best to hand this and *none* of them
are
right. No one has figured out an ideal solution and I'm not
convinced
an ideal solution exists. So we go with what we know works.
As people
experiment, what works changes. The problem is that
retrofitting is a
bear. (When's the last time you were able to get a $350k PO
approved
to update cooling to the datacenter? If you can't show a
direct ROI,
the money people don't like you. And on a more practical
line, how
many datacenters have you seen where it is physically
impossible to
remove the CRAC equipment for replacement without first
tearing out
entire rows of racks or even building walls?)

Anyway, my thoughts on the matter.

-Wayne

---
Wayne Bouchard
webtypo.org
Network Dude
http://www.typo.org/~web/