Re: Limit the space used by hadoop on a slave node

I think I have seen related bad behavior on 15.1.

On 1/8/08 11:49 AM, "Hairong Kuang"
<hairongyahoo-inc.com> wrote:

> Has anybody tried 15.0? Please check
> ht
tps://issues.apache.org/jira/browse/HADOOP-1463.
> 
> Hairong
> -----Original Message-----
> From: Joydeep Sen Sarma [mailto:jssarmafacebook.com]
> Sent: Tuesday, January 08, 2008 11:33 AM
> To: hadoop-userlucene.apache.org; hadoop-userlucene.apache.org
> Subject: RE: Limit the space used by hadoop on a slave
node
> 
> at least up until 14.4, these options are broken. see
> htt
ps://issues.apache.org/jira/browse/HADOOP-2549
> 
> (there's a trivial patch - but i am still testing).
> 
> 

We use, 

dfs.datanode.du.pct for 0.14 and dfs.datanode.du.reserved
for 0.15.

Change was made in the Jira Hairong mentioned.
htt
ps://issues.apache.org/jira/browse/HADOOP-1463

Koji

> -----Original Message-----
> From: Ted Dunning [mailto:tdunningveoh.com]
> Sent: Tuesday, January 08, 2008 1:13 PM
> To: hadoop-userlucene.apache.org
> Subject: Re: Limit the space used by hadoop on a slave
node
> 
> 
> I think I have seen related bad behavior on 15.1.
> 
> On 1/8/08 11:49 AM, "Hairong Kuang"
<hairongyahoo-inc.com> wrote:
> 
> > Has anybody tried 15.0? Please check
> > ht
tps://issues.apache.org/jira/browse/HADOOP-1463.
> >
> > Hairong
> > -----Original Message-----
> > From: Joydeep Sen Sarma [mailto:jssarmafacebook.com]
> > Sent: Tuesday, January 08, 2008 11:33 AM
> > To: hadoop-userlucene.apache.org;
hadoop-userlucene.apache.org
> > Subject: RE: Limit the space used by hadoop on a
slave node
> >
> > at least up until 14.4, these options are broken.
see
> > htt
ps://issues.apache.org/jira/browse/HADOOP-2549
> >
> > (there's a trivial patch - but i am still
testing).
> >
> >

Most of the time dfs and map/reduce share disks. Keep in
mind that du
options can not control how much space that map/reduce tasks
take.
Sometimes we get the out of disk space problem because data
intensive
map/reduce tasks take a lot of disk space.

Hairong

-----Original Message-----
From: Ted Dunning [mailto:tdunningveoh.com] 
Sent: Tuesday, January 08, 2008 1:13 PM
To: hadoop-userlucene.apache.org
Subject: Re: Limit the space used by hadoop on a slave node


I think I have seen related bad behavior on 15.1.

On 1/8/08 11:49 AM, "Hairong Kuang"
<hairongyahoo-inc.com> wrote:

> Has anybody tried 15.0? Please check
> ht
tps://issues.apache.org/jira/browse/HADOOP-1463.
> 
> Hairong
> -----Original Message-----
> From: Joydeep Sen Sarma [mailto:jssarmafacebook.com]
> Sent: Tuesday, January 08, 2008 11:33 AM
> To: hadoop-userlucene.apache.org; hadoop-userlucene.apache.org
> Subject: RE: Limit the space used by hadoop on a slave
node
> 
> at least up until 14.4, these options are broken. see
> htt
ps://issues.apache.org/jira/browse/HADOOP-2549
> 
> (there's a trivial patch - but i am still testing).
> 
>