Re: single output file

Output a constant key in the map function.


On 1/15/08 9:31 PM, "Vadim Zaliva"
<krokodilgmail.com> wrote:

> On Jan 15, 2008, at 17:56, Peter W. wrote:
> 
> That would output last 10 values for each key. I need
> to do this across all the keys in the set.
> 
> Vadim
> 
>> Hello,
>> 
>> Try using Java collection.
>> 
>> untested code follows...
>> 
>> public static class R extends MapReduceBase
implements Reducer
>>   {
>>   public void reduce(WritableComparable wc,Iterator
it,
>>      OutputCollector out,Reporter r)throws
IOException
>>      {
>>      Stack s=new Stack();
>>      int cnt=0;
>> 
>>      // everything
>>      while(it.hasNext())
>>         {
>>         s.push(((IntWritable)it.next()).get());
>>         }
>> 
>>      // last 10
>>      while((!s.empty())&&(cnt<=10))
>>         {
>>         out.collect(wc,(IntWritable)s.pop());
>>         cnt++;
>>         }
>>      }
>>   }
>> 
>> Good Luck,
>> 
>> Peter W.
>> 
>> Vadim Zaliva wrote:
>> 
>>> 
>>> Rui Shi wrote:
>>> 
>>>> As far as I understand, let mapper produce
top N records is not
>>>> working
>>>> as each mapper only has partial knowledge
of the data, which will
>>>> not lead to
>>>> global optimal... I think your mapper needs
to output all records
>>>> (combined) and let the reducer to pick the
top N values.
>>> 
>>> the question remains, how to return, say, last
10 records from
>>> Reducer.
>>> I need to know when last record is processed.
>>> 
>>> Vadim
>> 
>