List Info

Thread: Hadoop job question




Hadoop job question
user name
2006-08-29 23:17:24
Thanks for the pointer.
It does perfectly the job!


-----Original Message-----
From: Dennis Kubes [mailto:nutch-devdragonflymc.com] 
Sent: Tuesday, August 29, 2006 7:59 AM
To: nutch-devlucene.apache.org
Subject: Re: Hadoop job question

Although it is kinda hacking the system you may be able to
do it in the
map method by writing a custom MapRunner and having an
object that lives
in the MapRunner but that you set into each mapper instance.

Dennis

HUYLEBROECK Jeremy RD-ILAB-SSF wrote:
> I currently have a MR task that reads a SequenceFile
via the map 
> method to output some data.
>
> My goal is to output some data to MySQL but I'd like
to read several 
> records before doing the INSERT.
>
> But I can't figure out how to get several records...
> They all have different keys so the reduce task only
gets one at a
time.
>
>
> Thanks for any help!
>
>   
[1]

about | contact  Other archives ( Real Estate discussion Medical topics )