hi colleague,
Does hadoop just provide reading the files from line to
line?
How can I read many lines from a file?
thanks
--
Bin YANG
Department of Computer Science and Engineering
Fudan University
Shanghai, P. R. China
EMail: yangbinisme82gmail.com
Re: Dose hadoop just provide reading by
line
Germany
2007-12-05 04:32:32
Hi Bin YANG,
it can read as many lines/bytes as you want - but you need
to implement
your own RecordReader
(http://lucene.apache.org/hadoop/ap
i/org/apache/hadoop/mapred/RecordReader.html)
for this (and InputFormat in order to make it usable for
your jobs...)
Just take a look at the LineRecordReader as a starting point
on how to
implement a RecordReader.
Cu on the 'net,
Bye - bye,
<<<<<
André <<<< >>>> èrbnA
>>>>>
...you wrote:
> hi colleague,
>
> Does hadoop just provide reading the files from line to
line?
> How can I read many lines from a file?
>
> thanks
Re: Dose hadoop just provide reading by
line
United States
2007-12-11 18:26:26
Are you talking about reading arbitrary files from HDFS?
You can just get a regular InputStream to a Path:
JobConf conf = new JobConf(SomeJob.class);
FileSystem hdfs = FileSystem.get(conf);
FSDataInputStream inputStream = hdfs.open(new
Path("/my/path"));
BufferedReader reader = new BufferedReader(new
InputStreamReader(inputStream));
String s = reader.readLine();
Fabrice
Bin YANG wrote:
> hi colleague,
>
> Does hadoop just provide reading the files from line to
line?
> How can I read many lines from a file?
>
> thanks
>