[ https://issues.apache.org/jira/browse
/HADOOP-1952?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12540721 ]
Arun C Murthy commented on HADOOP-1952:
---------------------------------------
Lohit, I see your latest patch removes some combiner
options... are you sure about these changes? If so, are they
related to this jira?
Index:
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estStreamingFailure.java
============================================================
=======
---
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estStreamingFailure.java (revision 588543)
+++
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estStreamingFailure.java (working copy)
 -47,7
+47,6
"-input",
INVALID_INPUT_FILE.getAbsolutePath(),
"-output", OUTPUT_DIR.getAbsolutePath(),
"-mapper", map,
- "-combiner", combine,
"-reducer", reduce,
//"-verbose",
//"-jobconf", "stream.debug=set"
Index:
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estGzipInput.java
============================================================
=======
---
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estGzipInput.java (revision 588543)
+++
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estGzipInput.java (working copy)
-47,7
+47,6
"-input", INPUT_FILE.getAbsolutePath(),
"-output", OUTPUT_DIR.getAbsolutePath(),
"-mapper", map,
- "-combiner", combine,
"-reducer", reduce,
"-jobconf",
"stream.recordreader.compression=gzip"
};
Index:
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estStreaming.java
============================================================
=======
---
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estStreaming.java (revision 588543)
+++
src/contrib/streaming/src/test/org/apache/hadoop/streaming/T
estStreaming.java (working copy)
-40,7
+40,6
protected String map =
StreamUtil.makeJavaCommand(TrApp.class, new
String[]{".", "\n"});
// reduce behave like /usr/bin/uniq. But also prepend
lines with R.
// command-line combiner does not have any effect any
more.
- protected String combine =
StreamUtil.makeJavaCommand(UniqApp.class, new
String[]{"C"});
protected String reduce =
StreamUtil.makeJavaCommand(UniqApp.class, new
String[]{"R"});
protected String outputExpect =
"RaretnRbluetnRbunniestnRpinktnRredtnRrosest
nRvioletstn";
-66,7
+65,6
"-input", INPUT_FILE.getAbsolutePath(),
"-output", OUTPUT_DIR.getAbsolutePath(),
"-mapper", map,
- "-combiner", combine,
"-reducer", reduce,
//"-verbose",
//"-jobconf", "stream.debug=set"
> Streaming does not handle invalid -inputformat (typo
by users for example)
>
------------------------------------------------------------
---------------
>
> Key: HADOOP-1952
> URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1952
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.14.1
> Reporter: lohit vijayarenu
> Assignee: lohit vijayarenu
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: CatchInvalidInputFormat.patch,
HADOOP-1952-1.patch, HADOOP-1952-2.patch
>
>
> Hadoop Streaming does not handle invalid inputformat
class. For example -inputformat INVALID class would not be
thrown as an error. Instead it defaults to
StreamInputFormat. If an invalid inputformat is specified,
it is good to fail.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
|
