Created: (HADOOP-2164) Reducer sort failed due to wrong key class

Reducer sort failed due to wrong key class
------------------------------------------

                 Key: HADOOP-2164
                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-2164
             Project: Hadoop
          Issue Type: Bug
          Components: mapred
            Reporter: Runping Qi



One of my job's reducers failed due to the following
exception:

java.io.IOException: wrong key class: class
org.apache.hadoop.io.LongWritable is not class
org.apache.hadoop.io.Text
	at
org.apache.hadoop.io.SequenceFile$Sorter$SegmentDescriptor.n
extRawKey(SequenceFile.java:2733)
	at
org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(Se
quenceFile.java:2542)
	at
org.apache.hadoop.io.SequenceFile$Sorter.merge(SequenceFile.
java:2326)
	at
org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:291)

	at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.
java:1760)


I checked the segment files to be merged on the node running
the reducer, 
and founded that one of the map output file did record
org.apache.hadoop.io.LongWritable as its key class!
The question is how that happened? My job conf specified
Text as the map output key class, and other map output files
had 
the correct key class!




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse
/HADOOP-2164?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12540684 ] 

Enis Soztutar commented on HADOOP-2164:
---------------------------------------

I have seem similar errors when the input is a mapfile but i
am using SequenceFileInputFormat. Could you please check
your program if this is indeed the case. 

> Reducer sort failed due to wrong key class
> ------------------------------------------
>
>                 Key: HADOOP-2164
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-2164
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Runping Qi
>
> One of my job's reducers failed due to the following
exception:
> java.io.IOException: wrong key class: class
org.apache.hadoop.io.LongWritable is not class
org.apache.hadoop.io.Text
> 	at
org.apache.hadoop.io.SequenceFile$Sorter$SegmentDescriptor.n
extRawKey(SequenceFile.java:2733)
> 	at
org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(Se
quenceFile.java:2542)
> 	at
org.apache.hadoop.io.SequenceFile$Sorter.merge(SequenceFile.
java:2326)
> 	at
org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:291)

> 	at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.
java:1760)
> I checked the segment files to be merged on the node
running the reducer, 
> and founded that one of the map output file did record
org.apache.hadoop.io.LongWritable as its key class!
> The question is how that happened? My job conf
specified Text as the map output key class, and other map
output files had 
> the correct key class!

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse
/HADOOP-2164?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12540707 ] 

Devaraj Das commented on HADOOP-2164:
-------------------------------------

Runping, is it the case that only one attempt for the
reducer failed? Did another attempt for the task succeed?

> Reducer sort failed due to wrong key class
> ------------------------------------------
>
>                 Key: HADOOP-2164
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-2164
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Runping Qi
>
> One of my job's reducers failed due to the following
exception:
> java.io.IOException: wrong key class: class
org.apache.hadoop.io.LongWritable is not class
org.apache.hadoop.io.Text
> 	at
org.apache.hadoop.io.SequenceFile$Sorter$SegmentDescriptor.n
extRawKey(SequenceFile.java:2733)
> 	at
org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(Se
quenceFile.java:2542)
> 	at
org.apache.hadoop.io.SequenceFile$Sorter.merge(SequenceFile.
java:2326)
> 	at
org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:291)

> 	at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.
java:1760)
> I checked the segment files to be merged on the node
running the reducer, 
> and founded that one of the map output file did record
org.apache.hadoop.io.LongWritable as its key class!
> The question is how that happened? My job conf
specified Text as the map output key class, and other map
output files had 
> the correct key class!

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.