Created: (HADOOP-1042) Improve the handling of failed map output fetches

Thread: Created: (HADOOP-1042) Improve the handling of failed map output fetches

Search this list only Search all lists
Created: (HADOOP-1042) Improve the handling of failed map output fetches
United States	2007-02-26 23:35:06
Improve the handling of failed map output fetches ------------------------------------------------- Key: HADOOP-1042 URL: htt ps://issues.apache.org/jira/browse/HADOOP-1042 Project: Hadoop Issue Type: Improvement Components: mapred Affects Versions: 0.11.2 Reporter: Devaraj Das Assigned To: Devaraj Das Currently, whenever fetch of a map output fails the corresponding MapOutputLocation is added to a List datastructure for later retrial. But, if the failure was due to a lost task, the entry that was added is not deleted. For such cases, unnecessary retrials will happen. This situation should be prevented. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

Commented: (HADOOP-1042) Improve the handling of failed map output fetches
United States	2007-02-27 11:45:05
[ https://issues.apache.org/jira/browse /HADOOP-1042?page=com.atlassian.jira.plugin.system.issuetabp anels:comment-tabpanel#action_12476242 ] David Bowen commented on HADOOP-1042: ------------------------------------- A small point: in lines like this: Map<Integer, MapOutputLocation> knownOutputs = new HashMap(); it would be preferable (avoid a warning) to specify the types on the right hand side, like this: Map<Integer,MapOutputLocation> knownOutputs = new HashMap<Integer,MapOutputLocation>(); > Improve the handling of failed map output fetches > ------------------------------------------------- > > Key: HADOOP-1042 > URL: htt ps://issues.apache.org/jira/browse/HADOOP-1042 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.11.2 > Reporter: Devaraj Das > Assigned To: Devaraj Das > Attachments: 1042.patch > > > Currently, whenever fetch of a map output fails the corresponding MapOutputLocation is added to a List datastructure for later retrial. But, if the failure was due to a lost task, the entry that was added is not deleted. For such cases, unnecessary retrials will happen. This situation should be prevented. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

Updated: (HADOOP-1042) Improve the handling of failed map output fetches
United States	2007-02-27 12:54:05
[ https://issues.apache.org/jira/browse/HADOOP-1042?page=co m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HADOOP-1042: -------------------------------- Attachment: 1042.new.patch Ok, this patch incorporates David's comment (replaced only one line with what David suggested since that was the only place where such a thing was done). Actually, there are lots of places in the codebase where similar warnings can be removed. > Improve the handling of failed map output fetches > ------------------------------------------------- > > Key: HADOOP-1042 > URL: htt ps://issues.apache.org/jira/browse/HADOOP-1042 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.11.2 > Reporter: Devaraj Das > Assigned To: Devaraj Das > Attachments: 1042.new.patch, 1042.patch > > > Currently, whenever fetch of a map output fails the corresponding MapOutputLocation is added to a List datastructure for later retrial. But, if the failure was due to a lost task, the entry that was added is not deleted. For such cases, unnecessary retrials will happen. This situation should be prevented. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

Updated: (HADOOP-1042) Improve the handling of failed map output fetches
United States	2007-02-27 12:54:05
[ https://issues.apache.org/jira/browse/HADOOP-1042?page=co m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HADOOP-1042: -------------------------------- Status: Patch Available (was: Open) > Improve the handling of failed map output fetches > ------------------------------------------------- > > Key: HADOOP-1042 > URL: htt ps://issues.apache.org/jira/browse/HADOOP-1042 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.11.2 > Reporter: Devaraj Das > Assigned To: Devaraj Das > Attachments: 1042.new.patch, 1042.patch > > > Currently, whenever fetch of a map output fails the corresponding MapOutputLocation is added to a List datastructure for later retrial. But, if the failure was due to a lost task, the entry that was added is not deleted. For such cases, unnecessary retrials will happen. This situation should be prevented. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

Commented: (HADOOP-1042) Improve the handling of failed map output fetches
United States	2007-02-27 16:22:06
[ https://issues.apache.org/jira/browse /HADOOP-1042?page=com.atlassian.jira.plugin.system.issuetabp anels:comment-tabpanel#action_12476420 ] Hadoop QA commented on HADOOP-1042: ----------------------------------- +1, because http://issues.apache.org/jira/secure/atta chment/12352150/1042.new.patch applied and successfully tested against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/5 12006. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop -Patch > Improve the handling of failed map output fetches > ------------------------------------------------- > > Key: HADOOP-1042 > URL: htt ps://issues.apache.org/jira/browse/HADOOP-1042 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.11.2 > Reporter: Devaraj Das > Assigned To: Devaraj Das > Attachments: 1042.new.patch, 1042.patch > > > Currently, whenever fetch of a map output fails the corresponding MapOutputLocation is added to a List datastructure for later retrial. But, if the failure was due to a lost task, the entry that was added is not deleted. For such cases, unnecessary retrials will happen. This situation should be prevented. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

Updated: (HADOOP-1042) Improve the handling of failed map output fetches
United States	2007-02-28 12:52:57
[ https://issues.apache.org/jira/browse/HADOOP-1042?page=co m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1042: --------------------------------- Resolution: Fixed Fix Version/s: 0.12.0 Status: Resolved (was: Patch Available) I committed this yesterday. Thanks, Devaraj! > Improve the handling of failed map output fetches > ------------------------------------------------- > > Key: HADOOP-1042 > URL: htt ps://issues.apache.org/jira/browse/HADOOP-1042 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.11.2 > Reporter: Devaraj Das > Assigned To: Devaraj Das > Fix For: 0.12.0 > > Attachments: 1042.new.patch, 1042.patch > > > Currently, whenever fetch of a map output fails the corresponding MapOutputLocation is added to a List datastructure for later retrial. But, if the failure was due to a lost task, the entry that was added is not deleted. For such cases, unnecessary retrials will happen. This situation should be prevented. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[1-6]

about | contact Other archives ( Real Estate discussion Medical topics )