List Info

Thread: Created: (HADOOP-2086) ability to add dependencies to a job after construction




Created: (HADOOP-2086) ability to add dependencies to a job after construction
country flaguser name
United States
2007-10-22 05:44:50
ability to add dependencies to a job after construction
-------------------------------------------------------

                 Key: HADOOP-2086
                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-2086
             Project: Hadoop
          Issue Type: Improvement
          Components: mapred
    Affects Versions: 0.14.2
         Environment: n/a
            Reporter: Adrian Woodhead


The current Job API only allows for dependent jobs to be
passed in at object construction time. It would be nice if
there was an additional constructor which did not take any
depending jobs and then an "addDependingJob"
method which could be used to add depending jobs to a job at
a later point.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.


Updated: (HADOOP-2086) ability to add dependencies to a job after construction
country flaguser name
United States
2007-10-22 05:46:50
     [ https://issues.apache.org/jira/browse/HADOOP-2086?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrian Woodhead updated HADOOP-2086:
------------------------------------

    Status: Patch Available  (was: Open)

Here is a patch which fixes the issue I outlined in the
original submission. If you require me to make any changes
before you accept it please let me know and I will be happy
to make them.

> ability to add dependencies to a job after
construction
>
-------------------------------------------------------
>
>                 Key: HADOOP-2086
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-2086
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.14.2
>         Environment: n/a
>            Reporter: Adrian Woodhead
>
> The current Job API only allows for dependent jobs to
be passed in at object construction time. It would be nice
if there was an additional constructor which did not take
any depending jobs and then an "addDependingJob"
method which could be used to add depending jobs to a job at
a later point.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.


Updated: (HADOOP-2086) ability to add dependencies to a job after construction
country flaguser name
United States
2007-10-22 05:48:50
     [ https://issues.apache.org/jira/browse/HADOOP-2086?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrian Woodhead updated HADOOP-2086:
------------------------------------

    Status: Open  (was: Patch Available)

> ability to add dependencies to a job after
construction
>
-------------------------------------------------------
>
>                 Key: HADOOP-2086
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-2086
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.14.2
>         Environment: n/a
>            Reporter: Adrian Woodhead
>
> The current Job API only allows for dependent jobs to
be passed in at object construction time. It would be nice
if there was an additional constructor which did not take
any depending jobs and then an "addDependingJob"
method which could be used to add depending jobs to a job at
a later point.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.


Updated: (HADOOP-2086) ability to add dependencies to a job after construction
country flaguser name
United States
2007-10-22 05:54:51
     [ https://issues.apache.org/jira/browse/HADOOP-2086?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrian Woodhead updated HADOOP-2086:
------------------------------------

    Status: Patch Available  (was: Open)

Index:
/home/adrian/workspace/hadoop-0.14.2/src/java/org/apache/had
oop/mapred/jobcontrol/Job.java
============================================================
=======
---
/home/adrian/workspace/hadoop-0.14.2/src/java/org/apache/had
oop/mapred/jobcontrol/Job.java	(revision 584810)
+++
/home/adrian/workspace/hadoop-0.14.2/src/java/org/apache/had
oop/mapred/jobcontrol/Job.java	(working copy)
 -77,6
+77,15 
     this.message = "just initialized";
     this.jc = new JobClient(jobConf);
   }
+  
+  /**
+   * Construct a job that doesn't depend on any other
jobs.
+   * param jobConf a mapred job configuration representing
a job to be executed.
+   * throws IOException
+   */
+  public Job(JobConf jobConf) throws IOException {
+  	this(jobConf, null);
+  }
 	
   public String toString() {
     StringBuffer sb = new StringBuffer();
 -86,7
+95,7 
     sb.append("job mapred
id:t").append(this.mapredJobID).append("n")
;
     sb.append("job
message:t").append(this.message).append("n"
);
 		
-    if (this.dependingJobs == null) {
+    if (this.dependingJobs == null ||
this.dependingJobs.size() == 0) {
       sb.append("job has no depending
job:t").append("n");
     } else {
       sb.append("job has
").append(this.dependingJobs.size()).append("
dependeng jobs:n");
 -255,6
+264,18 
       }
     }
   }
+  
+  /**
+   * Add a job to this jobs' dependency list.
+   * param dependingJob Job that this job depends on.
+   * return <tt>true</tt> if this collection
changed as a result of the call.
+   */
+  public boolean addDependingJob(Job dependingJob) {
+  	if (this.dependingJobs == null) {
+  		this.dependingJobs = new ArrayList();
+  	}
+  	return this.dependingJobs.add(dependingJob);
+  }
 	
   /**
    * Check and update the state of this job. The state
changes  


> ability to add dependencies to a job after
construction
>
-------------------------------------------------------
>
>                 Key: HADOOP-2086
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-2086
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.14.2
>         Environment: n/a
>            Reporter: Adrian Woodhead
>
> The current Job API only allows for dependent jobs to
be passed in at object construction time. It would be nice
if there was an additional constructor which did not take
any depending jobs and then an "addDependingJob"
method which could be used to add depending jobs to a job at
a later point.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.


[1-4]

about | contact  Other archives ( Real Estate discussion Medical topics )