List Info

Thread: Relational Algrebra Operators
Relational Algrebra Operators
country flaguser name
United States		 2007-07-13 06:35:04 
[HbaseShell] Relational Algrebra Operators
------------------------------------------

                 Key: HADOOP-1608
                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
             Project: Hadoop
          Issue Type: Improvement
          Components: contrib/hbase
    Affects Versions: 0.13.0
         Environment: All environments 
            Reporter: edward yoon
            Priority: Minor
             Fix For: 0.14.0


Development of relational algebra operators has begun.

 * Projection 
 * Selection 
 * Product 
 * Rename
 * Group
 * Sort


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-13 06:52:05 
     [ https://issues.apache.org/jira/browse/HADOOP-1608?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

edward yoon updated HADOOP-1608:
--------------------------------

    Attachment: shell_r_operators_v01.patch

Abstract and Interface Classes.

> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.0
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-15 21:49:04 
    [ https://issues.apache.org/jira/browse
/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12512848 ] 

edward yoon commented on HADOOP-1608:
-------------------------------------

Hi, All.

I want to use hbase.mapred on shell.
But, when I try to submit a job to the JobTracker, I get
this exception error message.


2007-07-14 20:37:04,377 INFO
org.apache.hadoop.mapred.TaskInProgress: 
Error from task_200707142013_0004_r_000000_2: 
java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: 
org.apache.hadoop.hbase.mapred.IdentityTableReduce


How can i fix it without another changes ?
Please give me some advice.


> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.0
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-16 18:51:05 
    [ https://issues.apache.org/jira/browse
/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12513102 ] 

stack commented on HADOOP-1608:
-------------------------------

Try this Edward:


% $/bin/hbase org.apache.hadoop.mapred.JobClient
YOUR_JOB_JAR


The above does as '$/bin/hadoop jar
YOUR_JOB_JAR' does only it has hadoop-hbase.jar on the
CLASSPATH

Perhaps we should add to the hbase script support for a
'jar' argument?

> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.0
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-16 23:37:05 
    [ https://issues.apache.org/jira/browse
/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12513147 ] 

stack commented on HADOOP-1608:
-------------------------------

Thinking more on my above comment, it won't get you far.  It
may work for all running in single context but will fail
when you go to distribute your jar across multiple hosts.

Either copy the hadoop-hbase.jar to $HADOOP_HOME/lib
everywhere on your cluster or build the hadoop-hbase.jar
into your job jar under its lib directory.

Regards the latter, job jars submitted to hadoop are undone
by the RunJar class out on each tasktracker node.  Before
invoking the jar's Main-Class from its MANIFEST.MF (or
reading from the command-line if this is not set), it sets
up the CLASSPATH so all jars found in the jars' lib
directory, etc., are included.  See the RunJar source for
how this is done if you want to learn more (For an example,
see nutch).

I took a look at bin/hadoop to add contribs to the
classpath.  We could make a patch to add
'build/contrib/*/classes'.  This would work for development
environment but for binary distributions, the distribution
target could add contrib jars to the lib dir or into a
hadoop-contrib sub-directory that got added to the classpath
when bin/hadoop runs.

> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.0
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-23 03:50:31 
    [ https://issues.apache.org/jira/browse
/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12514591 ] 

edward yoon commented on HADOOP-1608:
-------------------------------------

>> We could make a patch to add
'build/contrib/*/classes'

I agree with you Micheal Stack.
because i don't think i should make a new JobTracker stater
all over again.
..
Then can this stack handle this issue?

> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.0
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>         Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-25 04:16:31 
     [ https://issues.apache.org/jira/browse/HADOOP-1608?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

edward yoon updated HADOOP-1608:
--------------------------------

    Attachment: shell_v02.patch

This patch 
 * adds (project, select, sort, substitute, store) command.


Examples.

HBase > A = table('movieLog_table');
HBase > B = A.projection('year','length');
HBase > [C = sort B by ('year');]
HBase > store B to table('temp_table');
...

HBase > A = table('movieLog_table');
HBase > B = A.selection(length > 100);
HBase > store B to table('temp_table2');
...

HBase > A = table('movieLog_table');
HBase > B = A.projection('year','length','studioName');
HBase > C = B.selection(length > 100);
HBase > D = sort C by ('studioName');
HBase > store D to table('temp_table3');
--Output Table Creating.
  --Job 'job_200707200959_0003' is submitted
    --Job job_200707200959_0003 is still running.
    --Job job_200707200959_0003 is still running.
    --Job job_200707200959_0003 is still running.
Job Successful.(47.86 sec)
HBase > select temp_table3;
+------+----------------------+----------------------+------
----------------+
| No.  | Row                  | Column               | Cell 
               |
+------+----------------------+----------------------+------
----------------+
| 1    | Disney               | length:              | 124  
               |
+------+----------------------+----------------------+------
----------------+
| 2    | Disney               | studioName:          |
Disney               |
+------+----------------------+----------------------+------
----------------+
| 3    | Disney               | year:                | 1991 
               |
+------+----------------------+----------------------+------
----------------+
| 4    | Fox                  | length:              | 124  
               |
+------+----------------------+----------------------+------
----------------+
| 5    | Fox                  | studioName:          | Fox  
               |
+------+----------------------+----------------------+------
----------------+
| 6    | Fox                  | year:                | 1977 
               |
+------+----------------------+----------------------+------
----------------+


but, I should make logical operators(AND, OR, NOT) in
selection command syntax.


> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.0
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>         Attachments: shell_r_operators_v01.patch,
shell_v02.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-26 07:49:41 
     [ https://issues.apache.org/jira/browse/HADOOP-1608?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

edward yoon updated HADOOP-1608:
--------------------------------

    Attachment: shell_v03.patch

 * Fixed Filter Operation

selection(length > 100 and studioName = 'Fox');
selection(studioName = 'Fox' or studioName = 'Paramount');
selection(length > 100 and studioName = 'Fox' or
studioName = 'Paramount');



> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.0
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>         Attachments: shell_r_operators_v01.patch,
shell_v02.patch, shell_v03.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-26 21:35:03 
     [ https://issues.apache.org/jira/browse/HADOOP-1608?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

edward yoon updated HADOOP-1608:
--------------------------------

    Attachment: shell_v04.patch

Add some comments in code, 
and if you agree, i would like to submit this patch.



> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.0
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>         Attachments: shell_r_operators_v01.patch,
shell_v02.patch, shell_v03.patch, shell_v04.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.
Relational Algrebra Operators
country flaguser name
United States		 2007-07-27 20:57:53 
     [ https://issues.apache.org/jira/browse/HADOOP-1608?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

edward yoon updated HADOOP-1608:
--------------------------------

        Fix Version/s: 0.14.0
    Affects Version/s:     (was: 0.13.0)
                       0.13.1
               Status: Patch Available  (was: Open)

Builds locally. Trying against hudson

> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
>                 Key: HADOOP-1608
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1608
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.13.1
>         Environment: All environments 
>            Reporter: edward yoon
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: shell_r_operators_v01.patch,
shell_v02.patch, shell_v03.patch, shell_v04.patch
>
>
> Development of relational algebra operators has begun.
>  * Projection 
>  * Selection 
>  * Product 
>  * Rename
>  * Group
>  * Sort

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

about | contact  Other archives ( Real Estate discussion Medical topics )