Created: (HADOOP-1185) dynamically change dfs log levels

dynamically change dfs log levels
---------------------------------

                 Key: HADOOP-1185
                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1185
             Project: Hadoop
          Issue Type: Improvement
          Components: dfs
            Reporter: dhruba borthakur
         Assigned To: dhruba borthakur


I would like to switch on the debug log level of the
namenode without restarting it. This is needed to analyze a
production system.

Can somebody please advice on how to set the loglevel
dyncamically on a running namenode? I was thinking of
enhancing dfsadmin to make an RPC to the namenode to set a
specified logging level. But the apache common logging APi
does not export an API to change logging levels.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse
/HADOOP-1185?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12488430 ] 

dhruba borthakur commented on HADOOP-1185:
------------------------------------------

The apache common logging API does not let me set logging
levels. Can somebody please suggest how I can get to the
underlying log4j object?

> dynamically change dfs log levels
> ---------------------------------
>
>                 Key: HADOOP-1185
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1185
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>         Assigned To: dhruba borthakur
>
> I would like to switch on the debug log level of the
namenode without restarting it. This is needed to analyze a
production system.
> Can somebody please advice on how to set the loglevel
dyncamically on a running namenode? I was thinking of
enhancing dfsadmin to make an RPC to the namenode to set a
specified logging level. But the apache common logging APi
does not export an API to change logging levels.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse
/HADOOP-1185?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12488432 ] 

Owen O'Malley commented on HADOOP-1185:
---------------------------------------

The best approach would be something along the lines of:

Log LOG = ...;

if (LOG instanceof Log4JLogger) {
  org.apache.log4j.Logger l4j = ((Log4JLogger)
LOG).getLogger();
  ... control the logger in log4j specific ways...
}

> dynamically change dfs log levels
> ---------------------------------
>
>                 Key: HADOOP-1185
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1185
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>         Assigned To: dhruba borthakur
>
> I would like to switch on the debug log level of the
namenode without restarting it. This is needed to analyze a
production system.
> Can somebody please advice on how to set the loglevel
dyncamically on a running namenode? I was thinking of
enhancing dfsadmin to make an RPC to the namenode to set a
specified logging level. But the apache common logging APi
does not export an API to change logging levels.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse
/HADOOP-1185?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12488451 ] 

Owen O'Malley commented on HADOOP-1185:
---------------------------------------

I'd suggest putting a little servlet into StatusHttpServer
so that all of the servers will respond to the same url.
Something like:

http://$host:$port/setLogLevel?log=org.apache.hadoop.mapred.
TaskTracker&level=debug

will push that server into logging the TaskTracker class at
the debug level.

> dynamically change dfs log levels
> ---------------------------------
>
>                 Key: HADOOP-1185
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1185
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>         Assigned To: dhruba borthakur
>
> I would like to switch on the debug log level of the
namenode without restarting it. This is needed to analyze a
production system.
> Can somebody please advice on how to set the loglevel
dyncamically on a running namenode? I was thinking of
enhancing dfsadmin to make an RPC to the namenode to set a
specified logging level. But the apache common logging APi
does not export an API to change logging levels.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

     [ https://issues.apache.org/jira/browse/HADOOP-1185?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur updated HADOOP-1185:
-------------------------------------

    Fix Version/s: 0.15.0

> dynamically change dfs log levels
> ---------------------------------
>
>                 Key: HADOOP-1185
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1185
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>             Fix For: 0.15.0
>
>
> I would like to switch on the debug log level of the
namenode without restarting it. This is needed to analyze a
production system.
> Can somebody please advice on how to set the loglevel
dyncamically on a running namenode? I was thinking of
enhancing dfsadmin to make an RPC to the namenode to set a
specified logging level. But the apache common logging APi
does not export an API to change logging levels.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

     [ https://issues.apache.org/jira/browse/HADOOP-1185?page=co
m.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur updated HADOOP-1185:
-------------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I just committed this. Thanks Nicholas!

> dynamically change log levels
> -----------------------------
>
>                 Key: HADOOP-1185
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1185
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: util
>    Affects Versions: 0.16.0
>            Reporter: dhruba borthakur
>            Assignee: Tsz Wo (Nicholas), SZE
>             Fix For: 0.16.0
>
>         Attachments: 1185_20071102.patch
>
>
> I would like to switch on the debug log level of the
namenode (or other components) without restarting it. This
is needed to analyze a production system.
> Can somebody please advice on how to set the loglevel
dyncamically on a running namenode? I was thinking of
enhancing dfsadmin to make an RPC to the namenode to set a
specified logging level. But the apache common logging APi
does not export an API to change logging levels.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse
/HADOOP-1185?page=com.atlassian.jira.plugin.system.issuetabp
anels:comment-tabpanel#action_12540477 ] 

Hudson commented on HADOOP-1185:
--------------------------------

Integrated in Hadoop-Nightly #295 (See [http://lucene.zones.apache.org:8080/hudson/job
/Hadoop-Nightly/295/])

> dynamically change log levels
> -----------------------------
>
>                 Key: HADOOP-1185
>                 URL: htt
ps://issues.apache.org/jira/browse/HADOOP-1185
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: util
>    Affects Versions: 0.16.0
>            Reporter: dhruba borthakur
>            Assignee: Tsz Wo (Nicholas), SZE
>             Fix For: 0.16.0
>
>         Attachments: 1185_20071102.patch
>
>
> I would like to switch on the debug log level of the
namenode (or other components) without restarting it. This
is needed to analyze a production system.
> Can somebody please advice on how to set the loglevel
dyncamically on a running namenode? I was thinking of
enhancing dfsadmin to make an RPC to the namenode to set a
specified logging level. But the apache common logging APi
does not export an API to change logging levels.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.