Created: (SOLR-320) DirectUpdateHandler2 threading issue

DirectUpdateHandler2 threading issue
------------------------------------

                 Key: SOLR-320
                 URL: https:
//issues.apache.org/jira/browse/SOLR-320
             Project: Solr
          Issue Type: Bug
          Components: update
    Affects Versions: 1.2
         Environment: Ubuntu 7.04, Java 1.6.0-b105
            Reporter: Stu Hood


While working on an embedded Solr solution, I noticed that
one of the threads created during typical usage of
(SolrCore, DocumentBuilder and UpdateHandler), was not
dying.

I wrote a small embedded Solr app, and running it under JDB
made it clear that the environment was not finishing cleanly
because of a thread called "pool-2-thread-1" in
"cond. waiting" state. After a quick grep, I saw
that only one class uses a thread pool, and that is the
DirectUpdateHandler2. It uses an instance of
ScheduledExecutorService to manage autocommit threads, but
it apparently isn't dieing correctly.

I'll start working on a patch, but the original author of
the handler probably has more knowledge (see https:/
/issues.apache.org/jira/browse/SOLR-65)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

     [ 
https://issues.apache.org/jira/browse/SOLR-320?page=com.atla
ssian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stu Hood updated SOLR-320:
--------------------------

    Attachment: solr-runner.tgz

Here is a little embedded Solr testcase that exhibits the
problem. See the README file for instructions.

> DirectUpdateHandler2 threading issue
> ------------------------------------
>
>                 Key: SOLR-320
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-320
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.2
>         Environment: Ubuntu 7.04, Java 1.6.0-b105
>            Reporter: Stu Hood
>         Attachments: solr-runner.tgz
>
>
> While working on an embedded Solr solution, I noticed
that one of the threads created during typical usage of
(SolrCore, DocumentBuilder and UpdateHandler), was not
dying.
> I wrote a small embedded Solr app, and running it under
JDB made it clear that the environment was not finishing
cleanly because of a thread called
"pool-2-thread-1" in "cond. waiting"
state. After a quick grep, I saw that only one class uses a
thread pool, and that is the DirectUpdateHandler2. It uses
an instance of ScheduledExecutorService to manage autocommit
threads, but it apparently isn't dieing correctly.
> I'll start working on a patch, but the original author
of the handler probably has more knowledge (see https:/
/issues.apache.org/jira/browse/SOLR-65)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-320?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12515746 ] 

Stu Hood commented on SOLR-320:
-------------------------------

Additionally, with the attached files, I noticed that
disabling autoCommit fixes the problem.

> DirectUpdateHandler2 threading issue
> ------------------------------------
>
>                 Key: SOLR-320
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-320
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.2
>         Environment: Ubuntu 7.04, Java 1.6.0-b105
>            Reporter: Stu Hood
>         Attachments: solr-runner.tgz
>
>
> While working on an embedded Solr solution, I noticed
that one of the threads created during typical usage of
(SolrCore, DocumentBuilder and UpdateHandler), was not
dying.
> I wrote a small embedded Solr app, and running it under
JDB made it clear that the environment was not finishing
cleanly because of a thread called
"pool-2-thread-1" in "cond. waiting"
state. After a quick grep, I saw that only one class uses a
thread pool, and that is the DirectUpdateHandler2. It uses
an instance of ScheduledExecutorService to manage autocommit
threads, but it apparently isn't dieing correctly.
> I'll start working on a patch, but the original author
of the handler probably has more knowledge (see https:/
/issues.apache.org/jira/browse/SOLR-65)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-320?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12515784 ] 

Mike Klaas commented on SOLR-320:
---------------------------------

Thanks for the bug report!  I should be able to fix this
momentarily.

> DirectUpdateHandler2 threading issue
> ------------------------------------
>
>                 Key: SOLR-320
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-320
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.2
>         Environment: Ubuntu 7.04, Java 1.6.0-b105
>            Reporter: Stu Hood
>         Attachments: solr-runner.tgz
>
>
> While working on an embedded Solr solution, I noticed
that one of the threads created during typical usage of
(SolrCore, DocumentBuilder and UpdateHandler), was not
dying.
> I wrote a small embedded Solr app, and running it under
JDB made it clear that the environment was not finishing
cleanly because of a thread called
"pool-2-thread-1" in "cond. waiting"
state. After a quick grep, I saw that only one class uses a
thread pool, and that is the DirectUpdateHandler2. It uses
an instance of ScheduledExecutorService to manage autocommit
threads, but it apparently isn't dieing correctly.
> I'll start working on a patch, but the original author
of the handler probably has more knowledge (see https:/
/issues.apache.org/jira/browse/SOLR-65)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-320?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12515788 ] 

Mike Klaas commented on SOLR-320:
---------------------------------

This fixes the problem for me:

Index:
src/java/org/apache/solr/update/DirectUpdateHandler2.java
============================================================
=======
---
src/java/org/apache/solr/update/DirectUpdateHandler2.java  
(revision 559884)
+++
src/java/org/apache/solr/update/DirectUpdateHandler2.java  
(working copy)
 -566,6
+566,7 
         tracker.pending.cancel( true );
         tracker.pending = null;
       }
+      tracker.scheduler.shutdown(); 
       doDeletions();
       closeSearcher();
       closeWriter();

> DirectUpdateHandler2 threading issue
> ------------------------------------
>
>                 Key: SOLR-320
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-320
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.2
>         Environment: Ubuntu 7.04, Java 1.6.0-b105
>            Reporter: Stu Hood
>             Fix For: 1.3
>
>         Attachments: solr-runner.tgz
>
>
> While working on an embedded Solr solution, I noticed
that one of the threads created during typical usage of
(SolrCore, DocumentBuilder and UpdateHandler), was not
dying.
> I wrote a small embedded Solr app, and running it under
JDB made it clear that the environment was not finishing
cleanly because of a thread called
"pool-2-thread-1" in "cond. waiting"
state. After a quick grep, I saw that only one class uses a
thread pool, and that is the DirectUpdateHandler2. It uses
an instance of ScheduledExecutorService to manage autocommit
threads, but it apparently isn't dieing correctly.
> I'll start working on a patch, but the original author
of the handler probably has more knowledge (see https:/
/issues.apache.org/jira/browse/SOLR-65)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

     [ 
https://issues.apache.org/jira/browse/SOLR-320?page=com.atla
ssian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mike Klaas resolved SOLR-320.
-----------------------------

       Resolution: Fixed
    Fix Version/s: 1.3
         Assignee: Mike Klaas

Fixed in r559887

> DirectUpdateHandler2 threading issue
> ------------------------------------
>
>                 Key: SOLR-320
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-320
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.2
>         Environment: Ubuntu 7.04, Java 1.6.0-b105
>            Reporter: Stu Hood
>            Assignee: Mike Klaas
>             Fix For: 1.3
>
>         Attachments: solr-runner.tgz
>
>
> While working on an embedded Solr solution, I noticed
that one of the threads created during typical usage of
(SolrCore, DocumentBuilder and UpdateHandler), was not
dying.
> I wrote a small embedded Solr app, and running it under
JDB made it clear that the environment was not finishing
cleanly because of a thread called
"pool-2-thread-1" in "cond. waiting"
state. After a quick grep, I saw that only one class uses a
thread pool, and that is the DirectUpdateHandler2. It uses
an instance of ScheduledExecutorService to manage autocommit
threads, but it apparently isn't dieing correctly.
> I'll start working on a patch, but the original author
of the handler probably has more knowledge (see https:/
/issues.apache.org/jira/browse/SOLR-65)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-320?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12515799 ] 

Stu Hood commented on SOLR-320:
-------------------------------

Thanks Mike!

> DirectUpdateHandler2 threading issue
> ------------------------------------
>
>                 Key: SOLR-320
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-320
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.2
>         Environment: Ubuntu 7.04, Java 1.6.0-b105
>            Reporter: Stu Hood
>            Assignee: Mike Klaas
>             Fix For: 1.3
>
>         Attachments: solr-runner.tgz
>
>
> While working on an embedded Solr solution, I noticed
that one of the threads created during typical usage of
(SolrCore, DocumentBuilder and UpdateHandler), was not
dying.
> I wrote a small embedded Solr app, and running it under
JDB made it clear that the environment was not finishing
cleanly because of a thread called
"pool-2-thread-1" in "cond. waiting"
state. After a quick grep, I saw that only one class uses a
thread pool, and that is the DirectUpdateHandler2. It uses
an instance of ScheduledExecutorService to manage autocommit
threads, but it apparently isn't dieing correctly.
> I'll start working on a patch, but the original author
of the handler probably has more knowledge (see https:/
/issues.apache.org/jira/browse/SOLR-65)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-320?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12515809 ] 

J.J. Larrea commented on SOLR-320:
----------------------------------

This was biting me too... thanks for filing the detailed
report + testcase, Stu, and the super-quick fix, Mike!

> DirectUpdateHandler2 threading issue
> ------------------------------------
>
>                 Key: SOLR-320
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-320
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.2
>         Environment: Ubuntu 7.04, Java 1.6.0-b105
>            Reporter: Stu Hood
>            Assignee: Mike Klaas
>             Fix For: 1.3
>
>         Attachments: solr-runner.tgz
>
>
> While working on an embedded Solr solution, I noticed
that one of the threads created during typical usage of
(SolrCore, DocumentBuilder and UpdateHandler), was not
dying.
> I wrote a small embedded Solr app, and running it under
JDB made it clear that the environment was not finishing
cleanly because of a thread called
"pool-2-thread-1" in "cond. waiting"
state. After a quick grep, I saw that only one class uses a
thread pool, and that is the DirectUpdateHandler2. It uses
an instance of ScheduledExecutorService to manage autocommit
threads, but it apparently isn't dieing correctly.
> I'll start working on a patch, but the original author
of the handler probably has more knowledge (see https:/
/issues.apache.org/jira/browse/SOLR-65)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.