Created: (SOLR-342) Add support for Lucene's new setRAMBufferSizeMB() method in IndexWriter

Add support for Lucene's new setRAMBufferSizeMB() method in
IndexWriter
------------------------------------------------------------
-----------

                 Key: SOLR-342
                 URL: https:
//issues.apache.org/jira/browse/SOLR-342
             Project: Solr
          Issue Type: Improvement
          Components: update
            Reporter: Grant Ingersoll
            Priority: Minor


LUCENE-843 adds support for new indexing capabilities using
the setRAMBufferSizeMB() method that should significantly
speed up indexing for many applications.  To fix this, we
will need trunk version of Lucene (or wait for the next
official release of Lucene)

Side effect of this is that Lucene's new, faster
StandardTokenizer will also be incorporated.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-342?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12531353 ] 

Grant Ingersoll commented on SOLR-342:
--------------------------------------

I think we should wait a bit more on this, as there still
are a fair number of issues related to these changes in
Lucene that need to be ironed out.

> Add support for Lucene's new setRAMBufferSizeMB()
method in IndexWriter
>
------------------------------------------------------------
-----------
>
>                 Key: SOLR-342
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-342
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Grant Ingersoll
>            Assignee: Grant Ingersoll
>            Priority: Minor
>
> LUCENE-843 adds support for new indexing capabilities
using the setRAMBufferSizeMB() method that should
significantly speed up indexing for many applications.  To
fix this, we will need trunk version of Lucene (or wait for
the next official release of Lucene)
> Side effect of this is that Lucene's new, faster
StandardTokenizer will also be incorporated.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-342?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12531588 ] 

Mike Klaas commented on SOLR-342:
---------------------------------

We could probably just wait for lucene 2.3 to be released
before releasing 1.3.  I wouldn't be averse to
pre-integrating the changes, though.

> Add support for Lucene's new setRAMBufferSizeMB()
method in IndexWriter
>
------------------------------------------------------------
-----------
>
>                 Key: SOLR-342
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-342
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Grant Ingersoll
>            Assignee: Grant Ingersoll
>            Priority: Minor
>
> LUCENE-843 adds support for new indexing capabilities
using the setRAMBufferSizeMB() method that should
significantly speed up indexing for many applications.  To
fix this, we will need trunk version of Lucene (or wait for
the next official release of Lucene)
> Side effect of this is that Lucene's new, faster
StandardTokenizer will also be incorporated.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-342?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12531591 ] 

Grant Ingersoll commented on SOLR-342:
--------------------------------------

yep, I agree with pre-integrating, just from watching the
Lucene discussions going on lately, I think it is worth
letting a few more things be worked out before using a
nightly build.

> Add support for Lucene's new setRAMBufferSizeMB()
method in IndexWriter
>
------------------------------------------------------------
-----------
>
>                 Key: SOLR-342
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-342
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Grant Ingersoll
>            Assignee: Grant Ingersoll
>            Priority: Minor
>
> LUCENE-843 adds support for new indexing capabilities
using the setRAMBufferSizeMB() method that should
significantly speed up indexing for many applications.  To
fix this, we will need trunk version of Lucene (or wait for
the next official release of Lucene)
> Side effect of this is that Lucene's new, faster
StandardTokenizer will also be incorporated.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.

    [ https://issues.apache.org/jira/browse/SO
LR-342?page=com.atlassian.jira.plugin.system.issuetabpanels:
comment-tabpanel#action_12537169 ] 

Yonik Seeley commented on SOLR-342:
-----------------------------------

Perhaps for now mergePolicy should be aligned with the
buffering limit (ram or docs)?
If/when we do add <mergePolicy> it seems like it
should be a class name.


> Add support for Lucene's new Indexing and merge
features (excluding Document/Field/Token reuse)
>
------------------------------------------------------------
-----------------------------------
>
>                 Key: SOLR-342
>                 URL: https:
//issues.apache.org/jira/browse/SOLR-342
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Grant Ingersoll
>            Assignee: Grant Ingersoll
>            Priority: Minor
>
> LUCENE-843 adds support for new indexing capabilities
using the setRAMBufferSizeMB() method that should
significantly speed up indexing for many applications.  To
fix this, we will need trunk version of Lucene (or wait for
the next official release of Lucene)
> Side effect of this is that Lucene's new, faster
StandardTokenizer will also be incorporated.  
> Also need to think about how we want to incorporate the
new merge scheduling functionality (new default in Lucene is
to do merges in a background thread)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
online.