Re: Lucene RAM Directory doesn't work for Index Size > 8 GB

hi Murali, I found a casting issue that can cause
this problem - see patch in
http:
//issues.apache.org/jira/browse/LUCENE-957
is the problem solved with this patch?
Doron

muraalee <muraaleegmail.com> wrote on 09/07/2007
14:15:04:

>
> Hi,
>
> We are facing a strange problem with RAMDirectory for
indices
> greater than 8
> GB. We have indexed around 6.5 million lucene documents
and theindex size
> is around 8 GB. Below is the contents of Index
Directory.
>
> 2236964197  _1x.fdt
>    51811488  _1x.fdx
>            293  _1x.fnm
> 2234929832  _1x.frq
>  369156856  _1x.nrm
> 2746607717  _1x.prx
>      4979893  _1x.tii
>  389652682  _1x.tis
>             42  segments_3x
>             20  segments.gen
>
> We could open this using FSDirectory and it seems to be
working
> fine. But if
> we switch to RAMDirectory, we get the following
exception.
>
> - We are using Lucene 2.2.0 currently
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException:
-369
>         at
>
org.apache.lucene.store.RAMInputStream.readByte(RAMInputStre
am.java:64)
>         at org.apache.lucene.store.IndexInput.
> readVLong(IndexInput.java:77)
>         at
> org.apache.lucene.index.MultiLevelSkipListReader.
> loadSkipLevels(MultiLevelSkipListReader.java:194)
>         at
> org.apache.lucene.index.MultiLevelSkipListReader.
> skipTo(MultiLevelSkipListReader.java:97)
>         at
>
org.apache.lucene.index.SegmentTermDocs.skipTo(SegmentTermDo
cs.
> java:164)
>         at
> org.apache.lucene.search.PhrasePositions.
> skipTo(PhrasePositions.java:52)
>         at
>
org.apache.lucene.search.PhraseScorer.doNext(PhraseScorer.ja
va:88)
>         at org.apache.lucene.search.PhraseScorer.
> next(PhraseScorer.java:81)
>         at
> org.apache.lucene.search.DisjunctionSumScorer.
> initScorerDocQueue(DisjunctionSumScorer.java:105)
>         at
> org.apache.lucene.search.DisjunctionSumScorer.
> next(DisjunctionSumScorer.java:144)
>         at
>
org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2
.
> java:327)
>         at
>
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.

> java:146)
>         at
org.apache.lucene.search.Searcher.search(Searcher.
> java:118)
>         at
org.apache.lucene.search.Searcher.search(Searcher.java:97)
>         at
> org.apache.lucene.search.QueryWrapperFilter.
> bits(QueryWrapperFilter.java:50)
>
>
> Any help to address this problem is greatly
appreciated.
>
>
> thanks
> Murali V


------------------------------------------------------------
---------
To unsubscribe, e-mail: java-user-unsubscribelucene.apache.org
For additional commands, e-mail: java-user-helplucene.apache.org