Re: Tagging

One way around this is to get support for ParallelReader (I
believe ParallelWriter is still in JIRA, contributed by
Chuck) into Solr.
http://lucene.apache.org/java/
docs/api/org/apache/lucene/index/ParallelReader.html

Then you'd keep your big fields in one index, and the
frequently modified and shorter fields in another index. 
But I never understood how you'd keep doc IDs in sync
between the two, which is something that ParallelReader
requires.

Otis
 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  - 
Tag  -  Search  -  Share

----- Original Message ----
From: Yonik Seeley <yonikapache.org>
To: solr-userlucene.apache.org
Sent: Tuesday, February 13, 2007 6:17:00 PM
Subject: Re: Tagging

On 2/13/07, Binkley, Peter <Peter.Binkleyualberta.ca> wrote:
> I still wonder if there's a good way of storing the
tags outside the
> Lucene index and using them via facets whose bitsets
are manipulated
> directly rather than being populated from the index. In
my project,
> reindexing a documents whenever a user adds a tag is
very very bad,
> since we're indexing potentially hundreds of pages of
full text in the
> body field of the document. A solution that gets the
tag into the system
> immediately without forcing a reindexing of the
document is essential.

Interesting... what are you indexing that is that large, the
book contents?
You could build a custom request handler and store tag info
outside
the index.  You could also store it inside the index in
separate
documents as Erik does with Collex.

For a more general solution, I'm thinking a separate lucene
index
might be ideal.

 -Yonik

On 2/15/07, Otis Gospodnetic <otis_gospodneticyahoo.com> wrote:
> One way around this is to get support for
ParallelReader (I believe ParallelWriter is still in JIRA,
contributed by Chuck) into Solr.
> http://lucene.apache.org/java/
docs/api/org/apache/lucene/index/ParallelReader.html
>
> Then you'd keep your big fields in one index, and the
frequently modified and shorter fields in another index. 
But I never understood how you'd keep doc IDs in sync
between the two, which is something that ParallelReader
requires.

Aye, that's the rub.

ParallelReader keeps popping into my head too, but then I
think about
what it takes to keep those id's in sync, and it seems like
everything
needs to be re-indexed in the smaller index on a change to
that index.
 It doesn't seem easy or fast/scalable.  I'd love to know
what Chuck
is doing with this stuff.

-Yonik

On Feb 15, 2007, at 2:55 AM, Otis Gospodnetic wrote:
> Then you'd keep your big fields in one index, and the
frequently  
> modified and shorter fields in another index.  But I
never  
> understood how you'd keep doc IDs in sync between the
two, which is  
> something that ParallelReader requires.

I've never understood that either.  I'd love to hear more
about how  
folks use it.  Doug elaborated on it once, but *woosh* over
my head. 

	Erik