Query optimisation - multiple filter caches?

I'm trying to understand how best to integrate directly with
Solr
(Java-to-Java in the same JVM) to make the most of its query
optimisation -
chiefly, its caching of queries which merely filter rather
than rank
results.
 
I notice that SolrIndexSearcher maintains a filter cache and
so does
LuceneQueryOptimiser.  Shouldn't they be contributing
to/using the same
cache, or are they used for different things?
 
Jon
 

On 8/15/07, Jonathan Woods <jonathan.woodsscintillance.com> wrote:
> I'm trying to understand how best to integrate directly
with Solr
> (Java-to-Java in the same JVM) to make the most of its
query optimisation -
> chiefly, its caching of queries which merely filter
rather than rank
> results.
>
> I notice that SolrIndexSearcher maintains a filter
cache and so does
> LuceneQueryOptimiser.  Shouldn't they be contributing
to/using the same
> cache, or are they used for different things?

LuceneQueryOptimiser is no longer used since one can
directly specify
filters via fq parameters.

-Yonik

I understand - thanks, Yonik.

I notice that LuceneQueryOptimizer is still used in
SolrIndexSearcher.search(Query, Filter, Sort) - is the idea
then that this
method is deprecated, or that the config parameter
query/boolTofilterOptimizer is no longer to be used?  As for
the other
search() methods, they just delegate directly to
org.apache.lucene.search.IndexSearcher, so no use of caches
there.

Jon

> -----Original Message-----
> From: Yonik Seeley [mailto:yonikapache.org] 
> Sent: 16 August 2007 01:40
> To: solr-userlucene.apache.org
> Subject: Re: Query optimisation - multiple filter
caches?
> 
> On 8/15/07, Jonathan Woods <jonathan.woodsscintillance.com> wrote:
> > I'm trying to understand how best to integrate
directly with Solr 
> > (Java-to-Java in the same JVM) to make the most of
its query 
> > optimisation - chiefly, its caching of queries
which merely filter 
> > rather than rank results.
> >
> > I notice that SolrIndexSearcher maintains a filter
cache 
> and so does 
> > LuceneQueryOptimiser.  Shouldn't they be
contributing to/using the 
> > same cache, or are they used for different
things?
> 
> LuceneQueryOptimiser is no longer used since one can
directly 
> specify filters via fq parameters.
> 
> -Yonik
> 
> 
> 

On 8/22/07, Jonathan Woods <jonathan.woodsscintillance.com> wrote:
> I notice that LuceneQueryOptimizer is still used in
> SolrIndexSearcher.search(Query, Filter, Sort) - is the
idea then that this
> method is deprecated,

Hmmm, so it is.  I hadn't noticed because that method is not
called
from any query handlers AFAIK (not since the first versions
of solr
before it went open source).
The method itself shouldn't be deprecated because it's part
of the
Lucene IndexSearcher interface.

> or that the config parameter
> query/boolTofilterOptimizer is no longer to be used?

That should probably be removed from the example schema...
thanks for
pointing that out.

-Yonik

Not high priority, but a few thoughts occur, then:

- perhaps it would be better to use
org.apache.lucene.search.Searcher by
composition and have SolrIndexSearcher merely implement
Searchable.

- or... perhaps search(...) should perform optimally
cache-aware searches -
else integrators might wrongly think they're getting the
full power of Solr.

Jon

> -----Original Message-----
> From: Yonik Seeley [mailto:yonikapache.org] 
> Sent: 22 August 2007 17:36
> 
> On 8/22/07, Jonathan Woods <jonathan.woodsscintillance.com> wrote:
> > I notice that LuceneQueryOptimizer is still used
in 
> > SolrIndexSearcher.search(Query, Filter, Sort) - is
the idea 
> then that 
> > this method is deprecated,
> 
> Hmmm, so it is.  I hadn't noticed because that method
is not 
> called from any query handlers AFAIK (not since the
first 
> versions of solr before it went open source).
> The method itself shouldn't be deprecated because it's
part 
> of the Lucene IndexSearcher interface.