Search performance question

Hi,

   I have an index which contains more than 20K documents.
Each document has
the following structure :

field : ID (Index and store)                         
typical value - "1000"
field : parameterName(index and store)          typical
value -
"/mcp/data/parent1/parent2/child1/child2/status"
field : parameterValue(index and not store)    typical value
- "draft"

When I search for a term which results in "all"
the documents getting
returned, the search time is more than 1 sec. I have still
not done
hits.doc(), which I understand, would be even worse.

My problem is, I am expecting the search itself to happen in
the order of a
few milliseconds irrespective of the number of documents it
matched. Am I
expecting too much ?
-- 
View this message in context: http://www.nabble.com/Search-performan
ce-question-tf4391551.html#a12520740
Sent from the Lucene - Java Users mailing list archive at
Nabble.com.


------------------------------------------------------------
---------
To unsubscribe, e-mail: java-user-unsubscribelucene.apache.org
For additional commands, e-mail: java-user-helplucene.apache.org

Your not expecting too much. On cheap hardware I watch
searches on over 
5 mil + docs that match every doc come back in under a
second. Able to 
post your search code?

makkhar wrote:
> Hi,
>
>    I have an index which contains more than 20K
documents. Each document has
> the following structure :
>
> field : ID (Index and store)                         
typical value - "1000"
> field : parameterName(index and store)          typical
value -
>
"/mcp/data/parent1/parent2/child1/child2/status"
> field : parameterValue(index and not store)    typical
value - "draft"
>
> When I search for a term which results in
"all" the documents getting
> returned, the search time is more than 1 sec. I have
still not done
> hits.doc(), which I understand, would be even worse.
>
> My problem is, I am expecting the search itself to
happen in the order of a
> few milliseconds irrespective of the number of
documents it matched. Am I
> expecting too much ?
>   

------------------------------------------------------------
---------
To unsubscribe, e-mail: java-user-unsubscribelucene.apache.org
For additional commands, e-mail: java-user-helplucene.apache.org

Have a look at http://wiki.apache.org/lucene-java/BasicsOfPerformance


Are you opening the IndexSearcher every time you search,
even when no  
documents have changed?

-Grant

On Sep 6, 2007, at 7:41 AM, makkhar wrote:

>
> Hi,
>
>    I have an index which contains more than 20K
documents. Each  
> document has
> the following structure :
>
> field : ID (Index and store)                         
typical value  
> - "1000"
> field : parameterName(index and store)          typical
value -
>
"/mcp/data/parent1/parent2/child1/child2/status"
> field : parameterValue(index and not store)    typical
value - "draft"
>
> When I search for a term which results in
"all" the documents getting
> returned, the search time is more than 1 sec. I have
still not done
> hits.doc(), which I understand, would be even worse.
>
> My problem is, I am expecting the search itself to
happen in the  
> order of a
> few milliseconds irrespective of the number of
documents it  
> matched. Am I
> expecting too much ?
> -- 
> View this message in context: http://www.nabble.com/S
earch- 
> performance-question-tf4391551.html#a12520740
> Sent from the Lucene - Java Users mailing list archive
at Nabble.com.
>
>
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: java-user-unsubscribelucene.apache.org
> For additional commands, e-mail: java-user-helplucene.apache.org
>

--------------------------
Grant Ingersoll
http://lucene.granti
ngersoll.com

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance

http://w
iki.apache.org/lucene-java/LuceneFAQ



------------------------------------------------------------
---------
To unsubscribe, e-mail: java-user-unsubscribelucene.apache.org
For additional commands, e-mail: java-user-helplucene.apache.org

On 6-Sep-07, at 4:41 AM, makkhar wrote:

>
> Hi,
>
>    I have an index which contains more than 20K
documents. Each  
> document has
> the following structure :
>
> field : ID (Index and store)                         
typical value  
> - "1000"
> field : parameterName(index and store)          typical
value -
>
"/mcp/data/parent1/parent2/child1/child2/status"
> field : parameterValue(index and not store)    typical
value - "draft"
>
> When I search for a term which results in
"all" the documents getting
> returned, the search time is more than 1 sec. I have
still not done
> hits.doc(), which I understand, would be even worse.
>
> My problem is, I am expecting the search itself to
happen in the  
> order of a
> few milliseconds irrespective of the number of
documents it  
> matched. Am I
> expecting too much ?

20K docs is not very many.  I would expect a simply
TermQuery to be  
on the order of milliseconds, _after_ the OS has cached the
index in  
memory.  Does the time improve after some warmup?

-MIke


------------------------------------------------------------
---------
To unsubscribe, e-mail: java-user-unsubscribelucene.apache.org
For additional commands, e-mail: java-user-helplucene.apache.org