Performance improvement for some cases of sorted search

     [
HTTPS://ISSUES.APACHE.ORG/JIRA/BROWSE/LUCENE-769?PAGE=COM.AT
LASSIAN.JIRA.PLUGIN.SYSTEM.ISSUETABPANELS:ALL-TABPANEL ]

ARTEM VASILIEV UPDATED LUCENE-769:
----------------------------------

    DESCRIPTION: 
IT'S A SMALL ADDITION TO LUCENE THAT SIGNIFICANTLY LOWERS
MEMORY CONSUMPTION AND IMPROVES PERFORMANCE FOR SORTED
SEARCHES WITH FREQUENT INDEX UPDATES AND RELATIVELY BIG
INDEXES (>1MLN DOCS) SCENARIO. THIS SOLUTION SUPPORTS
ONLY SINGLE-FIELD SORTING CURRENTLY (WHICH SEEM TO BE QUITE
POPULAR USE CASE). MULTIPLE FIELDS SUPPORT CAN BE ADDED
WITHOUT MUCH TROUBLE.

THE SOLUTION IS THIS: DOCUMENTS FROM THE SORTING SET
(INSTEAD OF GIVEN FIELD'S VALUES FROM THE WHOLE INDEX -
CURRENT FIELDCACHE APPROACH) ARE CACHED IN A WEAKHASHMAP SO
THE CACHED ITEMS ARE CANDIDATES FOR GC.  THEIR FIELDS VALUES
ARE THEN FETCHED FROM THE CACHE AND COMPARED WHILE SORTING.

  WAS:
THE ATTACHMENT „–5 FROM 31 JAN IS THE LAST AND THE WHOLE
PATCH FOR THIS ISSUE. GUYS WITH PERMISSIONS, PLEASE REMOVE
ALL THE OTHERS.

IT'S A SMALL ADDITION TO LUCENE THAT SIGNIFICANTLY LOWERS
MEMORY CONSUMPTION AND IMPROVES PERFORMANCE FOR SORTED
SEARCHES WITH FREQUENT INDEX UPDATES AND RELATIVELY BIG
INDEXES (>1MLN DOCS) SCENARIO. THIS SOLUTION SUPPORTS
ONLY SINGLE-FIELD SORTING CURRENTLY (WHICH SEEM TO BE QUITE
POPULAR USE CASE). MULTIPLE FIELDS SUPPORT CAN BE ADDED
WITHOUT MUCH TROUBLE.

THE SOLUTION IS THIS: DOCUMENTS FROM THE SORTING SET
(INSTEAD OF GIVEN FIELD'S VALUES FROM THE WHOLE INDEX -
CURRENT FIELDCACHE APPROACH) ARE CACHED IN A WEAKHASHMAP SO
THE CACHED ITEMS ARE CANDIDATES FOR GC.  THEIR FIELDS VALUES
ARE THEN FETCHED FROM THE CACHE AND COMPARED WHILE SORTING.


> [PATCH] PERFORMANCE IMPROVEMENT FOR SOME CASES OF
SORTED SEARCH
>
------------------------------------------------------------
---
>
>                 KEY: LUCENE-769
>                 URL:
HTTPS://ISSUES.APACHE.ORG/JIRA/BROWSE/LUCENE-769
>             PROJECT: LUCENE - JAVA
>          ISSUE TYPE: IMPROVEMENT
>    AFFECTS VERSIONS: 2.0.0
>            REPORTER: ARTEM VASILIEV
>         ATTACHMENTS: SELFCONTAINED.PATCH
>
>
> IT'S A SMALL ADDITION TO LUCENE THAT SIGNIFICANTLY
LOWERS MEMORY CONSUMPTION AND IMPROVES PERFORMANCE FOR
SORTED SEARCHES WITH FREQUENT INDEX UPDATES AND RELATIVELY
BIG INDEXES (>1MLN DOCS) SCENARIO. THIS SOLUTION SUPPORTS
ONLY SINGLE-FIELD SORTING CURRENTLY (WHICH SEEM TO BE QUITE
POPULAR USE CASE). MULTIPLE FIELDS SUPPORT CAN BE ADDED
WITHOUT MUCH TROUBLE.
> THE SOLUTION IS THIS: DOCUMENTS FROM THE SORTING SET
(INSTEAD OF GIVEN FIELD'S VALUES FROM THE WHOLE INDEX -
CURRENT FIELDCACHE APPROACH) ARE CACHED IN A WEAKHASHMAP SO
THE CACHED ITEMS ARE CANDIDATES FOR GC.  THEIR FIELDS VALUES
ARE THEN FETCHED FROM THE CACHE AND COMPARED WHILE SORTING.

-- 
THIS MESSAGE IS AUTOMATICALLY GENERATED BY JIRA.
-
YOU CAN REPLY TO THIS EMAIL TO ADD A COMMENT TO THE ISSUE
ONLINE.


------------------------------------------------------------
---------
TO UNSUBSCRIBE, E-MAIL: JAVA-DEV-UNSUBSCRIBELUCENE.APACHE.ORG
FOR ADDITIONAL COMMANDS, E-MAIL: JAVA-DEV-HELPLUCENE.APACHE.ORG