Sort by date with Lucene 2.2.0 ...

Hi,

I'm am trying to sort a date field in my index but I'm
seeing strange results.  I have searched the Lucene user
mail archive for Datetools but still couldn't figure out the
problem.

The date field is indexed as follows (i.e. DateTools is
used, date field is stored and untokenized):

String dateString = DateTools.dateToString (myDateObject,
DateTools.Resolution.SECOND);
Field field = new Field (DATE_FIELD, dateString,
Field.Store.YES, Field.Index.UN_TOKENIZED);
field.setOmitNorms (true);
document.add (field);

=====

I executed the search with a reverse sort on the
DATE_FIELD:

QueryParser queryParser = new QueryParser
("field1", new StandardAnalyzer ());
Query query = queryParser.parse
("txt:"sum"");
IndexSearcher searcher = new IndexSearcher
(indexDirectory);
TopFieldDocs topFieldDocs = searcher.search (query, null,
10000, new Sort (DATE_FIELD, true));

for (int i = 0; i < topFieldDocs.totalHits; i++)
{
  Document document = searcher.doc
(topFieldDocs.scoreDocs[i].doc);
  String date = document.get (DATE_FIELD);

  System.out.println (date);
}

====

As you can see, the output below is not sorted correctly
(e.g. 20070902233024 comes before 20070902233048).  Does
anyone know what the problem is? Thank you.


20070905140251
20070902233024
20070902233048
20070902234113
20070901040321
20070901040751
20070901040922
20070901042706
20070901042603
20070901050936
20070901050935
20070901050935
20070901080218


____________________________________________________________
_____
Windows Live Hotmail and Microsoft Office Outlook – together
at last.  Get it now.
http://office.microsoft.com/en-u
s/outlook/HA102225181033.aspx?pid=CL100626971033

Maybe I'm missing something, but that looks like the
correct
order to me, they are both on September 02, 2007, 11:30 P.M,
and
]24 seconds is before 48 seconds.....

Or is it just late and I'm missing the obvious (a specialty
of mine)...

Erick

On 10/18/07, Dragon Fly <dragon-fly999hotmail.com> wrote:
>
>
> Hi,
>
> I'm am trying to sort a date field in my index but I'm
seeing strange
> results.  I have searched the Lucene user mail archive
for Datetools but
> still couldn't figure out the problem.
>
> The date field is indexed as follows (i.e. DateTools is
used, date field
> is stored and untokenized):
>
> String dateString = DateTools.dateToString
(myDateObject,
> DateTools.Resolution.SECOND);
> Field field = new Field (DATE_FIELD, dateString,
Field.Store.YES,
> Field.Index.UN_TOKENIZED);
> field.setOmitNorms (true);
> document.add (field);
>
> =====
>
> I executed the search with a reverse sort on the
DATE_FIELD:
>
> QueryParser queryParser = new QueryParser
("field1", new StandardAnalyzer
> ());
> Query query = queryParser.parse
("txt:"sum"");
> IndexSearcher searcher = new IndexSearcher
(indexDirectory);
> TopFieldDocs topFieldDocs = searcher.search (query,
null, 10000, new Sort
> (DATE_FIELD, true));
>
> for (int i = 0; i < topFieldDocs.totalHits; i++)
> {
>   Document document = searcher.doc
(topFieldDocs.scoreDocs[i].doc);
>   String date = document.get (DATE_FIELD);
>
>   System.out.println (date);
> }
>
> ====
>
> As you can see, the output below is not sorted
correctly (e.g.
> 20070902233024 comes before 20070902233048).  Does
anyone know what the
> problem is? Thank you.
>
>
> 20070905140251
> 20070902233024
> 20070902233048
> 20070902234113
> 20070901040321
> 20070901040751
> 20070901040922
> 20070901042706
> 20070901042603
> 20070901050936
> 20070901050935
> 20070901050935
> 20070901080218
>
>
>
____________________________________________________________
_____
> Windows Live Hotmail and Microsoft Office Outlook –
together at last. Get
> it now.
>
> http://office.microsoft.com/en-u
s/outlook/HA102225181033.aspx?pid=CL100626971033

Thanks for the reply.  I expected the results to be sorted
in reverse chron (i.e. more recent time stamps first)
because my Sort object is:

new Sort (DATE_FIELD, true);

Using my previous example, I would expect to see the
following (because 48 seconds is more recent than 24
seconds):

September 05, 2007, 14:02:51 (20070905140251)
September 02, 2007, 23:30:48 (20070902233048)
September 02, 2007, 23:30:24 (20070902233024)
...
...

It looks to me like the Date part is sorted correctly but
not the Time part.  Am I missing something?

> Date: Thu, 18 Oct 2007 20:46:36 -0400
> From: erickericksongmail.com
> To: java-userlucene.apache.org
> Subject: Re: Sort by date with Lucene 2.2.0 ...
> 
> Maybe I'm missing something, but that looks like the
correct
> order to me, they are both on September 02, 2007, 11:30
P.M, and
> ]24 seconds is before 48 seconds.....
> 
> Or is it just late and I'm missing the obvious (a
specialty of mine)...
> 
> Erick
> 
> On 10/18/07, Dragon Fly <dragon-fly999hotmail.com> wrote:
> >
> >
> > Hi,
> >
> > I'm am trying to sort a date field in my index but
I'm seeing strange
> > results.  I have searched the Lucene user mail
archive for Datetools but
> > still couldn't figure out the problem.
> >
> > The date field is indexed as follows (i.e.
DateTools is used, date field
> > is stored and untokenized):
> >
> > String dateString = DateTools.dateToString
(myDateObject,
> > DateTools.Resolution.SECOND);
> > Field field = new Field (DATE_FIELD, dateString,
Field.Store.YES,
> > Field.Index.UN_TOKENIZED);
> > field.setOmitNorms (true);
> > document.add (field);
> >
> > =====
> >
> > I executed the search with a reverse sort on the
DATE_FIELD:
> >
> > QueryParser queryParser = new QueryParser
("field1", new StandardAnalyzer
> > ());
> > Query query = queryParser.parse
("txt:"sum"");
> > IndexSearcher searcher = new IndexSearcher
(indexDirectory);
> > TopFieldDocs topFieldDocs = searcher.search
(query, null, 10000, new Sort
> > (DATE_FIELD, true));
> >
> > for (int i = 0; i < topFieldDocs.totalHits;
i++)
> > {
> >   Document document = searcher.doc
(topFieldDocs.scoreDocs[i].doc);
> >   String date = document.get (DATE_FIELD);
> >
> >   System.out.println (date);
> > }
> >
> > ====
> >
> > As you can see, the output below is not sorted
correctly (e.g.
> > 20070902233024 comes before 20070902233048).  Does
anyone know what the
> > problem is? Thank you.
> >
> >
> > 20070905140251
> > 20070902233024
> > 20070902233048
> > 20070902234113
> > 20070901040321
> > 20070901040751
> > 20070901040922
> > 20070901042706
> > 20070901042603
> > 20070901050936
> > 20070901050935
> > 20070901050935
> > 20070901080218
> >
> >
> >
____________________________________________________________
_____
> > Windows Live Hotmail and Microsoft Office Outlook
– together at last. Get
> > it now.
> >
> > http://office.microsoft.com/en-u
s/outlook/HA102225181033.aspx?pid=CL100626971033

____________________________________________________________
_____
Peek-a-boo FREE Tricks & Treats for You!
http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us

ON THURSDAY 18 OCTOBER 2007 21:35, DRAGON FLY WROTE:

> I'M AM TRYING TO SORT A DATE FIELD IN MY INDEX BUT I'M
SEEING STRANGE
> RESULTS.  I HAVE SEARCHED THE LUCENE USER MAIL ARCHIVE
FOR DATETOOLS BUT
> STILL COULDN'T FIGURE OUT THE PROBLEM.

IT SHOULDN'T MAKE A DIFFERENCE BUT DOES IT HELP IF YOU
SPECIFY THE FIELD 
TYPE USING A SORTFIELD OBJECT? IF NOT, COULD YOU MAKE THIS A
STAND-ALONE 
TEST CASE SO PEOPLE CAN EASIER DEBUG IT?

REGARDS
 DANIEL

-- 
HTTP://WWW.DANIELNABER.DE

------------------------------------------------------------
---------
TO UNSUBSCRIBE, E-MAIL: JAVA-USER-UNSUBSCRIBELUCENE.APACHE.ORG
FOR ADDITIONAL COMMANDS, E-MAIL: JAVA-USER-HELPLUCENE.APACHE.ORG

I tried specifying the field type using a SortField object
but I got the same result.  I'll be glad to write a
stand-alone test case.  Should I post the code to this
thread when I'm done or should I submit some sort of bug
report? Thanks.

> From: lucenelist2007danielnaber.de
> To: java-userlucene.apache.org
> Subject: Re: Sort by date with Lucene 2.2.0 ...
> Date: Fri, 19 Oct 2007 19:50:46 +0200
> 
> On Thursday 18 October 2007 21:35, Dragon Fly wrote:
> 
> > I'm am trying to sort a date field in my index but
I'm seeing strange
> > results.  I have searched the Lucene user mail
archive for Datetools but
> > still couldn't figure out the problem.
> 
> It shouldn't make a difference but does it help if you
specify the field 
> type using a SortField object? If not, could you make
this a stand-alone 
> test case so people can easier debug it?
> 
> Regards
>  Daniel
> 
> -- 
> http://www.danielnaber.de
> 
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: java-user-unsubscribelucene.apache.org
> For additional commands, e-mail: java-user-helplucene.apache.org
> 

____________________________________________________________
_____
Help yourself to FREE treats served up daily at the
Messenger Café. Stop by today.
http://www.cafemessenger.com/
info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline

ON TUESDAY 23 OCTOBER 2007 15:57, DRAGON FLY WROTE:

> I TRIED SPECIFYING THE FIELD TYPE USING A SORTFIELD
OBJECT BUT I GOT THE
> SAME RESULT.  I'LL BE GLAD TO WRITE A STAND-ALONE TEST
CASE.  SHOULD I
> POST THE CODE TO THIS THREAD WHEN I'M DONE OR SHOULD I
SUBMIT SOME SORT
> OF BUG REPORT? THANKS.

I'M NOT SURE IF ATTACHMENTS WORK ON THIS LIST, SO PLEASE
OPEN A BUG REPORT.

REGARDS
 DANIEL

-- 
HTTP://WWW.DANIELNABER.DE

------------------------------------------------------------
---------
TO UNSUBSCRIBE, E-MAIL: JAVA-USER-UNSUBSCRIBELUCENE.APACHE.ORG
FOR ADDITIONAL COMMANDS, E-MAIL: JAVA-USER-HELPLUCENE.APACHE.ORG