Custom analyzer and custom queryparser

On Fri, 3 Mar 2006, Trond Aksel Myklebust wrote:

> Hello,
> I am trying to do the same as the
AdvancedQueryParserTest but with a custom
> analyzer.
>
> The test:
> self.analyzer = WhitespaceAnalyzer()
> parser = self.analyzer.queryParser(CustomQueryParser(),
"field")
>
> How do I do this with a custom analyzer?

You found a hole here. In order to create a custom query
parser you need an 
Analyzer instance. The issue is that a custom analyzer,
written in python, as 
in the test_PositionIncrement.py unit test for example,
doesn't become a 
proper Analyzer instance until it is passed to PyLucene
where it gets wrapped 
by a subclass of Analyzer, called PythonAnalyzer, whose sole
job is to invoke 
the extension points you implemented in python. This is done
automatically by 
any API declared to take an analyzer instance. So, I added a
simple function 
to PyLucene called wrapAnalyzer() that does this for you.
Now you can write:

         class _analyzer(object):
             def tokenStream(self, fieldName, reader):
                 class _tokenStream(object):
                     def __init__(self):
                         self.TOKENS = ["1",
"2", "3", "4",
"5"]
                         self.INCREMENTS = [1, 2, 1, 0, 1]
                         self.i = 0
                     def next(self):
                         if self.i == len(self.TOKENS):
                             return None
                         t = Token(self.TOKENS[self.i],
self.i, self.i)
                        
t.setPositionIncrement(self.INCREMENTS[self.i])
                         self.i += 1
                         return t
                 return _tokenStream()

         class _queryParser(object):
             def getFuzzyQuery(self, super, field, termText,
minSimilarity):
                 raise AssertionError, "Fuzzy queries
not allowed"

         qp =
wrapAnalyzer(_analyzer()).queryParser(_queryParser(),
"field")


This is checked in on the trunk.

Andi..
_______________________________________________
pylucene-dev mailing list
pylucene-devosafoundation.org
http://lists.osafoundation.org/mailman/listinfo/pylu
cene-dev