PyLucene

Hi,

Has anyone managed to get PyLucene working with Django?

It all works perfectly in unittests etc., but I using the
development
server import PyLucene causes python to crash (I suspect
because the
thread doesn't subclass PyLucene.PythonThread).

On OS X 10.4.8 with Apache 2.0.59 compiled for prefork,
mod_python
3.2.10 I can import PyLucene, open a store etc. but it
variously hangs
up when creating QueryParsers, closing stores etc. and never
returns to
the view.

I'm not doing anything radically different to what's in the
search-api
branch, so I'd be interested to see a working example
actually used
from a view - like I said, unittests over my index are fine
it only
breaks where the search method is called from a view whilst
running in
a webserver.

As a point of interest, anyone think it would be better to
create a
similar architecture to NXLucene?

Any insights appreciated.

cheers,
Cam

For completeness, here is some code:

--- search.py ---

class Base(object):
    """
    Base search class
    """

    def __init__(self, directory, create=False):
        """
        Constructor
        """
        self.create = create
        self.directory = directory
        if not os.path.exists(self.directory) and create:
            os.makedirs(self.directory)

        object.__init__(self)

    def open_store(self):
        """
        Create a datastore
        """
        return FSDirectory.getDirectory(self.directory,
self.create)

    def close_store(self, store, *args):
        """
        Close a datastore
        """
        for arg in args:
            if arg:
                arg.close()
        store.close()

class ListingSearcher(Base):
    """
    Searcher for Listing index
    """

    def search(self, query_string):
        """
        Parse and execute query_string against the index
        """
        fields =  ['name', 'id']
        parser = MultiFieldQueryParser(fields,
StandardAnalyzer())
        query = parser.parse(query_string)
        store = self.open_store()
        searcher = IndexSearcher(store)

        hits = searcher.search(query)

        hitz = []

        for i, doc in hits:
            hitz.append({'name': doc['name'], 'score':
hits.score(i), 
                         'url': doc['url'], 'id': doc['id'] 
})

        return hitz

--- views.py ---

def search(request):
    """
    Text search view.
    """
    if request.method == 'POST':
        new_data = request.POST.copy()
        form = TextSearchForm(new_data, auto_id=True)
    else:
        form = TextSearchForm(auto_id=True)

    if form.is_valid():
        searcher = ListingSearcher('/tmp')
        hits = searcher.search(form.clean_data['q'])
        return
render_to_response('listings/search_results.html',
{'results_list': hits})

    return render_to_response('listings/search.html',
{'form': form})


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Thu, Dec 28, 2006 at 08:53:43PM -0800, Cam wrote:
> 
> Hi,
> 
> Has anyone managed to get PyLucene working with Django?
> 
> It all works perfectly in unittests etc., but I using
the development
> server import PyLucene causes python to crash (I
suspect because the
> thread doesn't subclass PyLucene.PythonThread).

This was recently brought up on the pylucene mailing list:
        http://lists.osafoundation.org/pi
permail/pylucene-dev/2006-December/001468.html

It's to do with the incompatible threading models. I'd be
tempted to set
up a seperate server for the pylucene stuff and get django
to interface
with that via http or such, if you make a nice protocol then
you could
replace the backend search technology without having to
change anything
in the django code.

Also, it might be wirth having a look at xapian for search
stuff, it's
not got the java ties and looks really rather interesting:
        http://www.xapian.org/

Cheers,
-- 
Brett Parker

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Thanks Brett,

I found the mailing list after I posted, which always seems
the way 

I've got it up and running using fcgi & lighttpd which
benchmarks about
10% quicker than my mod_python install for the rest of the
app, with
the bonus that text searching actually works.

I've used xapian for a few things in the past and highly
recommend it.

On of the key benefits of PyLucene in a multi-platform
development shop
is index compatibility with Lucene - that's why I'm using it
here.

Thanks again.

cheers,
Cam.


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

I've used Lupy - old DivMod project. A howto is here:
http://w
ww.rkblog.rk.edu.pl/w/p/django-lupy/


--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---