Re: Moving to 1.0.x

On Tue, Oct 09, 2007 at 10:55:16PM -0400, Mike Boone wrote:
> Rerunning my indexer still gives the very large memory
increases of
> 25MB/1000 documents. I'm not sure what's causing it
now; I will try to
> isolate it tomorrow.

This is the code I was testing with, which reported growing
memory usage
before and now reports constant usage (for both PHP4 and
PHP5):

    for ($i = 0; $i < 1000000; ++$i) {
	print memory_get_usage()."n";
	$doc = new XapianDocument();
    }

It sounds like you are really using the new version -
perhaps see if
the above works, and if it does, try to find a similar
example which
doesn't.

Cheers,
    Olly

_______________________________________________
Xapian-discuss mailing list
Xapian-discusslists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-discuss

On 10/10/07, Olly Betts <ollysurvex.com> wrote:
> This is the code I was testing with, which reported
growing memory usage
> before and now reports constant usage (for both PHP4
and PHP5):
>
>     for ($i = 0; $i < 1000000; ++$i) {
>         print memory_get_usage()."n";
>         $doc = new XapianDocument();
>     }

OK, your code ran fine and the memory was constant. I also
threw in a
add_document call and it was still constant. I next tried
the stemmer
and that is at least part of the problem:

$stemmer = new XapianStem('english');

for ($i = 0; $i < 1000000; ++$i) {
  print memory_get_usage()."n";
  $doc = new XapianDocument();
  $stemmed_word = $stemmer->apply('testing');
}

With that code the memory usage grew from 456KB to 15.5MB.
That might
be all of my problem as I stem a lot of words per document.

_______________________________________________
Xapian-discuss mailing list
Xapian-discusslists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-discuss