Author: Zyuan
Email:
Message:
> Zyuan, many thanks for your reports!
>
> I think I managed to find the reason for this crash,
> which should hopefully fix problems reported in all
> these bugs:
>
> Bug#1366 - Segmentation fault / match.c
> Bug#1367 - Segmentation fault / robots.c
> Bug#1430 - Segmentation fault / robots.txt
> Bug#1443 - Segmentation fault
> Bug#1471 - Segmentation fault
> Bug#1633 - indexer crashed when multiple threads
> Bug#1893 - Indexer Segfault Using Threads Every Time!
>
> I committed a fix into both 3.2.x and 3.3.x trees.
>
> Please try the latest snapshots:
>
> http://www.mnogosearch.org/Download/snap
shot/mnogosearch32-latest.tar.gz
> http://www.mnogosearch.org/Download/snap
shot/mnogosearch33-latest.tar.gz
>
> (they will be updated within 30 hours).
>
> Please give feedback!
>
> Thanks!
>
Dear Alexander,
You are the MAN!!!! You fixed it. That's the way it should
run!!! Absolutely perfect!
But first let me tell you what I did in an effort to help
you find the problem (before your post).
The first thing I did was re-install Linux on the test
server (the one mentioned originally). This time I elected
to use CentOS 5 as this is mainstream Red Hat 5 Enterprise
release. I wanted to make absolutely sure the bug wasn't
with Fedora since Fedora 7 is "bleading edge". So
I wiped everything clean, re-installed the OS and all other
necessary files, programs and libs and started out fresh.
Here's a snapshot:
Kernel-PAE-2.6.18-8.1.8.el5 (SMP of course)
glibc-2.5-12
gcc-4.1.1-52.el5.2
gcc-c++-4.1.1-52.el5.2
make-3.81-1.1
mnogosearch-3.3.5 (today's snapshot)
mecab-0.96
mysql-5.0.45
I downloaded the most recent snapshot of mnoGoSearch and
compiled with the exact same options as I previously did.
All .conf files are also identical and it worked perfectly.
This time it indexed 1,000 URLs in less than 1.5 minutes and
NO segfault:
sbin/indexer -N 100 -a -m
Now that's how threads should work. This time the memory
consumed was more of what I expected averaging roughly
310MB. CPU load showed roughly 16% at peak. And I must say
that this is with a fresh DNS restart and NO cache! The
second time I ran things (utilizing DNS cacheing) it fetched
the 1,000 URLs in only 36 seconds. Now we're talking! I have
a lot of CPU power and a lot of RAM so I'll play with the
"-N nnn) to see what I can get out of it without
bringing the server to a crawl. I suspect I can get
somewhere around 140+ threads without any problem! Once I
see things are good and stable I'll probable compile an
optimized version without debug or syslog.
I am very happy I was able to help track down the bug and
I'm more than willing to assist in the future using any of
my test servers. If I were to have the opportunity to ask
for a "wish", the wish I would have would be a
threaded search daemon (without the bugs of aspseek of
course). If you can implement threads in search (ie,
searchd) as well as you did with the indexer, it would be an
awesome system I feel and I will be more than happy to test
on this end.
Now let me take a moment to personally thank you for your
hard efforts in solving this problem. To show that I'm not
your average "give me your software" person, I
wish to personally send you a donation/gift of US$100. I
also want to ask other users of mnoGoSearch to also donate
to this project. Anyone who benefits from the efforts of
others should in some way show their appreciation. If you
use mnoGoSearch to help you generate revenue (site search,
web search or ?), please donate something. It's us users
that can keep mnoGoSearch alive and a project that will get
better.
so Alexander, how would I go about sending you a donation?
Just let me know and I'll get to you.
Thanks again,
Zyuan
Reply: <http://www.mnogosearch.org/board/message.php?id=19360&g
t;
------------------------------------------------------------
---------
To unsubscribe, e-mail: general-unsubscribe mnogosearch.org
For additional commands, e-mail: general-helpmnogosearch.org
|