1000 socket connections send GC in an infinite loop

"Chaitanya Gupta" <mailchaitanyagupta.com>
writes:
> On 9/13/06, Martin Rydstr|m <rydiscd.chalmers.se> wrote:
> > On Wed, Sep 13, 2006 at 09:43:50AM +0100, Gisle
Sælensminde wrote:
> > > That limit can be adjusted with the ulimit
command in (in bash, limit in
> > > tcsh). For
> > >
> > > % ulimit -n  # prints the maximal number of
open files
> > > 1024
> > >
> > > % ulimit -n 4096 #sets the maximal number of
files to 4096
> >
> > That changes the maximum number of open files; it
doesn't change how
> > well select handles a set of files, when there are
many, if I have
> > understood it all correctly.
> 
> So even increasing this limit may not solve it? Is
there any fix?

This is really beyond my competence level, but I don't
think there's
an easy fix; FD_SETSIZE is rather low-level, if it is indeed
what is
causing your performance problem.

The easiest way might be to change whatever you're using to
use poll
rather than select; this might be a rather intrusive change
to CMUCL,
or (pretty much) a reimplementation of the machinery.

But, as I said, this is beyond what I actually know much
about, and
into the realm of vaguely recalled comments and conjecture.

However, if it was just bumping into the limit, I believe
you would
get an error message that you can't open a new socket,
containing
something about "too many open files".

',mr

-- 
[Emacs] is written in Lisp, which is the only computer
language that is
beautiful.  -- Neal Stephenson, _In the Beginning was the
Command Line_

Martin Rydstr|m wrote:
> "Chaitanya Gupta" <mailchaitanyagupta.com> writes:
>   
>> So even increasing this limit may not solve it? Is
there any fix?
>>     
>
> This is really beyond my competence level, but I don't
think there's
> an easy fix; FD_SETSIZE is rather low-level, if it is
indeed what is
> causing your performance problem.
>
> The easiest way might be to change whatever you're
using to use poll
> rather than select; this might be a rather intrusive
change to CMUCL,
> or (pretty much) a reimplementation of the machinery.
>
> But, as I said, this is beyond what I actually know
much about, and
> into the realm of vaguely recalled comments and
conjecture.
>
>   
Well, its way beyond what I know about. Too little I know
about Unix land.
> However, if it was just bumping into the limit, I
believe you would
> get an error message that you can't open a new socket,
containing
> something about "too many open files".
>   
I didn't see any such message in the logs. As I said, the
GC just went 
into an infinite loop.
Anyway, since the change to http-get, its ok for now. The
traffic is 
quite low. We will check whether bumping the limit up works
or not.

Thanks,
Chaitanya