Limiting connections per IP

Hi,

I am currently running Apache (prefork) 2.0.55 under Ubuntu
6.06 LTS. Apache
is running on a VPS with a limited amount of RAM so I have
been
investigating ways to limit the number of processes being
opened per
visitor.

My MaxClients setting is set to 38 which is just about all
the RAM I have
for. However 99% of the time only 8-10 processes are in use.
Occasionally
someone will run a site ripper which uses 30+ threads and my
MaxClients
setting is then reached. New visitors will then face a
delay.

To try and solve this I have tried 2 Apache modules,
mod_cband and
mod_limitipconn - both have the ability to limit the number
of connections
allowed per IP. However, both seem to need to launch an
Apache process
before it can reject the connection with a 503, by which
time the MaxClients
limit has already been reached.

Additionally I have investigated limiting the number of
simultaneous
connections per IP using iptables. However, due to a bug in
Ubuntu's
iptables, this option is not available to me. I cannot patch
the kernel
either, as I am using a VPS.

So my question in simple terms is this: is there a way of
limiting
simultaneous connections per IP without having to launch
additional
processes to handle the rejections?

Any help appreciated, Oliver.


------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org

On Thu, Sep 27, 2007 at 01:57:35PM +0100, Oliver wrote:
> So my question in simple terms is this: is there a way
of limiting
> simultaneous connections per IP without having to
launch additional
> processes to handle the rejections?

You may want to have a look into mod_qos:
http://sourc
eforge.net/projects/mod-qos/

It gives you just that.

Otherwise you can also try mod_security2 and play around
with
the guardian_log, which is meant to help you deal with DoS
attacks and your problem seems similar to a small DoS. 

regs,

Christian Folini


------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org

On 27 September 2007 at 14:06, Christian Folini wrote:
> On Thu, Sep 27, 2007 at 01:57:35PM +0100, Oliver
wrote:
> > So my question in simple terms is this: is there a
way of limiting
> > simultaneous connections per IP without having to
launch additional
> > processes to handle the rejections?
> 
> You may want to have a look into mod_qos:
> http://sourc
eforge.net/projects/mod-qos/
> It gives you just that.

Thanks for the pointer towards mod-qos. It did seem to be
very suitable, but I've done some testing with ab and it
seems to trigger the MaxClients setting just as
mod_limitipconn and mod_cband do, even when I set
QS_SrvMaxConnPerIP to 6. I have checked my settings are
configured correctly using mod_info. The module was
rejecting connections over 6, but not before a new process
was launched to handle the http rejection.
 
> Otherwise you can also try mod_security2 and play
around with
> the guardian_log, which is meant to help you deal with
DoS
> attacks and your problem seems similar to a small DoS.


I read the manual and it seems more aimed towards x requests
in y seconds, rather than x simultaneous connections. I was
thinking about xinetd to limit simultaneous connections, but
it's not supported anymore for apache, and would probably
hit performance badly anyway.

I'm starting to wonder whether limiting the number of
simultaneous connections is technically possible using
apache modules without launching a process for each
connection before rejecting it?

Oliver.


------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org

On 9/27/07, Oliver <oliver341gmail.com> wrote:
> On 27 September 2007 at 14:06, Christian Folini wrote:
> > On Thu, Sep 27, 2007 at 01:57:35PM +0100, Oliver
wrote:
> > > So my question in simple terms is this: is
there a way of limiting
> > > simultaneous connections per IP without
having to launch additional
> > > processes to handle the rejections?
> >
> > You may want to have a look into mod_qos:
> > http://sourc
eforge.net/projects/mod-qos/
> > It gives you just that.
>
> Thanks for the pointer towards mod-qos. It did seem to
be very suitable, but I've done some testing with ab and it
seems to trigger the MaxClients setting just as
mod_limitipconn and mod_cband do, even when I set
QS_SrvMaxConnPerIP to 6. I have checked my settings are
configured correctly using mod_info. The module was
rejecting connections over 6, but not before a new process
was launched to handle the http rejection.
>

Apache is a single-thread/process-per-connection server. In
order for
any apache module to handle a connection, it must be
accepted by a
thread/process and will thus count towards MaxClients until
it gets
dropped/responded to. There is no way to avoid that, short
of major
architectural changes (which are slowly appearing via the
event mpm).

If you need to make sure that requests don't make it through
to apache
at all (and therefore don't count towards MaxClients), you
really need
to use your OS firewall. This is by far the most efficient
way to
handle the problem as well. So you may want to go back to
looking at
ways to fix iptables.

Joshua.

------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org

On 27 September 2007 17:50, jslivegmail.com wrote:
> Apache is a single-thread/process-per-connection
server. In order for
> any apache module to handle a connection, it must be
accepted by a
> thread/process and will thus count towards MaxClients
until it gets
> dropped/responded to. There is no way to avoid that,
short of major
> architectural changes (which are slowly appearing via
the event mpm).
> 
> If you need to make sure that requests don't make it
through to apache
> at all (and therefore don't count towards MaxClients),
you really need
> to use your OS firewall. This is by far the most
efficient way to
> handle the problem as well. So you may want to go back
to looking at
> ways to fix iptables.

Ah thanks for the info, I suspected this might be the case,
as I've tried
several modules all with the same result. Even the event mpm
is not a
solution at this stage until PHP ceases to be un-recommended
with threaded
apache.

I'll have a word with my VPS provider but I suspect they'll
be unwilling to
make changes to the global system in order to patch the
iptables bug in
Ubuntu. The connlimit iptables match would be ideal.

Oliver.


------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org

I'm  using  apache 1.3.xx...I know it is way too old...
My situation is as follows :

Client   <----->   Apache      <------------> 
Apache
                         (Rev Proxy)                      
(Backend)

I have got access to CGI env vars on the Backend apache in
my application. I 
would like to get the client IP there.

I have tried this :

SetEnv REMOTE_IDENT %i

and tried to access REMOTE_IDENT  in my application...But i
get 
%i and not the value ...

Any help would be great .

Anand 


------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org

On 9/28/07, K Anand <kanandsail-steel.com> wrote:
> I'm  using  apache 1.3.xx...I know it is way too
old...
> My situation is as follows :
>
> Client   <----->   Apache     
<------------>  Apache
>                          (Rev Proxy)                   
   (Backend)
>
> I have got access to CGI env vars on the Backend apache
in my application. I
> would like to get the client IP there.
>
> I have tried this :
>
> SetEnv REMOTE_IDENT %i
>
> and tried to access REMOTE_IDENT  in my
application...But i get
> %i and not the value ...

SetEnv can't do that. SetEnvIf might be able to (with a
suitable
change in syntax -- see the docs), or you could use
mod_rewrite. But
why are you bothering? Your application should be able to
directly
access X-Forwarded-For.

(Also, careful using X-Forwarded-For because it could
contain more
than one IP address in the case of a chain of proxies.)

Joshua.

------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org

----- Original Message ----- 
From: "Joshua Slive" <joshuaslive.ca>


> On 9/28/07, K Anand <kanandsail-steel.com> wrote:
>> I'm  using  apache 1.3.xx...I know it is way too
old...
>> My situation is as follows :
>>
>> Client   <----->   Apache     
<------------>  Apache
>>                          (Rev Proxy)               
       (Backend)
>>
>> I have got access to CGI env vars on the Backend
apache in my 
>> application. I
>> would like to get the client IP there.
>>
>> I have tried this :
>>
>> SetEnv REMOTE_IDENT %i
>>
>> and tried to access REMOTE_IDENT  in my
application...But i get
>> %i and not the value ...
>
> SetEnv can't do that. SetEnvIf might be able to (with a
suitable
> change in syntax -- see the docs), or you could use
mod_rewrite. But
> why are you bothering? Your application should be able
to directly
> access X-Forwarded-For.
>
> (Also, careful using X-Forwarded-For because it could
contain more
> than one IP address in the case of a chain of
proxies.)
>
> Joshua.

I did try to use X-Forwarded-For directly. But I did not get
any value 
there.

Anand 


------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org

On 9/29/07, K Anand <kanandsail-steel.com> wrote:
> I finally managed to get the client IP. I had to use
> mod_proxy_add_forward.c  and
> mod_rpaf ....

Ahhh, that's because you're using the obsolete 1.3. It is
set in 2.x.

>
> X-Forwarded-For header was not getting set.  So I had
to use
> mod_proxy_add_forward.c .
> Once that header got set, my app was not able to use it
directly. As I had
> written earlier,
> my app can see only CGI vars...So then had to use
mod_rpaf  to move
> X-Forwarded-For into Remote IP.

All HTTP request headers are converted to CGI variables.
(X-Forwarded-For would become X_FORWARDED_FOR.)

------------------------------------------------------------
---------
The official User-To-User support forum of the Apache HTTP
Server Project.
See <URL:http://htt
pd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribehttpd.apache.org
   "   from the digest: users-digest-unsubscribehttpd.apache.org
For additional commands, e-mail: users-helphttpd.apache.org