fcgi-0013: Fixes to multiple concurrent connection failures

Hi,

This patch fixes the multiple concurrency failure in fastcgi
(against
rev. 128). Alo's previous patch sent to the list also
included in this
patch (Yes, Alo, you were right. It is better to have
integer, not the
pointer). The cause of the failure was because the manager
relied on
fcgi->id sent by the handler:

in read_fcgi():
return cherokee_fcgi_manager_step (fcgim, fcgi);

In fact, all data that received by the manager from the
server have
nothing to do with the current fcgi handler. Error happened
under
certain circumstance when the manager received END_QUERY for
fcgi->id =
X, but the current fcgi->id in the manager was Y. Hence,
the manager
will set the fcgi->phase of Y to 'finished' while it
should not be (it
should be X that is set to finished, not Y). Then, the Y
manager found
out that it should be finished while the incoming_buffer is
still empty.

This patch fixes the problem by resolving the current
fcgi->id based on
the record found inside the data coming from the server
instead of
getting it from the handler. Therefore, the manager could
set the phase
and fill the incoming_buffer for the correct (fcgi->id)
handler.

After applying the patch, there is still a problem:
I could never get 'ab -c 1 -n 1000' completed successfully
because
exactly in 501th request, the php interpreter always crashed
(probably
out of our scope, ie. PHP/libfcgi bug)

Btw, some amusements:
Concurrency Level:      1000
Time taken for tests:   0.878809 seconds
Complete requests:      300
Failed requests:        0
Write errors:           0
Total transferred:      64078 bytes
HTML transferred:       1194 bytes
Requests per second:    341.37 [#/sec] (mean)
Time per request:       2929.363 [ms] (mean)
Time per request:       2.929 [ms] (mean, across all
concurrent
requests)
Transfer rate:          70.55 [Kbytes/sec] received

_______________________________________________
Cherokee mailing list
Cherokeelists.alobbs.com
http://www.alobbs.com/cgi-bin/mailman/listinfo/cherokee

Hi Mohammad!

 > This patch fixes the multiple concurrency failure in
fastcgi
 > (against rev. 128). Alo's previous patch sent to the
list also
 > included in this patch (Yes, Alo, you were right. It
is better to
 > have integer, not the pointer). The cause of the
failure was because
 > the manager relied on fcgi->id sent by the handler:
 >
 > in read_fcgi():
 > return cherokee_fcgi_manager_step (fcgim, fcgi);
 >
 > In fact, all data that received by the manager from
the server have
 > nothing to do with the current fcgi handler. Error
happened under
 > certain circumstance when the manager received
END_QUERY for fcgi->id =
 > X, but the current fcgi->id in the manager was Y.
Hence, the manager
 > will set the fcgi->phase of Y to 'finished' while
it should not be (it
 > should be X that is set to finished, not Y). Then, the
Y manager found
 > out that it should be finished while the
incoming_buffer is still empty.
 >
 > This patch fixes the problem by resolving the current
fcgi->id based on
 > the record found inside the data coming from the
server instead of
 > getting it from the handler. Therefore, the manager
could set the phase
 > and fill the incoming_buffer for the correct
(fcgi->id) handler.

   Good stuff, this is actually great job!! 

   The patch is applied:
      http://www.0x5
0.org/bugs/changeset/140

 > After applying the patch, there is still a problem: I
could never
 > get 'ab -c 1 -n 1000' completed successfully because
exactly in
 > 501th request, the php interpreter always crashed
(probably out of
 > our scope, ie. PHP/libfcgi bug)

   Yeah, I got the same result yesterday. I will try to
figure out what
   is happening with PHP interpreter.

   We are on the way, now we are a step nearer to get the
FastCGI
   working alright, and hence to the 0.5.0 release! 

-- 
Greetings, alo.
http://www.alobbs.com
_______________________________________________
Cherokee mailing list
Cherokeelists.alobbs.com
http://www.alobbs.com/cgi-bin/mailman/listinfo/cherokee

Hi Mohammad!

 >  > After applying the patch, there is still a
problem: I could never
 >  > get 'ab -c 1 -n 1000' completed successfully
because exactly in
 >  > 501th request, the php interpreter always
crashed (probably out of
 >  > our scope, ie. PHP/libfcgi bug)
 >
 >   Yeah, I got the same result yesterday. I will try to
figure out what
 >   is happening with PHP interpreter.

   I have committed some changes that actually implement a
little piece
   we were missing here:

      http://www.0x5
0.org/bugs/changeset/142

   Basically it allows to define the environment variables
of the
   FastCGI we are launching. So with a configuration like
this (notice
   the PHP_FCGI_MAX_REQUESTS variable):

======
Extension php {
   Handler fastcgi {
     Server localhost:8002 {
       Env PHP_FCGI_MAX_REQUESTS "1010"
       Interpreter "/usr/bin/php4-cgi -b 8002"
     }
   }
}
======

   I have got this ab output:

======
Concurrency Level:      1
Time taken for tests:   1.452398 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      186000 bytes
HTML transferred:       24000 bytes
Requests per second:    688.52 [#/sec] (mean)
======

   now, the target is to make Cherokee relaunch and connect
gracefully
   with the FastCGI server when it exits (it wasn't a
segment fault or
   anything, it is the default PHP behaviour).

   Ah, and I also added support to add custom environment
variables
   inside the FastCGI request in the previous commit. 

-- 
Greetings, alo.
http://www.alobbs.com
_______________________________________________
Cherokee mailing list
Cherokeelists.alobbs.com
http://www.alobbs.com/cgi-bin/mailman/listinfo/cherokee

Hi Mohammad! (again)

 >   now, the target is to make Cherokee relaunch and
connect gracefully
 >   with the FastCGI server when it exits (it wasn't a
segment fault or
 >   anything, it is the default PHP behaviour).
 >
 >   Ah, and I also added support to add custom
environment variables
 >   inside the FastCGI request in the previous commit.


   What a productive evening I'm having.. 

     http://www.0x5
0.org/bugs/changeset/143

   Currently, with the following configuration (forcing down
the
   maximum PHP request to only 100)..

====
Extension php {
   Handler fastcgi {
     Server localhost:8002 {
          Env PHP_FCGI_MAX_REQUESTS "100"
          Interpreter "/usr/bin/php4-cgi -b 8002"
     }
   }
}
====

   the server is now able to gratefully complete 5,000
requests:

====
Concurrency Level:      1
Time taken for tests:   52.816527 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      930000 bytes
HTML transferred:       120000 bytes
Requests per second:    94.67 [#/sec] (mean)
Time per request:       10.563 [ms] (mean)
Time per request:       10.563 [ms] (mean, across all
concurrent requests)
Transfer rate:          17.19 [Kbytes/sec] received
====

   .. next stop, the concurrency issue! 

-- 
Greetings, alo.
http://www.alobbs.com
_______________________________________________
Cherokee mailing list
Cherokeelists.alobbs.com
http://www.alobbs.com/cgi-bin/mailman/listinfo/cherokee