cherrypy 3.0.1, lighttpd and fcgi through flup stop problem

Hi,

over the last few days I've been working on converting my
own WSGI
stuff to
just use cherrypy. After many frustrations due to incomplete
and
confusing
documentation of cherrypy I finally managed to pull
everything
together on how
to use fcgi through flup with RoutesDispatcher. (Once I wrap
my head
around
everything I hope to provide a few documentation patches.)

The only problem that I now have, after having patches my
3.0.1
install to not
die on the SIGTERM() argument issue (ticket #667) is that
whenever I
stop
lighttpd it leaves a stray process around connected to the
unix domain
socket.
I have to kill -9 it to even get rid of it.

It seems it has spawned 10 threads on this FreeBSD
6.2-STABLE machine
and
is waiting in a kse_release() loop (a wait state used inside
libkse
threads
meaning a thread is waiting for something to do). When I
stop lighttpd
it
sends a SIGTERM and subsequently 1 thread gets killed off,
but the
remaining 9
happily stay after lighttpd has stopped. It seems as if the
threads
are not
being stopped/the SIGTERM is not being propagated across all
threads.

This is FreeBSD 6.2-RELEASE, Python 2.5 with threads,
CherryPy 3.0.1 +
SIGTERM
fix, flup 0.5 (of a few days old), lighttpd 1.4.15.

Snippet from lighttpd.conf:

  $HTTP["url"] =~ "" {
    fastcgi.server = (
      "/" => (
        "kouyou-web.cgi" => (
          "bin-path" =>
"/home/asmodai/www/kouyou-web.fcgi",
          "socket"          =>
"/tmp/labs.sock",
          "check-local"     =>
"disable",
          "disable-time"    => 1,
          "min-procs"       => 1,
          "max-procs"       => 1,
        ),
      ),
    )
  } # end of $HTTP["url"] =~ "^/"

kouyou-web.fcgi:

#!/usr/local/bin/python

import cherrypy
import os

from flup.server.fcgi import WSGIServer

from kouyou.controllers import *

current_dir = os.path.dirname(os.path.abspath(__file__))

dispatcher = cherrypy.dispatch.RoutesDispatcher()
dispatcher.connect('home', '', controller=HomeController())

config = {'/': {'request.dispatch': dispatcher,
                'tools.staticdir.root':
os.path.join(current_dir,
'static')},
          '/css': {'tools.staticdir.on': True,
                   'tools.staticdir.dir': 'css'},
          '/img': {'tools.staticdir.on': True,
                   'tools.staticdir.dir': 'img'}
         }

app = cherrypy.tree.mount(None, config=config)
cherrypy.engine.start(blocking=False)
WSGIServer(app).run()

Any ideas, tips, hints, suggestions are more than welcome.
But could
it be
cherrypy does not do the right thing?

Thanks in advance,

--
Jeroen Ruigrok van der Werven
<asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル
ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
How are the mighty fallen...


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-usersgooglegroups.com
To unsubscribe from this group, send email to
cherrypy-users-unsubscribegooglegroups.com
For more options, visit this group at h
ttp://groups.google.com/group/cherrypy-users?hl=en
-~----------~----~----~----~------~----~------~--~---

asmodai wrote:
> over the last few days I've been working on converting
my
> own WSGI stuff to just use cherrypy. After many
frustrations
> due to incomplete and confusing documentation of
cherrypy
> I finally managed to pull everything together on how to
use
> fcgi through flup with RoutesDispatcher. (Once I wrap
my head
> around everything I hope to provide a few documentation
patches.)

Doc patches are always welcome.  If you
want a more informal place to
record your findings, http://tools.cherrypy.org is available for
immediate guest contributions anytime.

> The only problem that I now have, after having patches
my 3.0.1
> install to not die on the SIGTERM() argument issue
(ticket #667)
> is that whenever I stop lighttpd it leaves a stray
process around
> connected to the unix domain socket.
> I have to kill -9 it to even get rid of it.
> 
> It seems it has spawned 10 threads on this FreeBSD
6.2-STABLE
> machine and is waiting in a kse_release() loop (a wait
state
> used inside libkse threads meaning a thread is waiting
for
> something to do). When I stop lighttpd it sends a
SIGTERM and
> subsequently 1 thread gets killed off, but the
remaining 9
> happily stay after lighttpd has stopped. It seems as if
the
> threads are not being stopped/the SIGTERM is not being
> propagated across all threads.

The problem is most likely that your WSGI server is starting
its own
threads, but CP doesn't know how to tell it to shut down
those threads.
Ideally, you'd write a very small adapter so that, when the
SIGTERM is
handled, cherrypy.server (really a server manager) will send
a stop
message to the flup.server.fcgi.WSGIServer. See

http://www.cherrypy.org/wiki/ServerAPI#HTTPServerAPI for
a discussion of
what that adapter layer needs to look like. If that works
well, I think
we'd be happy to include such an adapter in the distro to
make it easier
for the next person.


Robert Brewer
System Architect
Amor Ministries
fumanchuamor.org

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-usersgooglegroups.com
To unsubscribe from this group, send email to
cherrypy-users-unsubscribegooglegroups.com
For more options, visit this group at h
ttp://groups.google.com/group/cherrypy-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Robert,

On May 30, 11:50 pm, "Robert Brewer"
<fuman...amor.org> wrote:
> Doc patches are always welcome.  If you
want a more informal place to
> record your findings,http://tools.cherrypy.org is available for
> immediate guest contributions anytime.

Ok cool. Will keep that in mind.

[snip]
> See
> 
http://www.cherrypy.org/wiki/ServerAPI#HTTPServerAPI for
a
> discussion of what that adapter layer needs to look
like. If that
> works well, I think we'd be happy to include such an
adapter in the
> distro to make it easier for the next person.

I will dig into that. I will need to fumble around on this
before I
get it right.
I will update when I got things sorted out.

Thanks thus far,

--
Jeroen Ruigrok van der Werven
<asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル
ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
What is to be, will be. And what isn't to be sometimes
happens...


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-usersgooglegroups.com
To unsubscribe from this group, send email to
cherrypy-users-unsubscribegooglegroups.com
For more options, visit this group at h
ttp://groups.google.com/group/cherrypy-users?hl=en
-~----------~----~----~----~------~----~------~--~---

[Resend, it seems my message didn't get through last
night.]

On May 30, 11:50 pm, "Robert Brewer"
<fuman...amor.org> wrote:
> The problem is most likely that your WSGI server is
starting its own
> threads, but CP doesn't know how to tell it to shut
down those threads.
> Ideally, you'd write a very small adapter so that, when
the SIGTERM is
> handled, cherrypy.server (really a server manager) will
send a stop
> message to the flup.server.fcgi.WSGIServer. See
> 
http://www.cherrypy.org/wiki/ServerAPI#HTTPServerAPI for
a discussion of
> what that adapter layer needs to look like.

Reading that URL and looking at my setup, I do not think it
matches.
Lighttpd is the one spawning the fcgi process which in turn
starts up
cherrypy. Cherrypy is not the process in command and as such
is not
the one in need to send the stop. Lighttpd sends flup a
SIGTERM, flup
then starts to tear down the process and clean up the
socket.

I suddenly realised the solution was fairly simple and I
tested this
with the stop/start cycle of lighttpd as well as a restart:

import cherrypy
import os

from flup.server.fcgi import WSGIServer

from kouyou.controllers import *

current_dir = os.path.dirname(os.path.abspath(__file__))

dispatcher = cherrypy.dispatch.RoutesDispatcher()
dispatcher.connect('home', '', controller=HomeController())

config = {'/': {'request.dispatch': dispatcher,
                'tools.staticdir.root':
os.path.join(current_dir,
'static')},
          '/css': {'tools.staticdir.on': True,
                   'tools.staticdir.dir': 'css'},
          '/img': {'tools.staticdir.on': True,
                   'tools.staticdir.dir': 'img'}
         }


app = cherrypy.tree.mount(None, config=config)
cherrypy.engine.start(blocking=False)
try:
    try:
        WSGIServer(app).run()
    except SystemExit:
        raise
    except Exception, e:
        print 'Content-Type: text/plainrnrn',
        print 'Oops...'
        print
        print 'Kouyou detected an internal error:'
        print
        print e
        print
        import traceback
        import StringIO
        tb = StringIO.StringIO()
        traceback.print_exc(file=tb)
        print tb.getvalue()
finally:
    cherrypy.engine.stop()

Comments/ideas/opinions?

--
Jeroen Ruigrok van der Werven
<asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル
ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
Ignorance is the opportunity to learn...


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-usersgooglegroups.com
To unsubscribe from this group, send email to
cherrypy-users-unsubscribegooglegroups.com
For more options, visit this group at h
ttp://groups.google.com/group/cherrypy-users?hl=en
-~----------~----~----~----~------~----~------~--~---