daemon tools

For some time, Zope has used a daemon-management tool we
wrote called  
zdaemon:

   http://www.python.
org/pypi/zdaemon

Until late last year, I found this tool a bit difficult to
use  
because it was essentially undocumented.  I was forced to
learn  
enough to mostly document it and have gained a new
appreciation of  
it. (I haven't documented its  interactive shell mode, which
I don't  
use. Maybe someone will document it or maybe I'll just rip
it out.)   
I considered making some enhancements to it and decided to
ask if  
some folks knew about alternative tools we might use
instead.  See  
the discussion at:

    http://mail.zope.org/pipermail/zope3-dev/2006-
December/021353.html

Ironically, this sort of tool isn't Python specific at all,
and the  
discussion highlighted some non-Python tools, notably
daeomontools  
and runit, neither of which seemed as appealing as zdaemon
for  
various reasons.  This discussion also noted a Python-based
tool  
named suoervisor2:

   http://www
.plope.com/software/supervisor2/

Which seems to be derived from zdaemon and has some
interesting  
features.  I think that both zdaemon and supervisor3 do a
better job  
of process management than daemontools or runit.

At the recent open-space discussion, another Python-based
tool was  
mentioned whos name I don't remember.

I ended up deciding to use zdaemon for our projects because
it met  
our needs very well.  I added a couple of enhancements:

- The ability to set environment variables.  This is really
important  
to us as it allows us to set LD_LIBRARY_PATH. This wants to
be done  
in a supervisor process. A Python program can't set
LD_LIBRARY_PATH  
for itself because it is too late for it to be used by the
library  
loaded.

- I finished the transcript log, making it rotatable. The
zdaemon  
transcript log consumes the standard error and output of the
program  
zdaemon manages, providing basic logging for applications
that have  
lacking or lame logging support.  (zdaemon has allowed us to
make the  
spread daemon far more manageable.)

Anyway, I share this for your consideration.  There are
probably  
better tools out there than zdaemon and supervisor2, but I'm
not  
aware of them.   I'm
curious what other people have found or use.

Jim

--
Jim Fulton			mailto:jimzope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org



_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

> Anyway, I share this for your consideration.  There are
probably  
> better tools out there than zdaemon and supervisor2,
but I'm not  
> aware of them.   I'm
curious what other people have found or use.

There's also monit:

   http://www.tildeslas
h.com/monit/



chad
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

Chad Whitacre wrote:
>> Anyway, I share this for your consideration.  There
are probably  
>> better tools out there than zdaemon and
supervisor2, but I'm not  
>> aware of them.   I'm
curious what other people have found or use.
> 
> There's also monit:
> 
>    http://www.tildeslas
h.com/monit/

I think monit overlaps some with supervisor2's featureset,
but not as 
much with zdaemon.  Having monit poll your process to check
it's alive 
isn't as solid a solution as having a real parent process to
do that. 
Monit would still be useful with zdaemon, because it can
poll things 
like HTTP responses, memory usage, etc.


-- 
Ian Bicking | ianbcolorstudy.com | http://blog.ianbicking.org

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Mar 3, 2007, at 6:19 PM, Robert Brewer wrote:

> Jim Fulton wrote:
> > For some time, Zope has used a daemon-management
tool
> > we wrote called zdaemon:
> >
> >    http://www.python.
org/pypi/zdaemon
> >
> > Ironically, this sort of tool isn't Python
specific at all,
> > and the discussion highlighted some non-Python
tools, notably
> > daemontools and runit, neither of which seemed as
appealing
> > as zdaemon for various reasons.
>
> The user interface isn't Python-specific, but the
interaction with  
> WSGI servers, middleware, applications, and frameworks
should be.
I don't think we are talking about the same thing. See my
comment at  
the end of this note.
> Components at all levels of the WSGI stack need to
interact with  
> "site-wide" events and settings. What I'm
envisioning (and writing  
> for CP at the moment) is a framework-neutral,
one-per-site Engine  
> object that is basically a publish/subscribe messenger;
when you  
> import a Python web framework, it registers listeners
for process  
> start, stop, and graceful restart. These would be
things that need  
> to happen regardless of the OS process invoker: whether
a common  
> 'webctl' script (that we author), or a
framework-specific function  
> (like cherrypy.quickstart), or Apache (via
mod_python).
I encourage you to look at the zope event system which
already  
supports this use case:

   http://www.pyth
on.org/pypi/zope.event
   ht
tp://www.python.org/pypi/zope.component#handlers

> The pub/sub model also supports plugins with their own
channel(s).  
> For example, frameworks would blindly call
engine.publish 
> ('autoreload.add', filename) as desired. If the invoker
(webctl,  
> quickstart, or Apache) plugs in an autoreloader, great;
it  
> subscribes to that channel, receives each message, and
adds each  
> filename to its list of files to monitor. If no
autoreloader has  
> been plugged in, the 'add' message is correctly
ignored. And when  
> the autoreloader detects a change, it would also
publish 'reload'  
> or 'reexec' messages, which would then be subscribed to
by a Reexec  
> plugin. Most of the plugins would be provided by the
invoker, but  
> frameworks would be free to use the Engine to register
their own  
> events and event listeners.
>
> This interface between a site-wide container and the
WSGI  
> components is far more important to me than the actual
details of  
> invocation (like forking, signal-handling, logging,
etc). The  
> latter can be written as Engine plugins, and can
compete in a  
> market created by a good "Web Site Engine
Interface" spec.
I think you're "sitewide container" is the main
program that loads  
the WSGI components.  This might be Apache, if mod_python is
used, or  
some Python script/program.  I was discussing a tool that
managed the  
main program in the later case. Something that started and
restarted  
it, provided status information, helped it to run as a
proper daemon  
and so on.

Jim

--
Jim Fulton			mailto:jimzope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org



_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Saturday 03 March 2007 11:08:24 Jim Fulton wrote:
>
> Anyway, I share this for your consideration.  There are
probably
> better tools out there than zdaemon and supervisor2,
but I'm not
> aware of them.   I'm
curious what other people have found or use.

ll.daemon (ht
tp://www.livinglogic.de/Python/daemon/index.html) seems
to be a 
straightforward and very simple library for core daemon
functionality.

Daemontools isn't very well respected by the SysV style
initscript crowd, and 
vice versa.  That's an external non python dependency, and
not commonly 
available.  Certainly not available on Windows.

I have written my own daemon base class (Pretty restrictive
license 
[reciprocal], but I'm sure I could get it loosened).  
http://hg.rpath.com/raa-1.1?f=9ac380d082f4;
file=raa/service/daemon.py I'm not 
married to it though, so would be happy to spin it out and
remove the conary 
requirements, or just junk it.


-- 
Joseph Tate
Software Engineer
rPath Inc.
http://www.rpath.com/r
builder/
(919) 851-3984 x2106
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

 > ll.daemon (ht
tp://www.livinglogic.de/Python/daemon/index.html)
 > seems to be a straightforward and very simple library
for core
 > daemon functionality.

I'm using this in Aspen, and I like it. Worth checking out.



chad
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Mar 5, 2007, at 12:25 PM, Joseph Tate wrote:
...
> ll.daemon (ht
tp://www.livinglogic.de/Python/daemon/index.html)  
> seems to be a
> straightforward and very simple library for core daemon
functionality.
...
> I have written my own daemon base class (Pretty
restrictive license
> [reciprocal], but I'm sure I could get it loosened).
> http://hg.rpath.com/raa-1.1?f=9ac380d082f4;file=raa/
service/ 
> daemon.py I'm not
> married to it though, so would be happy to spin it out
and remove  
> the conary
> requirements, or just junk it.

Are either of these useful on Windows?  IOW, do they map to
services  
on windows?

Jim

--
Jim Fulton			mailto:jimzope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org



_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com