Re: Web Site Process Bus

Chris McDonough wrote:
> On Jun 26, 2007, at 2:39 PM, Robert Brewer wrote:
> 
> > Chris McDonough wrote:
> >> There are also non-webbish processes like
postgres, mysql, 
> >> etc. that
> >> need to be treated as "part of the
application".
> >>
> >> I handle this currently by running all of the
processes 
> >> related to a
> >> specific project under a process controller
(which happens to be
> >> implemented in Python, but that's besides the
point, see http://
> >> www.plope.com/software/supervisor2/).  The
process controller is
> >> responsible for execing the child processes
upon its own startup.
> >> It is also responsible for restarting children
if they die,
> >> capturing their output (if any), and allowing
sufficiently
> >> privileged users to start and stop each one
independently.
> >> The only promise a subprocess must make to be
managed is that
> >> it must be possible to start the process
"in the foreground"
> >> (not under its down daemon manager).
> >>
> >> If a "process bus" is implemented I
suspect it should be 
> >> implemented at this kind of level.
> >
> > Ah, but there's the rub: we all have different
ideas about how to
> > *implement* IPC and control.
> 
> I'm confused by this in your earlier message,
describing example  
> scenarios:
> 
> """
> If I'm primarily a Zope user instead, I might start my
website with
> zdaemon. This would work exactly like the above, but
the Bus object
> would be instantiated and started by the zdaemon
package. If I'm using
> Graham's new mod_wsgi with Apache, I might expect it to
create and
> control the Bus.
> """
> 
> I think I'm mostly confused by the name "process
bus" because it  
> seems like the primary use case for something like this
is where all  
> of the applications share the same process space

I don't see why it should be limited by that. The primary
use case is
anywhere site components and application components are
interacting,
that could benefit from a shared understanding (and control)
of the
state of the site. To me, that requires a common set of
messages, but
the transport mechanism for those messages should be
flexible so that
it's useful in both multithread and multiprocess
architectures.

> ...and are all written in Python.  Am I right?

That's the initial target market, yes. But I think we can
design the
messaging spec to be useful with non-Python application
components.

> If so, maybe a different name is in order?   
> "Application Bus"?  Or even "WSGI
Bus", if its presumed that all of  
> the applications will be WSGI applications?

Sure, "application bus" is fine, although it's
just the other side of
the same coin: "applications" on one side,
"site" on the other.

I wouldn't want this to be "WSGI Bus", simply
because there's no benefit
for that relationship; the two specs should be useful
independently of
each other. In particular, we should be able to design a
site-messaging
bus which works for WSGI 1.0, 1.1, 2.0, and whatever might
obsolete the
current WSGI in the future.

> I'm confused because zdaemon is a generic process
controller, it  
> knows nothing in particular about the application
running under it  
> except that it's a UNIX process. It could start
postgres instead of  
> Zope if you configured it to.

Sorry, I was really thinking of zopectl when I wrote that.
I'm not sure
how zdaemon itself would fit into this whole scheme--it's
the process
which zdaemon invokes that should be directly involved. If
that process
is a good provider of bus-aware services, you might not need
zdaemon
anymore.

> If zdaemon creates a Bus object,
> nothing will be able to send messages to the bus except
zdaemon  
> itself, and there can't be any useful listeners because
it doesn't  
> share the same process space as its child.

If you use the example Bus implementation I posted, then
yes. That's why
I'm pitching WSPBus as a spec, not an implementation.
Multiprocess
controllers could implement the bus using any of various
forms of IPC;
they just need to arrange for each application component to
get a Bus
object that, behind the scenes, is specific to the chosen
method of IPC.

So, yes, interprocess communication is more complicated
than
intraprocess. But that's true whether you standardize on a
bus spec or
not.


Robert Brewer
System Architect
Amor Ministries
fumanchuamor.org
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Jun 26, 2007, at 5:07 PM, Robert Brewer wrote:
>> I think I'm mostly confused by the name
"process bus" because it
>> seems like the primary use case for something like
this is where all
>> of the applications share the same process space
>
> I don't see why it should be limited by that. The
primary use case is
> anywhere site components and application components are
interacting,
> that could benefit from a shared understanding (and
control) of the
> state of the site. To me, that requires a common set of
messages, but
> the transport mechanism for those messages should be
flexible so that
> it's useful in both multithread and multiprocess
architectures.

Thank you.  I see.  This is a little too abstract for me to
get my  
brain around, but I'll continue listening and maybe I'll get
 
religion. 

- C



_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com