WSGI diagram

Hi,

There's someone working on a mod_wsgi module for the Nginx
HTTP
server/proxy and Nginx's author was asking if there is a
diagram
somewhere outlining the WSGI protocol (I think perhaps he'd
help if he
understood WSGI).  I google'd a bit but came up
empty-handed.

Anyone know of anything?

TIA,

Cliff

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On 21/08/07, Cliff Wells <cliffdevelix.com> wrote:
> Hi,
>
> There's someone working on a mod_wsgi module for the
Nginx HTTP
> server/proxy and Nginx's author was asking if there is
a diagram
> somewhere outlining the WSGI protocol (I think perhaps
he'd help if he
> understood WSGI).  I google'd a bit but came up
empty-handed.
>
> Anyone know of anything?

Start with the PEP:

  http://www.p
ython.org/dev/peps/pep-0333/

Further information may be found on:

  http://www.wsgi.org/wsgi


The official site for the Apache mod_wsgi even references
this
information, although the www.wsgi.org site seems to be
unavailable as
I write this.

Other than that, the only thing is the various little
articles and
blog entries that people have written. There is no
exhaustive or
complete description of WSGI with even the PEP leaving
various things
unanswered. Generally people ask questions on this list to
fill out
the details on what is meant to occur, so perusing the
archives would
also be useful.

Some will no doubt say I am biased and trying to protect my
turf, but
from what I understand of the aims of Nginx HTTP, in as much
as it is
trying to create the fastest web server around, embedding a
Python
interpreter within the main server processes would work
against that.
This is because supporting Python in an embedded manner may
impact
performance in serving static files etc.

Thus, it may make more sense to be suggesting people use the
existing
FASTCGI support in Nginx HTTP for hosting Python WSGI
applications.
That way you aren't necessarily burdening the main server
processes
with the extra overhead. The same argument applies to
lighttpd as
well. As to Apache, many have the opinion that it is getting
quite
bloated anyway, so adding more stuff into the main server
processes
doesn't seem to worry people as much. It does though need to
be used
appropriately though.

Also note that WSGI isn't a wire protocol like FASTCGI is.
It is more
an programmatic interface definition for adapting a Python
application
on top of a specific web server.  If you specifically want
to run WSGI
applications in a separate daemon process, then you would
need to come
up with your own wire protocol for communicating between the
main
server processes and the daemon process. The daemon mode of
the Apache
mod_wsgi does this, but since this is what FASTCGI defines
anyway,
that is why it may make more sense to use the existing
FASTCGI support
in Nginx HTTP and not bother with a separate module. In
Apache
mod_wsgi, a new FASTCGI like daemon mode was specifically
implemented
to try and come up with an improvement over the sometimes
hard to
configure and get working FASTCGI implementations for
Apache.

BTW, I hope you would call it something other than mod_wsgi
so as not
to cause confusion with the Apache implementation.

Graham
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

At 02:08 PM 8/20/2007 -0700, Cliff Wells wrote:
>Hi,
>
>There's someone working on a mod_wsgi module for the
Nginx HTTP
>server/proxy and Nginx's author was asking if there is a
diagram
>somewhere outlining the WSGI protocol (I think perhaps
he'd help if he
>understood WSGI).  I google'd a bit but came up
empty-handed.
>
>Anyone know of anything?

I don't know of anything.  But it might help to explain that
WSGI 
isn't really a protocol, unless you consider e.g. Java
servlets to be 
a "protocol".

To implement WSGI in a non-Python web server, you need to
invoke 
Python code via the Python C API.

Pardon me if any of this is already obvious, but from your 
description it sounds like one or more parties might not be
aware of 
these points.

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Mon, 2007-08-20 at 18:42 -0400, Phillip J. Eby wrote:

> Pardon me if any of this is already obvious, but from
your 
> description it sounds like one or more parties might
not be aware of 
> these points.

One of the parties (Igor, Nginx's author) is certainly not
aware of
these points (hence his seeking a schematic so he could get
a quick
overview).  The author of the upcoming module is aware but
didn't offer
much in the way of explanation when Igor queried him.

Regards,
Cliff


_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Tue, 2007-08-21 at 08:33 +1000, Graham Dumpleton wrote:
> On 21/08/07, Cliff Wells <cliffdevelix.com> wrote:
> > Hi,
> >
> > There's someone working on a mod_wsgi module for
the Nginx HTTP
> > server/proxy and Nginx's author was asking if
there is a diagram
> > somewhere outlining the WSGI protocol (I think
perhaps he'd help if he
> > understood WSGI).  I google'd a bit but came up
empty-handed.
> >

> Some will no doubt say I am biased and trying to
protect my turf, but
> from what I understand of the aims of Nginx HTTP, in as
much as it is
> trying to create the fastest web server around,
embedding a Python
> interpreter within the main server processes would work
against that.
> This is because supporting Python in an embedded manner
may impact
> performance in serving static files etc.

The author of the module says the recommended deployment
method will be
running a second Nginx server dedicated to the WSGI process
and proxy to
that with a main Nginx server, probably for this very
reason.

Personally I think he'd do better implementing a mod_ajp and
then
utilizing the ajp-wsgi [1] server since this would probably
be easier to
implement and I suspect ajp-wsgi has a smaller footprint
than Nginx
(even though Nginx is pretty small), but it's not my project
=)

> Thus, it may make more sense to be suggesting people
use the existing
> FASTCGI support in Nginx HTTP for hosting Python WSGI
applications.
> That way you aren't necessarily burdening the main
server processes
> with the extra overhead. 

Lots of people do this or proxy to a Python HTTP server.  I
think the
idea is to get rid of one layer of relatively inefficient
processing (a
HTTP server in Python) by doing it in C.  How much
performance impact
this will actually have in real world applications I don't
know
(although Apache's mod_wsgi should give a clue, if there
were any
benchmarks published).

> The same argument applies to lighttpd as
> well. As to Apache, many have the opinion that it is
getting quite
> bloated anyway, so adding more stuff into the main
server processes
> doesn't seem to worry people as much. It does though
need to be used
> appropriately though.

Since it's a module, I don't think it will worry people too
much 
Nginx has plenty of optional stuff already.  The people who
need or want
it will use it, everyone else will skip it. 

> Also note that WSGI isn't a wire protocol like FASTCGI
is. It is more
> an programmatic interface definition for adapting a
Python application
> on top of a specific web server.  If you specifically
want to run WSGI
> applications in a separate daemon process, then you
would need to come
> up with your own wire protocol for communicating
between the main
> server processes and the daemon process. The daemon
mode of the Apache
> mod_wsgi does this, but since this is what FASTCGI
defines anyway,
> that is why it may make more sense to use the existing
FASTCGI support
> in Nginx HTTP and not bother with a separate module. 

Except that this still requires a fcgi->wsgi gateway that
would probably
be just as complicated as direct WSGI support (or done
inefficiently in
Python, such as flup).

> In Apache
> mod_wsgi, a new FASTCGI like daemon mode was
specifically implemented
> to try and come up with an improvement over the
sometimes hard to
> configure and get working FASTCGI implementations for
Apache.

Personally, I think FastCGI is too difficult on every server
and that
may be another motivation for the author.

> BTW, I hope you would call it something other than
mod_wsgi so as not
> to cause confusion with the Apache implementation.

Again, I'm not the author, but I'll bring this up.  However,
I'm not
certain it would be any worse than the plethora of
mod_fcgi's or
mod_proxy's or mod_xxx's in existing webservers.

Regards,
Cliff

[1] http://www.saddi.com/software/news/archives/49-ajp-w
sgi.html



_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

Cliff Wells wrote:
> Hi,
> 
> There's someone working on a mod_wsgi module for the
Nginx HTTP
> server/proxy and Nginx's author was asking if there is
a diagram
> somewhere outlining the WSGI protocol (I think perhaps
he'd help if he
> understood WSGI).  I google'd a bit but came up
empty-handed.
> 
> Anyone know of anything?

I've seen a few I think, but I'm not sure -- probably
nothing that would 
help him that much.  There's some links on 
http://wsgi.org/wsgi/
Learn_WSGI -- but I think they are all more for 
applications, not for people implementing servers.

If Nginx wanted to support Python really well, I think doing
process 
management would be the most ideal way to do that.  That's
kind of what 
FastCGI does, but with lots of different options and some
general 
confusion.  Nginx could potentially implement FastCGI (or a
protocol 
like SCGI or any other over-the-wire protocol they thought
best) with 
intelligently chosen process management, potentially with a
client 
library (maybe a customized Flup) that invoked everything
properly.

It's really more of an integration issue than a protocol
issue.  But if 
Nginx makes the integration clear, easy, and robust then
they'd create 
something quite useful.  And probably applicable to other
languages like 
Ruby as well.


-- 
Ian Bicking : ianbcolorstudy.com : http://blog.ianbicking.org

             : Write code, do good : http://topp.openpla
ns.org/careers
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On 21/08/07, Cliff Wells <cliffdevelix.com> wrote:
> On Mon, 2007-08-20 at 18:42 -0400, Phillip J. Eby
wrote:
>
> > Pardon me if any of this is already obvious, but
from your
> > description it sounds like one or more parties
might not be aware of
> > these points.
>
> One of the parties (Igor, Nginx's author) is certainly
not aware of
> these points (hence his seeking a schematic so he could
get a quick
> overview).  The author of the upcoming module is aware
but didn't offer
> much in the way of explanation when Igor queried him.

If the person doing this is called Igor, then they aren't
the only
person trying to implement an mod_wsgi for Nginx HTTP. After
dealing
with this email, found a message in my spam box from someone
else
doing the same thing. This other persons name is Manlio
Perillo. I
have pointed them to this discussion on Google groups, so
maybe they
will explain what they are up to and all parties can get
together. I
have already indicated to Manlio that I may be interested in
providing
him with assistance in doing for Nginx HTTP what I have
already done
in Apache mod_wsgi.

Graham
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On 8/20/07, Graham Dumpleton <graham.dumpletongmail.com> wrote:
> On 21/08/07, Cliff Wells <cliffdevelix.com> wrote:
> > On Mon, 2007-08-20 at 18:42 -0400, Phillip J. Eby
wrote:
> >
> > > Pardon me if any of this is already obvious,
but from your
> > > description it sounds like one or more
parties might not be aware of
> > > these points.
> >
> > One of the parties (Igor, Nginx's author) is
certainly not aware of
> > these points (hence his seeking a schematic so he
could get a quick
> > overview).  The author of the upcoming module is
aware but didn't offer
> > much in the way of explanation when Igor queried
him.
>
> If the person doing this is called Igor, then they
aren't the only
> person trying to implement an mod_wsgi for Nginx HTTP.
After dealing
> with this email, found a message in my spam box from
someone else
> doing the same thing. This other persons name is Manlio
Perillo.

>From the Nginx ML archives [1] it seems Manlio is the
person Cliff
refers to. Igor is the author of Nginx - not the one
writing
"mod_wsgi" for it.

Just to clarify..

Arnar

[1] ht
tp://news.gmane.org/gmane.comp.web.nginx.english
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Tue, 2007-08-21 at 09:29 +1000, Graham Dumpleton wrote:
> On 21/08/07, Cliff Wells <cliffdevelix.com> wrote:
> > On Mon, 2007-08-20 at 18:42 -0400, Phillip J. Eby
wrote:
> >
> > > Pardon me if any of this is already obvious,
but from your
> > > description it sounds like one or more
parties might not be aware of
> > > these points.
> >
> > One of the parties (Igor, Nginx's author) is
certainly not aware of
> > these points (hence his seeking a schematic so he
could get a quick
> > overview).  The author of the upcoming module is
aware but didn't offer
> > much in the way of explanation when Igor queried
him.
> 
> If the person doing this is called Igor, then they
aren't the only
> person trying to implement an mod_wsgi for Nginx HTTP.


No, Igor isn't trying to implement it.  He's the main Nginx
author.  I
think he was just trying to understand what it is people are
trying to
implement (and perhaps to understand if it's something he'd
want to
accept as part of Nginx proper - but I doubt that will
happen, for the
very reasons you outlined earlier).

> After dealing
> with this email, found a message in my spam box from
someone else
> doing the same thing. This other persons name is Manlio
Perillo. I
> have pointed them to this discussion on Google groups,
so maybe they
> will explain what they are up to and all parties can
get together. I
> have already indicated to Manlio that I may be
interested in providing
> him with assistance in doing for Nginx HTTP what I have
already done
> in Apache mod_wsgi.

Manlio is, in fact, the person I've been speaking of.   Your
help would
be terrific.  I'd actually meant to approach you about it
before, but
didn't know how interested you'd be in a Nginx port.

Cliff

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On 21/08/07, Cliff Wells <cliffdevelix.com> wrote:
> On Tue, 2007-08-21 at 09:29 +1000, Graham Dumpleton
wrote:
> > After dealing
> > with this email, found a message in my spam box
from someone else
> > doing the same thing. This other persons name is
Manlio Perillo. I
> > have pointed them to this discussion on Google
groups, so maybe they
> > will explain what they are up to and all parties
can get together. I
> > have already indicated to Manlio that I may be
interested in providing
> > him with assistance in doing for Nginx HTTP what I
have already done
> > in Apache mod_wsgi.
>
> Manlio is, in fact, the person I've been speaking of.  
Your help would
> be terrific.  I'd actually meant to approach you about
it before, but
> didn't know how interested you'd be in a Nginx port.

Because Apache mod_wsgi is to a degree quite wedded to the
Apache
APIs, not sure I'd really call it a port, although there are
certainly
things that could be learned from Apache mod_wsgi.

As I mentioned to Manlio one of the decisions that probably
has to be
made early on is whether you want to try and support
multiple Python
sub interpreters in a single process like mod_python and
mod_wsgi do.
Although this can be useful in some circumstances it isn't
always
practical as some third party extension modules will not
work in
secondary Python sub interpreters.

Thus, to make it a lot simpler, you might only support use
of the main
Python interpreter like would be the case if using pure
Python web
server. Doing this would eliminate a lot of the complexity
around
management of Python thread APIs and creation/destruction of
Python
sub interpreters.

But then, if you cut things back and go for a simpler
design, it is
arguable as to whether you may as well just use standard
Python
program and pure Python web server behind a proxy, of flup
behind
FASTCGI. Therefore comes down to what you want to achieve in
doing it
and whether the benefit is really there and that it offers
something
extra over these other existing approaches.

Graham
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com