Standardized template API

Phillip J. Eby wrote:
> [back to the Web-SIG]
> 
> At 09:39 PM 1/31/2006 -0600, Ian Bicking wrote:
> 
>> How do you pass in variables?
> 
> 
> environ, or a nested variable therein

OK, if you invert that (put the environ in the variables)
then you 
get... variables, like in the original spec.

>>   How do you get non-string output?
> 
> 
> What kind of non-string output?  For doing what?

Like ElementTree output, that you might pipe through other 
transformations.

>  >But there's a kind of templating that occurs in a
typical application
>  >that is much more intimately tied to the calling
code, and shouldn't be
>  >pushed through an HTTP-like interface.
> 
> But that HTTP-like interface can be concealed in a
library wrapper for 
> systems that don't orient that way, and if you *don't*
supply such an 
> interface, then template systems that orient that way
are put at a 
> disadvantage that *can't* be worked around by library
wrapping.

I just don't get it... why do I care about the status or
headers from 
the template?  I don't want the template returning those
things.  I 
probably want the template returning a unicode string. 
Providing a 
content-type would be fine, but hardly important.

So, I want to send variables in, and I want to get a string
out... and 
that's what the original spec does.  Why phrase it in terms
of WSGI?


-- 
Ian Bicking  |  ianbcolorstudy.com  |  http://blog.ianbicking.org

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

At 10:32 PM 1/31/2006 -0600, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>[back to the Web-SIG]
>>At 09:39 PM 1/31/2006 -0600, Ian Bicking wrote:
>>
>>>How do you pass in variables?
>>
>>environ, or a nested variable therein
>
>OK, if you invert that (put the environ in the
variables) then you get... 
>variables, like in the original spec.

Well, if you're going to include the environ, you're halfway
to WSGI, so 
why bother making a new spec?  


>>>   How do you get non-string output?
>>
>>What kind of non-string output?  For doing what?
>
>Like ElementTree output, that you might pipe through
other transformations.

But how's that a *template* any more, then?


>>  >But there's a kind of templating that occurs
in a typical application
>>  >that is much more intimately tied to the
calling code, and shouldn't be
>>  >pushed through an HTTP-like interface.
>>But that HTTP-like interface can be concealed in a
library wrapper for 
>>systems that don't orient that way, and if you
*don't* supply such an 
>>interface, then template systems that orient that
way are put at a 
>>disadvantage that *can't* be worked around by
library wrapping.
>
>I just don't get it... why do I care about the status or
headers from the 
>template?

Because in some frameworks, the template is the resource and
it's what 
controls these things.


>  I don't want the template returning those things.

And other people do.  Zope 2 certainly wants its templates
to be able to 
control such things.  peak.web wants it.  ASP and PHP-style
templating 
systems want it.  I'm sure there are others.


>I probably want the template returning a unicode string.
 Providing a 
>content-type would be fine, but hardly important.

You asked what was wrong with the spec; this is a good
example.  It's 
conceived in the context of a very narrow set of frameworks
whose paradigms 
differ so little that they might as well be one framework to
begin with, at 
least with respect to how they treat templates.

That's not to say that there's anything wrong with them,
just that there is 
a lot of useful diversity in Python web frameworks and I'd
prefer not to 
rule out entire *classes* of web frameworks from
consideration, just 
because some other ideas are more familiar to some people.

We should use the wider interface in this case, because it
means everybody 
can play.  If you make a variables-to-strings interface,
only 
variables-to-strings frameworks and templates can play. 
It's 
straightforward to wrap variables-to-strings templates in a
trivial WSGI 
interface and to pull string output from a WSGI app (you
can, after all, 
just use the body and ignore the headers), but it's not
possible to support 
WSGI functionality inside a variables-to-strings paradigm.

Notice, by the way, that a resources-as-WSGI-apps approach
also enables 
approaches like WSGI servers that run templates straight
from a directory, 
ASP/PHP-style.  This is just *one* kind of paradigm that's
enabled by my 
counter-proposal but isn't practical with the
variables-to-string approaches.

Another paradigm that's enabled: imagine having a
"template" whose text is 
actually a Paste Deploy specification.  Now imagine that
Zope implemented 
the WSGI-templating proposal using "WSGI Methods"
that allowed you to 
select whatever "template engine" you wanted, and
allowed editing the text 
through the web.  Now, you could deploy a Paste-based
application inside a 
folder in a Zope site!

So, those are just two ideas I pulled out of my hat for what
a 
WSGI-templating spec could permit, that
variables-and-strings can't match.

It's not so much that the vars-and-strings proposal is
*wrong* per se, it's 
just that compared to my counterproposal, it just ain't
right.  A "good is 
the enemy of the best" scenario is what I'm worried
about here, which is 
why even though the TG/B interface is a *good* thing in and
of itself, I 
don't want it to get promoted outside its area of expertise.
 The Web-SIG 
should put forth a WSGI-based templating API, because it's
better for the 
Python web platform as a whole.

The only cost that I see is disadvantaging frameworks that
don't make good 
use of WSGI, and that was always going to be a natural
consequence of 
having a standard in the first place.  (i.e., some will
always make better 
use of it than others, and the better ones will have a
better time of it.)

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

do either of these specs address the concept of
componentized  
templates ?  that is, when you are referencing a template,
compiling  
its output, and delivering its textual output, in reality
that  
template would consist of possibly dozens of smaller
sub-templates,  
to form different parts of the page, and also to form
various nesting  
schemes.  In myghty, the resolver stack, i.e. the thing that
takes  
some kind of string, runs it through some rules (which can
be a  
straight down list or a whole structure of conditionals and
groups),  
and produces a template object (actually called a
component), is not  
used only for handling the initial URL from an HTTP request.
 It is  
used entirely within the request for every sub-component,
nested- 
template, subrequest, template inheritance scheme,
what-have-you, so  
that a single request actually calls a lot of templates
which call  
each other, all using the same resolver stack to find
things.  Extra  
contextual information can also be passed to the resolver to
give it  
more things with which to make decisions; such as the
"context",  
which can say "im the top-level request calling
you", "im a sub- 
component calling you", "im your super-(inherited)
parent calling  
you", etc.   Information about the resolved component
is available as  
well, such as getting back the re.Match object that resulted
from a  
regexp call, or other parameters that indicate things about
the  
resolver rule that actually matched.   All of that stuff is
important.

A cross-template resolution/execution API introduces the
concept of  
one kind of template including snips from other kinds of
templates,  
or someone using a template-inheritance-enabled system to
enable that  
capability for a system that doesnt have that feature, etc. 
 So it  
would be nice if this system could support that use case,
and not  
just be request-oriented.

When I first wrote WSGI support for Myghty and saw what Ian
was doing  
with Paste, I thought, wow, why dont I take my component
model and  
make them all into little WSGI application() objects...but
in  
reality, for those sub-templates, the typical HTTP
request/response  
as well as the start_response doesnt really seem
appropriate,  
especially for the kinds of interactions those components
have with  
each other, such as intricate wrapping schemes, iterative
patterns.   
Also i dont believe any templating system should ever be
hardcoded to  
an "HTTP request" concept....too many places have
we had JSP or PHP  
sites that hit a brick wall when we want to use the same
templates to  
pre-generate files, or emails, etc. because those systems
have no  
notion of non-HTTP usage.

Forgive my rambling, I havent gotten to dig deeply into
Ian's spec  
and I'm still trying to get an idea of what the more
encompassing  
spec is about  (like what is template polymorphism exactly
?)

- mike

On Feb 1, 2006, at 12:22 AM, Phillip J. Eby wrote:

> At 10:32 PM 1/31/2006 -0600, Ian Bicking wrote:
>> Phillip J. Eby wrote:
>>> [back to the Web-SIG]
>>> At 09:39 PM 1/31/2006 -0600, Ian Bicking wrote:
>>>
>>>> How do you pass in variables?
>>>
>>> environ, or a nested variable therein
>>
>> OK, if you invert that (put the environ in the
variables) then you  
>> get...
>> variables, like in the original spec.
>
> Well, if you're going to include the environ, you're
halfway to  
> WSGI, so
> why bother making a new spec?  
>
>
>>>>   How do you get non-string output?
>>>
>>> What kind of non-string output?  For doing
what?
>>
>> Like ElementTree output, that you might pipe
through other  
>> transformations.
>
> But how's that a *template* any more, then?
>
>
>>>> But there's a kind of templating that
occurs in a typical  
>>>> application
>>>> that is much more intimately tied to the
calling code, and  
>>>> shouldn't be
>>>> pushed through an HTTP-like interface.
>>> But that HTTP-like interface can be concealed
in a library  
>>> wrapper for
>>> systems that don't orient that way, and if you
*don't* supply  
>>> such an
>>> interface, then template systems that orient
that way are put at a
>>> disadvantage that *can't* be worked around by
library wrapping.
>>
>> I just don't get it... why do I care about the
status or headers  
>> from the
>> template?
>
> Because in some frameworks, the template is the
resource and it's what
> controls these things.
>
>
>>  I don't want the template returning those things.
>
> And other people do.  Zope 2 certainly wants its
templates to be  
> able to
> control such things.  peak.web wants it.  ASP and
PHP-style templating
> systems want it.  I'm sure there are others.
>
>
>> I probably want the template returning a unicode
string.  Providing a
>> content-type would be fine, but hardly important.
>
> You asked what was wrong with the spec; this is a good
example.  It's
> conceived in the context of a very narrow set of
frameworks whose  
> paradigms
> differ so little that they might as well be one
framework to begin  
> with, at
> least with respect to how they treat templates.
>
> That's not to say that there's anything wrong with
them, just that  
> there is
> a lot of useful diversity in Python web frameworks and
I'd prefer  
> not to
> rule out entire *classes* of web frameworks from
consideration, just
> because some other ideas are more familiar to some
people.
>
> We should use the wider interface in this case, because
it means  
> everybody
> can play.  If you make a variables-to-strings
interface, only
> variables-to-strings frameworks and templates can play.
 It's
> straightforward to wrap variables-to-strings templates
in a trivial  
> WSGI
> interface and to pull string output from a WSGI app
(you can, after  
> all,
> just use the body and ignore the headers), but it's not
possible to  
> support
> WSGI functionality inside a variables-to-strings
paradigm.
>
> Notice, by the way, that a resources-as-WSGI-apps
approach also  
> enables
> approaches like WSGI servers that run templates
straight from a  
> directory,
> ASP/PHP-style.  This is just *one* kind of paradigm
that's enabled  
> by my
> counter-proposal but isn't practical with the
variables-to-string  
> approaches.
>
> Another paradigm that's enabled: imagine having a
"template" whose  
> text is
> actually a Paste Deploy specification.  Now imagine
that Zope  
> implemented
> the WSGI-templating proposal using "WSGI
Methods" that allowed you to
> select whatever "template engine" you wanted,
and allowed editing  
> the text
> through the web.  Now, you could deploy a Paste-based
application  
> inside a
> folder in a Zope site!
>
> So, those are just two ideas I pulled out of my hat for
what a
> WSGI-templating spec could permit, that
variables-and-strings can't  
> match.
>
> It's not so much that the vars-and-strings proposal is
*wrong* per  
> se, it's
> just that compared to my counterproposal, it just ain't
right.  A  
> "good is
> the enemy of the best" scenario is what I'm
worried about here,  
> which is
> why even though the TG/B interface is a *good* thing in
and of  
> itself, I
> don't want it to get promoted outside its area of
expertise.  The  
> Web-SIG
> should put forth a WSGI-based templating API, because
it's better  
> for the
> Python web platform as a whole.
>
> The only cost that I see is disadvantaging frameworks
that don't  
> make good
> use of WSGI, and that was always going to be a natural
consequence of
> having a standard in the first place.  (i.e., some will
always make  
> better
> use of it than others, and the better ones will have a
better time  
> of it.)
>
> _______________________________________________
> Web-SIG mailing list
> Web-SIGpython.org
> Web SIG: http://www.python.
org/sigs/web-sig
> Unsubscribe: http://mail.python.org/mailman/options/web-sig/mike_mp% 
> 40zzzcomputing.com

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

Michael Bayer wrote:
> do either of these specs address the concept of
componentized  templates 
> ?  that is, when you are referencing a template,
compiling  its output, 
> and delivering its textual output, in reality that 
template would 
> consist of possibly dozens of smaller sub-templates, 
to form different 
> parts of the page, and also to form various nesting 
schemes.  In 
> myghty, the resolver stack, i.e. the thing that takes 
some kind of 
> string, runs it through some rules (which can be a 
straight down list 
> or a whole structure of conditionals and groups),  and
produces a 
> template object (actually called a component), is not 
used only for 
> handling the initial URL from an HTTP request.  

I think you can do this with my second proposal, which uses 
find_resource.  In that model I imagine something like:

   find_resource = MyghtyResourceFinder(request info...)
   plugin = MyghtyPlugin()
   tmpl = find_resource(plugin, '/index.myt')
   plugin.render(tmpl, {})

The tmpl object knows that it is named '/index.myt', and it
has a 
reference to find_resource, so it can use that to find
sub-templates. 
The only context that is passed to find_resource in that
model is the 
(optional) template that is getting the new template. 
There's no 
subrequests, and maybe not some of the other things you are 
considering... but that might not be a problem, I'm not
sure.

Because when you get a subtemplate with find_resource you
are just 
*getting* it, you aren't filling it quite yet.  So if you
phrase that 
inclusion as a subrequest, maybe that just means that you
fill that 
subtemplate with different arguments that indicate that. 
Unless you 
need to know that it is a subrequest (beyond just checking
for the 
presence of the relative_to argument), or need to know the
chain of 
calls or something.


-- 
Ian Bicking  |  ianbcolorstudy.com  |  http://blog.ianbicking.org

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

At 02:03 AM 2/1/2006 -0500, Michael Bayer wrote:
>do either of these specs address the concept of
componentized
>templates ?  that is, when you are referencing a
template, compiling
>its output, and delivering its textual output, in
reality that
>template would consist of possibly dozens of smaller
sub-templates,
>to form different parts of the page, and also to form
various nesting
>schemes.  In myghty, the resolver stack, i.e. the thing
that takes
>some kind of string, runs it through some rules (which
can be a
>straight down list or a whole structure of conditionals
and groups),
>and produces a template object (actually called a
component), is not
>used only for handling the initial URL from an HTTP
request.  It is
>used entirely within the request for every
sub-component, nested- 
>template, subrequest, template inheritance scheme,
what-have-you, so
>that a single request actually calls a lot of templates
which call
>each other, all using the same resolver stack to find
things.  Extra
>contextual information can also be passed to the
resolver to give it
>more things with which to make decisions; such as the
"context",
>which can say "im the top-level request calling
you", "im a sub- component 
>calling you", "im your super-(inherited)
parent calling
>you", etc.   Information about the resolved
component is available as
>well, such as getting back the re.Match object that
resulted from a
>regexp call, or other parameters that indicate things
about the
>resolver rule that actually matched.   All of that stuff
is important.

My assumption is that framework-specific functionality of
this nature would 
be made available via standard WSGI "server extension
APIs", which is to 
say objects or callbacks inside of the 'environ' dict.  So,
you might have 
a 'myghty.resolver' key in environ to provide access to that
API.


>A cross-template resolution/execution API introduces the
concept of
>one kind of template including snips from other kinds of
templates,
>or someone using a template-inheritance-enabled system
to enable that
>capability for a system that doesnt have that feature,
etc.   So it
>would be nice if this system could support that use
case, and not
>just be request-oriented.
>
>When I first wrote WSGI support for Myghty and saw what
Ian was doing
>with Paste, I thought, wow, why dont I take my component
model and
>make them all into little WSGI application()
objects...but in
>reality, for those sub-templates, the typical HTTP
request/response
>as well as the start_response doesnt really seem
appropriate,
>especially for the kinds of interactions those
components have with
>each other, such as intricate wrapping schemes,
iterative patterns.

Note that different templating engines have different
notions of what a 
sub-template would consist of; some think of them as
functions, others as 
DOM trees, and still others as strings or objects
implementing some kind of 
rendering interface.  Some need subtemplates that are
effectively bound as 
methods of some object being rendered, so there's a need for
a "self" 
parameter to the subtemplate.

Thus, it seems to me that all this stuff needs to remain on
the template 
side of the API to start with, as it isn't currently
generalizable to some 
sort of generic API.  (Which is why my proposal doesn't
currently include a 
standard way to find other templates.)



>Also i dont believe any templating system should ever be
hardcoded to
>an "HTTP request" concept....too many places
have we had JSP or PHP
>sites that hit a brick wall when we want to use the same
templates to
>pre-generate files, or emails, etc. because those
systems have no
>notion of non-HTTP usage.

Sorry, you lost me there.  I see no reason why you can't use
a WSGI 
response to write to a file or send an email.


>Forgive my rambling, I havent gotten to dig deeply into
Ian's spec
>and I'm still trying to get an idea of what the more
encompassing
>spec is about  (like what is template polymorphism
exactly ?)

Specifying only a template's *name* in code, not its type. 
In peak.web, if 
I refer to a resource or template named "foo", the
system looks in the 
resource directory and determines the template type based on

extensions.  So if there's a foo.pwt or foo.html or foo.gif
or foo.jpeg, it 
determines the resource type from that.  When you combine
this feature with 
skinning, it means that deployers are free to change the
technology used to 
deliver a particular template or resource.  For example,
replacing a .gif 
or .jpeg with a .png or vice versa, or using a different
template engine 
for a template.

I don't know if any other frameworks support it; IIRC
TurboGears currently 
embeds the template engine name inside the code that refers
to the template.

Anyway, resource polymorphism is a deep assumption in
peak.web, and it 
obviously affects how it does subtemplate lookups -- another
reason why I 
think we should start with a spec that doesn't get into
subtemplating to 
start with.

There's nothing stopping the a WSGI-based templating
proposal from 
providing some kind of subtemplate support, though; it's
just an extension 
API after all.  What I'm opposed to is giving official
status to 
half-solutions to the subtemplating problem that only work
with a few 
string-oriented templates and frameworks.

However, I'm thinking maybe we could take a step in the
direction of making 
it easier for templating systems to play without needing
full WSGI 
support.  We could require a WSGI templating environ to
include a 
'wti.return_string' key, defined such that you could do
something like this:

      def template(environ, start_response):
          return environ['wti.return_string'](
              start_response, 'text/plain', "rendered
text"
          )

This would make it trivial to wrap string-based template
engines in WTI 
(the WSGI Template Interface).  Heck, it might not be a bad
idea to have a 
key like that in WSGI 1.1!

But I digress.  What I think we should do as we figure out
other models to 
return for things besides strings, we can add other
'wti.return_foo' keys 
to allow templates to return non-string things that can
nonetheless be 
reduced to strings if middleware is in play and needs them.

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Feb 1, 2006, at 11:07 AM, Phillip J. Eby wrote:
>> spec is about  (like what is template polymorphism
exactly ?)
>
> Specifying only a template's *name* in code, not its
type.  In  
> peak.web, if I refer to a resource or template named
"foo", the  
> system looks in the resource directory and determines
the template  
> type based on extensions.  So if there's a foo.pwt or
foo.html or  
> foo.gif or foo.jpeg, it determines the resource type
from that.   
> When you combine this feature with skinning, it means
that  
> deployers are free to change the technology used to
deliver a  
> particular template or resource.  For example,
replacing a .gif  
> or .jpeg with a .png or vice versa, or using a
different template  
> engine for a template.
>

It seems like myghty resolver does this as well...you send
in a URI,  
and as it walks down the chain, it can hit FileResolver
(returns file- 
based templates), ModuleResolver (returns python callables),
each of  
which are free to return whatever Component object they want
if they  
have something to offer for that URI; each Component just
has a run()  
method.   you can inject whatever customized rule you want
to return  
something else at some point.  rules could be constructed to
return  
an adapted version of any other template's template object
too.

_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

At 12:12 PM 2/1/2006 -0500, Michael Bayer wrote:

>On Feb 1, 2006, at 11:07 AM, Phillip J. Eby wrote:
>>>spec is about  (like what is template
polymorphism exactly ?)
>>
>>Specifying only a template's *name* in code, not its
type.  In
>>peak.web, if I refer to a resource or template named
"foo", the
>>system looks in the resource directory and
determines the template
>>type based on extensions.  So if there's a foo.pwt
or foo.html or
>>foo.gif or foo.jpeg, it determines the resource type
from that.
>>When you combine this feature with skinning, it
means that
>>deployers are free to change the technology used to
deliver a
>>particular template or resource.  For example,
replacing a .gif
>>or .jpeg with a .png or vice versa, or using a
different template
>>engine for a template.
>
>It seems like myghty resolver does this as well...you
send in a URI,
>and as it walks down the chain, it can hit FileResolver
(returns file- 
>based templates), ModuleResolver (returns python
callables), each of
>which are free to return whatever Component object they
want if they
>have something to offer for that URI; each Component
just has a run()
>method.   you can inject whatever customized rule you
want to return
>something else at some point.  rules could be
constructed to return
>an adapted version of any other template's template
object too.

So asking for "foo" can resolve to
"foo.gif" or "foo.jpeg" in the file 
system?  If you have to say in the code which it is, that's
not what I mean.





_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com

On Feb 1, 2006, at 12:59 PM, Phillip J. Eby wrote:

> At 1
>
> So asking for "foo" can resolve to
"foo.gif" or "foo.jpeg" in the  
> file system?  If you have to say in the code which it
is, that's  
> not what I mean.

you can write a resolver that searches the filesystem for
"foo.*" and  
does some type calculation based on the extension it
located, sure.   
or you get a similar effect with two resolvers in a chain,
one which  
is configured to look for "<name>.jpeg" and
return a jpeg-template,  
and one configured to look for "<name>.gif"
to return a gif- 
template.  you can switch the order of those resolvers to
specify one  
extension taking precendence over the other.  the common use
case in  
myghty is looking for ".py", which gives you a
ModuleComponent, or  
".myt" or another file-based extension which gives
you a FileComponent.
_______________________________________________
Web-SIG mailing list
Web-SIGpython.org
Web SIG: http://www.python.
org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/bo
nd%40yahoo.com