TT3 Reserved words could be less reserved

I have included a test program that tests what I am
requesting.  I would like 
to see the status of reserved words become a little less
reserved.  There is 
a lot of intuition that can be gathered based upon the place
you are in. 

Basically it seems the only place that a reserved word
should be reserved is 
anywhere that a directive could be placed - that is at the
beginning of a 
tag, or anywhere where a post directive can be used.

Below are some sample vars and what the optimum (in my
opinion) output should 
be.  Intermixed with the output I have included the current
status of what 
TT2 does (anything that says fails means that the TT2 engine
currently 
fails).

Again - this is all subject to the grammar engine being able
to tell if it 
should be looking for a directive at any given point or not
(sort of how if 
perl6 is looking for a term or expression or block at
various states of the 
parse).

my $vars = {
    GET => 'named_get',
    get => 'lower_named_get',
    named_get => 'value of named_get',
    hold_get => 'GET',
};


1..15
ok 1 - use Template;
    # good that this works 

ok 2 - "[% GET %]" => ""
   # correct - GET is in a space where a directive is
possible (usually
   #  expected)

ok 3 - "[% GET GET %]" => "named_get"
   # fails - There isn't any ambiguity here.  We want to get
a variable named
   # GET

ok 4 - "[% GET get %]" =>
"lower_named_get"
    # correct - so long as ANYCASE isn't set

ok 5 - "[% GET = 1 %][% GET GET %]" =>
""
    # correct - the variable name is in a spot where a
directive is expected  
    # can't fix this

ok 6 - "[% SET GET = 1 %][% GET GET %]" =>
"1"
    # fails - There isn't any ambiguity here

ok 7 - "[% foo = {GET => 1} %][% foo.$hold_get
%]" => "1"
    # fails - Even though the key should be autoquoted - it
still fails

ok 8 - "[% 'GET' %]" => "GET"
   # correct - this works most likely because '' and
"" are a single parsed
   # token

ok 9 - "[% GETS %]1" => "1"
   # correct - looks like undefined variable access

ok 10 - "[% GET $hold_get %]" =>
"named_get"
    # correct - we can access the hash via named access
    # you could also do [% name='G' _ 'ET' ; GET $name %]
    # but that is a hack

ok 11 - "[% GET $GET %]" => "value of
named_get"
    # fails - even though it looks like a named access

ok 12 - "[% BLOCK GET %]hi[% END %][% PROCESS GET
%]" => "hi"
    # fails - though the blockname and autoquoted process
name shouldn't
    # confused by reserved words

ok 13 - "[% BLOCK foo %]hi[% END %][% PROCESS foo a =
GET %]" => "hi"
    # fails - process is looking for a named argument value
- should be
    # unambiguous

ok 14 - "[% BLOCK foo %]hi[% END %][% PROCESS foo GET =
1 %]" => ""
    # correct - There could be a post operative directive at
that point - so
    # you can't use a directive there.  Arguably you could
keep the definitive
    # list of which items are post operative and then only
that set would be
    # reserved at that location.

ok 15 - "[% BLOCK foo %]hi[% END %][% PROCESS foo IF
GET %]" => "hi"
    # fails - This should be fine - as IF is expecting a
value at that
    # position

Again - I don't know if this request is even possible with
the grammar engine.  
It does seem like it gives the template writers the most
flexibility without 
imposing unnecessary restrictions on them.  Then there is
the argument that 
letting users do this could produce some very ambiguous edge
cases (which 
they will do) - but I don't know if that is a good enough
reason to prevent 
other users from using reserved words in unambiguous
positions.

Love to here any feed back - and no I don't have code
patches to bring the 
Grammar.pm into alignment - but before work is done on it
I'd like to see 
what people's opinions are.

Paul Seamons

mailseamons.com wrote:
> I have included a test program that tests what I am
requesting.  I would like 
> to see the status of reserved words become a little
less reserved.  There is 
> a lot of intuition that can be gathered based upon the
place you are in. 

Yep, that's all part of TT3.  You can't do it (easily, if at
all) in TT2 
because we tokenise everything up front.  We see GET and
mark it as a 
keyword, and use that the drive the grammar forward (using
YAPP).

In TT3 the parse is recursive descent and knows where to
look for a keyword
and where it can safely look for a variable that might
otherwise look
like a keyword.

Same goes for operators.  You should be able to have
variables called 
'and' and 'or' if you really want to confuse people.

  [% IF IF and and or or %]   
      =>   <key:IF> <var:IF> <op:and>
<var:and> <op:or> <var:or>
     ...
  [% ELSIF ELSIF or or and and %]  => 
      =>   <key:ELSIF> <var:ELSIF>
<op:or> <var:or> <op:and> <var:and>

Obfuscated TT3 anyone?  

Rewriting the parser was always the big part of TT3.  Apart
from some
of the code generation and handling some of the more complex
directives,
it's more-or-less done.  Certainly most of the low-level
stuff like 
parsing TT statements.

One particularly shiny thing in TT3 is that you can use
explicit namespaces 
to disambiguate the language elements if necessary.

For example, where you currently write:

   [% INCLUDE $filename %] 

the leading '$' in filename tells the parser that a variable
name follows
rather than the unquoted filename it usually expects.  In
TT3 you'll also
be able to say:

   [% INCLUDE var:filename %]

And if you want a template, file, image or some other kind
of thingy where
you would normally get a variable, you can write:

   [% t = template:header %]   # [% t.modtime %], [%
t.metadata %], etc.
   [% f = file:blah.txt   %]   # [% f.modtime %], [% f.owner
%], etc.
   [% i = image:foo.gif   %]   # [% i.width %], [% i.height
%]
   [% x = xml:example.xml %]   # [%
x.dom.getElementsByTagName('h1') %]

All depending on what providers you've bound to your various
namespaces,
of course.  In essence, all TT variables and other
identifiers will be
addressable via URIs which makes things nicely, er, uniform.

Cheers
A


_______________________________________________
templates mailing list
templatestemplate-toolkit.org
http://lists.template-toolkit.org/mailman/listinfo/t
emplates