Application packages

We briefly discussed this some time ago, just wanted to
bring it up 
again and maybe get some fresh ideas as it is still
something I am 
interested in doing. And also ask a sanity check from those
with better 
knowledge of the AOLserver core.

The goal would be to have simple deployable application
modules, quite 
like a "WAR" file in J2EE. Let's codename it
"AOLserver Application 
Package", or AAP for short. (yes, that is Dutch for
"monkey")

This would give the advantage that it would be easy to
deploy any open 
source applications written for AOLserver without having to
copy loads 
of files into the right location or do much editing the
global nsd.tcl. 
Just drop it in and go. You could even do it the Tomcat way;
just drop 
the .aap file in the "applications" directory
and it will unpack and 
bind it to a url automatically.

In the simplest form this would look something like:

/modules/tcl/ <-- Tcl library files
/pages/ <-- adp, html, images, etc.
/config.tcl <-- application level config file

You should then be able to deploy this application to a
root, like 
"/openacs", or simply "/".
(developers need to make sure they only use 
relative urls in their html!)

Initialy I thought this was something that needed to go into
the core, 
but now I am thinking a module could do it. You could load
the "ns_aap" 
module in the server of choice with a section like 
"ns/server/server1/aap" listing the applications
to be loaded and 
"ns/server/server1/aap/my_aap" can list config
values for the AAP in 
question.

Using functionality found in ns_rewrite (where did that
disappear to?) 
we should be able to map anything under the designated root
to the AAP's 
pages directory.

Now for the hard parts:

- Library loading: Could the module instruct the server to
source the 
Tcl library files from the application? Ns_TclInitModule
does something 
like it, but is limited to the server's modules/tcl
directory. It also 
stores it by name in a Tcl list, so I guess the path
expansion happens 
further down. That may need a core change after all.

- Application protection: an application that is a good
citizen could 
run in the normal shared interpreter pool, but applications
not using 
their own namespaces or non-prefixed nsv keys need to be
seperated. I 
know you can have different thread pools, but would it be
possible to a) 
create these at runtime by the ns_aap module and b) load
them with the 
application's Tcl lib, not the server global one? I do
consider this to 
be an optional one, I could live without it; developers just
have to 
follow the rules.

- registered procedures and traces. The root should be taken
into 
account when registering these in the the AAP's init.tcl. I
guess the 
module could overwrite the appropriate procedures with
wrapper that does 
just that before the application's Tcl lib is sourced.

I'd love to hear back from the usual suspects (and others!)
about their 
views on the implementation of this.

Cheers,
Bas.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<listservlistserv.aol.com> with the
body of "SIGNOFF AOLSERVER" in the email
message. You can leave the Subject: field of your email
blank.

Bas,

Good ideas! I'm using a 'package' structure which does
some of the things that 
you are talking about. In some ways the directory/file
structure is modeled 
on the ACS4/OACS structure. The big difference is that it is
entirely file 
based. That is configuration isn't in the database, you
don't even need to 
use a database with this configuration. The main reason is
so that you can 
quickly tar/gz up an application and move/replicate it
somewhere else with 
minor changes, if any.

In support of this effort, I also have a new configuration
file setup. Every 
virtual server has a new directory called config:
servers/server1/
                      modules/
                                  tcl/
                                      twt/
                                           init.tcl (sourced
to start)
                                           packages/
                                                        db/
                                                           
tcl/
                                                           
templates/
                                                           
queries/
                      config/
                              server.tcl (all sections which
have /$server/ )
                              nslog.tcl (referenced from
/ns/$server/modules)
                              nscgi.tcl ..
                              mymodule.tcl (tcl or c module
in modules sect)
                      pages/
                      log/
                           /supervise/
                           run (multilog script)
                      supervise/
                      run (startup file, serverlog =
/dev/fd/1)


Anyway, the packaging isn't to the point of dropping in and
having the server 
automatically notice stuff, but each package/module has a
separate set of 
configuation files, so it is much easier to add/remove stuff
without editing 
a monolithic config file. To see how to set the local
configuration of 
packages look at how things are sourced starting at:
twt/init.tcl:

<http://rmadilo.com/m2/servers/rmadilo/modules/tcl/twt/
>
(somewhat out of date, I'll freshed up this today)

I haven't settled on a standard way of 'mounting' a
package, although
the session package filters anything directed at /session/*
and runs templates
from the sessions/templates directory. I think this is how
standard Tomcat 
packages work, based upon filters. 

On Monday 20 February 2006 06:52, Bas Scheffers wrote:
> Using functionality found in ns_rewrite (where did that
disappear to?)
> we should be able to map anything under the designated
root to the AAP's
> pages directory.
>

Actually ns_rewriteurl only redirects under the set
pageroot, so you can't use 
it for this purpose.

What you can do is to write a simple filter which rewrites
the url to some 
other place under pageroot which is blocked from direct
access. You could, 
for instance, map a url /mypackage/* to
/packages/mypackage/*, and just block 
direct access to /packages/*.

<http://rmadilo.com/files/nsrewrite/doc/nsrewriteurl.
html>

> Now for the hard parts:
>
> - Library loading: Could the module instruct the server
to source the
> Tcl library files from the application?
Ns_TclInitModule does something
> like it, but is limited to the server's modules/tcl
directory. It also
> stores it by name in a Tcl list, so I guess the path
expansion happens
> further down. That may need a core change after all.
>

Loading C level modules using the AOLserver api will likely
need to be done at 
the correct time, before tcl modules are sourced, so I
don't see how to jump 
ahead and read a tcl configuration file to load an AOLserver
C module. 
However, if it is just a Tcl .so module, you can do this
easily right now.


tom jackson


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<listservlistserv.aol.com> with the
body of "SIGNOFF AOLSERVER" in the email
message. You can leave the Subject: field of your email
blank.

Tom Jackson wrote:
> Actually ns_rewriteurl only redirects under the set
pageroot, so you can't use 
> it for this purpose.
Ah, I see, Ns_ConnRedirect it must be. I looked into the
source and it 
goes very deep just by URL, before expanding it to a path,
so not very 
useful indeed.

> What you can do is to write a simple filter which
rewrites the url to some 
> other place under pageroot which is blocked from direct
access. You could, 
Sound like that would be harder (config wise) for the users,
not optimal.

Hmmm, I think I just hit the jackpot, Ns_UrlToFileProc in
the 
NsServer->fastpath struct. There even is an example on
how to use it: 
http://www.aolserver.com/docs/devel/c/c-examples.html
(example 2: "Alias")

Looks like that does what I want it to do. That makes it
even simpler 
than I thought it would be, I was afraid I might have to do
a lot of 
this stuff manually. (ie: Ns_ConnReturnFile and the like)

> Loading C level modules using the AOLserver api will
likely need to be done at 
> the correct time, before tcl modules are sourced, so I
don't see how to jump 
> ahead and read a tcl configuration file to load an
AOLserver C module. 
> However, if it is just a Tcl .so module, you can do
this easily right now.
Reading http://www.aolserver.com/docs/devel/c/api/c-ch393.htm,
it sounds 
like any of your own C modules load before nsd starts
sourcing the Tcl 
libraries (modules/tcl), so you should be able to tell it to
go look 
somewhere else for them too. I'll keep looking. One down,
one to go.

Cheers,
Bas.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<listservlistserv.aol.com> with the
body of "SIGNOFF AOLSERVER" in the email
message. You can leave the Subject: field of your email
blank.

On Monday 20 February 2006 09:45, Bas Scheffers wrote:
> Tom Jackson wrote:

> > What you can do is to write a simple filter which
rewrites the url to
> > some other place under pageroot which is blocked
from direct access. You
> > could,
>
> Sound like that would be harder (config wise) for the
users, not optimal.
>
Check out my package-name-init.tcl files, this is where you
put filters so 
that they load with the package. My session package has the
best example of 
this, and also even uses ns_rewriteurl for urls which
contain an embeded 
session id. (SID). 

I think it is a mistake to put all files under the package, 
outside of 
pageroot, or at least to force this to be the case. It
essentially means that 
every file must be served by a filter. You can still put
your package page 
files under pageroot and then include a filter to just
rewrite the url, which 
will let AOLserver take over. 

> > Loading C level modules using the AOLserver api
will likely need to be
> > done at the correct time, before tcl modules are
sourced, so I don't see
> > how to jump ahead and read a tcl configuration
file to load an AOLserver
> > C module. However, if it is just a Tcl .so module,
you can do this easily
> > right now.
>
> Reading http://www.aolserver.com/docs/devel/c/api/c-ch393.htm,
it sounds
> like any of your own C modules load before nsd starts
sourcing the Tcl
> libraries (modules/tcl), so you should be able to tell
it to go look
> somewhere else for them too. I'll keep looking. One
down, one to go.

Every C module is listed in /ns/server/$server/modules and
has an exact path 
to the shared library file. Tcl modules contain only the
name of the 
directory under the private tcl directory to look for an
init.tcl file.

ns_section "ns/server/$server/modules"
ns_param my_c_module /path/to/my_c_module.so
ns_param my_tcl_module tcl

so /path/to... could anywhere on the machine.

my_tcl_module must be located under
$private_tcl_lib/my_tcl_module.

In my case all my packages are part of a tcl module twt, so
I never have to 
add anything to the server.tcl file when I change the
configuration of my 
packages.

tom jackson


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
<listservlistserv.aol.com> with the
body of "SIGNOFF AOLSERVER" in the email
message. You can leave the Subject: field of your email
blank.