Proposal for new automation architecture

On Feb 10, 2006, at 10:04 AM, Philippe Bossut wrote:

>
>
> John Anderson wrote:
>> Having written my first functional test yesterday I
have some  
>> thoughts. The biggest problem I encountered when
trying to write  
>> and debug tests is navigating all the layers:
>>
>> my test <-> CATS <-> CPIA Script
<-> Chandler
>>
>> Fortunately I'm very familiar with Chandler,
somewhat familiar  
>> with CPIA Script and CATS is small enough to grock
without much  
>> effort. However, I suspect most developers would
find all the  
>> layers daunting and trying to debug things would
only make it worse.
> Agree with that.

One of the requirements is that the system be easy to use.
Obviously  
there is another layer of complexity over what we do with
CATS but it  
is still designed to be very easy for someone to pick up and
start  
writing scripts and to see legible output. Part of the
deliverables  
for the first version of this framework will be;

-Command line python wrapper (much like do_tests, a script
is  
imported and output is generated that is legible using a set
of  
default parameters for the framework)
-Sufficient Documentation ("Writting Chandler
automation in 10  
minutes" style doc, extended OAF documentation for
developers who  
wish to use non-default features in the system, and maybe
most  
importantly GOOD documentation for the chandler test library
that can  
facilitate both easy test script authoring and developer
improvements  
to the chandler testing library itself.

The output can be very customized using this framework, but
the  
default output will be humanly legible and go directly to a
file.

Also, a -debug flag can be set, which sets all output in the
 
framework to be processed as it comes in to the output
object. This  
is no good for performance tests but will make debugging
issues  
worlds easier than in CATS.

To finish up, many of the extra layers that developer might
find  
"daunting" will be transparent in the
implementation, but the output  
that developers depend on (such as a tracebacks in the log
if a  
failure occures) are made easy and reliable by this
abstraction.

I hope this alleviates your concerns.

-Mikeal

>> I think it would be preferable to make the small
changes necessary  
>> to CPIA Script to make it appropriate for testing
instead of  
>> adding another layer, e.g. CATS.
> Improving CPIA Script to make scripting easier is
indeed a good  
> idea. I don't think it will replace entirely a test
harness though  
> like CATS or, better, OAF (proposed by Mikeal). There's
a lot of  
> test functions (batch, log, data gathering and stats)
that have no  
> place in a Chandler level scripting language. John, I
suggest you  
> read Mikeal proposal (http://wiki.o
safoundation.org/bin/view/ 
> Projects/OpenAutomationFramework) first. Keep in mind
also that  
> Mikeal is trying to solve a problem that includes
Chandler and Cosmo.
>> Similarly, I think it's preferable to modify
Chandler to eliminate  
>> some of CPIA Script.
> What alternative to CPIA scripting do you propose? No
scripting at  
> all? Another script mechanism? Leverage an existing
one?
>
> Cheers,
> - Philippe
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "Dev"
mailing list
> h
ttp://lists.osafoundation.org/mailman/listinfo/dev

_______________________________________________
Cosmo mailing list
Cosmoosafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo